METHOD AND SYSTEM FOR EXEMPLARY CAMPAIGN MESSAGE GENERATION

Abstract

Methods and systems for improved and efficient exemplary campaign message generation are disclosed. By automating the exemplary campaign message generation process, the system can improve the selection efficiency and quality of these messages, which can be used to inspire prospective campaign generators. Furthermore, the selected exemplary messages are dynamically diversified to meet the needs of different campaign generators.

Description

CROSS-REFERENCES TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional Patent Application No. 63/393,914, entitled “METHOD AND SYSTEM FOR EXEMPLARY CAMPAIGN MESSAGE CLASSIFICATION, GENERATION AND MANAGEMENT,” filed Jul. 31, 2022, which is incorporated herein by reference for all purposes.

TECHNICAL FIELD

The present subject matter relates to systems and methods for data integration and management of electronic messages. More specifically, the present subject matter relates to data management in the context of computer-aided management of electronic messages.

BACKGROUND

Digital marketing campaign has become a powerful way to promote a business's products and services to its customers. It helps the customers to be aware of the business's latest offerings. Among digital marketing's various tools, computer-compiled campaign messages such as emails and mobile text messages can directly deliver the most relevant content to a customer.

Various providers have used message examples to motivate business marketers, such as ReallyGoodEmails or Milled. However, their selections of message examples are limited. For example, some examples are based on manual reviews and voluntary user submissions, which are time consuming and low-efficient; some examples lack a standardized selection process with quality control. In addition, these existing example providers do not provide insights into the performance of these message examples. Furthermore, these examples often are not diverse enough to offer a tailored experience for a specific business marketer.

As such, there are a lot of areas for improving the computing system performance in the classification, generation, and management of exemplary campaign messages via an improved approach.

SUMMARY OF THE INVENTION

The present subject matter describes improved methods for exemplary campaign message classification, generation, and management via a computer network. The system can dynamically select and showcase high-performance campaign messages with consideration for the messages' relevance, diversity, and breath. As such, these showcase exemplary messages can represent a broad range of successful marketing campaigns, which can be used to inspire and teach other campaigns that share similar traits and preferences with the previous campaigns.

Via an automated process, the system can generate a browsable collection of diverse, high-performing campaign messages, e.g., emails and SMS messages, that can prompt a user or a prospective campaign generator to create quality content for his/her own campaigns. Furthermore, varied exemplary campaigns can be shown to different users in response to his/her unique needs or expressed interests.

As disclosed in the present subject matter, the exemplary campaign message system can continuously extract performance metrics and data of all previously sent campaign messages. The system can preprocess the received message data, for example, to filter out non-compliant messages. The resulted messages are further subjected to a bucketing process that can segment the campaign messages into campaign peer groups and select respective exemplary campaign messages with performance metrics higher than a predetermined threshold. As the campaign messages are divided into different campaign groups, their performance data can be meaningfully compared for selecting the high-performance messages. Furthermore, to generate a diverse campaign message collection suitable for different campaign generators, the system can adopt a respective performance threshold for each respective campaign group.

Next, the system can remove unsupported campaign messages from the automatically generated exemplary campaign messages and, if needed, enable a manual review and moderation of the remaining messages. During the moderation process, a curator can remove inappropriate messages or poor design-quality messages. Furthermore, for each approved message, the curator can assign multiple tags, such as a campaign type, a discount code, a holiday, an industry type, a marketing channel, or a design quality grade, to facilitate the management and post-processing of the selected exemplary messages. According to some embodiments, instead of or in addition to a manual review, a moderation model can be trained to review and tag the selected campaign messages. For example, a moderation model that has been trained with messages with discount tags can automatically assign a discount code to a new message.

Furthermore, the system can initiate a consent process to obtain the previous campaign generator's consent. Upon receiving the consent to display or showcase, the selected and approved exemplary campaign messages can be displayed to a prospective campaign generator for creative inspiration.

During this process, the system can adopt a campaign type classifier to predicate any campaign message's category based on various data related to the message. Such data can comprise content components with layout data as well as metadata such as the subject line and the campaign name. The campaign type classifier can be an ensemble model that comprises at least a logic-rule model, a first language model, and a second language model configured to independently generate a predicted campaign type.

In addition, the system can rank, organize and present the selected campaign messages to showcase diversified campaign examples that have delivered excellent outcomes and matched the user's expressed interest or preference. Such sorting and displaying can be based on the assigned tags such as a campaign type, a discount code, a holiday tag, an industry tag, a quality-design tag, and a marketing channel tag. In particular, the system can selectively divide, merge and rank the campaign messages based on the assigned tags as well as other characteristic data of the messages.

Furthermore, the system can dynamically update and refresh the displayed campaign messages to the same user or marketer. For example, the prospective marketer can view different exemplary campaign messages every 24 hours, or at a specific interval predetermined by the system.

As such, the system can generate an expansive and growing list of relevant, diverse, and high-performing campaign messages to inspire a campaign generator to create engaging content of his/her own.

A computer implementation of the present subject matter comprises: receiving, at a campaign management server, a first plurality of campaign messages from previous campaign generators, extracting characteristics of the plurality of campaign messages, selecting a second plurality of campaign messages from the first plurality of campaign messages based on one or more filtering rules, segmenting the second plurality of campaign messages into campaign groups based on selected and extracted characteristics of the second plurality of campaign messages, selecting, for each of the campaign groups, exemplary campaign messages associated with performance metrics higher than a predetermine threshold, and displaying the exemplary campaign messages to a prospective campaign generator. Furthermore, the selected campaign messages also have breath and diversities in terms of their marketing channel, industry type, size of the campaign generator, targeting level, audience engagement, etc.

According to some embodiments, the characteristics comprise one or more performance metrics, campaign types, campaign attributes, campaign generator attributes, targeting levels, campaign timings, and campaign channels. These characteristics can be retrieved from databases associated with the first plurality of campaign messages.

According to some embodiments, a campaign type classifier can be adopted to predict a message's campaign type, which can be one of promotions, featured products, blogs and newsletters, notifications. Furthermore, an ensemble model can be used to predict a campaign type via a majority prediction based on the extracted image or text components data, the layout data, and available metadata associated with the campaign message. According to some embodiments, the ensemble model comprises at least one logic-rule model and one or more language models.

According to some embodiments, a scraper model is configured to generate the layout data by extracting size, location and visibility data of the image or text components of the campaign message.

According to some embodiments, the filtering rules are configured to remove non-compliant messages in a preprocessing stage. For example, the filtering rules can remove messages that are not from US companies, or from a specific country, to avoid a foreign language, or to limit the selection within one country; the filtering rules can also remove messages with obviously incorrect performance metrics, e.g., too high, or with missing values for features of interest; the filtering rules can further remove messages that do not pass a minimal performance threshold.

According to some embodiments, for the segmentation purpose during the bucketing process, some characteristics can be the campaign types, the campaign generator attributes, e.g., sizes and industries, and the targeting levels, or any other characteristics that can be used to divide the messages into campaign groups or peer groups. Furthermore, the campaign generator attributes comprise industry types and sizes of the previous campaign generators.

According to some embodiments, a number of parameters are adopted to determine the output of the bucketing process, such as the percentage of selected or kept messages per peer group/bucket, the minimum and/or maximum number of kept emails per peer group. Furthermore, the maximum number of kept messages can avoid excessive message volume from the largest peer groups. The minimum number of kept message can avoid skipping messages from the smaller peer groups.

According to some embodiments, the system can receive a performance threshold for selecting the exemplary campaign messages. Furthermore, the predetermine threshold for each of the campaign groups can be individually determined based on the campaign group size or other factors. According to some embodiments, the exemplary campaign messages are ranked in an ascending order according to the performance metrics.

According to some embodiments, the system can further filter or remove unsupported campaign messages from the selected exemplary campaign messages. One goal of the filtering process is to reduce the number of messages for further moderation or manual review. Examples of such unsupported campaign messages can be GIF-based messages, or whether the system can reach the previous campaign generator for obtaining the consent to display.

According to some embodiments, the system can enable a manual review and moderation of the remaining selected campaign messages. To make the review more efficient, the system can automatically generate a screenshot image of the campaign message. Additional information such as the message's subject line, the preview text, is also available for the curator. The curator can assign a number of tags to the reviewed messages for additional sorting, ranking and management of the messages. Examples of such tags can comprise a campaign type, a discount code, a holiday, an industry type, a marketing channel, or a design quality grade. According to some embodiments, instead of a manual review, a moderation model can be trained to review and tag the selected campaign messages. For example, a moderation model that has been trained with messages with discount tags can automatically assign a discount code to a new message. The output of the moderation process can be a list of the selected campaign messages with assigned tag, which can be further saved as JSON (JavaScript Object Notation) data structures in a database.

According to some embodiments, for each of the campaign message candidate, the system can initiate a consent process to obtain the previous campaign generator's consent for display. Upon receiving the consent to display or showcase, the selected and approved exemplary campaign messages can be displayed to a prospective campaign generator for creativity inspiration. Furthermore, the system can generate images of theses campaign messages for displaying.

According to some embodiments, to provide a constantly fresh inspiration for the showcase messages, the system can update exemplary campaign messages at a predetermined interval, and update the displayed campaign messages. According to some embodiments, the system can receive sorting input from the prospective campaign generator, and generate updated campaign messages pursuant to the input.

Other aspects and advantages of the present subject matter will become apparent from the following detailed description taken in conjunction with the accompanying drawings, which illustrate, by way of example, the principles of the present subject matter.

DESCRIPTION OF DRAWINGS

The present subject matter is illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings in which:

FIG. 1 shows an exemplary diagram of a campaign management system for exemplary campaign message generation, according to one or more embodiments of the present subject matter;

FIG. 2A shows an exemplary display of the exemplary campaign messages to a campaign generator, according to one or more embodiments of the present subject matter;

FIG. 2B shows another exemplary display of the exemplary campaign messages to the campaign generator, according to one or more embodiments of the present subject matter;

FIG. 3 shows some exemplary processes for exemplary campaign messages generation, according to one or more embodiments of the present subject matter;

FIG. 4 shows some exemplary processes for exemplary campaign messages generation, according to one or more embodiments of the present subject matter;

FIG. 5A shows a server system of rack-mounted blades, according to one or more embodiments of the present subject matter;

FIG. 5B shows a diagram of a networked data center server, according to one or more embodiments of the present subject matter;

FIG. 6A shows a packaged system-on-chip device, according to one or more embodiments of the present subject matter; and

FIG. 6B shows a block diagram of a system-on-chip, according to one or more embodiments of the present subject matter.

DETAILED DESCRIPTION

The present subject matter pertains to improved approaches for an exemplary campaign message system. Embodiments of the present subject matter are discussed below with reference to FIGS. 1-6.

In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present subject matter. It will be apparent, however, to one skilled in the art that the present subject matter may be practiced without some of these specific details. In addition, the following description provides examples, and the accompanying drawings show various examples for the purposes of illustration. Moreover, these examples should not be construed in a limiting sense as they are merely intended to provide examples of embodiments of the subject matter rather than to provide an exhaustive list of all possible implementations. In other instances, well-known structures and devices are shown in block diagram form in order to avoid obscuring the details of the disclosed features of various described embodiments.

The following sections describe systems of process steps and systems of machine components for efficiently generating and managing exemplary high-performing campaign messages with diversity and relevance from a large number of available messages. These can be implemented with computers that execute software instructions stored on non-transitory computer-readable media. An improved exemplary campaign message system can have one or more of the features described below.

FIG. 1A shows an exemplary diagram of an exemplary campaign message system 100 for efficiently extracting high-performing and diverse campaign messages according to the present subject matter. A campaign message can be any computer-compiled electronic messages such as emails and Short Message Service (SMS) messages that can deliver relevant content to its targeted customers by a marketer or campaign generator.

Campaign management server 101 can comprise a number of modules or units to implement functions of the present subject matter. According to some embodiments, campaign management server 101 can implement functions related to message characteristics extraction 120, preprocessing 122, filtering 124, campaign type classification 126, bucketing 128 and moderation 130. Other relevant functions, such as consent flow initiation and management, message management and presentation, can also be implemented by campaign management server 101.

As shown in FIG. 1, campaign management server 101 can receive a large number of previous campaign messages 138 from its numerous campaign generators. According to some embodiments, the campaign management server 101 can extract all campaign messages that were created or sent through the server. Alternatively, the campaign management server 101 can extract all campaign messages in a fixed or preferred timeline, for example, messages sent in the past 30 days. The previous campaign messages 138 can comprise, for example, text message 132 sent to a mobile device, and marketing emails 134 and 136 sent to a computing device.

Network 114 can comprise a single network or a combination of multiple networks, such as the Internet or intranets, wireless cellular networks, local area network (LAN), wide area network (WAN), WiFi, Bluetooth, near-field communication (NFC), etc. Network 114 can comprise a mixture of private and public networks or one or more local area networks (LANs) and wide-area networks (WANs) that may be implemented by various technologies and standards.

According to some embodiments, campaign management server 101 can extract performance metrics and characteristics data of all previously sent campaign messages 138 via executing a management command. According to some embodiments, performance metrics can be related to high click-through rate or high open rate of the campaign messages. In addition, performance metrics can also comprise conversion rate or average revenue generated by the conversion. According to some embodiments, the system can define a threshold for high performance metrics based on collected data, such as over 30% open rate, or over 2% click-through rate. According to some embodiments, different types of campaigns can be associated with different performance metrics. For example, a promotion campaign's predetermined high performance metrics could be different from that of a blog/newsletter campaign. According to some embodiments, a weighted and hybrid performance metrics with consideration of various rates and factors, e.g., the conversion rate, the click-through rate, the open rate, the average revenue, can be adopted by the system.

According to some embodiments, campaign management server 101 can remove messages with evident metric errors. An example can be a campaign message that was sent to a small number of recipients since their performance metrics can be skewed due to excessive targeting or statistical randomness.

According to some embodiments, the characteristics comprise, for example, performance metrics, campaign types, campaign attributes, campaign generator attributes, targeting levels, campaign timings, and campaign channels.

According to some embodiments, based on its intended goal, a campaign message can be categorized by campaign management server 101 as one of the following types: promotions, featured products, blogs and newsletters, and notifications. For example, promotions and featured products campaigns are created to generate revenue for the marketers, whereas blogs and newsletters campaigns aim to deliver blog/newsletter/entertainment/educational content to its audiences, and notifications campaigns are intended to deliver specific information to the audiences.

According to some embodiments, the campaign type classification 126 can be implemented by a campaign type classifier that can predict any campaign message's category/type based on various data related to the message. Such data can comprise, for example, content components with layout data, and metadata such as the subject line and the campaign name. The campaign type classifier can be an ensemble model that comprises at least a logic-rule model, a first language model, and a second language model configured to independently generate a predicted campaign type.

According to some embodiments, a scraper model is configured to generate the layout data by extracting size, location, and visibility data of the image or text components of the campaign message.

Campaign management server 101 can preprocess the received message data, for example, to filter out non-compliant messages via preprocessing 122. According to some embodiments, the filtering rules are configured to remove non-compliant messages in this stage. A non-compliant message can be geographically specific messages, messages with incorrect performance metrics, messages with missing interest feature values, and messages with performances lower than a predetermined threshold.

For example, to avoid a foreign language or to limit the selection within one country, the filtering rules can remove messages that are not from US companies or from a specific country. The filtering rules can also remove messages with obviously incorrect performance metrics, e.g., too high, or with missing values for features of interest. The filtering rules can further remove messages that do not pass a minimal performance threshold.

Next, the filtered messages are further subjected to bucketing 128 that can segment the campaign messages into campaign peer groups and select respective exemplary campaign messages with performance metrics higher than a predetermined threshold for each campaign peer group. As the campaign messages are divided into different campaign groups, their performance data can be meaningfully compared for selecting the high-performance messages.

According to some embodiments, the segmentation characteristics for grouping the campaign messages can comprise the campaign types, the campaign generator attributes and the targeting levels. Furthermore, the campaign generator attributes can comprise industry types, e.g., apparel & accessories, automotive, electronics, and sizes of the previous campaign generators, e.g., employees, or over 100 employees. In addition, other characteristics can be used to divide the messages into different campaign groups.

Furthermore, in addition to the performance metrics, other factors such as varieties of the industry types, sizes of the campaign generators can also be considered in selecting the campaign messages. This way, any prospective campaign generator can find relevant and engaging showcase messages that share similar traits with the generator. For example, a small company with less than employees in the healthcare industry can view exemplary messages that have been proven successful for a similar company in the healthcare industry.

According to some embodiments, campaign management server 101 can adopt a number of parameters to determine the output of the bucketing process. For example, such parameters can define the percentage of kept messages per peer group/bucket, the minimum and/or maximum number of kept emails per peer group. Furthermore, setting the maximum number of kept messages can avoid excessive message volume from the largest peer groups. In addition, the parameters can define the minimum number of kept messages to avoid skipping messages from the smaller peer group. As such, different sized bucket or campaign group can have different parameter rules.

For example, when a campaign segmentation group has less than 10 campaign messages, the system can keep all of them as exemplary messages. When the group has 10-100 campaign messages, the system can keep the top 10 messages with the best performance metrics. When the group has 100-4000 campaign messages, the system can keep the top 10% performance messages. When the group has over 4000 campaign messages, the system can keep the top 400 messages in performance metrics and disregard the rest.

To generate a diversified and balanced campaign message collection, different campaign groups can have different performance thresholds. According to some embodiments, the system can adopt a respective performance threshold for each campaign group. For example, a small campaign group can have a low-performance threshold, whereas a large campaign group can have a high-performance threshold. Furthermore, the threshold values can be empirically predetermined or dynamically adapted. According to some embodiments, the selected exemplary campaign messages are ranked in ascending order according to the performance metrics.

In addition, according to some embodiments, campaign management server 101 can avoid selecting multiple campaign messages from the same campaign generator so that it does not send numerous consent requests to one campaign generator.

According to some embodiments, campaign management server 101 can further filter or remove unsupported campaign messages from the selected exemplary campaign messages via filtering 124. One goal of the filtering process is to reduce the number of messages for further moderation or manual review. Examples of such unsupported campaign messages can be GIF-based messages, or whether the system can reach the previous campaign generator to obtain the consent to display.

According to some embodiments, campaign management server 101 can enable a manual review and moderation of the remaining selected campaign messages via moderation 130 by a curator. To make the review more efficient, the system can automatically generate a screenshot image of the campaign message. Additional information such as the message's subject line, the preview text, is also available for the curator.

For additional sorting, ranking and management of the messages, the curator can assign a number of tags to the reviewed messages. Examples of such tags can comprise a campaign type, a discount code, a holiday, an industry type, a marketing channel, or a design quality grade. According to some embodiments, instead of a manual review, a moderation model can be trained to review and tag the selected campaign messages. For example, a moderation model that has been trained with messages with discount tags can automatically assign a discount code to a new message. The output of the moderation process can be a list of the selected campaign messages with assigned tags, which can be further saved as JSON (JavaScript Object Notation) data structures in database 113.

According to some embodiments, for each campaign message candidate, the system can initiate a consent process to obtain the previous campaign generator's consent for displaying the message. Upon receiving the consent to display or showcase, the selected and approved exemplary campaign messages can be organized and displayed to a new campaign generator 106. Furthermore, the system can generate images 104 of these campaign messages for displaying on a client device 102.

FIG. 2A shows an exemplary display 200 of the exemplary campaign messages to a campaign generator or a user. A campaign generator can view the exemplary campaign messages via a display 252 of a client device. As shown in FIG. 2A, the campaign generator can provide sorting input to select and view the most relevant showcase campaign messages. For example, various sorting inputs can comprise marketing channel 260, campaign type 270, industries 280, holidays 290, as well as other factors such as performance metrics and discounts.

According to some embodiments, marketing channel 260 can comprise an SMS message 262 and an email message 264. Campaign type 270 can comprise a newsletter 272, notifications 274, product announcements 276, and promotions 278. Industries 280 can comprise a number of industry types such as apparel & accessories 282, automotive 284, etc. Furthermore, holidays 290 can comprise a listing of consecutive months, e.g., January 292 and February 294. As an alternative, holidays 290 can comprise a list of specific holidays or foreign holidays e.g., Labor Day, Thanksgiving, Earth Day, Canada Day, etc.

Upon receiving the sorting input, the system can generate relevant and updated exemplary campaign messages and display the updated campaign messages such as email messages 254, 258 and 259, and text message 256 on display 252, in a browsable image grid.

FIG. 2B shows another exemplary display 250 of the exemplary campaign messages to the campaign generator. A campaign generator can click an email message 258 to initiate a full view. In addition, more characteristic information related to email message 258 can be displayed upon clicking. For example, the system can display the subject line, preview text, and assigned tags of email message 258.

According to some embodiments, the system can update shown exemplary campaign messages at a predetermined interval, e.g., every 24 hours, for the same prospective campaign generator. This way, the system can constantly provide fresh inspiration to the campaign generator or user.

FIG. 3 shows some exemplary processes 300 for exemplary campaign message generation. At step 302, the exemplary campaign message system can receive a first plurality of campaign messages from previous campaign generators at a campaign management server. According to some embodiments, the campaign management server can extract all campaign messages that were ever created or sent through the server. Alternatively, the campaign management server can extract all campaign messages in a fixed timeline. The first plurality of campaign messages can comprise, for example, text messages and marketing emails.

At step 304, the system can extract characteristics of the plurality of campaign messages. According to some embodiments, the characteristics comprise, for example, performance metrics, campaign types, campaign attributes, campaign generator attributes, targeting levels, campaign timings, and campaign channels. According to some embodiments, a campaign type classifier can predicate any campaign message's category based on various data related to the message. Such data can comprise content components with layout data as well as metadata such as the subject line and the campaign name. The campaign type classifier can be an ensemble model that comprises at least a logic-rule model, a first language model and a second language model configured to independently generate a predicted campaign type.

At step 306, the system can select a second plurality of campaign messages from the first plurality of campaign messages based on one or more filtering rules. The system can preprocess the received message data, for example, to filter out non-compliant messages. According to some embodiments, the filtering rules are configured to remove non-compliant messages in a preprocessing stage. A non-compliant message can be geographically specific messages, messages with incorrect performance metrics, messages with missing interest feature values, and messages with performances lower than a predetermined threshold.

At step 308, the system can segment the second plurality of campaign messages into campaign groups based on selected and extracted characteristics of the second plurality of campaign messages. At step 310, the system can select exemplary campaign messages associated with performance metrics higher than a predetermined threshold for each of the campaign groups. According to some embodiments, the segmentation characteristics for grouping the campaign messages can comprise the campaign types, the campaign generator attributes, and the targeting levels. Furthermore, the campaign generator attributes can comprise the industry types and sizes of the previous campaign generators. In addition, other characteristics can be used to divide the messages into different campaign groups. As such, any prospective campaign generator can find relevant and engaging showcase messages for inspiration. For example, a small company in the healthcare industry can view example messages that have been proven successful for a similar company.

According to some embodiments, the system can adopt a number of parameters to determine the output of the bucketing process, such as the percentage of kept messages per peer group/bucket, the minimum and/or maximum number of kept emails per peer group. Accordingly, to generate a diversified and balanced campaign message collection, the system can adopt a respective performance threshold for each respective campaign group. For example, a small campaign group can have a low-performance threshold, whereas a large campaign group can have a high-performance threshold. Furthermore, the threshold values can be empirically predetermined or dynamically adapted.

In addition, according to some embodiments, the system can avoid selecting multiple campaign messages from the same campaign generator so that he/she does not receive numerous consent requests.

At step 312, the system can remove unsupported campaign messages from the exemplary campaign messages. The system can further filter or remove unsupported campaign messages from the selected exemplary campaign messages. This process can further reduce the number of messages for further moderation or manual review. Examples of such unsupported campaign messages can be GIF-based messages, or whether the system can reach the previous campaign generator for obtaining the consent to display.

At step 314, the system can enable a manual review of the exemplary campaign messages. The system can enable a manual review and moderation of the remaining selected campaign messages by a curator. To render the review more efficient, the system can automatically generate a screenshot image of the campaign message. Additional information such as the message's subject line, and the preview text is also available for the curator.

According to some embodiments, the curator can assign a number of tags to messages for additional sorting, ranking, and management of them. Examples of such tags can comprise a campaign type, a discount code, a holiday, an industry type, a marketing channel, or a design quality grade. According to some embodiments, instead of a manual review, a moderation model can be trained to review and tag the selected campaign messages. The output of the moderation process can be a list of the selected campaign messages with assigned tags, which can be further saved as JSON (JavaScript Object Notation) data structures in a database.

According to some embodiments, for each campaign message candidate approved by a curator, the system can initiate a consent process to obtain the previous campaign generator's consent for displaying the message. Upon receiving the consent to display or showcase, the selected and approved exemplary campaign messages can be displayed to a prospective campaign generator. Furthermore, the system can generate images of these campaign messages for displaying.

At step 316, the system can display the exemplary campaign messages to a prospective campaign generator. According to some embodiments, the system can receive sorting input from the prospective campaign generator and generate updated campaign messages pursuant to the input. Furthermore, to provide constantly fresh inspiration for the showcase messages, the system can update exemplary campaign messages at a predetermined interval and frequently update the displayed campaign messages.

FIG. 4 shows some exemplary processes 400 for exemplary campaign message generation. At step 402, the exemplary campaign message system can receive a first plurality of campaign messages from previous campaign generators at a campaign management server. According to some embodiments, the campaign management server can extract all campaign messages that were ever created or sent through the server. Alternatively, the campaign management server can extract all campaign messages in a fixed timeline. The first plurality of campaign messages can comprise, for example, text messages and marketing emails.

According to some embodiments, the system can extract characteristics of the plurality of campaign messages. According to some embodiments, the characteristics comprise, for example, performance metrics, campaign types, campaign attributes, campaign generator attributes, targeting levels, campaign timings, and campaign channels. According to some embodiments, a campaign type classifier can predicate any campaign message's category based on various data related to the message. Such data can comprise content components with layout data as well as metadata such as the subject line and the campaign name. The campaign type classifier can be an ensemble model that comprises at least a logic-rule model, a first language model and a second language model configured to independently generate a predicted campaign type.

At step 404, the system can select a second plurality of campaign messages from the first plurality of campaign messages based on one or more filtering rules. The system can preprocess the received message data, for example, to filter out non-compliant messages. According to some embodiments, the filtering rules are configured to remove non-compliant messages in a preprocessing stage. A non-compliant message can be geographically specific messages, messages with incorrect performance metrics, messages with missing interest feature values, and messages with performances lower than a predetermined threshold.

At step 406, the system can segment the second plurality of campaign messages into campaign groups based on selected and extracted characteristics of the second plurality of campaign messages. At step 408, the system can select exemplary campaign messages associated with performance metrics higher than an individual predetermined threshold for each of the campaign groups. According to some embodiments, the segmentation characteristics for grouping the campaign messages can comprise the campaign types, the campaign generator attributes, and the targeting levels. Furthermore, the campaign generator attributes can comprise the industry types and sizes of the previous campaign generators. In addition, other characteristics can be used to divide the messages into different campaign groups.

According to some embodiments, the system can adopt a number of parameters to determine the output of the bucketing process, such as the percentage of kept messages per peer group/bucket, the minimum and/or maximum number of kept emails per peer group. Accordingly, to generate a diversified and balanced campaign message collection, the system can adopt a respective performance threshold for each respective campaign group. For example, a small campaign group can have a low-performance threshold, whereas a large campaign group can have a high-performance threshold. Furthermore, the threshold values can be empirically predetermined or dynamically adapted.

According to some embodiments, the system can remove unsupported campaign messages from the exemplary campaign messages. The system can further filter or remove unsupported campaign messages from the selected exemplary campaign messages. This process can further reduce the number of messages for further moderation or manual review. Examples of such unsupported campaign messages can be GIF-based messages, or whether the system can reach the previous campaign generator for obtaining the consent to display.

According to some embodiments, the system can enable a manual review of the exemplary campaign messages. The system can enable a manual review and moderation of the remaining selected campaign messages by a curator. To render the review more efficient, the system can automatically generate a screenshot image of the campaign message. Additional information such as the message's subject line, and the preview text is also available for the curator.

According to some embodiments, the curator or a moderation model can assign a number of tags to messages for additional sorting, ranking, and management of them. Examples of such tags can comprise a campaign type, a discount code, a holiday, an industry type, a marketing channel, or a design quality grade. For example, a moderation model such as a campaign type classifier can be trained to tag the selected campaign messages with its corresponding campaign type. Other moderation models can be adopted to automatically tag a message, for example, a discount code tag. The output of the moderation process can be a list of the selected campaign messages with assigned tags, which can be further saved as JSON (JavaScript Object Notation) data structures in a database.

According to some embodiments, for each campaign message candidate approved by a curator, the system can initiate a consent process to obtain the previous campaign generator's consent for displaying the message. Upon receiving the consent to display or showcase, the selected and approved exemplary campaign messages can be displayed to a prospective campaign generator. Furthermore, the system can generate images of these campaign messages for displaying.

At step 410, the system can display the exemplary campaign messages to a prospective campaign generator. According to some embodiments, the system can receive sorting input from the prospective campaign generator and generate updated campaign messages pursuant to the input. Furthermore, to provide constantly fresh inspiration for the showcase messages, the system can update exemplary campaign messages at a predetermined interval and frequently update the displayed campaign messages.

FIG. 5A shows a server system of rack-mounted blades for implementing the present subject matter. Various examples are implemented with cloud servers, such as ones implemented by data centers with rack-mounted server blades. FIG. 5A shows a rack-mounted server blade multi-processor server system 511. Server system 511 comprises a multiplicity of network-connected computer processors that run software in parallel.

FIG. 5B shows a diagram of a server system 511. It comprises a multicore cluster of computer processors (CPU) 512 and a multicore cluster of graphics processors (GPU) 513. The processors connect through a board-level interconnect 514 to random-access memory (RAM) devices 515 for program code and data storage. Server system 511 also comprises a network interface 516 to allow the processors to access the Internet, non-volatile storage, and input/output interfaces. By executing instructions stored in RAM devices 515, the CPUs 512 and GPUs 513 perform steps of methods described herein.

FIG. 6A shows the bottom side of a packaged system-on-chip device 631 with a ball grid array for surface-mount soldering to a printed circuit board. Various package shapes and sizes are possible for various chip implementations. System-on-chip (SoC) devices control many embedded systems, IoT device, mobile, portable, and wireless implementations.

FIG. 6B shows a block diagram of the system-on-chip 631. It comprises a multicore cluster of computer processor (CPU) cores 632 and a multicore cluster of graphics processor (GPU) cores 633. The processors connect through a network-on-chip 634 to an off-chip dynamic random access memory (DRAM) interface 635 for volatile program and data storage and a Flash interface 636 for non-volatile storage of computer program code in a Flash RAM non-transitory computer readable medium. SoC 631 also has a display interface for displaying a graphical user interface (GUI) and an I/O interface module 637 for connecting to various I/O interface devices, as needed for different peripheral devices. The I/O interface enables sensors such as touch screen sensors, geolocation receivers, microphones, speakers, Bluetooth peripherals, and USB devices, such as keyboards and mice, among others. SoC 631 also comprises a network interface 638 to allow the processors to access the Internet through wired or wireless connections such as WiFi, 3G, 4G long-term evolution (LTE), 5G, and other wireless interface standard radios as well as Ethernet connection hardware. By executing instructions stored in RAM devices through interface 635 or Flash devices through interface 636, the CPU cores 632 and GPU cores 633 perform functionality as described herein.

Examples shown and described use certain spoken languages. Various embodiments work, similarly, for other languages or combinations of languages. Examples shown and described use certain domains of knowledge and capabilities. Various systems work similarly for other domains or combinations of domains.

Some systems are screenless, such as an earpiece, which has no display screen. Some systems are stationary, such as a vending machine. Some systems are mobile, such as an automobile. Some systems are portable, such as a mobile phone. Some systems are for implanting in a human body. Some systems comprise manual interfaces such as keyboards or touchscreens.

Some systems function by running software on general-purpose programmable processors (CPUs) such as ones with ARM or x86 architectures. Some power-sensitive systems and some systems that require especially high performance, such as ones for neural network algorithms, use hardware optimizations. Some systems use dedicated hardware blocks burned into field-programmable gate arrays (FPGAs). Some systems use arrays of graphics processing units (GPUs). Some systems use application-specific-integrated circuits (ASICs) with customized logic to give higher performance.

Some physical machines described and claimed herein are programmable in many variables, combinations of which provide essentially an infinite variety of operating behaviors. Some systems herein are configured by software tools that offer many parameters, combinations of which support essentially an infinite variety of machine embodiments.

Several aspects of implementations and their applications are described. However, various implementations of the present subject matter provide numerous features including, complementing, supplementing, and/or replacing the features described above. In addition, the foregoing description, for purposes of explanation, used specific nomenclature to provide a thorough understanding of the embodiments of the invention. However, it will be apparent to one skilled in the art that the specific details are not required in order to practice the embodiments of the invention.

It is to be understood that even though numerous characteristics and advantages of various embodiments of the present invention have been set forth in the foregoing description, together with details of the structure and function of various embodiments of the invention, this disclosure is illustrative only. In some cases, certain subassemblies are only described in detail with one such embodiment. Nevertheless, it is recognized and intended that such subassemblies may be used in other embodiments of the invention. Practitioners skilled in the art will recognize many modifications and variations. Changes may be made in detail, especially matters of structure and management of parts within the principles of the embodiments of the present invention to the full extent indicated by the broad general meaning of the terms in which the appended claims are expressed.

Having disclosed exemplary embodiments and the best mode, modifications and variations may be made to the disclosed embodiments while remaining within the scope of the embodiments of the invention as defined by the following claims.

Claims

1. A computer-implemented method of generating exemplary campaign messages, the method comprising:

receiving, at a campaign management server, a first plurality of campaign messages from previous campaign generators;

extracting characteristics of the plurality of campaign messages;

selecting a second plurality of campaign messages from the first plurality of campaign messages based on one or more filtering rules;

segmenting the second plurality of campaign messages into campaign groups based on selected characteristics of the second plurality of campaign messages;

selecting, for each of the campaign groups, exemplary campaign messages associated with performance metrics higher than a predetermined threshold; and

displaying the exemplary campaign messages to a prospective campaign generator.

2. The computer-implemented method of claim 1, wherein the characteristics comprise one or more performance metrics, campaign types, campaign attributes, campaign generator attributes, targeting levels, campaign timings, and campaign channels.

3. The computer-implemented method of claim 2, wherein the campaign types are predicted by a campaign type classifier.

4. The computer-implemented method of claim 1, further comprising:

determining, individually, the predetermined threshold for each of the campaign groups.

5. The computer-implemented method of claim 1, wherein the exemplary campaign messages are further associated with message diversities.

6. The computer-implemented method of claim 1, wherein the one or more filtering rules are configured to remove non-compliant messages.

7. The computer-implemented method of claim 2, wherein the selected characteristics for segmenting comprise the campaign types, the campaign generator attributes and the targeting levels.

8. The computer-implemented method of claim 7, wherein the campaign generator attributes comprise industry types and sizes of the previous campaign generators.

9. The computer-implemented method of claim 1, wherein the exemplary campaign messages are ranked in an ascending order according to the performance metrics.

10. The computer-implemented method of claim 1, further comprising:

removing unsupported campaign messages from the exemplary campaign messages.

11. The computer-implemented method of claim 1, further comprising:

enabling a manual review of the exemplary campaign messages, wherein tags can be assigned to the exemplary campaign messages.

12. The computer-implemented method of claim 1, further comprising:

generating images of the exemplary campaign messages for displaying to the prospective campaign generator.

13. The computer-implemented method of claim 1, further comprising:

initiating consent flows to obtain consent for displaying the exemplary campaign messages from the previous campaign generators.

14. The computer-implemented method of claim 1, further comprising:

updating, at a predetermined interval, exemplary campaign messages for each of the campaign groups; and

displaying the updated exemplary campaign messages to the prospective campaign generator.

15. The computer-implemented method of claim 1, further comprising:

receiving one or more sorting input from the prospective campaign generator;

generating updated exemplary campaign messages based on the one or more sorting input; and

displaying the updated exemplary campaign messages to the prospective campaign generator.

16. A computer-implemented method of generating exemplary campaign messages, the method comprising:

receiving, at a campaign management server, a first plurality of campaign messages from previous campaign generators;

selecting a second plurality of campaign messages from the first plurality of campaign messages based on one or more filtering rules;

segmenting the second plurality of campaign messages into campaign groups based on selected characteristics of the second plurality of campaign messages;

selecting, for each of the campaign groups, exemplary campaign messages; and

displaying the exemplary campaign messages to a prospective campaign generator.

17. The computer-implemented method of claim 16, further comprising:

extracting characteristics of the plurality of campaign messages.

18. The computer-implemented method of claim 16, wherein the exemplary campaign messages are associated with performance metrics higher than a predetermined threshold.

19. The computer-implemented method of claim 18, further comprising:

determining, individually, a predetermined threshold for each of the campaign groups.

20. The computer-implemented method of claim 16, further comprising:

updating, at a predetermined interval, exemplary campaign messages for each of the campaign groups; and

displaying the updated exemplary campaign messages to the prospective campaign generator.