Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods for generating and utilizing a touchpoint attribution attention neural network to identify and measure performance of touchpoints in digital content campaigns. For example, a deep learning attribution system trains a touchpoint attribution attention neural network using touchpoint sequences, which include user interactions with content via one or more digital media channels. In one or more embodiments, the deep learning attribution system utilizes the trained touchpoint attribution attention neural network to determine touchpoint attributions of touchpoints in a target touchpoint sequence. In addition, the deep learning attribution system can utilize the trained touchpoint attribution attention neural network to generate conversion predictions for target touchpoint sequences and to provide targeted digital content over specific digital media channels to client devices of individual users.

Abstract: An improved analytics system generates actionable KPI-based customer segments. The analytics system determines predicted outcomes for a key performance indicator (KPI) of interest and a contribution value for each variable indicating an extent to which each variable contributes to predicted outcomes. Topics are generated by applying a topic model to the contribution values for the variables. Each topic comprises a group of variables with a contribution level for each variable that indicates the importance of each variable to the topic. User segments are generated by assigning each user to a topic based on attribution levels output by the topic model.

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for determining send times to provide electronic communications based on predicted response rates by utilizing a Bayesian approach and multi-armed bandit algorithms. For example, the disclosed systems can generate predicted response rates by training and utilizing one or more response rate prediction models to generate a weighted combination of user-specific response information and population-specific response information. The disclosed systems can further utilize a Bayes upper-confidence-bound send time model to determine send times that are more likely to elicit user responses based on the predicted response rates and further based on exploration and exploitation considerations.

Abstract: An improved analytics system generates actionable KPI-based customer segments. The analytics system determines predicted outcomes for a key performance indicator (KPI) of interest and a contribution value for each variable indicating an extent to which each variable contributes to predicted outcomes. Topics are generated by applying a topic model to the contribution values for the variables. Each topic comprises a group of variables with a contribution level for each variable that indicates the importance of each variable to the topic. User segments are generated by assigning each user to a topic based on attribution levels output by the topic model.

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for generating a target distribution schedule for providing electronic communications based on predicted behavior rates by utilizing a genetic algorithm and one or more objective functions. For example, the disclosed systems can generate predicted behavior rates by training and utilizing one or more behavior prediction models. Based on the predicted behavior rates, the disclosed systems can further utilize a genetic algorithm to apply objective functions to generate one or more candidate distribution schedules. In accordance with the genetic algorithm, the disclosed systems can select a target distribution schedule for a particular user/client device. The disclosed systems can thus provide one or more electronic communications to individual users based on respective target distribution schedules.

Abstract: Quantitative rating systems and techniques are described that prioritize customers by propensity to buy and buy size to generate customer ratings. In one example, a propensity model is used to determine a likelihood of a potential customer to purchase a product, and a projected timeframe buy size for the potential customer is determined. An expected value for the potential customer is generated by combining the likelihood of the potential customer to purchase the product and the projected timeframe buy size. In another example, a ratio model of annualized recurring revenue (ARR) is used to determine a timeframe buy size for an existing customer in consecutive time frames. An upsell opportunity for the existing customer is determined based on the timeframe buy size less an ARR for a current time frame for the existing customer. A rating of the potential or existing customer is output in a user interface.

Abstract: Methods, systems, and non-transitory computer readable storage media are disclosed for using reject inference to generate synthetic data for modifying lead scoring models. For example, the disclosed system identifies an original dataset corresponding to an output of a lead scoring model that generates scores for a plurality of prospects to indicate a likelihood of success of prospects of the plurality of prospects. In one or more embodiments, the disclosed system selects a reject inference model by performing simulations on historical prospect data associated with the original dataset. Additionally, the disclosed system uses the selected reject inference model to generate an imputed dataset by generating synthetic outcome data representing simulated outcomes of rejected prospects in the original dataset. The disclosed system then uses the imputed dataset to modify the lead scoring model by modifying at least one parameter of the lead scoring model using the synthetic outcome data.

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods for generating and utilizing a touchpoint attribution attention neural network to identify and measure performance of touchpoints in digital content campaigns. For example, a deep learning attribution system trains a touchpoint attribution attention neural network using touchpoint sequences, which include user interactions with content via one or more digital media channels. In one or more embodiments, the deep learning attribution system utilizes the trained touchpoint attribution attention neural network to determine touchpoint attributions of touchpoints in a target touchpoint sequence. In addition, the deep learning attribution system can utilize the trained touchpoint attribution attention neural network to generate conversion predictions for target touchpoint sequences and to provide targeted digital content over specific digital media channels to client devices of individual users.

Abstract: Quantitative rating systems and techniques are described that prioritize customers by propensity to buy and buy size to generate customer ratings. In one example, a propensity model is used to determine a likelihood of a potential customer to purchase a product, and a projected timeframe buy size for the potential customer is determined. An expected value for the potential customer is generated by combining the likelihood of the potential customer to purchase the product and the projected timeframe buy size. In another example, a ratio model of annualized recurring revenue (ARR) is used to determine a timeframe buy size for an existing customer in consecutive time frames. An upsell opportunity for the existing customer is determined based on the timeframe buy size less an ARR for a current time frame for the existing customer. A rating of the potential or existing customer is output in a user interface.

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods for generating and utilizing a touchpoint attribution attention neural network to identify and measure performance of touchpoints in digital content campaigns. For example, a deep learning attribution system trains a touchpoint attribution attention neural network using touchpoint sequences, which include user interactions with content via one or more digital media channels. In one or more embodiments, the deep learning attribution system utilizes the trained touchpoint attribution attention neural network to determine touchpoint attributions of touchpoints in a target touchpoint sequence. In addition, the deep learning attribution system can utilize the trained touchpoint attribution attention neural network to generate conversion predictions for target touchpoint sequences and to provide targeted digital content over specific digital media channels to client devices of individual users.

Abstract: The present disclosure includes systems, methods, and non-transitory computer readable media that train and utilize multi-phase learning models to predict performance during digital content campaigns and provide digital content to client devices in a real-time bidding environment. In particular, one or more embodiments leverage organizational structure of digital content campaigns to train two learning models, utilizing different data sources, to predict performance, generate bid responses, and provide digital content to client devices. For example, the disclosed systems can train a first performance learning model in an offline mode utilizing parent-level historical data. Then, in an online mode, the disclosed systems can train a second performance learning model utilizing child-level historical data and utilize the first performance learning model and the second performance learning model to generate bid responses and bid amounts in a real-time bidding environment.

Abstract: Quantitative rating systems and techniques are described that prioritize customers by propensity to buy and buy size to generate customer ratings. In one example, a propensity model is used to determine a likelihood of a potential customer to purchase a product, and a projected timeframe buy size for the potential customer is determined. An expected value for the potential customer is generated by combining the likelihood of the potential customer to purchase the product and the projected timeframe buy size. In another example, a ratio model of annualized recurring revenue (ARR) is used to determine a timeframe buy size for an existing customer in consecutive time frames. An upsell opportunity for the existing customer is determined based on the timeframe buy size less an ARR for a current time frame for the existing customer. A rating of the potential or existing customer is output in a user interface.

Abstract: The present disclosure relates to a media attribution system that improves multi-channel media attribution by employing discrete-time survival modeling. In particular, the media attribution system uses event data (e.g., interactions and conversions) to generate positive and negative conversion paths, which the media attribution system uses to train an algorithmic attribution model. The media attribution system also uses the trained algorithmic attribution model to determine attribution scores for each interaction used in the conversion paths. Generally, the attribution score for an interaction indicates the effect the interaction has in influencing a user toward conversion.

Abstract: A bid management system generates estimated performance metrics at the bid unit level to facilitate bid optimization. The bid management system includes a hierarchical feature selection and prediction approach. Feature selection is performed by aggregating historical performance metrics to a higher hierarchical level and testing features for statistical significance. Features for which a significance level satisfies a significance threshold are selected for prediction analysis. The prediction analysis uses a statistical model based on selected features to generate estimated performance metrics at the bid unit level.

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for determining send times for distributing digital content to client devices utilizing a recommendation system approach. For example, the disclosed systems can utilize a recommendation system model such as a matrix factorization model, a factorization machine model, and/or a neural network to implement collaborative filtering to generate predicted response rates for particular candidate send times. Based on the predicted response rates indicating likelihoods of receiving responses for particular send times, the disclosed system can generate a distribution schedule to provide electronic communications at one or more of the send times.

Abstract: Systems and methods are disclosed herein for distributing online ads with electronic content according to online ad request targeting parameters. One embodiment of this technique involves placing online test ads across multiple online ad request dimensions and tracking a performance metric for the online test ads. The performance of the online ad request dimensions is estimated based on the tracking of the performance metric for the online test ads and online ad request targeting parameters are established for spending a budget of a campaign to place online ads in response to online ad requests having particular online ad request dimensions. Online ads are then distributed based on using the online ad request targeting parameters to select online ad requests.

Abstract: Certain embodiments involve a model for predicting user behavior. For example, a system accesses user behavior data indicating various users' behaviors during intervals over various periods of time and target behavior data indicating a particular user behavior. The system associates each user with a label that indicates whether a user performed a particular action during or after a time period based on the target behavior data. The system uses the user behavior data to train various deep Restricted Boltzmann Machines (“RBM”) to generate representations of each user over each period of time that indicate the user behavior over the time period. The system generates a predictive model by connecting the RBMs into a deep recurrent neural network and uses the target behavior data associated with each user, along with the representations of each user, as input data to train the deep recurrent neural network to predict user behavior.

Abstract: Dynamic Hierarchical Empirical Bayes techniques and systems are described that are implemented to control output of digital content. In one example, a system identifies splitting variables included in data. An amount of loss is then determined for each of the identified splitting variables by the system using a loss function. Based on the determined amounts of loss, the system selects at least one splitting variable from the plurality of splitting variables that are to be used to partition data in a respective node, e.g., a parent node to form a plurality of child nodes. The system, for instance, may select the splitting variable that minimizes the cost, i.e., has the lowest amount of cost. The selected splitting variable is then employed by the system to generate at least one hierarchical level of the hierarchical structure of the statistical model by partitioning data from the parent node into respective child nodes.

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for determining send times for distributing digital content to client devices utilizing a recommendation system approach. For example, the disclosed systems can utilize a recommendation system model such as a matrix factorization model, a factorization machine model, and/or a neural network to implement collaborative filtering to generate predicted response rates for particular candidate send times. Based on the predicted response rates indicating likelihoods of receiving responses for particular send times, the disclosed system can generate a distribution schedule to provide electronic communications at one or more of the send times.

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for generating a target distribution schedule for providing electronic communications based on predicted behavior rates by utilizing a genetic algorithm and one or more objective functions. For example, the disclosed systems can generate predicted behavior rates by training and utilizing one or more behavior prediction models. Based on the predicted behavior rates, the disclosed systems can further utilize a genetic algorithm to apply objective functions to generate one or more candidate distribution schedules. In accordance with the genetic algorithm, the disclosed systems can select a target distribution schedule for a particular user/client device. The disclosed systems can thus provide one or more electronic communications to individual users based on respective target distribution schedules.