Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for determining send times to provide electronic communications based on predicted response rates by utilizing a Bayesian approach and multi-armed bandit algorithms. For example, the disclosed systems can generate predicted response rates by training and utilizing one or more response rate prediction models to generate a weighted combination of user-specific response information and population-specific response information. The disclosed systems can further utilize a Bayes upper-confidence-bound send time model to determine send times that are more likely to elicit user responses based on the predicted response rates and further based on exploration and exploitation considerations.

Abstract: In various implementations, analytics data is received that indicates performance of bid targets for historical bids made in one or more content delivery auctions. Baseline prediction models are maintained for the bid targets. The baseline prediction models use the analytics data to predict performance of the bid targets in one or more future instances of at least one content delivery auction. A presentation context factor model is maintained that provides an adjustment factor that quantifies a contribution of a subset of a plurality of presentation context factors associated with the bid targets to performance of the bid targets based on predicted values from the baseline prediction models. A contextual predicted value is computed using the adjustment factor for the subset of the plurality of presentation context factors. A performance prediction is transmitted to a user device and is based on at least the contextual predicted value.

Abstract: An improved analytics system generates actionable KPI-based customer segments. The analytics system determines predicted outcomes for a key performance indicator (KPI) of interest and a contribution value for each variable indicating an extent to which each variable contributes to predicted outcomes. Topics are generated by applying a topic model to the contribution values for the variables. Each topic comprises a group of variables with a contribution level for each variable that indicates the importance of each variable to the topic. User segments are generated by assigning each user to a topic based on attribution levels output by the topic model.

Abstract: A hybrid deep-learning action prediction architecture system is described that predicts actions. The architecture includes a main path and an auxiliary path. The main path may contain multiple layers of convolutional neural networks for further aggregation to coarser time spans. The resultant data produced by the convolutional neural networks is passed to multiple layers of LSTMs. The outputs from LSTMs are then combined with the profile in the auxiliary path to predict an action label.

Abstract: Systems and methods are disclosed herein for distributing online ads with electronic content according to online ad request targeting parameters. One embodiment of this technique involves placing online test ads across multiple online ad request dimensions and tracking a performance metric for the online test ads. The performance of the online ad request dimensions is estimated based on the tracking of the performance metric for the online test ads and online ad request targeting parameters are established for spending a budget of a campaign to place online ads in response to online ad requests having particular online ad request dimensions. Online ads are then distributed based on using the online ad request targeting parameters to select online ad requests.

Abstract: Techniques and systems are described that employ survival analysis and classification to predict occurrence of future events by a digital analytics system. Survival analysis involves modeling time to event data. Survival analysis is used by digital analytics systems to analyze an expected duration of time until an event happens. In the techniques described herein, survival analysis is employed as part of a classification technique by a digital analytics system. In one example, a digital analytics system generates training data from a dataset in accordance with a survival analysis technique such that, after generated, the training data is usable to train a classification model.

Abstract: Methods, systems, and non-transitory computer readable storage media are disclosed for using reject inference to generate synthetic data for modifying lead scoring models. For example, the disclosed system identifies an original dataset corresponding to an output of a lead scoring model that generates scores for a plurality of prospects to indicate a likelihood of success of prospects of the plurality of prospects. In one or more embodiments, the disclosed system selects a reject inference model by performing simulations on historical prospect data associated with the original dataset. Additionally, the disclosed system uses the selected reject inference model to generate an imputed dataset by generating synthetic outcome data representing simulated outcomes of rejected prospects in the original dataset. The disclosed system then uses the imputed dataset to modify the lead scoring model by modifying at least one parameter of the lead scoring model using the synthetic outcome data.

Abstract: Quantitative rating systems and techniques are described that prioritize customers by propensity to buy and buy size to generate customer ratings. In one example, a propensity model is used to determine a likelihood of a potential customer to purchase a product, and a projected timeframe buy size for the potential customer is determined. An expected value for the potential customer is generated by combining the likelihood of the potential customer to purchase the product and the projected timeframe buy size. In another example, a ratio model of annualized recurring revenue (ARR) is used to determine a timeframe buy size for an existing customer in consecutive time frames. An upsell opportunity for the existing customer is determined based on the timeframe buy size less an ARR for a current time frame for the existing customer. A rating of the potential or existing customer is output in a user interface.

Abstract: Prioritization techniques and systems are described that utilize a historical purchase sequence and customer features to prioritize products and services to generate product and service recommendations. In an example, feature data describing a customer and historical purchase data for the customer is received that indicates products or services purchased by the customer. The historical purchase data further includes indicators of when the products or services were purchased by the customer. Then, probabilities of future purchases by the customer of additional products are determined by classifying the additional products using a multiclass classification. The multiclass classification is based on the historical purchase data and the feature data describing the customer. Next, a ranking of the additional products is generated based on the determined probabilities of future purchases. The ranking of the additional products is output in a user interface based on the determined probabilities.

Abstract: Dynamic Hierarchical Empirical Bayes techniques and systems are described that are implemented to control output of digital content. In one example, a system identifies splitting variables included in data. An amount of loss is then determined for each of the identified splitting variables by the system using a loss function. Based on the determined amounts of loss, the system selects at least one splitting variable from the plurality of splitting variables that are to be used to partition data in a respective node, e.g., a parent node to form a plurality of child nodes. The system, for instance, may select the splitting variable that minimizes the cost, i.e., has the lowest amount of cost. The selected splitting variable is then employed by the system to generate at least one hierarchical level of the hierarchical structure of the statistical model by partitioning data from the parent node into respective child nodes.

Abstract: Systems and methods are disclosed herein for distributing online ads with electronic content according to online ad request targeting parameters. One embodiment of this technique involves placing online test ads across multiple online ad request dimensions and tracking a performance metric for the online test ads. The performance of the online ad request dimensions is estimated based on the tracking of the performance metric for the online test ads and online ad request targeting parameters are established for spending a budget of a campaign to place online ads in response to online ad requests having particular online ad request dimensions. Online ads are then distributed based on using the online ad request targeting parameters to select online ad requests.

Abstract: The present disclosure includes systems, methods, and non-transitory computer readable media that train and utilize multi-phase learning models to predict performance during digital content campaigns and provide digital content to client devices in a real-time bidding environment. In particular, one or more embodiments leverage organizational structure of digital content campaigns to train two learning models, utilizing different data sources, to predict performance, generate bid responses, and provide digital content to client devices. For example, the disclosed systems can train a first performance learning model in an offline mode utilizing parent-level historical data. Then, in an online mode, the disclosed systems can train a second performance learning model utilizing child-level historical data and utilize the first performance learning model and the second performance learning model to generate bid responses and bid amounts in a real-time bidding environment.

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods for generating and utilizing a touchpoint attribution attention neural network to identify and measure performance of touchpoints in digital content campaigns. For example, a deep learning attribution system trains a touchpoint attribution attention neural network using touchpoint sequences, which include user interactions with content via one or more digital media channels. In one or more embodiments, the deep learning attribution system utilizes the trained touchpoint attribution attention neural network to determine touchpoint attributions of touchpoints in a target touchpoint sequence. In addition, the deep learning attribution system can utilize the trained touchpoint attribution attention neural network to generate conversion predictions for target touchpoint sequences and to provide targeted digital content over specific digital media channels to client devices of individual users.

Abstract: Certain embodiments involve a model for predicting user behavior. For example, a system accesses user behavior data indicating various users' behaviors during intervals over various periods of time and target behavior data indicating a particular user behavior. The system associates each user with a label that indicates whether a user performed a particular action during or after a time period based on the target behavior data. The system uses the user behavior data to train various deep Restricted Boltzmann Machines (“RBM”) to generate representations of each user over each period of time that indicate the user behavior over the time period. The system generates a predictive model by connecting the RBMs into a deep recurrent neural network and uses the target behavior data associated with each user, along with the representations of each user, as input data to train the deep recurrent neural network to predict user behavior.

Abstract: Systems and methods disclosed herein use one or more auxiliary time series to more accurately identify change points in a target time series. This involves receiving data for the target time series and one or more auxiliary time series, where the one or more auxiliary time series have a relationship with the target time series. A combined auxiliary time series is generated based on the relationship between the target time series and the one or more auxiliary time series and the change point is detected for the target time series based on the target time series and the combined auxiliary time series. In one embodiment, time series data is received on an on-going basis. Recent time series data for the target time series and the one or more auxiliary time series is identified and used to detect the change point. The change point can be detected without using time series data older than the recent time series data.

Abstract: A bid management system generates estimated performance metrics at the bid unit level to facilitate bid optimization. The bid management system includes a hierarchical feature selection and prediction approach. Feature selection is performed by aggregating historical performance metrics to a higher hierarchical level and testing features for statistical significance. Features for which a significance level satisfies a significance threshold are selected for prediction analysis. The prediction analysis uses a statistical model based on selected features to generate estimated performance metrics at the bid unit level.

Abstract: The present disclosure relates to a media attribution system that improves multi-channel media attribution by employing discrete-time survival modeling. In particular, the media attribution system uses event data (e.g., interactions and conversions) to generate positive and negative conversion paths, which the media attribution system uses to train an algorithmic attribution model. The media attribution system also uses the trained algorithmic attribution model to determine attribution scores for each interaction used in the conversion paths. Generally, the attribution score for an interaction indicates the effect the interaction has in influencing a user toward conversion.

Abstract: Systems and methods are disclosed herein for distributing online ads with electronic content according to online ad request targeting parameters. One embodiment of this technique involves placing online test ads across multiple online ad request dimensions and tracking a performance metric for the online test ads. The performance of the online ad request dimensions is estimated based on the tracking of the performance metric for the online test ads and online ad request targeting parameters are established for spending a budget of a campaign to place online ads in response to online ad requests having particular online ad request dimensions. Online ads are then distributed based on using the online ad request targeting parameters to select online ad requests.

Abstract: In various implementations, analytics data is received that indicates performance of bid targets for historical bids made in one or more content delivery auctions. Baseline prediction models are maintained for the bid targets. The baseline prediction models use the analytics data to predict performance of the bid targets in one or more future instances of at least one content delivery auction. A presentation context factor model is maintained that provides an adjustment factor that quantifies a contribution of a subset of a plurality of presentation context factors associated with the bid targets to performance of the bid targets based on predicted values from the baseline prediction models. A contextual predicted value is computed using the adjustment factor for the subset of the plurality of presentation context factors. A performance prediction is transmitted to a user device and is based on at least the contextual predicted value.

Abstract: Systems and methods disclosed herein use one or more auxiliary time series to more accurately identify change points in a target time series. This involves receiving data for the target time series and one or more auxiliary time series, where the one or more auxiliary time series have a relationship with the target time series. A combined auxiliary time series is generated based on the relationship between the target time series and the one or more auxiliary time series and the change point is detected for the target time series based on the target time series and the combined auxiliary time series. In one embodiment, time series data is received on an on-going basis. Recent time series data for the target time series and the one or more auxiliary time series is identified and used to detect the change point. The change point can be detected without using time series data older than the recent time series data.