Abstract: Machine-learning based multi-step engagement strategy generation and visualization is described. Rather than rely heavily on human involvement to create delivery strategies, the described learning-based engagement system generates multi-step engagement strategies by leveraging machine-learning models trained using data describing historical user interactions with content delivered in connection with historical campaigns. Initially, the learning-based engagement system obtains data describing an entry condition and an exit condition for a campaign. Based on the entry and exit condition, the learning-based engagement system utilizes the machine-learning models to generate a multi-step engagement strategy, which describes a sequence of content deliveries that are to be served to a particular client device user (or segment of client device users).

Abstract: Machine-learning based multi-step engagement strategy generation and visualization is described. Rather than rely heavily on human involvement to create delivery strategies, the described learning-based engagement system generates multi-step engagement strategies by leveraging machine-learning models trained using data describing historical user interactions with content delivered in connection with historical campaigns. Initially, the learning-based engagement system obtains data describing an entry condition and an exit condition for a campaign. Based on the entry and exit condition, the learning-based engagement system utilizes the machine-learning models to generate a multi-step engagement strategy, which describes a sequence of content deliveries that are to be served to a particular client device user (or segment of client device users).

Abstract: Machine-learning based multi-step engagement strategy modification is described. Rather than rely heavily on human involvement to manage content delivery over the course of a campaign, the described learning-based engagement system modifies a multi-step engagement strategy, originally created by an engagement-system user, by leveraging machine-learning models. In particular, these leveraged machine-learning models are trained using data describing user interactions with delivered content as those interactions occur over the course of the campaign. Initially, the learning-based engagement system obtains a multi-step engagement strategy created by an engagement-system user. As the multi-step engagement strategy is deployed, the learning-based engagement system randomly adjusts aspects of the sequence of deliveries for some users.

Abstract: Machine-learning based multi-step engagement strategy generation and visualization is described. Rather than rely heavily on human involvement to create delivery strategies, the described learning-based engagement system generates multi-step engagement strategies by leveraging machine-learning models trained using data describing historical user interactions with content delivered in connection with historical campaigns. Initially, the learning-based engagement system obtains data describing an entry condition and an exit condition for a campaign. Based on the entry and exit condition, the learning-based engagement system utilizes the machine-learning models to generate a multi-step engagement strategy, which describes a sequence of content deliveries that are to be served to a particular client device user (or segment of client device users).

Abstract: In implementations of digital image search based on arbitrary image features, a server computing device maintains an images database of digital images, and includes an image search system that receives a search input as a digital image depicting image features, and receives search criteria of depicted image features in the digital image. The image search system can then determine similar images to the received digital image based on similarity criterion corresponding to the search criteria. A trained image model of the image search system is applied to determine an image feature representation of the received digital image. A feature mask model of the image search system is applied to the image feature representation to determine a masked feature representation of the received digital image. The masked feature representation of the received digital image is compared to a masked feature representation of each respective database image to identify the similar images.

Abstract: Some implementations provide a feature recommendation system that receives sequences from user sessions with applications, where each sequence is of features of the applications in an order the features were used by a user. The sequences are applied to a feature embedding model that learns semantic similarities between the features based on occurrences of the features in the sequences in a same user session. A request is received for a feature recommendation that identifies a feature of an application used by a given user in a user session. A recommended feature for the feature recommendation is determined from a set of the semantic similarities that are between the identified feature and others of the features. The feature recommendation is presented on a user device associated with the given user.

Abstract: Learning vector-space representations of items for recommendations using word embedding models is described. In one or more embodiments, a word embedding model is used to produce item vector representations of items based on considering items interacted with as words and items interacted with during sessions as sentences. The item vectors are used to produce item recommendations similar to currently or recently viewed items.

Abstract: In implementations of digital image search based on arbitrary image features, a server computing device maintains an images database of digital images, and includes an image search system that receives a search input as a digital image depicting image features, and receives search criteria of depicted image features in the digital image. The image search system can then determine similar images to the received digital image based on similarity criterion corresponding to the search criteria. A trained image model of the image search system is applied to determine an image feature representation of the received digital image. A feature mask model of the image search system is applied to the image feature representation to determine a masked feature representation of the received digital image. The masked feature representation of the received digital image is compared to a masked feature representation of each respective database image to identify the similar images.

Abstract: Collaborative-filtered content recommendations with justification in real-time is described. A recommendation system determines these recommendations, in part, by identifying digital content items of a catalog that are associated with a single attribute used to describe digital content. The attribute used for the identification is based on affinity scores computed for a client device user to which the recommendations are being provided. These affinity scores indicate the client device user's affinity for different attributes used to describe the digital content. Once the digital content items are identified based on the one attribute, the recommendation system is then limited to ranking and selecting from the identified digital content items to provide the recommendations. The recommendation system does not process the entire catalog of digital content items at once to rank and select the items.

Abstract: A technique is disclosed for generating class level rules that globally explain the behavior of a machine learning model, such as a model that has been used to solve a classification problem. Each class level rule represents a logical conditional statement that, when the statement holds true for one or more instances of a particular class, predicts that the respective instances are members of the particular class. Collectively, these rules represent the pattern followed by the machine learning model. The techniques are model agnostic, and explain model behavior in a relatively easy to understand manner by outputting a set of logical rules that can be readily parsed. Although the techniques can be applied to any number of applications, in some embodiments, the techniques are suitable for interpreting models that perform the task of classification. Other machine learning model applications can equally benefit.

Abstract: Systems and methods are disclosed herein for determining user segments created by a predictive model based on user behavioral data. A data analysis application executing on a computing device receives training data and a user input defining an outcome of interest. The data analysis application trains a predictive model with the training data and the outcome of interest. The data analysis application generates input data for each of a set of conditions determined from the training data. The data analysis application receives predicted outcome from the predictive model based on the input data. The data analysis application determines the relevance of the condition based on a comparison of the predicted outcome and the outcome of interest. The data analysis application generates a user segment that comprises a condition from the set of conditions based on the relevance of the condition.

Abstract: Techniques are disclosed for identifying the same online user across different communication networks, and further creating a unified profile for that user. The unified profile is an aggregation of publicly available user profile attributes across the different networks. In an embodiment, the techniques are implemented as a computer implemented methodology, including: (1) feature space analysis to identify relevant user features that allows for clusterization of the given target network(s), (2) unsupervised candidate selection to identify one or more candidate user profiles from each target network and that are likely belonging to a target user or so-called queried user, and (3) supervised user identification to identify a likely matching user profile for that target user from each target network. A unified user profile can then be built from data taken from all matched user profiles, and effectively allows a marketer to better understand that user and hence execute more informed targeting.

Abstract: Rule determination for black-box machine-learning models (BBMLMs) is described. These rules are determined by an interpretation system to describe operation of a BBMLM to associate inputs to the BBMLM with observed outputs of the BBMLM and without knowledge of the logic used in operation by the BBMLM to make these associations. To determine these rules, the interpretation system initially generates a proxy black-box model to imitate the behavior of the BBMLM based solely on data indicative of the inputs and observed outputs—since the logic actually used is not available to the system. The interpretation system generates rules describing the operation of the BBMLM by combining conditions—identified based on output of the proxy black-box model—using a genetic algorithm. These rules are output as if-then statements configured with an if-portion formed as a list of the conditions and a then-portion having an indication of the associated observed output.

Abstract: A digital medium environment is described to facilitate recommendations based on vectors generated using feature word embeddings. A recommendation system receives data that describes at least one attribute for a user profile, at least one item, and an interaction between the user profile and the at least one item. The recommendation system associates each user profile attribute, each item, and each interaction between a user profile and an item as a word, using natural language processing, and combines the words into sentences. The sentences are input to a word embedding model to determine feature vector representations describing relationships between the user profile attributes, items, and explicit and implicit interactions. From the feature vector representations, the recommendation system ascertains a similarity between different features.

Abstract: Digital experience targeting techniques are disclosed which serve digital experiences that have a high probability of conversion with regard to a given user visit profile. In some examples, a method may include predicting a probability of each digital experience in a campaign being served based on a user visit profile and an indication that a user exhibiting the user visit profile is going to convert, predicting a probability of each digital experience in the campaign being served based on the user visit profile and an indication that the user exhibiting the user visit profile is not going to convert, and deriving, for the user visit profile, a probability of conversion for each digital experience in the campaign. The probability of conversion for each digital experience in the campaign for the user visit profile may be derived using a Bayesian framework.

Abstract: Some implementations provide a feature recommendation system that receives sequences from user sessions with applications, where each sequence is of features of the applications in an order the features were used by a user. The sequences are applied to a feature embedding model that learns semantic similarities between the features based on occurrences of the features in the sequences in a same user session. A request is received for a feature recommendation that identifies a feature of an application used by a given user in a user session. A recommended feature for the feature recommendation is determined from a set of the semantic similarities that are between the identified feature and others of the features. The feature recommendation is presented on a user device associated with the given user.

Abstract: Learning vector-space representations of items for recommendations using word embedding models is described. In one or more embodiments, a word embedding model is used to produce item vector representations of items based on considering items interacted with as words and items interacted with during sessions as sentences. The item vectors are used to produce item recommendations similar to currently or recently viewed items.

Abstract: Recommendation control techniques using incremental matrix factorization and clustering are described. User latent factors and item latent factors are computed from data that denotes ratings associated with the users regarding respective ones of the plurality of items of digital content. Data is obtained that describes interaction of a particular one of the users with at least one respective item of the digital content. A plurality of clusters is formed using the user latent factors. The recommendations are generated using the user latent factors and the item latent factors for each of the plurality of clusters. Further, at least one of recommendations is located based on comparison of a user identifier of a subsequent user with the plurality of clusters. Interaction of the subsequent user with the digital content is controlled based on the located at least one of the recommendations.

Abstract: Techniques are disclosed for identifying the same online user across different communication networks, and further creating a unified profile for that user. The unified profile is an aggregation of publicly available user profile attributes across the different networks. In an embodiment, the techniques are implemented as a computer implemented methodology, including: (1) feature space analysis to identify relevant user features that allows for clusterization of the given target network(s), (2) unsupervised candidate selection to identify one or more candidate user profiles from each target network and that are likely belonging to a target user or so-called queried user, and (3) supervised user identification to identify a likely matching user profile for that target user from each target network. A unified user profile can then be built from data taken from all matched user profiles, and effectively allows a marketer to better understand that user and hence execute more informed targeting.