Abstract: Methods and apparatuses are described for digital content classification and recommendation using constraint-based predictive machine learning. A server trains a machine learning (ML) recommendation model to generate digital content layouts each comprising digital content item slots arranged according to one or more digital content selection constraints. The server receives user profile information for a first user. The server executes the trained ML recommendation model to generate a plurality of digital content item displays, each including a selected digital content item placed in each slot. The server determines, for each digital content item display, an interaction prediction score for each digital content item in the display. The server selects a digital content item display based upon the interaction predictions scores. The server transmits the selected digital content item display to a client device.

Abstract: A method for generating cross-channel recommendations for a customer includes receiving customer data associated with the customer, content data, and clickstream data; encoding the content data using a text encoder, the encoding resulting in content embeddings; encoding the clickstream data using a clickstream encoder, the encoding resulting in clickstream embeddings; providing the content data, the clickstream data, and the clickstream embeddings as inputs to a hybrid latent model; causing execution of the hybrid latent model, the execution resulting in cross-system user and item interaction embeddings; retrieving application features corresponding to the clickstream data from an application feature store; providing the customer data, the content embeddings, the clickstream embeddings, the cross-system user and item interaction embeddings, and the application features as inputs to a recommendation model; causing execution of the recommendation model, the execution resulting in a plurality of ranked recommend

Abstract: Methods and apparatuses for evaluating probabilistic fairness of machine learning (ML) classification models include a server that generates a first input data set, including assigning a class membership label to each of a plurality of participants based upon a probability of class membership derived from a surrogate class variable. The server generates a second input data set, including assigning a class membership label to each of the plurality of participants based upon ground truth class values. The server executes a binary classification model on the first input data set to generate inferred fairness metrics for the binary classification model. The server executes the binary classification model on the second input data set to generate actual fairness metrics for the binary classification model. The server determines a disparity in one or more fairness metrics for the binary classification model based upon a comparison of the inferred fairness metrics to the actual fairness metrics.

Abstract: A method for generating cross-channel recommendations for a customer includes receiving customer data associated with the customer, content data, and clickstream data; encoding the content data using a text encoder, the encoding resulting in content embeddings; encoding the clickstream data using a clickstream encoder, the encoding resulting in clickstream embeddings; providing the content data, the clickstream data, and the clickstream embeddings as inputs to a hybrid latent model; causing execution of the hybrid latent model, the execution resulting in cross-system user and item interaction embeddings; retrieving application features corresponding to the clickstream data from an application feature store; providing the customer data, the content embeddings, the clickstream embeddings, the cross-system user and item interaction embeddings, and the application features as inputs to a recommendation model; causing execution of the recommendation model, the execution resulting in a plurality of ranked recommend

Abstract: Methods and apparatuses are described for automatic data-driven optimization of a target outcome using machine learning. A server generates a first feature dataset and applies a trained outcome prediction model to the first feature dataset as input to generate a second feature data set and a first predicted value for a target outcome. The server displays the first predicted value on a client device. The server receives input corresponding to one or more preferences or constraints from the client device and adjusts the trained outcome prediction model based upon the received input to incorporate the one or more preferences or constraints. The server applies the adjusted outcome prediction to the second feature dataset as input to generate a third feature data set a second predicted value for the target outcome. The server displays the second predicted value on the client device.

Abstract: Methods and apparatuses are described for automatic data-driven optimization of a target outcome using machine learning. A server generates a first feature dataset and applies a trained outcome prediction model to the first feature dataset as input to generate a second feature data set and a first predicted value for a target outcome. The server displays the first predicted value on a client device. The server receives input corresponding to one or more preferences or constraints from the client device and adjusts the trained outcome prediction model based upon the received input to incorporate the one or more preferences or constraints. The server applies the adjusted outcome prediction to the second feature dataset as input to generate a third feature data set a second predicted value for the target outcome. The server displays the second predicted value on the client device.

Abstract: Methods and apparatuses are described for digital content classification and recommendation using constraint-based predictive machine learning. A server trains a machine learning (ML) recommendation model to generate digital content layouts each comprising digital content item slots arranged according to one or more digital content selection constraints. The server receives user profile information for a first user. The server executes the trained ML recommendation model to generate a plurality of digital content item displays, each including a selected digital content item placed in each slot. The server determines, for each digital content item display, an interaction prediction score for each digital content item in the display. The server selects a digital content item display based upon the interaction predictions scores. The server transmits the selected digital content item display to a client device.

Abstract: Methods and apparatuses are described for determining future user actions using time-based featurization of clickstream data. A server captures clickstream data corresponding to web browsing sessions and converts the clickstream data into tokens by identifying each unique URL and parsing each unique URL into tokens. The server generates a frequency matrix based upon the tokens, and generates a latent feature vector for each URL in the session based upon the frequency matrix. The server merges the latent feature vectors and the clickstream data into an aggregate clickstream vector set for a user. The server assigns time-decayed weight values to each latent feature vector in the aggregate clickstream vector set. The server combines the time-decayed latent feature vectors to generate a clickstream embedding for the user, and executes a machine learning model using the clickstream embedding to generate one or more predicted actions of the user.

Abstract: A computer-implemented method is provided for determining surrogate ground truth to enable fairness evaluation after completion of a campaign of interest. The surrogate ground truth indicates individuals who should have been contacted by the campaign of interest. The method includes receiving data for the campaign of interest and data for a previous campaign in relation to a population group selected for the previous campaign. The method also includes generating, before commencement of the campaign of interest, control and treatment models trained based on data collected from the previous campaign. The method further includes calculating, after completion of the campaign of interest, the surrogate ground truth using the trained control and treatment models and data collected from the campaign of interest.

Abstract: Methods and apparatuses are described for digital content classification and recommendation based upon reinforcement learning. A server converts unstructured text corresponding to each digital content item into a content item feature set. The server generates a user context vector associated with a plurality of users. The server trains a linear multi-armed bandit (MAB) classification model based upon the user context vectors and historical user content recommendation information. The server receives a new user context vector associated with a new user. The server executes the MAB model using the new user context vector as input to generate content interaction prediction scores. The server selects the content interaction prediction scores above a predetermined threshold and identifies the associated digital content item. The server presents the identified digital content items on a client device and receives a response. The server updates linear UCB coefficient vectors of the MAB model based upon the response.

Abstract: Methods and apparatuses are described for digital content classification and recommendation based upon reinforcement learning. A server converts unstructured text corresponding to each digital content item into a content item feature set. The server generates a user context vector associated with a plurality of users. The server trains a linear multi-armed bandit (MAB) classification model based upon the user context vectors and historical user content recommendation information. The server receives a new user context vector associated with a new user. The server executes the MAB model using the new user context vector as input to generate content interaction prediction scores. The server selects the content interaction prediction scores above a predetermined threshold and identifies the associated digital content item. The server presents the identified digital content items on a client device and receives a response. The server updates linear UCB coefficient vectors of the MAB model based upon the response.

Abstract: Methods and apparatuses are described for automated predictive product recommendations using reinforcement learning. A server captures historical activity data associated with a plurality of users. The server generates a context vector for each user, the context vector comprising a multidimensional array corresponding to historical activity data. The server transforms each context vector into a context embedding. The server assigns each context embedding to an embedding cluster. The server determines, for each context embedding, (i) an overall likelihood of successful attempt and (ii) an incremental likelihood of success associated products available for recommendation. The server calculates, for each context embedding, an incremental income value associated with each of the likelihoods of success. The server aggregates (i) the overall likelihood of successful attempt, (ii) the likelihoods of success, and (iii) the incremental income values into a recommendation matrix.

Abstract: Techniques for selecting test configurations associated with a particular coverage strength using a constraint solver are disclosed. A set of parameters are configurable for conducting a test on a particular target application. A data model generator identifies one or more candidate test configurations based on the set of parameters. The data model generator determines a set of interactions based on a desired coverage strength. The data model generator specifies a selection variable indicating the candidate test configurations that are selected for testing the particular target application. The data model generator specifies constraint(s) minimizing the number of selected test configurations. The data model generator specifies constraint(s) requiring that each interaction be covered by at least one selected test configuration.

Abstract: Computerized embodiments are disclosed for simulating requests and resources to be assigned to the requests by assignment logic. In one embodiment, a simulation session is initiated by generating test data that includes resource data, request data, and simulation state parameters. The test data is communicated to the assignment logic programmed to generate an assignment solution between resources and requests as represented by the resource data and the request data, respectively. The assignment solution is received from the assignment logic and the test data is updated. The test data can be updated by one or more of updating the simulation state parameters based on the assignment solution, adding at least one new request, or adding at least one new resource. The test data, as updated, is again communicated to the assignment logic and the process repeats until the simulation session is stopped.

Abstract: Techniques for assigning applications to virtual machines (VMs) using constraint programming are disclosed. The applications are associated with application groups. The assignment problem requires that each application of a particular application group be assigned to a VM hosted by a same machine. A data model, for application to a constraint programming solver, formulates the assignment problem as a set of constraints for a solution to be found. The data model includes a set of data model elements corresponding to the applications to be processed. The data model includes a set of data model elements corresponding to the VMs. The data model includes a set of constraints that limits assignment of any particular application to a VM corresponding to a tenant associated with the particular application. The set of constraints further limits assignment of applications corresponding to a same application group to VMs executing on a same machine.

Abstract: Techniques for selecting test configurations associated with a particular coverage strength using a constraint solver are disclosed. A set of parameters are configurable for conducting a test on a particular target application. A data model generator identifies one or more candidate test configurations based on the set of parameters. The data model generator determines a set of interactions based on a desired coverage strength. The data model generator specifies a selection variable indicating the candidate test configurations that are selected for testing the particular target application. The data model generator specifies constraint(s) minimizing the number of selected test configurations. The data model generator specifies constraint(s) requiring that each interaction be covered by at least one selected test configuration.

Abstract: Computerized embodiments are disclosed for simulating requests and resources to be assigned to the requests by assignment logic. In one embodiment, a simulation session is initiated by generating test data that includes resource data, request data, and simulation state parameters. The test data is communicated to the assignment logic programmed to generate an assignment solution between resources and requests as represented by the resource data and the request data, respectively. The assignment solution is received from the assignment logic and the test data is updated. The test data can be updated by one or more of updating the simulation state parameters based on the assignment solution, adding at least one new request, or adding at least one new resource. The test data, as updated, is again communicated to the assignment logic and the process repeats until the simulation session is stopped.

Abstract: Techniques for assigning applications to virtual machines (VMs) using constraint programming are disclosed. The applications are associated with application groups. The assignment problem requires that each application of a particular application group be assigned to a VM hosted by a same machine. A data model, for application to a constraint programming solver, formulates the assignment problem as a set of constraints for a solution to be found. The data model includes a set of data model elements corresponding to the applications to be processed. The data model includes a set of data model elements corresponding to the VMs. The data model includes a set of constraints that limits assignment of any particular application to a VM corresponding to a tenant associated with the particular application. The set of constraints further limits assignment of applications corresponding to a same application group to VMs executing on a same machine.

Abstract: A data model for application to a constraint programming solver is generated. The data model includes a set of data model elements. A particular data model element corresponds to a particular request. The particular data model element also corresponds to one or more resources that may be assigned to the request. The data model also includes a set of constraints. One or more sort/search algorithms may be applied with the data model to the constraint programming solver. The sort/search algorithms may direct the constraint programming solver to output certain preferred assignments of resources to requests.

Abstract: A data model for application to a constraint programming solver is generated. The data model includes a set of data model elements. A particular data model element corresponds to a particular request. The particular data model element also corresponds to one or more resources that may be assigned to the request. The data model also includes a set of constraints. One or more sort/search algorithms may be applied with the data model to the constraint programming solver. The sort/search algorithms may direct the constraint programming solver to output certain preferred assignments of resources to requests.