Abstract: Systems and methods for diversity auditing are described. The systems and methods include identifying a plurality of images; detecting a face in each of the plurality of images using a face detection network; classifying the face in each of the plurality of images based on a sensitive attribute using an image classification network; generating a distribution of the sensitive attribute in the plurality of images based on the classification; and computing a diversity score for the plurality of images based on the distribution.

Abstract: Certain aspects and features of this disclosure relate to providing anytime-valid confidence sequences for multiple messaging treatments in an experiment. A process controls and/or corrects statistical error when multiple messaging treatments are being evaluated together. Messages can be stored, formatted, and transmitted from a communication server or other computing system. In one example, each test message from among multiple test messages is sent to an independent group of recipients over some period of time. An analytics application programmatically evaluates a metric related to message responses over time and determines a difference in the metric for each of several unique messages as compared to a baseline message. The analytics application also determines a confidence value and can display the changing confidence value in sequence over time along with the current difference, or lift, while maintaining the accuracy of the values.

Abstract: Methods and systems disclosed herein relate generally to systems and methods for generating simulated images for enhancing socio-demographic diversity. An image-generating application receives a request that includes a set of target socio-demographic attributes. The set of target socio-demographic attributes can define a gender, age, and/or race of a subject that are non-stereotypical for a particular occupation. The image-generating application applies the a machine-learning model to the set of target socio-demographic attributes. The machine-learning model generates a simulated image depicts a subject having visual characteristics that are defined by the set of target socio-demographic attributes.

Abstract: The present disclosure relates to a feature contribution system that accurately and efficiently provides the influence of features utilized in machine-learning models with respect to observed model results. In particular, the feature contribution system can utilize an observed model result, initial contribution values, and historical feature values to determine a contribution value correction factor. Further, the feature contribution system can apply the correction factor to the initial contribution values to determine correction-factor adjusted contribution values of each feature of the model with respect to the observed model result.

Abstract: Multi-modal machine-learning model training techniques for search are described that overcome conventional challenges and inefficiencies to support real time output, which is not possible in conventional training techniques. In one example, a search system is configured to support multi-modal machine-learning model training. This includes use of a preview mode and an expanded mode. In the preview mode, a preview segment is generated as part of real time training of a machine learning model. In the expanded mode, the preview segment is persisted as an expanded segment that is used to train and utilize an expanded machine-learning model as part of search.

Abstract: A cluster generation system identifies data elements, from a first binary record, that each have a particular value and correspond to respective binary traits. A candidate description function describing the binary traits is generated, the candidate description function including a model factor that describes the data elements. Responsive to determining that a second record has additional data elements having the particular value and corresponding to the respective binary traits, the candidate description function is modified to indicate that the model factor describes the additional elements. The candidate description function is also modified to include a correction factor describing an additional binary trait excluded from the respective binary traits. Based on the modified candidate description function, the cluster generation system generates a data summary cluster, which includes a compact representation of the binary traits of the data elements and additional data elements.

Abstract: A method, apparatus, and non-transitory computer readable medium for data analytics are described. Embodiments of the method, apparatus, and non-transitory computer readable medium include monitoring online activity corresponding to a plurality of users; receiving aggregate marketing data for a marketing activity; identifying online activity data for a time period corresponding to the marketing activity based on the monitoring; generating a regression model based on the aggregate marketing data and the online activity data using Bayesian regression, wherein the regression model represents a relationship between the marketing activity and the online activity, comprises a time effect coefficient, and is based on a prior distribution of the time effect coefficient that decays to zero as time increases; and estimating a treatment effect for the marketing activity on the online activity based on the regression model, wherein the treatment effect comprises a rate of effect decay.

Abstract: Machine-learning model retargeting techniques are described. In one example, training data is generated by extrapolating feedback data collected from entities. These techniques supports an ability to identify a wider range of thresholds and corresponding entities than those available in the feedback data. This also provides an opportunity to explore additional thresholds than those used in the past through extrapolating operations outside of a range used to define a segment, for which, the feedback data is captured. These techniques also support retargeting of a machine-learning model for a secondary label that is different than a primary label used to initially train the machine-learning model.

Abstract: A system and method for content distribution without tracking is described. The system and method includes determining that device identifiers are not available for a first digital content channel; identifying a first cluster of users and a second cluster of users based on the determination that device identifiers are not available; providing first content and second content via the first digital content channel; monitoring user interactions on the first digital content channel to obtain a first conversion rate for users in the first cluster that receive the first content and a second conversion rate for users in the second cluster that receive the second content; computing a cross-cluster treatment effect based on the first conversion rate and the second conversion rate; computing a treatment effect for the first content based on the cross-cluster treatment effect; and providing the first content to a subsequent user based on the treatment effect.

Abstract: A cluster generation system identifies data elements, from a first binary record, that each have a particular value and correspond to respective binary traits. A candidate description function describing the binary traits is generated, the candidate description function including a model factor that describes the data elements. Responsive to determining that a second record has additional data elements having the particular value and corresponding to the respective binary traits, the candidate description function is modified to indicate that the model factor describes the additional elements. The candidate description function is also modified to include a correction factor describing an additional binary trait excluded from the respective binary traits. Based on the modified candidate description function, the cluster generation system generates a data summary cluster, which includes a compact representation of the binary traits of the data elements and additional data elements.

Abstract: Systems and methods for diversity auditing are described. The systems and methods include identifying a plurality of images; detecting a face in each of the plurality of images using a face detection network; classifying the face in each of the plurality of images based on a sensitive attribute using an image classification network; generating a distribution of the sensitive attribute in the plurality of images based on the classification; and computing a diversity score for the plurality of images based on the distribution.

Abstract: Multi-modal machine-learning model training techniques for search are described that overcome conventional challenges and inefficiencies to support real time output, which is not possible in conventional training techniques. In one example, a search system is configured to support multi-modal machine-learning model training. This includes use of a preview mode and an expanded mode. In the preview mode, a preview segment is generated as part of real time training of a machine learning model. In the expanded mode, the preview segment is persisted as an expanded segment that is used to train and utilize an expanded machine-learning model as part of search.

Abstract: Methods and systems disclosed herein relate generally to systems and methods for generating simulated images for enhancing socio-demographic diversity. An image-generating application receives a request that includes a set of target socio-demographic attributes. The set of target socio-demographic attributes can define a gender, age, and/or race of a subject that are non-stereotypical for a particular occupation. The image-generating application applies the a machine-learning model to the set of target socio-demographic attributes. The machine-learning model generates a simulated image depicts a subject having visual characteristics that are defined by the set of target socio-demographic attributes.

Abstract: This disclosure involves detecting biases in predictive models and the root cause of those biases. For example, a processing device receives test data and training data from a client device. The processing device identifies feature groups from the training data and the test data generates performance metrics and baseline metrics for a feature group. The processing device detects biases through a comparison of the performance metrics and the baseline metrics the feature group. The processing device then isolates a portion of the training data that corresponds to the detected bias. The processing device generates a model correction usable to remove the bias from the predictive model.

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media for generating pairs of natural language queries and corresponding query-language representations. For example, the disclosed systems can generate a contextual representation of a prior-generated dialogue sequence to compare with logical-form rules. In some implementations, the logical-form rules comprise trigger conditions and corresponding logical-form actions for constructing a logical-form representation of a subsequent dialogue sequence. Based on the comparison to logical-form rules indicating satisfaction of one or more trigger conditions, the disclosed systems can perform logical-form actions to generate a logical-form representation of a subsequent dialogue sequence.

Abstract: This disclosure relates to methods, non-transitory computer readable media, and systems that generate and visualize bias scores within segment-generation-user interfaces prior to executing proposed actions with regard to target segments. For example, the disclosed systems can generate a bias score indicating a measure of bias for a characteristic within a segment of users selected for a proposed action and visualize the bias score and corresponding characteristic in a segment-generation-user interface. In some implementations, the disclosed systems can further integrate detecting and visualizing bias as a bias score with selectable options for a segmentation-bias system to generate and modify segments of users to reduce detected bias.

Abstract: The present disclosure relates applying a survival analysis to model when a particular recipient will view an electronic message. For example, one or more embodiments train a survivor function to model the time that will elapse, on a continuous scale, before a recipient will open an electronic message. For example, one or more embodiments involve accessing analytics training data and extracting a first set of features affecting the time that elapsed before past recipients opened an electronic message and a second set of features affecting whether the recipients opened the electronic message at all. The system then generates a mixture model modified survivor function and determines the effect of each feature set on its corresponding outcome to learn parameters for the mixture model modified survivor function.

Abstract: A method, apparatus, and non-transitory computer readable medium for data analytics are described. Embodiments of the method, apparatus, and non-transitory computer readable medium include monitoring online activity corresponding to a plurality of users; receiving aggregate marketing data for a marketing activity; identifying online activity data for a time period corresponding to the marketing activity based on the monitoring; generating a regression model based on the aggregate marketing data and the online activity data using Bayesian regression, wherein the regression model represents a relationship between the marketing activity and the online activity, comprises a time effect coefficient, and is based on a prior distribution of the time effect coefficient that decays to zero as time increases; and estimating a treatment effect for the marketing activity on the online activity based on the regression model, wherein the treatment effect comprises a rate of effect decay.

Abstract: The present disclosure is directed toward systems and methods for generating an un-subscription model and predicting whether a potential customer will un-subscribe from receiving electronic marketing content from a marketing source. For example, systems and methods described herein involve generating a prediction un-subscription model that predicts whether a potential customer is prone to un-subscribe from receiving future communications about a product or merchant in response to receiving a communication for the product or merchant. The systems and methods further involve determining an appropriate action to take with regard to a potential customer based on whether the potential customer is prone to un-subscribe from receiving future communications.

Abstract: This disclosure involves detecting biases in predictive models and the root cause of those biases. For example, a processing device receives test data and training data from a client device. The processing device identifies feature groups from the training data and the test data generates performance metrics and baseline metrics for a feature group. The processing device detects biases through a comparison of the performance metrics and the baseline metrics the feature group. The processing device then isolates a portion of the training data that corresponds to the detected bias. The processing device generates a model correction usable to remove the bias from the predictive model.