Abstract: Techniques for evaluating a user experience experiment designed to use one user experience variant selected from a number of user experience variants as a global-best user experience variant to be used across users relative to a machine model trained to use user data to identify a user-preferred user experience variant. Disclosed systems and methods provide techniques for optimizing user response. In one embodiment, a global-best user experience variant is evaluated by comparing an aggregate user response determined for the global-best user experience variant to an aggregate user response determined using user response predictions determined using the trained machine model, and using the outcome of the comparison to make a recommendation as to which one of the global-best user experience variant and the trained machine model to adopt for providing a user experience to users.

Abstract: Techniques for assigning users to buckets for use in bucket experiments are disclosed. Disclosed systems and methods provide systems and methods for making automatic bucket assignments using Nearest Neighbor Matching (NNM). In one embodiment, an iterative approach is used in assigning users to buckets, such that in a given iteration selected users are assigned to a number of buckets, the selected users being an initial user selected from a pool of users and other users selected using pairwise distances associated with the initial user and the other users.