Abstract: An online system generates a set of genomic representations, each including multiple genes, in which each gene represents users assigned to a control or test group for performing a test. A metric is identified based on a treatment associated with the test group and a score for each representation is computed based on a difference between two values, in which each value is based on the metric associated with users assigned to the test or control group. A propagation process is executed by identifying representations having at least a threshold score, propagating genes included in the representations to an additional set of representations through recombination and/or mutation, and computing the score for each additional representation. The propagation process is repeated for each additional set of representations until stopping criteria are met and a representation is selected based on scores associated with one or more representations.

Abstract: A variation testing system environment for simulating adaptive experiments of objects is disclosed. An experiment system conducts one or more simulations of an adaptive experiment that includes a plurality of variants of an object. Simulation results based on the one or more simulations are generated that are indicative of at least an estimated amount of time to conduct a real-world adaptive experiment based on the one or more simulations.

Abstract: An online system generates a set of genomic representations, each including multiple genes, in which each gene represents users assigned to a control or test group for performing a test. A metric is identified based on a treatment associated with the test group and a score for each representation is computed based on a difference between two values, in which each value is based on the metric associated with users assigned to the test or control group. A propagation process is executed by identifying representations having at least a threshold score, propagating genes included in the representations to an additional set of representations through recombination and/or mutation, and computing the score for each additional representation. The propagation process is repeated for each additional set of representations until stopping criteria are met and a representation is selected based on scores associated with one or more representations.

Abstract: A user treatment engine uses user data describing characteristics of a user to evaluate a set of treatments that the user treatment engine may apply to the user. The user treatment engine generates treatment cost predictions for the treatments and generates treatment scores for the set of treatments based on the treatment cost predictions for the treatments and the user data for the user. The user treatment engine selects and applies a treatment from the set of treatments based on the generated treatment scores. The user treatment engine determines a reward to the online concierge system for the application of the treatment to the user and updates treatment selection parameters for the applied treatment based on the determined reward.

Abstract: An online concierge system evaluates different parameters for order fulfillment in different geographic regions and evaluates order fulfillment in the different geographic regions to assess the different parameters. However, shoppers may fulfill orders in different geographic regions, causing spillover where order fulfillment by a shopper is affected by different parameters. To reduce spillover but still obtain a large number of zones for testing, the online concierge system generates a graph of geographic regions, with connections between geographic regions based on shoppers who fulfill orders in different geographic regions. The online concierge system generates a cluster including geographic regions connected to each other with a connection satisfying one or more criteria indicating at least a threshold percentage of shoppers fulfilling orders in each of the connected geographic regions.

Abstract: An online concierge system receives orders from users that include items from one or more warehouses. The online concierge system identifies the orders to shoppers, who select one or more orders to fulfill. The online concierge system uses models to estimate orders likely to be received at different times and shoppers likely to be available to fulfill orders at different times. Responsive to greater than a threshold difference between estimated orders and estimated shoppers during a time interval, the online concierge system selects one or more incentives for shoppers to select orders during the time interval to entice shoppers to select orders during the time interval. An interface displayed to the shoppers by the online concierge system may present a map of warehouses and their estimated number of orders and allow shoppers to identify incentives offered for fulfilling orders at different warehouses during the time interval.