Abstract: Techniques for generating unbiased estimates of causal effects in online experiments are provided. In one technique, campaign data is received that includes a first set of targeting criteria and a second set of targeting criteria. An online experiment is established that comprises a content delivery campaign that is associated with a treatment group and a control group. Afterward, a content request is received and a first entity that initiated the content request is identified. In response to determining that the first entity is targeted by the campaign based on the first set of targeting criteria, the first entity is randomly assigned to the control group. In responses to second content request, a second entity is identified. In response to determining that the second entity is targeted by the campaign based on the first set of targeting criteria, the second entity is randomly assigned to the treatment group.

Abstract: The disclosed embodiments provide a system for performing unified management of targeting attributes in A/B tests. During operation, the system obtains attribute configurations for attributes to be used in subsequent targeting of users by A/B tests. Next, the system configures, based on the attribute configurations, onboarding of the attributes from an offline environment, a near-real-time environment, and an online environment. During an A/B test, the system retrieves values of one or more of the attributes for a user from locations specified in the attribute configurations. Finally, the system outputs the values with targeting conditions for the A/B test for use in selecting a treatment assignment for the user in the A/B test.

Abstract: The disclosed embodiments provide a system for managing an A/B test. During operation, the system calculates a first risk associated with ramping up exposure to a first A/B test by a first ramp amount. Next, the system uses a first sequential hypothesis test to compare the first risk with a first risk tolerance for the first A/B test. When the first sequential hypothesis test indicates that the first risk is within the first risk tolerance, the system automatically triggers a ramp-up of exposure to the first A/B test by the first ramp amount.