Abstract: Technologies for on-device experimentation include embodiments that receive a request to provide digital content for display in a slot of a user interface display screen of a first device. By a secure execution environment, an identifier received with the request is anonymized. The request is determined to be associated with a content distribution test. The anonymized identifier is assigned to a test group associated with the content distribution test. The secure execution environment receives user interface event data generated by the first device in response to the content distribution test. The received user interface event data is attributed to the test group. An association of the user interface event data with the test group and the content distribution test is provided by the secure execution environment to a second device different than the first device while the identifier and the anonymized identifier are not provided to the second device.

Abstract: Technologies for on-device experimentation include embodiments that receive a request to provide digital content for display in a slot of a user interface display screen of a first device. By a secure execution environment, an identifier received with the request is anonymized. The request is determined to be associated with a content distribution test. The anonymized identifier is assigned to a test group associated with the content distribution test. The secure execution environment receives user interface event data generated by the first device in response to the content distribution test. The received user interface event data is attributed to the test group. An association of the user interface event data with the test group and the content distribution test is provided by the secure execution environment to a second device different than the first device while the identifier and the anonymized identifier are not provided to the second device.

Abstract: Techniques for generating unbiased estimates of causal effects in online experiments are provided. In one technique, campaign data is received that includes a first set of targeting criteria and a second set of targeting criteria. An online experiment is established that comprises a content delivery campaign that is associated with a treatment group and a control group. Afterward, a content request is received and a first entity that initiated the content request is identified. In response to determining that the first entity is targeted by the campaign based on the first set of targeting criteria, the first entity is randomly assigned to the control group. In responses to second content request, a second entity is identified. In response to determining that the second entity is targeted by the campaign based on the first set of targeting criteria, the second entity is randomly assigned to the treatment group.

Abstract: Techniques for generating a multidimensional forecast are provided. In one technique, multiple segments are generated, each comprising a different set of attribute values. For each segment, a set of prior content requests for the segment is determined based on historical data, a forecasted number of content requests is determined based on the set of prior content requests, and the forecasted number of content requests is stored in association with a set of attribute values corresponding to the segment. A request is received to forecast performance of a content delivery campaign based on a particular set of attribute values. In response to receiving the request, multiple segments that share the particular set of attribute values are identified. The forecasted number of content requests associated with each segment of the multiple segments are aggregated to generate aggregated performance data. A portion of the aggregated performance data is caused to be displayed.