Abstract: Performance-based digital content delivery in a digital medium environment is described. Initially, different items of a collection of digital content are delivered to a substantially equal number of users. The collection is then iteratively tested to identify which content item achieves a desired action (e.g., conversion) at a highest rate. During the iterative test, data describing user interaction with the delivered content is collected. Based on the collected data, measures of achievement are determined for the different content items. Measures of statistical guarantee are also computed that indicate an estimated accuracy of the achievement measures. Responsive to determining that a condition for ending the test has not yet occurred, an optimized allocation is computed for delivery of the content by applying one of multiple allocation optimization techniques. The particular technique applied is based on the condition for ending the test and a type of statistical guarantee associated with the test.

Abstract: Performance-based digital content delivery in a digital medium environment is described. Initially, different items of a collection of digital content are delivered to a substantially equal number of users. The collection is then iteratively tested to identify which content item achieves a desired action (e.g., conversion) at a highest rate. During the iterative test, data describing user interaction with the delivered content is collected. Based on the collected data, measures of achievement are determined for the different content items. Measures of statistical guarantee are also computed that indicate an estimated accuracy of the achievement measures. Responsive to determining that a condition for ending the test has not yet occurred, an optimized allocation is computed for delivery of the content by applying one of multiple allocation optimization techniques. The particular technique applied is based on the condition for ending the test and a type of statistical guarantee associated with the test.

Abstract: Techniques for allocating test offers to determine a best offer in a set of offers with a desired confidence level. An allocatiton of additional test offers is determined by allocating the additional test offers unevenly, e.g., allocating more additional test offers to offer A than to offer B, etc. The test offers are allocated to test the offers in the set of offers to attain the desired confidence level that one of the offers is the best offer for the campaign. For example, this can involve allocating test offers in a way that is expected to minimize the number of additional test offers needed to attain the desired confidence level in the best offer. Additional test offers are made until a best offer is identified with the desired confidence level based on the outcome information.