Abstract: This application sets forth techniques for variant testing at scale. In particular, the embodiments set forth provide systems and methods for testing, on a large-scale software application store, visual aspects of one or more variants of representative data associated with an application available through the software application store. According to some embodiments, a method may include using a subset of conversion data associated with a control object and a subset of the conversion data associated with at least one variant object to compute at least one conversion metric for the control object and at least one conversion metric for the at least one variant object. The method may also include generating a performance measurement by applying at least one statistical hypothesis testing function to the at least one conversion metric for the control object and the at least one conversion metric for the at least one variant object.

Abstract: Providing peer group benchmarking with differential privacy obtaining one or more app store metrics for a first application, determining an application peer group for the first application, wherein the application peer group is determined based on a plurality of common traits, and obtaining one or more peer group app store metrics for the application peer group based on the one or more app store metrics. A user interface is displayed to indicate a relative placement of at least one of the one or more app store metrics for the first application among the peer group app store metrics, and the user interface element identifies the application peer group metrics with a minimum level of accuracy without revealing the performance of individual apps within the peer group.

Abstract: Aspects of the subject technology receiving, by a campaign client module and via an advertisement network server, a hierarchical campaign identifier including one or more sub-identifiers. The campaign client module also accesses, from an aggregation server, anonymity data. The campaign client module also selects a sub-identifier from the one or more sub-identifiers of the hierarchical campaign identifier based on the anonymity data, generates a reporting data structure including a set of data based on the sub-identifier, and transmits the reporting data structure to the advertisement network server.