Abstract: A machine may be configured to facilitate following metrics generated by an A/B testing system. For example, the machine determines, based on a metric identifier of a metric that measures an aspect of a service provided on a Social Networking Service, one or more experiment identifiers of experiments that impact the metric. The machine accesses, based on the metric identifier, a subscription record in a database. The subscription record identifies one or more users who requested to follow the particular metric. The machine identifies, based on the subscription record, a user identifier of a user who requested to follow the metric. The machine generates a digital content item that references the metric and the experiments that impact the metric. The machine causes a presentation of the digital content item in a user interface of a client device associated with the user.

Abstract: A machine may be configured to generate A/B test reports on demand. For example, the machine causes a display of a user interface for receiving a request of a customized report of result data of an A/B test. The machine receives, via the user interface, an identifier of the A/B test, a specification of a metric associated with the result data, a specification of a dimension of the metric, a specification of a location of the result data, and a request to generate the customized report. The machine generates the metric based on the identifier of the A/B test, the specification of the metric, and the result data. The machine generates the customized report pertaining to the dimension of the metric based on the generated metric and the specification of the dimension of the metric. The machine causes a display of the customized report in the user interface.

Abstract: A machine may be configured to manage alerts related to ramping A/B experiments. For example, the machine identifies an A/B experiment that targets users of a social networking service (SNS). The machine accesses a first value of a metric associated with operation of the SNS. The first value of the metric is generated as a result of a previous execution of the A/B experiment targeting a first segment of users. The machine generates a predicted second value of the metric based on executing a prediction model associated with the A/B experiment. The executing of the prediction model targets a second segment of users that is greater than the first segment. The machine determines that the predicted second value of the metric indicates an inferred negative impact of the A/B experiment on the metric. The machine causes a display of an alert in a user interface displayed on a client device.

Abstract: Techniques for conducting A/B experimentation of online content are described. According to various embodiments, a user specification of a metric being recorded as a result of an online A/B experiment of online content is received, the online A/B experiment being targeted at a segment of members of an online social networking service. Thereafter, a power value for the A/B experiment that is associated with the metric is calculated, the power value indicating an inferred ability to detect changes in a value of the metric during performance of the A/B experiment. The power value for the A/B experiment is then displayed via a user interface displayed on a client device.

Abstract: Techniques for conducting A/B experimentation of online content are described. According to various embodiments, a user specification of a metric associated with operation of an online social networking service is received. A set of one or more A/B experiments of online content is then identified, each A/B experiment being targeted at a segment of members of the online social networking service. Thereafter, each of the A/B experiments is ranked, based on an inferred impact on the value of the metric in response to application of a treatment variant of each A/B experiment to the online social networking service. A list of one or more of the ranked A/B experiments is then displayed, via a user interface displayed on a client device.