Abstract: A machine may be configured to generate A/B test reports on demand. For example, the machine causes a display of a user interface for receiving a request of a customized report of result data of an A/B test. The machine receives, via the user interface, an identifier of the A/B test, a specification of a metric associated with the result data, a specification of a dimension of the metric, a specification of a location of the result data, and a request to generate the customized report. The machine generates the metric based on the identifier of the A/B test, the specification of the metric, and the result data. The machine generates the customized report pertaining to the dimension of the metric based on the generated metric and the specification of the dimension of the metric. The machine causes a display of the customized report in the user interface.

Abstract: Techniques for conducting A/B experimentation of online content are described. According to various embodiments, a user specification of a metric being recorded as a result of an online A/B experiment of online content is received, the online A/B experiment being targeted at a segment of members of an online social networking service. Thereafter, a power value for the A/B experiment that is associated with the metric is calculated, the power value indicating an inferred ability to detect changes in a value of the metric during performance of the A/B experiment. The power value for the A/B experiment is then displayed via a user interface displayed on a client device.

Abstract: Techniques for conducting A/B experimentation of online content are described. According to various embodiments, a user specification of a metric associated with operation of an online social networking service is received. A set of one or more A/B experiments of online content is then identified, each A/B experiment being targeted at a segment of members of the online social networking service. Thereafter, each of the A/B experiments is ranked, based on an inferred impact on the value of the metric in response to application of a treatment variant of each A/B experiment to the online social networking service. A list of one or more of the ranked A/B experiments is then displayed, via a user interface displayed on a client device.

Abstract: Techniques for conducting A/B experimentation of online content are described. According to various embodiments, it is determined that a member of an online social networking service is accessing electronic content that is associated with an A/B experiment, the member being associated with a given member ID. The given member ID and a request for a correct variant of the A/B experiment for display for the given member ID is transmitted, from a service associated with the online content, to an A/B experiment server. The correct variant of the A/B experiment for display for the given member ID is accessed from a database, and information regarding the correct variant of the A/B experiment is transmitted to the service associated with the online content. Thereafter, the correct variant of the A/B experiment in conjunction with the online content is displayed, to a client device associated with the member.

Abstract: Techniques for conducting A/B experimentation of online content are described. According to various embodiments, a site-wide impact value for an A/B experiment that is associated with a metric is calculated, the site-wide impact value indicating a predicted percentage change in the value of a metric responsive to application of a treatment variant to an entire portion of a targeted segment of members, in comparison to application of a control variant to an entire portion of the targeted segment of members.

Abstract: Techniques for conducting A/B experimentation of online content are described. According to various embodiments, a user specification of targeting criteria defining a targeted segment of members of an online social networking service for an A/B experiment is received. A set of members of the online social networking service satisfying the user-specified targeting criteria is then identified. Thereafter, a user specification of allocation criteria defining one or more variants of an A/B experiment and one or more corresponding allocation percentages is received. A subset of the set of members (corresponding to the user-specific allocation percentage associated with the respective user-specified variant) is then assigned to each of the user-specified variants. Further, at least one data record associated with each of the variants is recorded in a database, each data record indicating the subset of the set of members assigned to the respective variant.

Abstract: Techniques for conducting A/B testing are described. According to various embodiments, a user request to configure an A/B test is received, and a targeting user interface is displayed. The targeting user interface includes a group of one or more windows, each of the windows being configured to display one or more user-specified targeting criteria statements that identify members of an online social networking service based on member attributes of the members. Thereafter, a user specification of one or more targeting criteria statements is received in conjunction with one or more of the windows. Targeting criteria information defining a set of members of the online social networking service to be targeted for the A/B test is then generated, the targeting criteria information indicating that all members satisfying all the targeting criteria statements for any window in the group of windows are to be targeted for the A/B test.