Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for creating experiment content distribution campaigns (“experiment campaign”) and allocating content in accordance with the experiment campaign. In one aspect, a control campaign or an experiment campaign are selected as an available campaign for providing content in response to receiving a content request. The selection can be a probability constrained selection where the probability with which the experiment campaign is selected is specified as a portion of a total number of requests received during an experiment period for which the experiment campaign is available to provide content. Content is provided in response to the request, where the provided content being content corresponding to the selected campaign.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for selecting controlling keywords. In one aspect, a method includes determining that two or more targeting keywords for a same content item source are matched by one or more terms in a content item request, where a first targeting keyword matches the one or more terms with a more specific type of match than the second targeting keyword. If the bid associated with the second targeting keyword does not exceed the bid associated with the first targeting keyword, the first targeting keyword and the second targeting keyword are ranked based on selection scores. The targeting keywords are ranked based at least in part on the quality scores when the bid associated with the second targeting keyword does exceed the bid associated with the first targeting keyword.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for selecting controlling keywords. In one aspect, a method includes determining that two or more targeting keywords for a same content item source are matched by one or more terms in a content item request, where a first targeting keyword matches the one or more terms with a more specific type of match than the second targeting keyword. If the bid associated with the second targeting keyword does not exceed the bid associated with the first targeting keyword, the first targeting keyword and the second targeting keyword are ranked based on selection scores. The targeting keywords are ranked based at least in part on the quality scores when the bid associated with the second targeting keyword does exceed the bid associated with the first targeting keyword.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for printing online resources. In one aspect, a method includes receiving a request to print an online resource that is presented at a user device according to a presentation format in which a first advertisement is presented at a first presentation position. A print format for a print-out of the online resource is selected. The print format specifies a second advertisement position at which a second advertisement is included in the print-out of the online resource. The second advertisement is selected based, at least in part, on characteristics of the second advertisement position. The second advertisement can be a different advertisement than the first advertisement. Data that cause the resource and the second advertisement to be printed according to the print format are provided.

Abstract: This specification describes technologies relating to displaying online content. In general, one aspect of the subject matter described in this specification can be embodied in methods that include receiving a collection of advertisement candidates for display in an online medium, the advertisement candidates each assigned a quality score calculated based at least in part on a measure indicative of relevance of the respective advertisement candidate to online content for concurrent display in the online medium, determining a score threshold based at least in part on relationships among multiple quality scores of the quality scores associated with the advertisement candidates in the collection of advertisement candidates, and based on the determined score threshold, identifying a subset of advertisement candidates of the collection for display. Other embodiments of this aspect include corresponding systems, apparatus, and computer program products.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for creating experiment content distribution campaigns (“experiment campaign”) and allocating content in accordance with the experiment campaign. In one aspect, a control campaign or an experiment campaign are selected as an available campaign for providing content in response to receiving a content request. The selection can be a probability constrained selection where the probability with which the experiment campaign is selected is specified as a portion of a total number of requests received during an experiment period for which the experiment campaign is available to provide content. Content is provided in response to the request, where the provided content being content corresponding to the selected campaign.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for creating experiment content distribution campaigns (“experiment campaign”) and allocating content in accordance with the experiment campaign. In one aspect, a control campaign or an experiment campaign are selected as an available campaign for providing content in response to receiving a content request. The selection can be a probability constrained selection where the probability with which the experiment campaign is selected is specified as a portion of a total number of requests received during an experiment period for which the experiment campaign is available to provide content. Content is provided in response to the request, where the provided content being content corresponding to the selected campaign.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for creating experiment content distribution campaigns (“experiment campaign”) and allocating content in accordance with the experiment campaign. In one aspect, a control campaign or an experiment campaign are selected as an available campaign for providing content in response to receiving a content request. The selection can be a probability constrained selection where the probability with which the experiment campaign is selected is specified as a portion of a total number of requests received during an experiment period for which the experiment campaign is available to provide content. Content is provided in response to the request, where the provided content being content corresponding to the selected campaign.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for selecting controlling keywords. In one aspect, a method includes determining that two or more targeting keywords for a same content item source are matched by one or more terms in a content item request, where a first targeting keyword matches the one or more terms with a more specific type of match than the second targeting keyword. If the bid associated with the second targeting keyword does not exceed the bid associated with the first targeting keyword, the first targeting keyword and the second targeting keyword are ranked based on selection scores. The targeting keywords are ranked based at least in part on the quality scores when the bid associated with the second targeting keyword does exceed the bid associated with the first targeting keyword.

Abstract: In general, in one aspect, a first request to provide one or more advertisements on a first web page is received, the first web page comprising one or more items of web content related to a query. A first quality score is calculated for a first advertisement included in a set of one or more candidate advertisements. The first advertisement is presented on the first web page based at least in part on the first quality score. A second request is received to provide one or more advertisements on a second web page, the second web page comprising one or more different items of web content related to the query. A second quality score for the first advertisement is calculated based at least in part on the previous presentation of the first advertisement. It is determined whether to present the first advertisement on the second web page based at least in part on the second quality score.