Abstract: Methods, systems, and apparatus include computer programs encoded on a computer-readable storage medium, including a method for discovering unique entities over multiple devices. A virtual pool of entities is created and divided into subpools, each including fewer than all entities. Subpools are subdivided into delta pools. Cookies are recorded for each delta pool when the particular portion of content is presented to or accessed by entities in the delta pool. Recorded cookies are divided into cookie types based on cookie characteristics. Machine learning and statistical analysis algorithms are used to automatically determine sizes of delta pools and probabilities of each cookie type being classified as belonging to particular delta pools. Virtual entities are assigned from the virtual pool to each of the recorded cookies that were recorded when the particular portion of content was presented. A number of unique entities that accessed the particular portion of content is determined.

Abstract: A computer-implemented method includes generating a test group for a pointer to information; generating a control group for the pointer to information; receiving conversion information from the test group and the control group; and determining, based on the conversion information, a number of incremental conversions that are attributable to the pointer.

Abstract: Methods, systems, and apparatus, including computer program products, for determining a mixture ratio for allocating portions of an advertising budget among different advertising channels (e.g., print, online, radio, television) to optimize a performance measure, such as cost-per-action. A mixture space is used to define the available advertising channels and any constraints placed on those channels, such as no more than fifty percent of the advertising budget being allocated to a particular channel, and test mixture ratios are selected according to an optimality criterion. The selected test mixture ratios are used during a testing period on live traffic. The performance measures from the test mixture ratios are used to select a preferred mixture ratio from the mixture space.

Abstract: Methods, systems, and apparatus, including computer program products, for determining a mixture ratio for allocating portions of an advertising budget among different advertising channels (e.g., print, online, radio, television) to optimize a performance measure, such as cost-per-action. A mixture space is used to define the available advertising channels and any constraints placed on those channels, such as no more than fifty percent of the advertising budget being allocated to a particular channel, and test mixture ratios are selected according to an optimality criterion. The selected test mixture ratios are used during a testing period on live traffic. The performance measures from the test mixture ratios are used to select a preferred mixture ratio from the mixture space.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer-readable storage medium, for determining performance for a campaign.

Abstract: Systems, methods and computer program products for determining lost opportunities resulting from changes to advertising spending are described. To assist advertisers in evaluating and allocating a proper budget to advertising, an analyzer can be used to develop an analytical model that gathers data pertaining to the incremental value of search advertising. (e.g., the true cost of the additional click lost or gained), which can be presented to the advertisers when changes have been made/proposed to the advertiser's advertising spending. The analyzer can detect large changes in advertising spending, and indicate (e.g., by prediction) how many total clicks were lost or gained as a result of the change in advertising spending to allow the advertisers to visualize the impact to changes in advertising spending, and determine when to decrease advertising budget on ads that yield low return-on-investment or to increase advertising budget to maximize the effectiveness of an active ad campaign.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for performing geography-based advertising experiments. One method includes receiving pre-spend data for geographic regions, identifying the geographic regions as control or treatment regions, obtaining change in ad spend data for each region, estimating a variance in a return on ad spend according to the pre-spend and the change in ad spend data. The method further includes determining whether the variance satisfies an acceptance criterion, and either allocating the change in ad spend data for use in an advertising experiment or selecting different change in ad spend data. Another method includes receiving pre-spend data for geographic regions, determining a change in ad spend for each geographic region, fitting a model to the pre-spend data, the change in ad spend, and test data, and determining a return on ad spend from the fitted model.

Abstract: A method and system for generating an accurate model of noise created through near end crosstalk (NEXT) when qualifying DSL loops. The model is configured such that it may account for loop characteristics such as fill and gauge of a particular loop, and disturbers which may create a noise environment within the particular binder group. The noise model may further be employed in an automated system which, based on stored characteristics for a particular loop, is employed to generate a noise value which is then further employable in a qualification analysis.

Abstract: A system and method provides for the identification of termination telephone numbers in a telephone network which are being employed as a data network modem. When a local exchange carrier (LEC) routes a call to another telecommunications network, such as one used by a competitive local exchange carriers (CLECs), it may be advantageous for reciprocal compensation purposes to identify which termination telephone numbers serviced by the CLECs are acting as modems. As such, all telephone traffic to a particular CLEC is monitored over a predetermined period of time. The information gathered is then processed and segregated according to terminating telephone number. At that point, comparisons may be made against reference information to determine whether the terminating telephone number is a modem or a non-modem.

Abstract: A method and system for generating an accurate model of noise created through near end crosstalk (NEXT) when qualifying DSL loops. The model is configured such that it may account for loop characteristics such as fill and gauge of a particular loop, and disturbers which may create a noise environment within the particular binder group. The noise model may further be employed in an automated system which, based on stored characteristics for a particular loop, is employed to generate a noise value which is then further employable in a qualification analysis.