Abstract: Methods and apparatus are described for evaluating a binary classification system operable to classify each of a plurality of events as a first event type or a second event type. At least some of the events of the first event type are independently verifiable with reference to verification data. The binary classification system is susceptible to a first error type in which events of the first event type are classified as the second event type, and a second error type in which events of the second event type are classified as the first event type. Operation of a first configuration of the binary classification system is evaluated with reference to an objective function. The objective function is derived by expressing a number of errors of the second error type in terms of a number of errors of the first error type with reference to the verification data, and by assuming relative proportions of the first and second event types within the plurality of events.

Abstract: Methods and apparatus are described for evaluating a binary classification system operable to classify each of a plurality of events as a first event type or a second event type. At least some of the events of the first event type are independently verifiable with reference to verification data. The binary classification system is susceptible to a first error type in which events of the first event type are classified as the second event type, and a second error type in which events of the second event type are classified as the first event type. Operation of a first configuration of the binary classification system is evaluated with reference to an objective function. The objective function is derived by expressing a number of errors of the second error type in terms of a number of errors of the first error type with reference to the verification data, and by assuming relative proportions of the first and second event types within the plurality of events.

Abstract: Methods and apparatus are described for evaluating a binary classification system operable to classify each of a plurality of events as a first event type or a second event type. At least some of the events of the first event type are independently verifiable with reference to verification data. The binary classification system is susceptible to a first error type in which events of the first event type are classified as the second event type, and a second error type in which events of the second event type are classified as the first event type. Operation of a first configuration of the binary classification system is evaluated with reference to an objective function. The objective function is derived by expressing a number of errors of the second error type in terms of a number of errors of the first error type with reference to the verification data, and by assuming relative proportions of the first and second event types within the plurality of events.

Abstract: Methods and apparatus are described for evaluating a binary classification system operable to classify each of a plurality of events as a first event type or a second event type. At least some of the events of the first event type are independently verifiable with reference to verification data. The binary classification system is susceptible to a first error type in which events of the first event type are classified as the second event type, and a second error type in which events of the second event type are classified as the first event type. Operation of a first configuration of the binary classification system is evaluated with reference to an objective function. The objective function is derived by expressing a number of errors of the second error type in terms of a number of errors of the first error type with reference to the verification data, and by assuming relative proportions of the first and second event types within the plurality of events.

Abstract: An indication is generated of desirability of user's activity relative to a website that includes a plurality of web pages. Indications of the user's activity with respect to the plurality of web pages are processed. At least a portion of the indications are of user's activity with respect to the website other than a conversion activity with respect to the website. Based on the processed indications, generating an overall indication of desirability of the user's activity with respect to the web site. Theses steps may be repeated for a plurality of users. Each user's activity may be categorized (not based on the determined overall indication of desirability), and a measure of the quality of the category determination may be determined using the determined overall indications of desirability.