Abstract: An online experiment system is described that generates a randomization evaluation for an online experiment, while the experiment is ongoing, indicating whether a distribution of experiment participants allocated to one or more participant groups satisfies an expected distribution. The online experiment system analyzes one of the experiment groups to obtain an observed distribution of the subset of experiment participants included in the experiment group. The online experiment system then evaluates the observed distribution relative to the expected distribution for the experiment according to a decision criteria of a population stability index test. The decision criteria is influenced by a tuning parameter that represents a ratio of experiment participants included in the observed experiment group to experiment participants included in a different experiment group.

Abstract: Sample delta monitoring is described herein. In one or more implementations, first and second experimental groups are defined in a randomized controlled trial. A first set of data indicative of a first experimental group in the randomized controlled trial is received. A second set of data indicative of a second experimental group in the randomized controlled trial is received. Based on sample counts of the first and second sets of data, an indication is generated that the experimental groups in the randomized controlled trial are skewed.

Abstract: Methods and systems for enhancing a user interface of a client device are described. In an example embodiment, a machine processes a query to identify one or more item listings that satisfy the query. Each item listing is associated with at least one item being offered for sale. The machine assigns one or more scores to one or more item listings that satisfy the query based on an observed demand metric derived from historical performance data of the one or more item listings. The machine causes a presentation of the one or more item listings that satisfy the query in a search results page displayed in the user interface of the client device. The one or more item listings are positioned in the search results page based on the one or more scores assigned to the one or more item listings.

Abstract: A system and method for generating a score for a listing are described. A data field of a listing is parsed and at least one element from the data field is generated. A score for the listing is calculated based on the at least one element, the listing score representing a probability of the listing being in one of two binary classifications. The listing score and one or more listing attribute values are inputted into a binary classifier. An output is generated using the binary classifier, the output representing a refined score for the listing based on the listing score and at least one of the listing attribute values.

Abstract: Methods and systems for assessing the quality of an item listing are described. In an example embodiment, a listing quality score for an item listing is derived as a weighted sum of first and second parts. The first part represents a predicted score based on a comparison of item attributes for the item listing that are known at listing time, with item attributes of similar item listings that have historical data available for assessing their actual performance. The second part is based on one or more observed demand metrics representing actual historical performance of the item listing. The weighting factor is derived, such that over time, the emphasis shifts from the predicted to the observed score.

Abstract: A system and method for assessing excessive accessory listings in search results includes a processor-implemented textual mining module that parses a data field of a document and generates at least one token from the data field. A processor-implemented scoring module calculates a score for the at least one token, with the at least one token score representing a likelihood that the at least one token belongs to one of two binary classifications. The processor-implemented scoring module also calculates a score for the document based on the at least one token score, with the document score representing a probability of the document being in one of the two binary classifications. A processor-implemented decision tree module inputs the document score and document attribute values into a decision tree and generates an output representing a refined score based on the document score and at least one of the document attribute values.

Abstract: In an example embodiment, a pseudo product group for an item to be listed in an online marketplace having unstructured data is determined. Then, a fair market price for the item is determined using past data from listings in the pseudo product group. The fair market price may be used as a reference price in a curve of price elasticity versus price to obtain a price elasticity value. A price range may then be recommended to a user for the listing based upon the fair market price and the price elasticity value.

Abstract: A system and method for assessing excessive accessory listings in search results includes a processor-implemented textual mining module that parses a data field of a document and generates at least one token from the data field. A processor-implemented scoring module calculates a score for the at least one token, with the at least one token score representing a likelihood that the at least one token belongs to one of two binary classifications. The processor-implemented scoring module also calculates a score for the document based on the at least one token score, with the document score representing a probability of the document being in one of the two binary classifications. A processor-implemented decision tree module inputs the document score and document attribute values into a decision tree and generates an output representing a refined score based on the document score and at least one of the document attribute values.

Abstract: Methods and systems for assessing the quality of an item listing are described. In an example embodiment, a listing quality score for an item listing is derived as a weighted sum of first and second parts. The first part represents a predicted score based on a comparison of item attributes for the item listing that are known at listing time, with item attributes of similar item listings that have historical data available for assessing their actual performance. The second part is based on one or more observed demand metrics representing actual historical performance of the item listing. The weighting factor is derived, such that over time, the emphasis shifts from the predicted to the observed score.