Abstract: Methods, systems, and computer programs are presented for eliminating bias while training an ML model using training data that includes past experimental data. One method includes accessing experiment results, for A/B testing of a first model, that comprise information regarding engagement with a first set of items presented to users, each item being presented within an ordered list of results. A position bias is calculated for positions within the ordered list of results where the items were presented. A machine-learning program is trained to obtain a second model using a training set comprising values for features that include the calculated position bias. The method includes detecting a second set of items to be ranked for presentation to a first user, and calculates, using the second model, a relevance score for the second set of items, which are ranked based on the respective relevance score and presented on a display.

Abstract: Methods, systems, and computer programs are presented for eliminating bias while training an ML model using training data that includes past experimental data. One method includes accessing experiment results, for A/B testing of a first model, that comprise information regarding engagement with a first set of items presented to users, each item being presented within an ordered list of results. A position bias is calculated for positions within the ordered list of results where the items were presented. A machine-learning program is trained to obtain a second model using a training set comprising values for features that include the calculated position bias. The method includes detecting a second set of items to be ranked for presentation to a first user, and calculates, using the second model, a relevance score for the second set of items, which are ranked based on the respective relevance score and presented on a display.