Abstract: A system can include a data processing system that can include memory and one or more processors to generate, by a first model trained using machine learning and compatible with first data having a first type and second data having a second type, a first metric based on the first data and indicating a first fault probability in a second model, generate, by the first model, a second metric based on the second data and indicating a second fault probability in a third model, determine, based on the first metric and the second metric, that an aggregate model that includes the second model and the third model satisfies a heuristic indicating a third fault probability in the aggregate model, and instruct, in response to a determination that the aggregate model satisfies the heuristic, a user interface to present an indication that the aggregate model satisfies the heuristic.

Abstract: A Bayesian test of demographic parity for learning to rank may be applied to determine ranking modifications. A fairness control system receiving a ranking of items may apply Bayes factors to determine a likelihood of bias for the ranking. These Bayes factors may include a factor for determining bias in each item and a factor for determining bias in the ranking of the items. An indicator of bias may be generated using the applied Bayes factors and the fairness control system may modify the ranking if the determines likelihood of bias satisfies modification criteria for the ranking.