Abstract: An electronic system stores metadata for a plurality of media items, including, for each media item of the plurality of media items, at least one categorical identifier from a set of categorical identifiers. For a user of the media-providing service, the electronic system (i) determines a distribution of interests of the user with respect to the set of categorical identifiers; (ii) generates a network graph configured to represent a calibrated media item selection task, wherein the network graph represents respective relevance scores for each respective media item of the plurality of media items and the distribution of interests of the user with respect to the categorical identifiers; (iii) selects a set of media items from the plurality of media items to recommend to the user by solving for a maximum flow of the network graph; and (iv) provides the set of media items as recommendations to the user.

Abstract: Disclosed examples include an automated online experimentation mechanism that can perform model selection from a large pool of models with a relatively small number of online experiments. The probability distribution of the metric of interest that contains the model uncertainty is derived from a Bayesian surrogate model trained using historical logs. Disclosed techniques can be applied to identify a superior model by sequentially selecting and deploying a list of models from the candidate set that balance exploration-exploitation.