Abstract: An online concierge system selects picker expertise tags that showcase abilities or experiences of pickers that fulfill orders for the system. The online concierge system establishes a set of user-order cohorts based on characteristics of orders and users placing the orders. When an order is received, the online concierge system identifies a relevant user-order cohort and applies a trained model to predict, in the context of the user-order cohort, the performance of various candidate picker expertise tags applicable to the order. The trained model may be generated via a training and testing process in which different picker expertise tags are tested in the context of a user-order cohort, and performance metrics are observed to learn which picker expertise tags perform best in the context of a user-order cohort.