Abstract: Aspects of the technology described herein can improve the efficiency of a multi-result set ranking model by selecting a better exploration strategy. The technology described herein can improve the use of the result set opportunities by running offline simulations of different exploration policies to compare the different exploration policies. A better exploration policy for a given ranking model can then be implemented. In addition to allocating an efficient amount of result set opportunities to exploration, the selection of exploration results can help reduce performance drop during exploration. Thus, the technology described herein can provide valuable exploration data to improve ranking performance in the long run, and at the same time increase performance while exploration lasts.