Abstract: A method includes: accessing a user query; and accessing a quality prediction model that predicts quality scores of responses from a set of language models according to queries. The method also includes, for each language model in the set of language models, calculating a predicted quality score, in a set of predicted quality scores, of a projected response by the language model based on the user query and the quality prediction model; and calculating a resource estimate, in a set of resource estimates, for the projected response by the language model. The method further includes: selecting a target language model, in the set of language models, for the user query based on the set of predicted quality scores and the set of resource estimates; and invoking the target language model to generate a response for the user query.