Abstract: Devices and techniques are generally described for application shortlisting for language models. An LLM may determine a first goal of first query data. The LLM may generate first code data representing the first query data. First encoded data including a first encoded representation of the first goal and the first query data may be generated. A first application may be determined based at least in part on analysis of the first encoded data with respect to second encoded data representing the first application. A second application may be determined based at least in part on analysis of the first encoded data with respect to third encoded data representing the second application. First prompt data including the first query data, first data identifying the first application, and second data identifying the second application may be generated. The LLM may generate output data based at least in part on the first prompt data.

Abstract: Techniques for identifying components (e.g., application program interfaces (APIs)) relevant for a large language model (LLM) prompt are described. The shortlister includes one or more proposers that each select n components for performing an instant task. The n components from each proposer may be merged and reranked to produce a ranked list of K components for inclusion in the LLM prompt. The shortlister proposers can include, among others, a request-based proposer, query-based proposer, and/or rule-based proposer. In particular, the query-based proposer may utilize a LLM to generate a query to search component descriptions to identify n components the proposer determines are usable to perform the instant task.

Abstract: Techniques for improving query generation (or generation of an input to another component) by a language model are described. In some embodiments, the generated query is used to retrieve API calls relevant for responding to a user input. The language model can be finetuned using language model generated queries. Results retrieved using the generated query are evaluated against ground truth data to determine performance metrics data, which may be based on a ranking of the ground truth API call in the retrieved results. Based on the performance metrics data, training data including the generated query can be used to update/finetune the language model may be determined. Updating of the language model may be initiated based on feedback corresponding to the generated query.

Abstract: Devices and techniques are generally described for incremental, asynchronous machine learning inference. In various examples, first feature data generated from a first process during a first time period may be input into a first subgraph of a neural network. Second feature data generated from the first process during a second time period may be input into a second subgraph of the neural network. First intermediate data may be generated by the first subgraph based on the first feature data. Second intermediate data may be generated by the second subgraph based on the second feature data. First output data may be generated by the neural network based on the first intermediate data and the second intermediate data.