Abstract: Techniques for cache management for LLM processing are described. Example embodiments include a signal hashing model that generates a key for particular context data. An LLM output corresponding to the context data is stored in a cache along with the key. For a user input received by the system, a cache lookup is performed using a key for context data corresponding to the received user input. For a cache hit, the stored output is used to respond to the user input. For a cache miss, a LLM processes the context data and the user input to generate an output within a first timeout. If the LLM is unable to generate an output within the first timeout, then in some cases, the LLM is allowed to continue processing until a second timeout, and a final or partial output from the LLM is stored in the cache.

Abstract: Techniques for determining alternative input representations using entity expansion and entity weighting are described. An entity expansion knowledge base is built by extracting entities from user input-system response pairs that resulted in satisfactory experiences. An extracted entity is associated with an initial score based on it being included in the user input only, in the system response only or both the user input and the system response. Entities co-occurring in the user input-system response pair are connected in the knowledge base. An overall score is associated with the connections based on the initial scores of the connected entities. Using the knowledge base, expansion entities related to an entity included in a user input are determined, and the expansion entities and user input entity are weighted. The weighting of the entities involves assigning a level to each entity based on pairs of user input-alternative input representation.

Abstract: Techniques for generating tasks to be completed in order to perform an action responsive to a user input and, for a given task, shortlisting available components to those that are relevant for the task are described. The system processes a user input to determine tasks to be completed in order to perform an action responsive to the user input. The system determines a priority of the tasks and selects a top-ranked task. The system determines descriptions of processing performable by components that are semantically similar to the current task, and requests a description of the function the corresponding components would perform for the current task. Based on the received descriptions, the system selects one or more components to perform the task. Thereafter, the system causes the action to be performed and outputs a response to the user input.

Abstract: Techniques for generating tasks to be completed in order to perform an action responsive to a user input and, for a given task, shortlisting available components to those that are relevant for the task are described. The system processes a user input to determine tasks to be completed in order to perform an action responsive to the user input. The system determines a priority of the tasks and selects a top-ranked task. The system determines descriptions of processing performable by components that are semantically similar to the current task, and requests a description of the function the corresponding components would perform for the current task. Based on the received descriptions, the system selects one or more components to perform the task. Thereafter, the system causes the action to be performed and outputs a response to the user input.

Abstract: Techniques for presenting an alternative input representation to a user for testing and collecting processing data are described. A system may determine that a received spoken input triggers an alternative input representation for presenting. The system may output data corresponding to the alternative input representation in response to the received spoken input, and the system may receive user feedback from the user. The system may store the user feedback and processing data corresponding to processing of the alternative input representation, which may be later used to update an alternative input component configured to determine alternative input representations for spoken inputs.