Abstract: Techniques for using multiple machine learning (ML) models, with varying compute costs, for ASR processing is described. The system may include an arbitrator component configured to determine which ML model is to be used to process an audio frame from a sequence of audio frames representing a spoken natural language input. The arbitrator component may switch between the ML models, on a frame-by-frame basis, to reduce an overall compute cost for the entire spoken natural language input. The outputs of the different ML models may be combined to determine the final output for the entire spoken natural language input.

Abstract: Techniques for determining and storing relevant context information for a user input, such as a spoken input, are described. In some embodiments, context information is determined to be relevant on an audio frame basis. Context scores for different types of context data (e.g., prior dialog turn data, user profile data, device information, etc.) are determined for individual audio frames corresponding to a spoken input. Based on the corresponding context scores, the most relevant context is stored in a local context cache. The local context cache is updated as subsequent audio frames, of the user input, are processed. The data stored in the context cache is provided to downstream components to perform tasks such as ASR, NLU and SLU.

Abstract: Techniques for determining and storing relevant context information for a user input, such as a spoken input, are described. In some embodiments, context information is determined to be relevant on an audio frame basis. Context scores for different types of context data (e.g., prior dialog turn data, user profile data, device information, etc.) are determined for individual audio frames corresponding to a spoken input. Based on the corresponding context scores, the most relevant context is stored in a local context cache. The local context cache is updated as subsequent audio frames, of the user input, are processed. The data stored in the context cache is provided to downstream components to perform tasks such as ASR, NLU and SLU.

Abstract: Some devices may perform processing using machine learning models trained at a centralized system and distributed to the device. The centralized system may update the machine learning model and distribute the update to the device (or devices). To reduce the size of an update, the centralized system may train a model update object, which may be smaller in size than the model itself and thus more suitable for sending to the device(s). A device may receive the model update object and use it to update the on-device machine learning model; for example, by changing some parameters of the model. Parameters left unchanged during the update may retain their previous value. Thus, using the model update object to update the on-device model may result in a more accurate updated model when compared to sending an updated model compressed to a size similar to that of the model update object.

Abstract: Devices and techniques are generally described for compression of natural language processing models. A first index value to a first address of a weight table may be stored in a hash table. The first address may store a first weight associated with a first feature of a natural language processing model. A second index value to a second address of the weight table may be stored in the hash table. The second address may store a second weight associated with a second feature of the natural language processing model. A first code associated with the first feature and comprising a first number of bits may be generated. A second code may be generated associated with the second feature and comprising a second number of bits greater than the first number of bits based on a magnitude of the second weight being greater than a magnitude of the first weight.

Abstract: Devices and techniques are generally described for compression of natural language processing models. A first index value to a first address of a weight table may be stored in a hash table. The first address may store a first weight associated with a first feature of a natural language processing model. A second index value to a second address of the weight table may be stored in the hash table. The second address may store a second weight associated with a second feature of the natural language processing model. A first code associated with the first feature and comprising a first number of bits may be generated. A second code may be generated associated with the second feature and comprising a second number of bits greater than the first number of bits based on a magnitude of the second weight being greater than a magnitude of the first weight.