Abstract: Techniques for performing automatic speech recognition (ASR) are described. In some embodiments, an ASR component integrates contextual information from user profile data into audio encoding data to predict a token(s) corresponding to a spoken input. The user profile data may include personalized words, such as, contact names, device names, etc. The ASR component determines word embedding data using the personalized words. The ASR component is configured to apply attention to audio frames that are relevant to the personalized words based on processing the audio encoding data and the word embedding data.

Abstract: Techniques for performing automatic speech recognition (ASR) are described. In some embodiments, an ASR component integrates contextual information from user profile data into audio encoding data to predict a token(s) corresponding to a spoken input. The user profile data may include personalized words, such as, contact names, device names, etc. The ASR component determines word embedding data using the personalized words. The ASR component is configured to apply attention to audio frames that are relevant to the personalized words based on processing the audio encoding data and the word embedding data.

Abstract: A multi-channel transformer acoustic model that processes a plurality of audio signals output by microphones of a microphone array and outputs probabilities for acoustic units of an utterance represented in the audio signals. The audio signals represent the individual microphones' respective capturing of the utterance. The multi-channel model may perform self-attention on embeddings of the audio signals and then cross-channel attention across the attended audio signals. The cross-channel attention may involve processing of signals relative to each other to model the relationships across channels within and across time frames. The multi-channel model may include a transducer to perform processing frame-by-frame.