Abstract: A system configured to improve audio processing by performing dereverberation and noise reduction during a communication session. In some examples, the system may include a deep neural network (DNN) configured to perform speech enhancement, which is located after an Acoustic Echo Cancellation (AEC) component. For example, the DNN may process isolated audio data output by the AEC component to jointly mitigate additive noise and reverberation. In other examples, the system may include a DNN configured to perform acoustic interference cancellation, which may jointly mitigate additive noise, reverberation, and residual echo, removing the need to perform residual echo suppression processing. The DNN is configured to process complex-valued spectrograms corresponding to the isolated audio data and/or estimated echo data generated by the AEC component.

Abstract: A system configured to perform user orientation estimation to determine a direction a user is facing using a deep neural network (DNN). As a directionality of human speech increases with frequency, the DNN may estimate the user orientation by comparing high-frequency components detected by each of the multiple devices. For example, a group of devices may individually generate feature data, which represents audio features and spatial information, and send the feature data to the other devices. Thus, each device in the group receives feature data generated by the other devices and processes this feature data using a DNN to determine an estimate of user orientation. In some examples, the DNN may also generate sound source localization (SSL) data and/or a confidence score associated with the user orientation estimate. A post-processing step may process the individual user orientation estimates generated by the individual devices and determine a final user orientation estimate.

Abstract: Techniques for improving sound source localization (SSL) are provided. A method for probabilistic SSL using a deep neural network (DNN) may include receiving audio data including a representation of audio such as a wakeword from a microphone array. The audio data may be processed by a DNN to output a plurality of values where each value indicates a probability that the audio originated from a direction corresponding to that value. A sensor may provide computer vision or other data which may be used to inform the plurality of values based on detecting presence of a human or obstacle. A probability that the audio originated from one of the directions of the plurality of directions may be determined based at least in part on the DNN output and the computer vision or other data.