Abstract: A system configured to improve sound source localization (SSL) processing by reducing a number of direction vectors and grouping the direction vectors into direction cells is provided. The system performs clustering to generate a smaller set of direction vectors included in a delay-direction codebook, reducing a size of the codebook to the number of unique delay vectors. In addition, the system groups the direction vectors into direction cells having a regular structure (e.g., predetermined uniformity and/or symmetry), which simplifies SSL processing and results in a substantial reduction in computational cost. The system may also select between multiple codebooks and/or dynamically adjust the codebook to compensate for changes to the microphone array. For example, a device with a microphone array fixed to a display that can tilt may adjust the codebook based on a tilt angle of the display to improve accuracy.

Abstract: Techniques for encoding audio data with metadata are described. In an example, a device receives audio data corresponding to audio detected by a microphone and receives metadata associated with the audio. The device generates encoded data based at least in part on encoding the audio data with the metadata. The encoding involves replacing a portion of the audio data with the metadata, such that the encoded data includes the metadata and a remaining portion of the audio data. The device sends the encoded data to an audio processing application.

Abstract: Techniques for improving adaptive interference cancellation (AIC) using cascaded AIC algorithms are described. To improve an accuracy of detecting speech, a device may perform a first stage of AIC to generate isolated audio data and may generate speech mask data indicating time windows when speech is detected in the isolated audio data. Based on the speech mask data, the device may perform second AIC to generate output audio data, with adaptation of the adaptive filter enabled when the speech is not detected and disabled when the speech is detected. Thus, the first AIC improves the accuracy with which the device detects that speech is present and the second AIC reduces distortion in the output audio data by not updating filter coefficient values when the speech is present. The first AIC may use playback audio data, microphone audio data or beamformed audio data as reference signals.

Abstract: This disclosure describes techniques for detecting voice commands from a user of an ear-based device. The ear-based device may include an in-ear facing microphone to capture sound emitted in an ear of the user, and an exterior facing microphone to capture sound emitted in an exterior environment of the user. The in-ear microphone may generate an inner audio signal representing the sound emitted in the ear, and the exterior microphone may generate an outer audio signal representing sound from the exterior environment. The ear-based device may compute a ratio of a power of the inner audio signal to the outer audio signal and may compare this ratio to a threshold. If the ratio is larger than the threshold, the ear-based device may detect the voice of the user. Further, the ear-based device may set a value of the threshold based on a level of acoustic seal of the ear-based device.

Abstract: A system configured to perform microphone occlusion event detection. When a device detects a microphone occlusion event, the device will modify audio processing performed prior to speech processing, such as by disabling spatial processing and only processing audio data from a single microphone. The device detects the microphone occlusion event by determining inter-level difference (ILD) values between two microphone signals and using the ILD values as input features to a classifier. For example, when a far-end reference signal is inactive, the classifier may process a first ILD value within a high frequency band. However, when the far-end reference signal is active, the classifier may process the first ILD value and a second ILD value within a low frequency band.

Abstract: A system configured to improve audio processing by performing dereverberation and noise reduction during a communication session. The system may apply a two-channel dereverberation algorithm by calculating coherence-to-diffuse ratio (CDR) values and calculating dereverberation (DER) gain values based on the CDR values. While the DER gain values may be calculated at a first stage within the pipeline, the device may apply the DER gain values at a second stage within the pipeline. For example, the device may calculate the DER gain values prior to performing residual echo suppression (RES) processing but may apply the DER gain values after performing RES processing, in order to avoid excessive attenuation of the local speech. In addition to removing reverberation, the DER gain values also remove diffuse noise components, reducing an amount of noise reduction required. Thus, the device may soften noise reduction when the DER gain values are applied.

Abstract: Techniques for encoding audio data with metadata are described. In an example, a device receives audio data corresponding to audio detected by a microphone and receives metadata associated with the audio. The device generates encoded data based at least in part on encoding the audio data with the metadata. The encoding involves replacing a portion of the audio data with the metadata, such that the encoded data includes the metadata and a remaining portion of the audio data. The device sends the encoded data to an audio processing application.

Abstract: Techniques for improving adaptive interference cancellation (AIC) using cascaded AIC algorithms are described. To improve an accuracy of detecting speech, a device may perform a first stage of AIC to generate isolated audio data and may generate speech mask data indicating time windows when speech is detected in the isolated audio data. Based on the speech mask data, the device may perform second AIC to generate output audio data, with adaptation of the adaptive filter enabled when the speech is not detected and disabled when the speech is detected. Thus, the first AIC improves the accuracy with which the device detects that speech is present and the second AIC reduces distortion in the output audio data by not updating filter coefficient values when the speech is present. The first AIC may use playback audio data, microphone audio data or beamformed audio data as reference signals.

Abstract: Techniques for detecting a voice command from a user of a hearable device. The hearable device may include an in-ear facing microphone to capture sound emitted from an ear of the user, and an exterior facing microphone to capture sound emitted from an exterior environment of the user. The in-ear microphone may generate an in-ear audio signal representing the sound emitted from the ear, and the exterior microphone may generate an exterior audio signal representing sound from the exterior environment. The hearable device may include components to determine correlations or similarities between the in-ear audio signal and exterior audio signal, which indicate that the audio signals represent sound emitted from the user. Further, the components may perform voice activity detection to determine that the sound emitted from the user is a voice command, and proceed to perform further voice-processing techniques.

Abstract: Techniques for improving adaptive interference cancellation (AIC) using cascaded AIC algorithms are described. To improve an accuracy of detecting speech, a device may perform a first stage of AIC to generate isolated audio data and may generate speech mask data indicating time windows when speech is detected in the isolated audio data. Based on the speech mask data, the device may perform second AIC to generate output audio data, with adaptation of the adaptive filter enabled when the speech is not detected and disabled when the speech is detected. Thus, the first AIC improves the accuracy with which the device detects that speech is present and the second AIC reduces distortion in the output audio data by not updating filter coefficient values when the speech is present. The first AIC may use playback audio data, microphone audio data or beamformed audio data as reference signals.

Abstract: A system configured to improve audio processing by performing dereverberation and noise reduction during a communication session. The system may apply a two-channel dereverberation algorithm by calculating coherence-to-diffuse ratio (CDR) values and calculating dereverberation (DER) gain values based on the CDR values. While the device calculates the DER gain values prior to performing acoustic echo cancellation (AEC) processing, the device applies the DER gain values after performing residual echo suppression (RES) processing in order to avoid excessive attenuation of the local speech. To improve output speech quality, the device does not apply the DER gain values for nonreverberant signals, when a signal-to-noise ratio (SNR) value is too low, and/or when far-end talk (e.g., remote speech) is present. Dereverberation processing is further improved by using frequency dependent parameters to calculate the DER gain values and by adjusting other gain values when the DER gain values are applied.

Abstract: A device capable of autonomous motion may move in response to a user speaking an utterance, such as a command. Before moving, the device processes audio data received from a microphone array to identify different audio signals arriving at the device from different directions. Based on properties of the audio signals, the device determines which of the audio signals are merely reflections of other audio.

Abstract: A multi-channel acoustic echo cancellation (AEC) system that includes a residual echo suppressor (RES) that dynamically controls an amount of attenuation to reduce distortion of local speech during double-talk conditions. The RES determines when double-talk conditions are present based on an echo return loss enhancement (ERLE) value. When the ERLE value is above a first threshold value but below a second threshold value, the RES reduces an amount of attenuation applied while generating an RES mask to pass local speech without distortion. When the ERLE value is below the first threshold value or above the second threshold value, the RES applies full attenuation while generating the RES mask in order to suppress a residual echo signal. To further improve RES processing, the RES may apply smoothing across time, smoothing across frequencies, or apply extra echo suppression processing to further attenuate the residual echo signal.

Abstract: This disclosure describes, in part, techniques to process audio signals to lessen the impact that wind and/or other environmental noise has upon the resulting quality of these audio signals. For example, the techniques may determine a level of wind and/or other noise in an environment and may determine how best to process the signals to lessen the impact of the noise, such that one or more users that hear audio based on output of the signals hear higher-quality audio.

Abstract: A system that performs automatic gain control (AGC) using different decay rates. The system may select a slow decay rate to track a loudness level within speech (e.g., within an utterance), improving audio quality and maintaining dynamic range for an individual voice, while selecting a fast decay rate to track the loudness level after a gap of silence (e.g., no voice activity detected for a duration of time) or during large level changes (e.g., actual speech loudness is lower than estimated speech loudness for a duration of time). This improves an accuracy of the loudness estimate and therefore a responsiveness of the automatic gain control, resulting in an improved user experience.

Abstract: This disclosure describes techniques for detecting voice commands from a user of an ear-based device. The ear-based device may include an in-ear facing microphone to capture sound emitted in an ear of the user, and an exterior facing microphone to capture sound emitted in an exterior environment of the user. The in-ear microphone may generate an inner audio signal representing the sound emitted in the ear, and the exterior microphone may generate an outer audio signal representing sound from the exterior environment. The ear-based device may compute a ratio of a power of the inner audio signal to the outer audio signal and may compare this ratio to a threshold. If the ratio is larger than the threshold, the ear-based device may detect the voice of the user. Further, the ear-based device may set a value of the threshold based on a level of acoustic seal of the ear-based device.

Abstract: This disclosure describes, in part, techniques for processing audio data. For instance, an electronic device may include an automatic gain controller (AGC) that determines AGC gains for amplifying or attenuating an audio data. To determine the AGC gains, the AGC uses information from a residual echo suppressor (RES) and/or a noise reductor (NR). The information may indicate RES gains applied to the audio data by the RES and/or NR gains applied to the audio data by the NR. In some instances, to determine the AGC gain, the AGC determines time-constant parameter(s) using the information. The AGC then uses the time-constant parameter(s) to determine an input signal level for the audio data and/or the AGC gain. In some instances, to determine the AGC gain, the AGC operates in an attack mode or a release mode based on the information.

Abstract: A sensor processing unit comprises a sensor processor. The sensor processor is configured to communicatively couple with a microphone. The sensor processor is configured to acquire, from the microphone, a sample captured by the microphone from an environment in which the microphone is disposed. The sensor processor is configured to perform music activity detection on the audio sample to detect for music within the audio sample. Responsive to detection of music within the audio sample, the sensor processor is configured to send a music detection signal to an external processor located external to the sensor processing unit, the music detection signal indicating that music has been detected in the environment.

Abstract: In a method of adaptive buffering in a mobile device having a host processor and a sensor processor coupled with the host processor, the sensor processor is used to buffer data received from a sensor that is operated by the sensor processor. The data is buffered by the sensor processor into a circular data buffer. Responsive to the sensor processor detecting triggering data within the received data: a first adaptive data buffering action is initiated with respect to the data received from the sensor operated by the sensor processor; a second adaptive data buffering action is initiated with respect to second data received from a second sensor of the mobile device; and a command is sent from the sensor processor to a second processor.

Abstract: A system configured to perform power normalization for voice enhancement. The system may identify active intervals corresponding to voice activity and may selectively amplify the active intervals in order to generate output audio data at a near uniform loudness. The system may determine a variable gain for each of the active intervals based on a desired output loudness and a flatness value, which indicates how much a signal envelope is to be modified. For example, a low flatness value corresponds to no modification, with peak active interval values corresponding to the desired output loudness and lower active intervals being lower than the desired output loudness. In contrast, a high flatness value corresponds to extensive modification, with peak active interval values and lower active interval values both corresponding to the desired output loudness. Thus, individual words may share the same peak power level.