Abstract: An acoustic echo cancellation (AEC) system that dynamically controls an adaptation speed of an adaptive filter, enabling the adaptive filter to converge quickly while protecting near-end speech. For example, a device may control the adaptation speed of the adaptive filter to adapt quickly when near-end speech is not present and adapt slowly when near-end speech is present. The device controls the adaptation speed by dynamically determining step-size values and/or performing error normalization to limit the rate of adaptation. In some examples, the device determines the variable step-size parameter based on a relative strength of a microphone signal and a reference signal over time. For example, the device can compare current energy levels of the microphone signal and the reference signal to a range of energy levels to determine a microphone step-size value, a reference step-size value, and an AEC step-size value.

Abstract: A system that performs beam selection and beam merging using beam-specific signal quality metrics corresponding to a minimum noise floor. For example, a device may track a minimum noise floor for each beam, determine a highest minimum noise floor across the beams, and determine a noise floor ratio between the beam-specific minimum noise floor and the highest minimum noise floor. Using a combination of the noise floor ratio and signal-to-noise ratio (SNR) values, the device may perform beam selection by prioritizing low background noise as well as high SNR to select a pre-defined beam group. In addition, the device may use the noise floor ratio to perform beam merging and generate single-channel output audio data using the selected beam group. For example, the device may scale the beams based on a combination of the SNR value and the noise floor ratio.

Abstract: Techniques for improving microphone noise suppression are provided. When a playback signal is available, a device may use the playback signal as a reference while performing Acoustic Echo Cancellation (AEC) processing. When the playback signal is not available, the device may instead use microphone audio signal(s) as the reference while performing Adaptive Interference Cancellation (AIC) processing. For example, the device may use first audio data generated by first microphones as the reference while performing AIC processing on the first audio data and second audio data generated by second microphones. As the adaptive filter used to perform AIC processing is updated at a slower rate, the reference signal only cancels stationary portions of the first audio data (e.g., noise), leaving a representation of transient sounds such as speech. In some examples, the device may perform AIC processing and AEC processing in series.

Abstract: A system that performs beam selection and beam merging using beam-specific signal quality metrics corresponding to a minimum noise floor. For example, a device may track a minimum noise floor for each beam, determine a highest minimum noise floor across the beams, and determine a noise floor ratio between the beam-specific minimum noise floor and the highest minimum noise floor. Using a combination of the noise floor ratio and signal-to-noise ratio (SNR) values, the device may perform beam selection by prioritizing low background noise as well as high SNR to select a pre-defined beam group. In addition, the device may use the noise floor ratio to perform beam merging and generate single-channel output audio data using the selected beam group. For example, the device may scale the beams based on a combination of the SNR value and the noise floor ratio.

Abstract: Techniques for improving adaptive interference cancellation (AIC) using cascaded AIC algorithms are described. To improve an accuracy of detecting speech, a device may perform a first stage of AIC to generate isolated audio data and may generate speech mask data indicating time windows when speech is detected in the isolated audio data. Based on the speech mask data, the device may perform second AIC to generate output audio data, with adaptation of the adaptive filter enabled when the speech is not detected and disabled when the speech is detected. Thus, the first AIC improves the accuracy with which the device detects that speech is present and the second AIC reduces distortion in the output audio data by not updating filter coefficient values when the speech is present. The first AIC may use playback audio data, microphone audio data or beamformed audio data as reference signals.

Abstract: Techniques for improving microphone noise suppression are provided. As wind noise may disproportionately impact a subset of microphones, a method for processing audio data using two adaptive reference algorithm (ARA) paths in parallel is provided. For example, first ARA processing performs noise cancellation using all microphones, while second ARA processing performs noise cancellation using only a portion of the microphones. As the first ARA processing and the second ARA processing are performed in parallel, beam merging can be performed using beams from the first ARA, the second ARA, and/or a combination of each. In addition, beam merging can be performed using beam sections instead of individual beams to further improve performance and reduce attenuation to speech.

Abstract: Techniques for improving beamforming using filter coefficient values corresponding to virtual microphones are described. A system may define “virtual” microphone positions and determine corresponding filter coefficient values. These filter coefficient values may be applied to input audio data captured by actual physical microphones, enabling the system to improve performance of beamforming and/or to reduce a number of physical microphones without degrading performance. Offline testing and simulations may be performed to identify the best combination of virtual microphones and/or filter coefficient values for a particular look-direction. For example, the simulations may identify that a first filter coefficient corresponding to a first virtual microphone and a first direction will be associated with a first physical microphone and the first direction.

Abstract: Techniques for improving adaptive interference cancellation (AIC) using cascaded AIC algorithms are described. To improve an accuracy of detecting speech, a device may perform a first stage of AIC to generate isolated audio data and may generate speech mask data indicating time windows when speech is detected in the isolated audio data. Based on the speech mask data, the device may perform second AIC to generate output audio data, with adaptation of the adaptive filter enabled when the speech is not detected and disabled when the speech is detected. Thus, the first AIC improves the accuracy with which the device detects that speech is present and the second AIC reduces distortion in the output audio data by not updating filter coefficient values when the speech is present. The first AIC may use playback audio data, microphone audio data or beamformed audio data as reference signals.

Abstract: Techniques for improving adaptive interference cancellation (AIC) using cascaded AIC algorithms are described. To improve an accuracy of detecting speech, a device may perform a first stage of AIC to generate isolated audio data and may generate speech mask data indicating time windows when speech is detected in the isolated audio data. Based on the speech mask data, the device may perform second AIC to generate output audio data, with adaptation of the adaptive filter enabled when the speech is not detected and disabled when the speech is detected. Thus, the first AIC improves the accuracy with which the device detects that speech is present and the second AIC reduces distortion in the output audio data by not updating filter coefficient values when the speech is present. The first AIC may use playback audio data, microphone audio data or beamformed audio data as reference signals.

Abstract: A system configured to improve echo cancellation for nonlinear systems. The system generate reference audio data by isolating portions of microphone audio data that correspond to playback audio data. For example, the system may determine a correlation between the playback audio data and the microphone audio data in individual time-frequency bands in a frequency domain. In some examples, the system may substitute microphone audio data associated with output audio for the playback audio data. The system may generate the reference audio data based on portions of the microphone audio data that have a strong correlation with the playback audio data. The system may generate the reference audio data by selecting these portions of the microphone audio data or by performing beamforming. This results in precise time alignment between the reference audio data and the microphone audio data, improving performance of the echo cancellation.

Abstract: A system configured to improve noise cancellation by using portions of multiple reference signals instead of using a complete reference signal. The system divides a frequency spectrum into frequency bands and selects a single reference signal from a group of potential reference signals for every frequency band. For example, a first reference signal is selected for a first frequency band while a second reference signal is selected for a second frequency band. The system may generate a combined reference signal using portions of each of the selected reference signals, such as a portion of the first reference signal corresponding to the first frequency band and a portion of the second reference signal corresponding to the second frequency band. Additionally or alternatively, the system may perform noise cancellation using each of the selected reference signals and filter the outputs based on the corresponding frequency band to generate combined audio output data.

Abstract: Techniques for improving acoustic echo cancellation to attenuate an echo signal generated by a loudspeaker included in a device are described. A system may determine a loudspeaker canceling beam (LCB) (e.g., fixed beam directed to the loudspeaker) and may use the LCB to generate LCB audio data that corresponds to the echo signal. For example, based on a configuration of the loudspeaker relative to microphone(s) of the device, the system may perform simulation(s) to generate a plurality of filter coefficient values corresponding to the loudspeaker. By subtracting the LCB audio data during acoustic echo cancellation, the system may attenuate the echo signal even when there is distortion or nonlinearity or the like caused by the loudspeaker. In some examples, the system may perform acoustic echo cancellation using the LCB audio data and playback audio data.

Abstract: A system configured to improve noise cancellation by reducing attenuation of local speech in proximity to a device. When the local speech is present in both a target signal and a reference signal, performing noise cancellation to remove the reference signal inadvertently attenuates the local speech. To prevent this, the system may perform first noise cancellation to identify frequency bands associated with the local speech and may generate a modified reference signal based on the frequency bands. For example, the system may generate the modified reference signal by applying attenuation to first frequencies associated with the local speech and/or gain to second frequencies that are not associated with the local speech. The system may generate final output audio data by performing noise cancellation using the modified reference signal.

Abstract: A system configured to improve beamforming by using deep neural networks (DNNs). The system can use one trained DNN to focus on a first person speaking an utterance (e.g., target user) and one or more trained DNNs to focus on noise source(s) (e.g., wireless loudspeaker(s), a second person speaking, other localized sources of noise, or the like). The DNNs may generate time-frequency mask data that indicates individual frequency bands that correspond to the particular source detected by the DNN. Using this mask data, a beamformer can generate beamformed audio data that is specific to a source of noise. The system may perform noise cancellation to isolate first beamformed audio data associated with the target user by removing second beamformed audio data associated with noise source(s).

Abstract: Systems and methods for disabling adaptive echo cancellation functionality for a temporal window are provided herein. In some embodiments, audio data may be received by a voice activated electronic device, where the audio data may include an utterance of a wakeword that may be subsequently followed by additional speech. A start time of when the wakeword began to be uttered may be determined by the voice activated electronic device, and the voice activated electronic device may also send the audio data to a backend system. Adaptive echo cancellation functionality may be disabled at the start time. The backend system may determine an end time of the speech, and may provide an indication to the voice activated electronic device of the end time, which in turn may cause the voice activated electronic device to enable the adaptive echo cancellation functionality at the end time.

Abstract: An acoustic echo cancellation (AEC) system that detects and compensates for differences in delay times between the AEC system and a set of wireless speakers. The filter coefficients used for AEC are adjusted based on the determined delay time to correct for frequency domain signal rotation.

Abstract: A system may be configured to interact with a user through speech using a first and second audio devices, where the first device produces audio and the second device captures audio. The second device may be configured to perform acoustic echo cancellation with respect to a microphone signal based on a reference signal provided by the first device. The reference and microphone signals may have the same nominal signal rates. However, the signal rates may drift from each other over time. In order to synchronize the rates of the signals, each of the devices maintains a signal index. The second device compares the values of the two signal indexes over time to determine rate differences between the reference and microphone signals and then corrects for the rate differences.

Abstract: An acoustic interference cancellation system that performs beamforming using a subset of microphones from a microphone array. For example, a first group of microphones from an array can be used to generate target signals that focus on the direction of the desired speech in the audio and a second group of microphones from the array can be used to generate reference signals that include the environmental noise, audio from a loudspeaker, etc. The reference signals of the second group of microphones can then be used to isolate the actual speech from the target signals of the first group of microphones. The microphone array can be three dimensional, allowing a device to simplify beamforming calculations by selecting subsets of microphones along different planes. In addition, directional microphones and remote microphones may be used to improve a quality of the reference signals.

Abstract: An echo cancellation system performs audio beamforming to separate audio input into multiple directions (e.g., target signals) and generates multiple audio outputs using two acoustic echo cancellation (AEC) circuits. A first AEC removes a playback reference signal (generated from a signal sent a loudspeaker) to isolate speech included in the target signals. A second AEC removes an adaptive reference signal (generated from microphone inputs corresponding to audio received from the loudspeaker) to isolate speech included in the target signals. A beam selector receives the multiple audio outputs and selects the first AEC or the second AEC based on a linearity of the system. When linear (e.g., no distortion or variable delay between microphone input and playback signal), the beam selector selects an output from the first AEC based on signal to noise (SNR) ratios. When nonlinear, the beam selector selects an output from the second AEC.

Abstract: An echo cancellation system that detects and compensations for differences in sample rates between the echo cancellation system and a set of wireless speakers based on a frequency-domain analysis of estimated impulse response coefficients. The system tracks the real and imaginary number components of the coefficients, and determines a “rotation” of the coefficients over time caused by a frequency offset between the audio sent to the speakers and the audio received from a microphone. Based on the rotation, samples of the audio are added or dropped when echo cancellation is performed, compensating for the frequency offset.