Abstract: Techniques for improving adaptive interference cancellation (AIC) using cascaded AIC algorithms are described. To improve an accuracy of detecting speech, a device may perform a first stage of AIC to generate isolated audio data and may generate speech mask data indicating time windows when speech is detected in the isolated audio data. Based on the speech mask data, the device may perform second AIC to generate output audio data, with adaptation of the adaptive filter enabled when the speech is not detected and disabled when the speech is detected. Thus, the first AIC improves the accuracy with which the device detects that speech is present and the second AIC reduces distortion in the output audio data by not updating filter coefficient values when the speech is present. The first AIC may use playback audio data, microphone audio data or beamformed audio data as reference signals.

Abstract: Techniques for improving microphone noise suppression are provided. As wind noise may disproportionately impact a subset of microphones, a method for processing audio data using two adaptive reference algorithm (ARA) paths in parallel is provided. For example, first ARA processing performs noise cancellation using all microphones, while second ARA processing performs noise cancellation using only a portion of the microphones. As the first ARA processing and the second ARA processing are performed in parallel, beam merging can be performed using beams from the first ARA, the second ARA, and/or a combination of each. In addition, beam merging can be performed using beam sections instead of individual beams to further improve performance and reduce attenuation to speech.

Abstract: Techniques for improving beamforming using filter coefficient values corresponding to virtual microphones are described. A system may define “virtual” microphone positions and determine corresponding filter coefficient values. These filter coefficient values may be applied to input audio data captured by actual physical microphones, enabling the system to improve performance of beamforming and/or to reduce a number of physical microphones without degrading performance. Offline testing and simulations may be performed to identify the best combination of virtual microphones and/or filter coefficient values for a particular look-direction. For example, the simulations may identify that a first filter coefficient corresponding to a first virtual microphone and a first direction will be associated with a first physical microphone and the first direction.

Abstract: Techniques for improving adaptive interference cancellation (AIC) using cascaded AIC algorithms are described. To improve an accuracy of detecting speech, a device may perform a first stage of AIC to generate isolated audio data and may generate speech mask data indicating time windows when speech is detected in the isolated audio data. Based on the speech mask data, the device may perform second AIC to generate output audio data, with adaptation of the adaptive filter enabled when the speech is not detected and disabled when the speech is detected. Thus, the first AIC improves the accuracy with which the device detects that speech is present and the second AIC reduces distortion in the output audio data by not updating filter coefficient values when the speech is present. The first AIC may use playback audio data, microphone audio data or beamformed audio data as reference signals.

Abstract: Techniques for improving adaptive interference cancellation (AIC) using cascaded AIC algorithms are described. To improve an accuracy of detecting speech, a device may perform a first stage of AIC to generate isolated audio data and may generate speech mask data indicating time windows when speech is detected in the isolated audio data. Based on the speech mask data, the device may perform second AIC to generate output audio data, with adaptation of the adaptive filter enabled when the speech is not detected and disabled when the speech is detected. Thus, the first AIC improves the accuracy with which the device detects that speech is present and the second AIC reduces distortion in the output audio data by not updating filter coefficient values when the speech is present. The first AIC may use playback audio data, microphone audio data or beamformed audio data as reference signals.

Abstract: A system configured to improve echo cancellation for nonlinear systems. The system generate reference audio data by isolating portions of microphone audio data that correspond to playback audio data. For example, the system may determine a correlation between the playback audio data and the microphone audio data in individual time-frequency bands in a frequency domain. In some examples, the system may substitute microphone audio data associated with output audio for the playback audio data. The system may generate the reference audio data based on portions of the microphone audio data that have a strong correlation with the playback audio data. The system may generate the reference audio data by selecting these portions of the microphone audio data or by performing beamforming. This results in precise time alignment between the reference audio data and the microphone audio data, improving performance of the echo cancellation.

Abstract: A system configured to improve noise cancellation by using portions of multiple reference signals instead of using a complete reference signal. The system divides a frequency spectrum into frequency bands and selects a single reference signal from a group of potential reference signals for every frequency band. For example, a first reference signal is selected for a first frequency band while a second reference signal is selected for a second frequency band. The system may generate a combined reference signal using portions of each of the selected reference signals, such as a portion of the first reference signal corresponding to the first frequency band and a portion of the second reference signal corresponding to the second frequency band. Additionally or alternatively, the system may perform noise cancellation using each of the selected reference signals and filter the outputs based on the corresponding frequency band to generate combined audio output data.

Abstract: Techniques for improving acoustic echo cancellation to attenuate an echo signal generated by a loudspeaker included in a device are described. A system may determine a loudspeaker canceling beam (LCB) (e.g., fixed beam directed to the loudspeaker) and may use the LCB to generate LCB audio data that corresponds to the echo signal. For example, based on a configuration of the loudspeaker relative to microphone(s) of the device, the system may perform simulation(s) to generate a plurality of filter coefficient values corresponding to the loudspeaker. By subtracting the LCB audio data during acoustic echo cancellation, the system may attenuate the echo signal even when there is distortion or nonlinearity or the like caused by the loudspeaker. In some examples, the system may perform acoustic echo cancellation using the LCB audio data and playback audio data.

Abstract: A system configured to improve noise cancellation by reducing attenuation of local speech in proximity to a device. When the local speech is present in both a target signal and a reference signal, performing noise cancellation to remove the reference signal inadvertently attenuates the local speech. To prevent this, the system may perform first noise cancellation to identify frequency bands associated with the local speech and may generate a modified reference signal based on the frequency bands. For example, the system may generate the modified reference signal by applying attenuation to first frequencies associated with the local speech and/or gain to second frequencies that are not associated with the local speech. The system may generate final output audio data by performing noise cancellation using the modified reference signal.

Abstract: A system configured to improve beamforming by using deep neural networks (DNNs). The system can use one trained DNN to focus on a first person speaking an utterance (e.g., target user) and one or more trained DNNs to focus on noise source(s) (e.g., wireless loudspeaker(s), a second person speaking, other localized sources of noise, or the like). The DNNs may generate time-frequency mask data that indicates individual frequency bands that correspond to the particular source detected by the DNN. Using this mask data, a beamformer can generate beamformed audio data that is specific to a source of noise. The system may perform noise cancellation to isolate first beamformed audio data associated with the target user by removing second beamformed audio data associated with noise source(s).

Abstract: Systems and methods for disabling adaptive echo cancellation functionality for a temporal window are provided herein. In some embodiments, audio data may be received by a voice activated electronic device, where the audio data may include an utterance of a wakeword that may be subsequently followed by additional speech. A start time of when the wakeword began to be uttered may be determined by the voice activated electronic device, and the voice activated electronic device may also send the audio data to a backend system. Adaptive echo cancellation functionality may be disabled at the start time. The backend system may determine an end time of the speech, and may provide an indication to the voice activated electronic device of the end time, which in turn may cause the voice activated electronic device to enable the adaptive echo cancellation functionality at the end time.

Abstract: An acoustic echo cancellation (AEC) system that detects and compensates for differences in delay times between the AEC system and a set of wireless speakers. The filter coefficients used for AEC are adjusted based on the determined delay time to correct for frequency domain signal rotation.

Abstract: An echo cancellation system performs audio beamforming to separate audio input into multiple directions (e.g., target signals) and generates multiple audio outputs using two acoustic echo cancellation (AEC) circuits. A first AEC removes a playback reference signal (generated from a signal sent a loudspeaker) to isolate speech included in the target signals. A second AEC removes an adaptive reference signal (generated from microphone inputs corresponding to audio received from the loudspeaker) to isolate speech included in the target signals. A beam selector receives the multiple audio outputs and selects the first AEC or the second AEC based on a linearity of the system. When linear (e.g., no distortion or variable delay between microphone input and playback signal), the beam selector selects an output from the first AEC based on signal to noise (SNR) ratios. When nonlinear, the beam selector selects an output from the second AEC.

Abstract: An acoustic interference cancellation system that performs beamforming using a subset of microphones from a microphone array. For example, a first group of microphones from an array can be used to generate target signals that focus on the direction of the desired speech in the audio and a second group of microphones from the array can be used to generate reference signals that include the environmental noise, audio from a loudspeaker, etc. The reference signals of the second group of microphones can then be used to isolate the actual speech from the target signals of the first group of microphones. The microphone array can be three dimensional, allowing a device to simplify beamforming calculations by selecting subsets of microphones along different planes. In addition, directional microphones and remote microphones may be used to improve a quality of the reference signals.

Abstract: A system may be configured to interact with a user through speech using a first and second audio devices, where the first device produces audio and the second device captures audio. The second device may be configured to perform acoustic echo cancellation with respect to a microphone signal based on a reference signal provided by the first device. The reference and microphone signals may have the same nominal signal rates. However, the signal rates may drift from each other over time. In order to synchronize the rates of the signals, each of the devices maintains a signal index. The second device compares the values of the two signal indexes over time to determine rate differences between the reference and microphone signals and then corrects for the rate differences.

Abstract: An echo cancellation system that detects and compensations for differences in sample rates between the echo cancellation system and a set of wireless speakers based on a frequency-domain analysis of estimated impulse response coefficients. The system tracks the real and imaginary number components of the coefficients, and determines a “rotation” of the coefficients over time caused by a frequency offset between the audio sent to the speakers and the audio received from a microphone. Based on the rotation, samples of the audio are added or dropped when echo cancellation is performed, compensating for the frequency offset.

Abstract: A multi-channel echo cancellation system that dynamically adapts to changes in acoustic conditions. The system does not require a sequence of “start-up” tones to determine the impulse responses. Rather, the adaptive filters approximate estimated transfer functions for each channel. A secondary adaptive filter adjusts cancellation to adapt to changes in the actual transfer functions over time after the adaptive filters have been trained, even if the reference signals are not unique relative to each other.

Abstract: An echo cancellation system that generates multiple output paths, enabling Automatic Speech Recognition (ASR) processing in parallel with voice communication. For single direction AEC (e.g., ASR processing), the system prioritizes speech from a single user and ignores other speech by selecting a single directional output from a plurality of directional outputs as a first output path. For multi-directional AEC (e.g., voice communication), the system includes all speech by combining the plurality of directional outputs as a second output path. The system may use a weighted sum technique, such that each directional output is represented in the combined output based on a corresponding signal metric, or an equal weighting technique, such that a first group of directional outputs having a higher signal metric may be equally weighted using a first weight while a second group of directional outputs having a lower signal metric may be equally weighted using a second weight.

Abstract: An acoustic echo cancellation (AEC) system that detects and compensates for differences in sample rates between the AEC system and a set of wireless speakers based on a search-based trial-and-error technique. The system individually determines a frequency offset for each microphone-speaker pair using an iterative process, determining an echo-return loss enhancement (ERLE) value for each offset that is tried, and selecting the frequency offset associated with the largest ERLE value.

Abstract: A multi-channel audio communication system is configured to receive highly correlated input audio signals, generated as an example by multiple microphones at a far-end site. Each input audio signal is cyclically stepped through a range of discrete delay amounts, between upper and lower limits, using a step size that is a fraction of the sample period of the input audio signals. Delay cycles applied to the different input audio signals are configured to have different phases, thereby reducing the inter-signal correlation of the input audio signals. The delayed input audio signals are then played by loudspeakers. Microphone output, which may contain sound generated by the loudspeakers, is then subjected to multi-channel acoustic echo cancellation.