Abstract: A distributed voice controlled system has a primary assistant and at least one secondary assistant. The primary assistant has a housing to hold one or more microphones, one or more speakers, and various computing components. The secondary assistant is similar in structure, but is void of speakers. The voice controlled assistants perform transactions and other functions primarily based on verbal interactions with a user. The assistants within the system are coordinated and synchronized to perform acoustic echo cancellation, selection of a best audio input from among the assistants, and distributed processing.

Abstract: Techniques for improving adaptive interference cancellation (AIC) using cascaded AIC algorithms are described. To improve an accuracy of detecting speech, a device may perform a first stage of AIC to generate isolated audio data and may generate speech mask data indicating time windows when speech is detected in the isolated audio data. Based on the speech mask data, the device may perform second AIC to generate output audio data, with adaptation of the adaptive filter enabled when the speech is not detected and disabled when the speech is detected. Thus, the first AIC improves the accuracy with which the device detects that speech is present and the second AIC reduces distortion in the output audio data by not updating filter coefficient values when the speech is present. The first AIC may use playback audio data, microphone audio data or beamformed audio data as reference signals.

Abstract: A speech-based audio device may be configured to detect a user-uttered wake expression. For example, the audio device may generate a parameter indicating whether output audio is currently being produced by an audio speaker, whether the output audio contains speech, whether the output audio contains a predefined expression, loudness of the output audio, loudness of input audio, and/or an echo characteristic. Based on the parameter, the audio device may determine whether an occurrence of the predefined expression in the input audio is a result of an utterance of the predefined expression by a user.

Abstract: A speech-based audio device may be configured to detect a user-uttered wake expression. For example, the audio device may generate a parameter indicating whether output audio is currently being produced by an audio speaker, whether the output audio contains speech, whether the output audio contains a predefined expression, loudness of the output audio, loudness of input audio, and/or an echo characteristic. Based on the parameter, the audio device may determine whether an occurrence of the predefined expression in the input audio is a result of an utterance of the predefined expression by a user.

Abstract: A distributed voice controlled system has a primary assistant and at least one secondary assistant. The primary assistant has a housing to hold one or more microphones, one or more speakers, and various computing components. The secondary assistant is similar in structure, but is void of speakers. The voice controlled assistants perform transactions and other functions primarily based on verbal interactions with a user. The assistants within the system are coordinated and synchronized to perform acoustic echo cancellation, selection of a best audio input from among the assistants, and distributed processing.

Abstract: Techniques for improving beamforming using filter coefficient values corresponding to virtual microphones are described. A system may define “virtual” microphone positions and determine corresponding filter coefficient values. These filter coefficient values may be applied to input audio data captured by actual physical microphones, enabling the system to improve performance of beamforming and/or to reduce a number of physical microphones without degrading performance. Offline testing and simulations may be performed to identify the best combination of virtual microphones and/or filter coefficient values for a particular look-direction. For example, the simulations may identify that a first filter coefficient corresponding to a first virtual microphone and a first direction will be associated with a first physical microphone and the first direction.

Abstract: Techniques for improving adaptive interference cancellation (AIC) using cascaded AIC algorithms are described. To improve an accuracy of detecting speech, a device may perform a first stage of AIC to generate isolated audio data and may generate speech mask data indicating time windows when speech is detected in the isolated audio data. Based on the speech mask data, the device may perform second AIC to generate output audio data, with adaptation of the adaptive filter enabled when the speech is not detected and disabled when the speech is detected. Thus, the first AIC improves the accuracy with which the device detects that speech is present and the second AIC reduces distortion in the output audio data by not updating filter coefficient values when the speech is present. The first AIC may use playback audio data, microphone audio data or beamformed audio data as reference signals.

Abstract: Techniques for improving adaptive interference cancellation (AIC) using cascaded AIC algorithms are described. To improve an accuracy of detecting speech, a device may perform a first stage of AIC to generate isolated audio data and may generate speech mask data indicating time windows when speech is detected in the isolated audio data. Based on the speech mask data, the device may perform second AIC to generate output audio data, with adaptation of the adaptive filter enabled when the speech is not detected and disabled when the speech is detected. Thus, the first AIC improves the accuracy with which the device detects that speech is present and the second AIC reduces distortion in the output audio data by not updating filter coefficient values when the speech is present. The first AIC may use playback audio data, microphone audio data or beamformed audio data as reference signals.

Abstract: A system that performs wall detection, range estimation, and/or corner detection to determine a position of a device relative to acoustically reflective surfaces. The device generates output audio using a loudspeaker, generates microphone audio data using a microphone array, performs beamforming to generate directional audio data and then generates impulse response data for each of a plurality of directions. The device may detect a peak in the impulse response data and determine a distance and/or direction to a reflective surface based on the peak. Based on a number of reflected surfaces and/or direction(s) of the reflected surfaces detected by the device, the device may classify the different directions and estimate where it is in the room, such as whether the device is in a corner, along one wall, or in an open area. By knowing its position relative to the room surfaces, the device may improve sound equalization.

Abstract: A system configured to improve audio processing by adaptively selecting target signals based on current system conditions. For example, a device may select a target signal based on a highest signal quality metric when only the local speech is present (e.g., during near-end single-talk conditions), as this maximizes an amount of energy included in the output audio signal. In contrast, the device may select the target signal based on a lowest signal quality metric when only the remote speech is present (e.g., during far-end single-talk conditions), as this minimizes an amount of energy included in the output audio signal. In addition, the device may track positions of the local speech and the remote speech over time, enabling the device to accurately select the target signal when both local speech and remote speech is present (e.g., during double-talk conditions).

Abstract: A speech-based audio device may be configured to detect a user-uttered wake expression. For example, the audio device may generate a parameter indicating whether output audio is currently being produced by an audio speaker, whether the output audio contains speech, whether the output audio contains a predefined expression, loudness of the output audio, loudness of input audio, and/or an echo characteristic. Based on the parameter, the audio device may determine whether an occurrence of the predefined expression in the input audio is a result of an utterance of the predefined expression by a user.

Abstract: A speech-based audio device may be configured to detect a user-uttered wake expression. For example, the audio device may generate a parameter indicating whether output audio is currently being produced by an audio speaker, whether the output audio contains speech, whether the output audio contains a predefined expression, loudness of the output audio, loudness of input audio, and/or an echo characteristic. Based on the parameter, the audio device may determine whether an occurrence of the predefined expression in the input audio is a result of an utterance of the predefined expression by a user.

Abstract: A distributed voice controlled system has a primary assistant and at least one secondary assistant. The primary assistant has a housing to hold one or more microphones, one or more speakers, and various computing components. The secondary assistant is similar in structure, but is void of speakers. The voice controlled assistants perform transactions and other functions primarily based on verbal interactions with a user. The assistants within the system are coordinated and synchronized to perform acoustic echo cancellation, selection of a best audio input from among the assistants, and distributed processing.

Abstract: A wearable computer is configured to use beamforming techniques to isolate a user's speech from extraneous audio signals occurring within a physical environment. A microphone array of the wearable computer may generate audio signal data from an utterance from a user's mouth. A motion sensor(s) of the wearable computer may generate motion data from movement of the wearable computer. This motion data may be used to determine a direction vector pointing from the wearable computer to the user's mouth, and a beampattern may be defined that has a beampattern direction in substantial alignment with the determined direction vector to focus the microphone array on the user's mouth for speech isolation.

Abstract: A system that performs wall detection, range estimation, and/or corner detection to determine a position of a device relative to acoustically reflective surfaces. The device generates output audio using loudspeaker(s), generates microphone audio data using a microphone array, and generates impulse response data for each of the microphones. The device may generate the impulse response data using an acoustic echo cancellation (AEC) component or multi-channel AEC (MC-AEC). The device may detect a peak in the impulse response data and determine a distance to a reflective surface based on the peak. Based on a number of reflected surfaces detected by the device, the device may classify a position of the device within the room, such as whether the device is in a corner, along one wall, or in an open area. By knowing the position relative to the room surfaces, the device may improve sound equalization and other processing.

Abstract: A system configured to improve noise cancellation by using portions of multiple reference signals instead of using a complete reference signal. The system divides a frequency spectrum into frequency bands and selects a single reference signal from a group of potential reference signals for every frequency band. For example, a first reference signal is selected for a first frequency band while a second reference signal is selected for a second frequency band. The system may generate a combined reference signal using portions of each of the selected reference signals, such as a portion of the first reference signal corresponding to the first frequency band and a portion of the second reference signal corresponding to the second frequency band. Additionally or alternatively, the system may perform noise cancellation using each of the selected reference signals and filter the outputs based on the corresponding frequency band to generate combined audio output data.

Abstract: A speech-based audio device may be configured to detect a user-uttered wake expression. For example, the audio device may generate a parameter indicating whether output audio is currently being produced by an audio speaker, whether the output audio contains speech, whether the output audio contains a predefined expression, loudness of the output audio, loudness of input audio, and/or an echo characteristic. Based on the parameter, the audio device may determine whether an occurrence of the predefined expression in the input audio is a result of an utterance of the predefined expression by a user.

Abstract: A system configured to improve double-talk detection. The system detects when double-talk is present in a voice conversation using two or more speaker models. The system extracts feature data from microphone audio data and compares the feature data to each speaker model. For example, the system may generate a first distance score indicating a likelihood that the feature data corresponds to a far-end speaker model and a second distance score indicating a likelihood that the feature data corresponds to a universal speaker model. The system may determine current system conditions based on the distance scores and may change settings to improve speech quality during the voice conversation. For example, during far-end single-talk the system may aggressively reduce an echo signal, whereas during near-end single-talk and double-talk the system may apply minimal echo cancellation to improve a quality of the local speech.

Abstract: A system configured to improve noise cancellation by reducing attenuation of local speech in proximity to a device. When the local speech is present in both a target signal and a reference signal, performing noise cancellation to remove the reference signal inadvertently attenuates the local speech. To prevent this, the system may perform first noise cancellation to identify frequency bands associated with the local speech and may generate a modified reference signal based on the frequency bands. For example, the system may generate the modified reference signal by applying attenuation to first frequencies associated with the local speech and/or gain to second frequencies that are not associated with the local speech. The system may generate final output audio data by performing noise cancellation using the modified reference signal.

Abstract: A system configured to improve beamforming by using deep neural networks (DNNs). The system can use one trained DNN to focus on a first person speaking an utterance (e.g., target user) and one or more trained DNNs to focus on noise source(s) (e.g., wireless loudspeaker(s), a second person speaking, other localized sources of noise, or the like). The DNNs may generate time-frequency mask data that indicates individual frequency bands that correspond to the particular source detected by the DNN. Using this mask data, a beamformer can generate beamformed audio data that is specific to a source of noise. The system may perform noise cancellation to isolate first beamformed audio data associated with the target user by removing second beamformed audio data associated with noise source(s).