Abstract: A device capable of autonomous motion may move in an environment and may receive audio data from a microphone. A model may be trained to process the audio data to suppress noise from the audio data. The model may include an encoder that includes one or more convolutional layers, one or more recurrent layers, and a decoder that includes one or more convolutional layers.

Abstract: A device capable of autonomous motion may move in an environment and may receive audio data from a microphone. A model may be trained to process the audio data to determine mask data, which may be used to mask noise in the audio data. Training data for the model may be normalized before training, and different loss functions may be used for different types of training data.

Abstract: The systems, devices, and processes described herein may identify a beam of a voice-controlled device that is directed toward a reflective surface, such as a wall. The beams may be created by a beamformer. An acoustic echo canceller (AEC) may create filter coefficients for a reference sound. The filter coefficients may be analyzed to identify beams that include multiple peaks. The multiple peaks may indicate presence of one or more reflective surfaces. Using the amplitude and the time delay between the peaks, the device may determine that it is close to a reflective surface in a direction of the beam.

Abstract: A computing device may receive audio data from a microphone representing audio in an environment of the device, which may correspond to an utterance and noise. A model may be trained to process the audio data to cancel noise from the audio data. The model may include an encoder that includes one or more dense layers, one or more recurrent layers, and a decoder that includes one or more dense layers.

Abstract: A device capable of motion includes a beamformer for determining audio data corresponding to one or more directions. The beamformer includes a target beamformer that boosts audio from a target direction and a null beamformer that suppresses audio from that direction. When the device outputs sound while moving, the target and null beamformers capture and compensate for Doppler effects in output audio that reflects from nearby surfaces back to the device.

Abstract: A device capable of autonomous motion includes a residual echo suppressor for suppressing echoes caused by an output reference signal. When the device outputs audio while moving with a velocity, it may receive echoes that are Doppler-shifted due to the motion. The residual echo suppressor generates estimated residual error data based on phase-shifted reference data to account for and suppress the Doppler-shifted echoes.

Abstract: A mobile device capable of capturing voice commands includes a beamformer for determining audio data corresponding to one or more directions and a beam selector for selecting in which direction a source of target audio lies. The device determines, based on data from one or more sensors, an angle through which the device has rotated. Based on the angle and one or more rotation-compensation functions, the device interpolates audio data corresponding to the one or more directions to compensate for the rotation such that the direction corresponding to the source of target audio remains selected.

Abstract: The systems, devices, and processes described herein may identify a beam of a voice-controlled device that is directed toward a reflective surface, such as a wall. The beams may be created by a beamformer. An acoustic echo canceller (AEC) may create filter coefficients for a reference sound. The filter coefficients may be analyzed to identify beams that include multiple peaks. The multiple peaks may indicate presence of one or more reflective surfaces. Using the amplitude and the time delay between the peaks, the device may determine that it is close to a reflective surface in a direction of the beam.

Abstract: A wearable computer is configured to use beamforming techniques to isolate a user's speech from extraneous audio signals occurring within a physical environment. A microphone array of the wearable computer may generate audio signal data from an utterance from a user's mouth. A motion sensor(s) of the wearable computer may generate motion data from movement of the wearable computer. This motion data may be used to determine a direction vector pointing from the wearable computer to the user's mouth, and a beampattern may be defined that has a beampattern direction in substantial alignment with the determined direction vector to focus the microphone array on the user's mouth for speech isolation.

Abstract: The systems, devices, and processes described herein may identify a beam of a voice-controlled device that is directed toward a reflective surface, such as a wall. The beams may be created by a beamformer. An acoustic echo canceller (AEC) may create filter coefficients for a reference sound. The filter coefficients may be analyzed to identify beams that include multiple peaks. The multiple peaks may indicate presence of one or more reflective surfaces. Using the amplitude and the time delay between the peaks, the device may determine that it is close to a reflective surface in a direction of the beam.

Abstract: A decision engine executing on an electronic device may determine, using sensor data captured by multiple sensors of the device, whether a user is present in an environment that includes the device. If the user is determined to be present in the environment, the device may transition from a first state to a second state. The first state may be a first power state of the device in which the device is powered off or an idle or dormant state in which the device is powered on but a display of the device is powered off. Correspondingly, the second state may be a second power state of the device in which the device and the display are powered on and content is being rendered on the display. If the decision engine cannot make a determination based on the sensor data, a context engine may adjudicate the user presence determination.

Abstract: An audio capture device that incorporates a beamformer and beam-specific trigger word detection. Audio data from each beam is processed by a low power trigger word detector, such as a neural network or other trained model to detect if audio data (such as an audio frame or feature vector corresponding thereto) likely includes part of a trigger word. The beam that either most strongly represents a trigger word portion or represents a trigger word portion most early in time may be selected for further processing such as speech processing or confirmation by a more robust power intensive trigger word detector.

Abstract: A wearable computer is configured to use beamforming techniques to isolate a user's speech from extraneous audio signals occurring within a physical environment. A microphone array of the wearable computer may generate audio signal data from an utterance from a user's mouth. A motion sensor(s) of the wearable computer may generate motion data from movement of the wearable computer. This motion data may be used to determine a direction vector pointing from the wearable computer to the user's mouth, and a beampattern may be defined that has a beampattern direction in substantial alignment with the determined direction vector to focus the microphone array on the user's mouth for speech isolation.

Abstract: A beamformer system that can isolate a desired portion of an audio signal resulting from a microphone array. A combination of beamformers is used to dampen undesired noise, whether diffuse or coherent. A fixed beamformer is used to dampen diffuse noise while an adaptive beamformer is used to cancel directional coherent noise. The adaptive beamformer isolates and weights audio from various directions. The weights may vary depending on the isolated desired audio signal, dynamically adjusting the step-size adjustments to the weights.

Abstract: An acoustic interference cancellation system that combines acoustic echo cancellation and an adaptive beamformer to cancel acoustic interference from an audio output. The system uses a fixed beamformer to generate a target signal in a look direction and an adaptive beamformer to generate noise reference signals corresponding to non-look directions. The noise reference signals are used to estimate acoustic noise using an acoustic interference canceller (AIC), while reference signals associated with loudspeakers are used to estimate an acoustic echo using a multi-channel acoustic echo canceller (MC-AEC). The system cancels the acoustic echo and the acoustic noise simultaneously by adding the estimate of the acoustic noise and the estimate of the acoustic echo to generate an interference reference signal and cancelling the interference reference signal from the target signal. The system jointly updates adaptive filters for the AIC and the MC-AEC logic to improve a robustness of the system.

Abstract: A speech-capturing device that can modulate its output audio data volume based on environmental sound conditions at the location of a user speaking to the device. The device detects the sound pressure of a spoken utterance at the device location and determines the distance of the user from the device. The device also detects the sound pressure of noise at the device and uses information about the location of the noise source and user to determine the sound pressure of noise at the location of the talker. The device can then adjust the gain for output audio (such as a spoken response to the utterance) to ensure that the output audio is at a certain desired sound pressure when it reaches the location of the user.

Abstract: An audio device may have an output speaker that produces audio within the environment of a user and one or more input microphones that capture speech and other sounds from the user environment. The audio device may use acoustic echo cancellation (AEC) to suppress echoed components of the speaker output that may be present in audio captured by the input microphones. The AEC may be implemented using an adaptive filter that estimates echoing based on an output reference signal. The output reference signal may be generated by a reference microphone placed near the speaker of the audio device.

Abstract: A device that includes a first beam selection component that selects a portion of the multiple beams prior to performing adaptive noise cancellation, in addition to a second beam selection component that selects a beam after adaptive noise cancellation is performed. The device only performs adaptive noise cancellation on the portion of the multiple beams that are selected, reducing a complexity of performing adaptive noise cancellation. In addition, as the first beam selection component selects from the multiple beams before adaptive noise cancellation, the quality of the signals are not reduced and/or desired signals are not suppressed, resulting in the first beam selection component increasing a likelihood that the second beam selection component selects the desired beam. The first beam selection component may use a different algorithm and/or selection criteria than the second beam selection component in order to select beams associated with desired signals such as speech.

Abstract: Embodiments of systems and methods are described for determining which of a plurality of beamformed audio signals to select for signal processing. In some embodiments, a plurality of audio input signals are received from a microphone array comprising a plurality of microphones. A plurality of beamformed audio signals are determined based on the plurality of input audio signals, the beamformed audio signals comprising a direction. A plurality of signal features may be determined for each beamformed audio signal. Smoothed features may be determined for each beamformed audio signal based on at least a portion of the plurality of signal features. The beamformed audio signal corresponding to the maximum smoothed feature may be selected for further processing.

Abstract: Sound is banked laterally over an array of microphones arranged on a rear surface of a device. Sound enters a duct behind the device from different directions via inlets along the sides of the device. The duct directs the sound waves across the microphone array. An effective direction from which the banked sounds originated is determined, relative to a front of the device. Based on the determined effective direction, the device applies spatial filtering to isolate the received sound waves, selectively increasing a signal-to-noise ratio of sound from the selected source and at least partially occluding sounds from other sources.