Abstract: A method for audio beam steering, tracking, and audio effects for an immersive reality application is provided. The method includes receiving, from an immersive reality application, a first audio waveform from a first acoustic source to provide to a user of a headset, identifying a perceived direction for the first acoustic source relative to the headset based on a location of the first acoustic source, and providing, to a first speaker in a client device, an audio signal including the first audio waveform, wherein the audio signal includes a time delay and an amplitude of the first audio waveform based on the perceived direction. A non-transitory, computer-readable medium storing instructions which, when executed by a processor, cause a system to perform the above method, and the system, are also provided.

Abstract: A wearable device may include a processor configured to detect a self-voice signal, based on one or more transducers. The processor may be configured to separate the self-voice signal from a background signal in an external audio signal based on using a multi-microphone speech generative network. The processor may also be configured to apply a first filter to an external audio signal, detected by at least one external microphone on the wearable device, during a listen through operation based on an activation of the audio zoom feature to generate a first listen-through signal that includes the external audio signal. The processor may be configured to produce an output audio signal that is based on at least the first listen-through signal that includes the external signal, and is based on the detected self-voice signal.

Abstract: A device includes a memory configured to store instructions and one or more processors configured execute the instructions. The one or more processors are configured execute the instructions to receive audio data including first audio data corresponding to a first output of a first microphone and second audio data corresponding to a second output of a second microphone. The one or more processors are also configured to execute the instructions to provide the audio data to a dynamic classifier. The dynamic classifier is configured to generate a classification output corresponding to the audio data. The one or more processors are further configured to execute the instructions to determine, at least partially based on the classification output, whether the audio data corresponds to user voice activity.

Abstract: A device includes one or more processors configured to receive an audio data sample and to provide the audio data sample to a dynamic classifier. The dynamic classifier is configured to generate a classification output corresponding to the audio data sample. The one or more processors are further configured to selectively access a particular device based on the classification output.

Abstract: A wearable device may include a processor configured to perform active noise cancelation (ANC) applied to an input audio signal received by at least one microphone, and detect a self-voice signal, based on one or more transducers. The processor may also be configured to apply a first filter to an external audio signal, detected by at least one external microphone on the wearable device, during a listen through operation based on an activation of the audio zoom feature to generate a first listen-through signal that includes the external audio signal. The processor may also be configured to after the activation of the audio zoom feature terminate a second filter that provides low frequency compensation. The processor may be configured to produce an output audio signal that is based on at least the first listen-through signal that includes the external signal, and is based on the detected self-voice signal.

Abstract: A wearable device may include a processor configured to perform active noise cancelation (ANC) applied to an input audio signal received by at least one microphone, and detect a self-voice signal, based on one or more transducers. The processor may also be configured to apply a first filter to an external audio signal, detected by at least one external microphone on the wearable device, during a listen through operation based on an activation of the audio zoom feature to generate a first listen-through signal that includes the external audio signal. The processor may also be configured to after the activation of the audio zoom feature terminate a second filter that provides low frequency compensation. The processor may be configured to produce an output audio signal that is based on at least the first listen-through signal that includes the external signal, and is based on the detected self-voice signal.

Abstract: Methods, systems, and devices for signal processing are described. Generally, as provided for by the described techniques, a wearable device may receive an input audio signal (e.g., including both an external signal and a self-voice signal). The wearable device may detect the self-voice signal in the input audio signal based on a self-voice activity detection (SVAD) procedure, and may implement the described techniques based thereon. The wearable device may perform beamforming operations or other separation procedures to isolate the external signal and the self-voice signal from the input audio signal. The wearable device may apply a first filter to the external signal, and a second filter to the self-voice signal. The wearable device may then mix the filtered signals, and generate an output signal that sounds natural to the user.

Abstract: A method for audio beam steering, tracking, and audio effects for an immersive reality application is provided. The method includes receiving, from an immersive reality application, a first audio waveform from a first acoustic source to provide to a user of a headset, identifying a perceived direction for the first acoustic source relative to the headset based on a location of the first acoustic source, and providing, to a first speaker in a client device, an audio signal including the first audio waveform, wherein the audio signal includes a time delay and an amplitude of the first audio waveform based on the perceived direction. A non-transitory, computer-readable medium storing instructions which, when executed by a processor, cause a system to perform the above method, and the system, are also provided.

Abstract: A method to combine contact and acoustic microphones in a headset for voice wake and voice processing in immersive reality applications is provided. The method includes receiving, from a contact microphone, a first acoustic signal, determining a fidelity and a quality of the first acoustic signal, receiving, from an acoustic microphone, a second acoustic signal, and when the fidelity and quality of the first acoustic signal exceeds a pre-selected threshold, combining the first acoustic signal and the second acoustic signal to provide an enhanced acoustic signal to a smart glass user. A non-transitory, computer-readable medium storing instructions to cause a headset to perform the above method, and the headset, are also provided.

Abstract: A device includes a memory configured to store instructions and also includes one or more processors configured to execute the instructions to obtain audio data corresponding to a sound source and metadata indicative of a direction of the sound source. The one or more processors are configured to execute the instructions to obtain direction data indicating a viewing direction associated with a user of a playback device. The one or more processors are configured to execute the instructions to determine a resolution setting based on a similarity between the viewing direction and the direction of the sound source. The one or more processors are also configured to execute the instructions to process the audio data based on the resolution setting to generate processed audio data.

Abstract: In general, techniques are described by which to correlate scene-based audio data for psychoacoustic audio coding. A device comprising a memory and one or more processors may be configured to perform the techniques. The memory may store a bitstream including a plurality of encoded correlated components of a soundfield represented by scene-based audio data. The one or more processors may perform psychoacoustic audio decoding with respect to one or more of the plurality of encoded correlated components to obtain a plurality of correlated components, and obtain, from the bitstream, an indication representative of how the one or more of the plurality of correlated components were reordered in the bitstream. The one or more processors may reorder, based on the indication, the plurality of correlated components to obtain a plurality of reordered components, and reconstruct, based on the plurality of reordered components, the scene-based audio data.

Abstract: A device includes a memory configured to store instructions and one or more processors configured to execute the instructions. The one or more processors are configured to execute the instructions to obtain link data corresponding to a communication link to a second device. The one or more processors are configured to execute the instructions to select, at least partially based on the link data, between an ambisonics mode and a stereo mode.

Abstract: A device includes one or more processors configured to receive an audio data sample and to provide the audio data sample to a dynamic classifier. The dynamic classifier is configured to generate a classification output corresponding to the audio data sample. The one or more processors are further configured to selectively access a particular device based on the classification output.

Abstract: In general, techniques are described by which to code scaled spatial components. A device comprising a memory and one or more processors may be configured to perform the techniques. The memory may store a bitstream including an encoded foreground audio signal and a corresponding quantized spatial component. The one or more processors may perform psychoacoustic audio decoding with respect to the encoded foreground audio signal to obtain a foreground audio signal, and determine, when performing psychoacoustic audio decoding, a bit allocation for the encoded foreground audio signal. The one or more processors may dequantize the quantized spatial component to obtain a scaled spatial component, and descale, based on the bit allocation, the scaled spatial component to obtain a spatial component. The one or more processors may reconstruct, based on the foreground audio signal and the spatial component, scene-based audio data.

Abstract: An example device configured to obtain image data includes a memory configured to store one or more priority values, each of the one or more priority values being associated with a type of image object associated with the image data. The device includes one or more processors coupled to the memory, and configured to associate image objects in the image data with one or more audio sources represented in one or more audio streams. The one or more processors are also configured to assign a respective priority value to each of the one or more audio sources represented in the one or more streams and code ambisonic coefficients based on the assigned priority value.

Abstract: An example device includes a memory configured to store at least one spatial component and at least one audio source within a plurality of audio streams. The device also includes one or more processors coupled to the memory. The one or more processors are configured to receive, from motion sensors, rotation information. The one or more processors are configured to rotate the at least one spatial component based on the rotation information to form at least one rotated spatial component. The one or more processors are also configured to reconstruct ambisonic signals from the at least one rotated spatial component and the at least one audio source, wherein the at least one spatial component describes spatial characteristics associated with the at least one audio source in a spherical harmonic domain representation.

Abstract: A device includes a memory configured to store instructions and one or more processors configured execute the instructions. The one or more processors are configured execute the instructions to receive audio data including first audio data corresponding to a first output of a first microphone and second audio data corresponding to a second output of a second microphone. The one or more processors are also configured to execute the instructions to provide the audio data to a dynamic classifier. The dynamic classifier is configured to generate a classification output corresponding to the audio data. The one or more processors are further configured to execute the instructions to determine, at least partially based on the classification output, whether the audio data corresponds to user voice activity.

Abstract: Methods, systems, and devices for signal processing are described. Generally, as provided for by the described techniques, a wearable device may receive an input audio signal (e.g., including both an external signal and a self-voice signal). The wearable device may detect the self-voice signal in the input audio signal based on a self-voice activity detection (SVAD) procedure, and may implement the described techniques based thereon. The wearable device may perform beamforming operations or other separation procedures to isolate the external signal and the self-voice signal from the input audio signal. The wearable device may apply a first filter to the external signal, and a second filter to the self-voice signal. The wearable device may then mix the filtered signals, and generate an output signal that sounds natural to the user.

Abstract: An example device includes a memory configured to store at least one spatial component and at least one audio source within a plurality of audio streams. The device also includes one or more processors coupled to the memory. The one or more processors are configured to receive, from motion sensors, rotation information. The one or more processors are configured to rotate the at least one spatial component based on the rotation information to form at least one rotated spatial component. The one or more processors are also configured to reconstruct ambisonic signals from the at least one rotated spatial component and the at least one audio source, wherein the at least one spatial component describes spatial characteristics associated with the at least one audio source in a spherical harmonic domain representation.

Abstract: An example device configured to obtain image data includes a memory configured to store one or more priority values, each of the one or more priority values being associated with a type of image object associated with the image data. The device includes one or more processors coupled to the memory, and configured to associate image objects in the image data with one or more audio sources represented in one or more audio streams. The one or more processors are also configured to assign a respective priority value to each of the one or more audio sources represented in the one or more streams and code ambisonic coefficients based on the assigned priority value.