Abstract: Various implementations disclosed herein include devices, systems, and methods that provide an improved remote device user interface for enabling accurate text input into a user device. For example, a user device (e.g., an HMD) may be enabled to receive touchscreen input via input controls displayed by a remote device (e.g., a mobile phone, a tablet computer, etc.) using a display mode enhanced for pass-through viewing. For example, an enhanced display mode may enable a larger than normal display to enhance display mode attributes associated with input entry with respect to the user device. In some implementations display mode attributes may be enabled based on context such as distance, lens distortion, lighting, minimum character resolution, etc. Likewise, some implementations may augment input controls (on an HMD) such as 3 dimensional bubbles, auto-correct controls, auto-fill controls, spatialized audio controls, virtual touch/function bar controls, etc.

Abstract: Aspects of the subject technology provide improved point-to-point audio communications based on human variable sensitivity to latency differences in multipath communications. In aspects, improved techniques may include measuring a level of ambient noise, and then selecting processing for a received electronic audio based on the measured level of ambient noise before emitting the processed audio signal at a loudspeaker worn by a listener.

Abstract: Various implementations disclosed herein include devices, systems, and methods that provide video content (e.g., a TV show, a recorded sporting event, a movie, a 3D video, etc.) within a 3D environment using parameters selected based on one or more contextual factors. Such contextual factors may be determined based on an attribute of the content (e.g., its intended purpose or viewing environment), the user (e.g., the user's visual quality, interpupillary distance, etc.), the 3D environment (e.g., current lighting conditions, spatial considerations, etc.), or other context attributes.

Abstract: An audio processing system may obtain a size of a visual object to present to a display. The audio processing system may determine a virtual placement for each of a plurality of virtual speakers at least based on the size of the visual object. Each of the plurality of virtual speakers may be spatially rendered at each virtual placement through binaural audio, for playback through head-worn speakers. Other aspects are also described and claimed.

Abstract: Sound sources can be spatially rendered in a setting and shown through a display. In response to satisfaction of a threshold criterion that is satisfied based on relative distance between the sound sources and a position of a listener, the rendering of the sound sources can be adjusted to maintain spatial integrity of the sound sources. The adjustment can be performed to prevent one of the sound sources from arriving at the listener earlier than another of the sound sources.

Abstract: A method for self-calibrating a sound pickup process that uses a microphone array in a wearable device that also includes a loudspeaker, where the microphone array being in a physical arrangement with respect to the loudspeaker. The method obtains, for each of several microphones of the microphone array, one or more transfer functions that each represent a response of the microphone to sound from a position in an acoustic space. The method determines whether a physical arrangement of the microphone array with respect to the loudspeaker has changed and adjusts the transfer function, for at least one of the microphones of the several microphones, in response to determining that the current physical arrangement of the microphone array with respect to the loudspeaker has changed.

Abstract: In one implementation, a method of visualizing a combined audio pick-up pattern is performed at a first device in a physical environment, the first device including a display, one or more processors, and non-transitory memory. The method includes determining a first audio pick-up pattern of the first device. The method includes determining one or more second audio pick-up patterns of a respective one or more second devices. The method includes determining a combined audio pick-up pattern of the first device and the one or more second devices based on the first audio pick-up pattern and the one or more second audio pick-up patterns. The method includes displaying, on the display, a representation of the combined audio pick-up pattern. In one implementation, a method of determining an audio emission pattern is performed at a first device at a first location, the first device having a microphone, one or more processors, and non-transitory memory.

Abstract: A method for self-calibrating a sound pickup process that uses a microphone array in a wearable device that also includes a loudspeaker, where the microphone array being in a physical arrangement with respect to the loudspeaker. The method obtains, for each of several microphones of the microphone array, one or more transfer functions that each represent a response of the microphone to sound from a position in an acoustic space. The method determines whether a physical arrangement of the microphone array with respect to the loudspeaker has changed and adjusts the transfer function, for at least one of the microphones of the several microphones, in response to determining that the current physical arrangement of the microphone array with respect to the loudspeaker has changed.

Abstract: Various implementations disclosed herein include devices, systems, and methods that that modify audio of played back AV content based on context in accordance with some implementations. In some implementations audio-visual content of a physical environment is obtained, and the audio-visual content includes visual content and audio content that includes a plurality of audio portions corresponding to the visual content. In some implementations, a context for presenting the audio-visual content is determined, and a temporal relationship between one or more audio portions of the plurality of audio portions and the visual content is determined based on the context. Then, synthesized audio-visual content is presented based on the temporal relationship.

Abstract: Each of a plurality of virtual loudspeaker arrays and their channels are produced, based on a corresponding microphone array and microphone signals thereof. Channels of a hallucinated loudspeaker array are determined based on the channels of the plurality of virtual loudspeaker arrays. The plurality of virtual loudspeaker arrays and the hallucinated loudspeaker array share a common geometry and orientation. Spatial audio is rendered based on the channels of the hallucinated loudspeaker array.

Abstract: An object can represent computer applications that play audio. Audio parameters associated with the audio can be determined based on size of the object so that when the object is large, the audio sounds like it originates from a large sound source or sources. When the object is small, the audio parameters are determined so that the audio sounds like it originates from a small sound source. Other aspects are described.

Abstract: Various implementations disclosed herein include devices, systems, and methods that display visual content as part of a 3D environment and add audio corresponding to the visual content. The audio may be spatialized to be from one or more audio source locations within the 3D environment. For example, a video may be presented on a virtual surface within an extended reality (XR) environment while audio associated with the video is spatialized to sound as if it is produced from an audio source location corresponding to that virtual surface. How the audio is provided may be determined based on the position of the viewer (e.g., the user or his/her device) relative to the presented visual content.

Abstract: Various implementations disclosed herein include devices, systems, and methods that that modify audio of played back AV content based on context in accordance with some implementations. In some implementations audio-visual content of a physical environment is obtained, and the audio-visual content includes visual content and audio content that includes a plurality of audio portions corresponding to the visual content. In some implementations, a context for presenting the audio-visual content is determined, and a temporal relationship between one or more audio portions of the plurality of audio portions and the visual content is determined based on the context. Then, synthesized audio-visual content is presented based on the temporal relationship.

Abstract: A machine learning model can determine output frequency response at different directions relative to a target audio output format, based on input including frequency response at directions relative to microphones of a capture device. A spatial filter determined based on the output frequency responses is applied to one or more of the microphone signals to map the spatial information from the microphone signals to the target audio output.

Abstract: Spatial filters are generated that map response of an audio capture device to head related transfer functions (HRTFs) for different positions of the audio capture device relative to the HRTFs. A current set of spatial filters are determined based on the plurality of spatial filters and a head position of a user. The microphone signals are convolved with the current set of spatial filters, resulting in a left audio channel and right audio channel that form output binaural audio channels. The binaural audio channels can be used to drive speakers of a headphone set to generate sound that is perceived to have a spatial quality. Other aspects are described and claimed.

Abstract: Various implementations disclosed herein include devices, systems, and methods that that modify audio of played back AV content based on context in accordance with some implementations. In some implementations audio-visual content of a physical environment is obtained, and the audio-visual content includes visual content and audio content that includes a plurality of audio portions corresponding to the visual content. In some implementations, a context for presenting the audio-visual content is determined, and a temporal relationship between one or more audio portions of the plurality of audio portions and the visual content is determined based on the context. Then, synthesized audio-visual content is presented based on the temporal relationship.

Abstract: Various implementations disclosed herein include devices, systems, and methods that that modify audio of played back AV content based on context in accordance with some implementations. In some implementations audio-visual content of a physical environment is obtained, and the audio-visual content includes visual content and audio content that includes a plurality of audio portions corresponding to the visual content. In some implementations, a context for presenting the audio-visual content is determined, and a temporal relationship between one or more audio portions of the plurality of audio portions and the visual content is determined based on the context. Then, synthesized audio-visual content is presented based on the temporal relationship.

Abstract: A method for self-calibrating a sound pickup process that uses a microphone array in a wearable device that also includes a loudspeaker, where the microphone array being in a physical arrangement with respect to the loudspeaker. The method obtains, for each of several microphones of the microphone array, one or more transfer functions that each represent a response of the microphone to sound from a position in an acoustic space. The method determines whether a physical arrangement of the microphone array with respect to the loudspeaker has changed and adjusts the transfer function, for at least one of the microphones of the several microphones, in response to determining that the current physical arrangement of the microphone array with respect to the loudspeaker has changed.

Abstract: Various implementations disclosed herein include devices, systems, and methods that that modify audio of played back AV content based on context in accordance with some implementations. In some implementations audio-visual content of a physical environment is obtained, and the audio-visual content includes visual content and audio content that includes a plurality of audio portions corresponding to the visual content. In some implementations, a context for presenting the audio-visual content is determined, and a temporal relationship between one or more audio portions of the plurality of audio portions and the visual content is determined based on the context. Then, synthesized audio-visual content is presented based on the temporal relationship.

Abstract: A glove interface object, including: a flex sensor configured to generate flex sensor data identifying a flex of a finger portion of the glove interface object; a trackable object configured to be illuminated during interactivity, the trackable object being positioned at a wrist portion of the glove interface object; a communications module configured to transmit the flex sensor data to a computing device for processing to determine a finger position pose of the glove interface object, applied for rendering a virtual hand in a view of a virtual environment on a head-mounted display (HMD), the virtual hand rendered based on the identified finger position pose, the computing device identifying the trackable object from captured image data to track a location of the glove interface object; wherein the virtual hand is rendered at a location in the virtual environment that is substantially defined by the location of the glove interface object.