Abstract: Aspects of the subject technology provide improved techniques for determining a state of an audio device, including reduced power techniques for determining if an earbud is currently being worn in a user's ear or not. In aspects, a potential state change in an audio device may be detected, and in response measurement of the state of the device may be initiated, such as by emitting an audio signal and then determining the state of the audio device based on a sensed version of the emitted audio signal.

Abstract: A method for self-calibrating a sound pickup process that uses a microphone array in a wearable device that also includes a loudspeaker, where the microphone array being in a physical arrangement with respect to the loudspeaker. The method obtains, for each of several microphones of the microphone array, one or more transfer functions that each represent a response of the microphone to sound from a position in an acoustic space. The method determines whether a physical arrangement of the microphone array with respect to the loudspeaker has changed and adjusts the transfer function, for at least one of the microphones of the several microphones, in response to determining that the current physical arrangement of the microphone array with respect to the loudspeaker has changed.

Abstract: Loudspeakers are described that may reduce comb filtering effects perceived by a listener by either 1) moving transducers closer to a sound reflective surface (e.g., a baseplate, a tabletop or a floor) through vertical (height) or rotational adjustments of the transducers or 2) guiding sound produced by the transducers to be released into the listening area proximate to the reflective surface through the use of horns and openings that are at a prescribed distance from the reflective surface. The reduction of this distance between the reflective surface and the point at which sound emitted by the transducers is released into the listening area may lead to shorter reflected path that reduces comb filtering effects caused by reflected sounds that are delayed relative to the direct sound. Accordingly, the loudspeakers shown and described may be placed on reflective surfaces without severe audio coloration caused by reflected sounds.

Abstract: In one implementation, a method of playing an audio notification is performed at a device including one or more image sensors, one or more speakers, one or more processors, and non-transitory memory. The method includes receiving, from the one or more image sensors, an image of a physical environment having a device field-of-view different than a user field-of-view. The method includes detecting, in the image of the physical environment, an object at a location in the physical environment. The method includes determining that the location in the physical environment is outside an area of the user field-of-view. The method includes, in response to determining that the location in the physical environment is outside the area of the user field-of-view, playing, via the speaker, an audio notification of the detection.

Abstract: In one implementation, a method of performing an action is performed at a device including an image sensor, one or more processors, and non-transitory memory. The method includes receiving, from the image sensor, one or more images of a physical environment. The method includes detecting, in the one or more images of the physical environment, one or more actionable items respectively associated with one or more actions. The method includes detecting, in the one or more images of the physical environment, a hand gesture indicating a particular actionable item. The method includes in response to detecting the hand gesture, performing an action associated with the particular actionable item.

Abstract: In one implementation, a method of detecting an occlusion of an image sensor is performed at a device including one or more image sensors, one or more speakers, one or more processors, and non-transitory memory. The method includes receiving, from the one or more image sensors, an image of a physical environment. The method includes detecting, based on the image of the physical environment, an occlusion at least partially occluding the one or more image sensors. The method includes classifying the occlusion as one of a plurality of occlusion types. The method includes playing, via the one or more speakers, an occlusion notification based on the classification.

Abstract: In one implementation, a method of playing audio data is performed at a device including a frame configured for insertion into an outer ear, a speaker coupled to the frame, an image sensor coupled to the frame, one or more processors, and non-transitory memory. The method includes capturing, using the image sensor, one or more images of a physical environment. The method includes generating audio data based on the one or more images of the physical environment. The method includes playing, via the speaker, the audio data.

Abstract: The present disclosure generally relates techniques for audio-assisted enrollment of biometric features. In some embodiments, methods and devices for assisting users with enrollment of biometric features, using spatial audio cues, are described.

Abstract: A method for self-calibrating a sound pickup process that uses a microphone array in a wearable device that also includes a loudspeaker, where the microphone array being in a physical arrangement with respect to the loudspeaker. The method obtains, for each of several microphones of the microphone array, one or more transfer functions that each represent a response of the microphone to sound from a position in an acoustic space. The method determines whether a physical arrangement of the microphone array with respect to the loudspeaker has changed and adjusts the transfer function, for at least one of the microphones of the several microphones, in response to determining that the current physical arrangement of the microphone array with respect to the loudspeaker has changed.

Abstract: A method performed a local device that is communicatively coupled with several remote devices, the method includes: receiving, from each remote device with which the local device is engaged in a communication session, an input audio stream; receiving, for each remote device, a set parameters; determining, for each input audio stream, whether the input audio stream is to be 1) rendered individually or 2) rendered as a mix of input audio streams based on the set of parameters; for each input audio stream that is determined to be rendered individually, spatial rendering the input audio stream as an individual virtual sound source that contains only that input audio stream; and for input audio streams that are determined to be rendered as the mix of input audio streams, spatial rendering the mix of input audio streams as a single virtual sound source that contains the mix of input audio streams.

Abstract: Loudspeakers are described that may reduce comb filtering effects perceived by a listener by either 1) moving transducers closer to a sound reflective surface (e.g., a baseplate, a tabletop or a floor) through vertical (height) or rotational adjustments of the transducers or 2) guiding sound produced by the transducers to be released into the listening area proximate to the reflective surface through the use of horns and openings that are at a prescribed distance from the reflective surface. The reduction of this distance between the reflective surface and the point at which sound emitted by the transducers is released into the listening area may lead to shorter reflected path that reduces comb filtering effects caused by reflected sounds that are delayed relative to the direct sound. Accordingly, the loudspeakers shown and described may be placed on reflective surfaces without severe audio coloration caused by reflected sounds.

Abstract: The present disclosure generally relates techniques for audio-assisted enrollment of biometric features. In some embodiments, methods and devices for assisting users with enrollment of biometric features, using spatial audio cues, are described.

Abstract: In a device or method for rendering a sound program for headphones, a location is received for placing the sound program with respect to first and second ear pieces. If the location is between the first ear piece and the second ear piece, the sound program is filtered to produce low-frequency and high-frequency portions. The high-frequency portion is panned according to the location to produce first and second high-frequency signals. The low-frequency portion and the first high-frequency signal are combined to produce a first headphone driver signal to drive the first ear piece. A second headphone driver signal is similarly produced. The sound program may be a stereo sound program. The device or method may also provide for a location that is between the first ear piece and a near-field boundary. Other aspects are also described.

Abstract: A system for enhancing audio including a plurality of sensors, an output device, and a processor in communication with the plurality of sensors and the output device. The processor is configured to process data captured by the plurality of sensors, and based on that, modify an output of the output device. The processor also is configured to determine whether there are a plurality of users associated with a video conferencing session, determine which user of the plurality of users is speaking, and enhance the audio or video output of the speaking user on the output device.

Abstract: An output device that includes a housing, a first speaker driver that is integrated within the housing at a first location and is arranged to project sound into an ambient environment in a first direction, and a second speaker driver that is integrated within the housing at a second location that is different than the first location and is arranged to project sound into the ambient environment, where the first and second speaker drivers are integrated within the housing and share a common back volume within the housing.

Abstract: Disclosed are techniques for a multimedia device with audio and video capturing capability to identify an audio device based on acoustic playback signal if the audio device cannot be identified from captured video. The multimedia device may assemble a list of candidate audio devices that are a possible match for the observed audio device from a database of previously recognized audio devices and may transmit commands to the candidate audio devices to play acoustic identification signals. The acoustic identification signals may be audible sound or ultrasonic tone sequences with embedded identification information unique to each audio device. The multimedia device may record and analyze the acoustic identification signals received from any of the candidate audio devices to construct metrics to select the most likely candidate for the observed audio device. The metrics may include time of flight, direction of arrival, received amplitude, direct-to-reverberant ratio (DRR) of the acoustic identification signals.

Abstract: The present disclosure generally relates to user interfaces for managing spatial audio. Some exemplary techniques include user interfaces for transitioning between visual elements. Some exemplary techniques include user interfaces for previewing audio. Some exemplary techniques include user interfaces for discovering music. Some exemplary techniques include user interfaces for managing headphone transparency. Some exemplary techniques include user interfaces for manipulating multiple audio streams of an audio source.

Abstract: Implementations of the subject technology provide user tracking headrest audio control. For example, a seat may have a headrest and one or more speakers mounted to the headrest for providing audio output to an occupant of the seat. Because the head of the occupant may be disposed in the near field of the headrest speakers when the occupant is seated in the seat, movements of the occupant's head and/or ears may affect the acoustic experience of the occupant. Aspects of the subject technology provide for modifications to audio output(s) from one or more speaker(s) mounted in a headrest, based on tracking of the location of the occupant's head and/or ears relative to the location(s) of the speaker(s).

Abstract: A system for enhancing audio including a plurality of sensors, an output device, and a processor in communication with the plurality of sensors and the output device. The processor is configured to process data captured by the plurality of sensors, and based on that, modify an output of the output device. The processor also is configured to determine whether there are a plurality of users associated with a video conferencing session, determine which user of the plurality of users is speaking, and enhance the audio or video output of the speaking user on the output device.

Abstract: The present disclosure generally relates to user interfaces for managing spatial audio. Some exemplary techniques include user interfaces for transitioning between visual elements. Some exemplary techniques include user interfaces for previewing audio. Some exemplary techniques include user interfaces for discovering music. Some exemplary techniques include user interfaces for managing headphone transparency. Some exemplary techniques include user interfaces for manipulating multiple audio streams of an audio source.