Abstract: An electronic system is described that is responsive to non-verbal commands. The system may include a display component for presenting visual content to a user and an audio system, such as an in-ear or over-the-ear speaker system. The speaker system may be equipped with a bone conduction sensor or in-ear microphone that generates data associated with vibrations of a facial region of the user. The system may use the data to detect non-verbal command in the form of grunts, hums, coughs, tongue clicks, and the like. The system may perform various predetermined operations based on the detected vibrations.

Abstract: A method to combine contact and acoustic microphones in a headset for voice wake and voice processing in immersive reality applications is provided. The method includes receiving, from a contact microphone, a first acoustic signal, determining a fidelity and a quality of the first acoustic signal, receiving, from an acoustic microphone, a second acoustic signal, and when the fidelity and quality of the first acoustic signal exceeds a pre-selected threshold, combining the first acoustic signal and the second acoustic signal to provide an enhanced acoustic signal to a smart glass user. A non-transitory, computer-readable medium storing instructions to cause a headset to perform the above method, and the headset, are also provided.

Abstract: An audio system determines latency for each client device of users participating in an artificial reality or augmented reality environment. The latency determined for a client device accounts for latency of a communication channel used by the client device to exchange data, as well as time for the client device to perform computational tasks. The audio system determines a distance in the artificial reality environment between a user's avatar and an audio source for audio data and adjusts one or more transfer functions for generating audio content for the user to account for effects of the latency determined for the user's client device on the audio content. In some embodiments, the audio system modifies a distance in the artificial reality environment between the user's avatar and the audio source to compensate for latency of the user's client device.

Abstract: A method for audio beam steering, tracking, and audio effects for an immersive reality application is provided. The method includes receiving, from an immersive reality application, a first audio waveform from a first acoustic source to provide to a user of a headset, identifying a perceived direction for the first acoustic source relative to the headset based on a location of the first acoustic source, and providing, to a first speaker in a client device, an audio signal including the first audio waveform, wherein the audio signal includes a time delay and an amplitude of the first audio waveform based on the perceived direction. A non-transitory, computer-readable medium storing instructions which, when executed by a processor, cause a system to perform the above method, and the system, are also provided.

Abstract: A communication system actively monitors a conversation between users in an environment. Once a threshold condition occurs (e.g., client device within a threshold distance of a room where the conversation is occurring, the user enters a chatroom via the client device), the client device receives a summary from the communication system. The communication system may customize the summary in accordance with user preferences. User preferences may include level of detail in summary or time range associated with summary. The client device then presents the summary to the user. The summary may be audio or text. In some embodiments, the summary may be historical audio that is overlaid over current audio at faster speed and higher volume, and stops once the historical audio reaches real time.

Abstract: An electronic system is described that is responsive to non-verbal commands. The system may include a display component for presenting visual content to a user and an audio system, such as an in-ear or over-the-ear speaker system. The speaker system may be equipped with a bone conduction sensor or in-ear microphone that generates data associated with vibrations of a facial region of the user. The system may use the data to detect non-verbal command in the form of grunts, hums, coughs, tongue clicks, and the like. The system may perform various predetermined operations based on the detected vibrations.

Abstract: A communication system provides audio content to one or more client devices capable of playing spatialized audio content. For example, the communication system receives audio content from a client device and transmits the audio content to other client devices to be played for users. The communication system dynamically modifies audio content transmitted to different client devices based on a payload including audio parameters (e.g., local area acoustic properties, an audiogram for a user, a head related transfer function for a user, etc.) received from a client device.

Abstract: A method to combine contact and acoustic microphones in a headset for voice wake and voice processing in immersive reality applications is provided. The method includes receiving, from a contact microphone, a first acoustic signal, determining a fidelity and a quality of the first acoustic signal, receiving, from an acoustic microphone, a second acoustic signal, and when the fidelity and quality of the first acoustic signal exceeds a pre-selected threshold, combining the first acoustic signal and the second acoustic signal to provide an enhanced acoustic signal to a smart glass user. A non-transitory, computer-readable medium storing instructions to cause a headset to perform the above method, and the headset, are also provided.

Abstract: A communication system provides audio content to one or more client devices capable of playing spatialized audio content. For example, the communication system receives audio content from a client device and transmits the audio content to other client devices to be played for users. The communication system dynamically modifies audio content transmitted to different client devices based on a payload including audio parameters (e.g., local area acoustic properties, an audiogram for a user, a head related transfer function for a user, etc.) received from a client device.

Abstract: A method of tracking sound sources in an environment by monitoring the local area with a primary sensor system comprising a microphone array. The location of sound sources in the local area are tracked using the monitored sound. If an ambiguity is determined in the tracked location of the sound source, a secondary sensor system is activated. The secondary sensor system has a larger power draw than the primary sensor system. The secondary sensor system determines the updated location of the sound source. The tracked location of the sound sources is updated to be the updated location. Once the location is updated, the secondary sensor system is deactivated.

Abstract: The disclosed computer-implemented method may include applying, via a sound reproduction system, sound cancellation that reduces an amplitude of various sound signals. The method further includes identifying, among the sound signals, an external sound whose amplitude is to be reduced by the sound cancellation. The method then includes analyzing the identified external sound to determine whether the identified external sound is to be made audible to a user and, upon determining that the external sound is to be made audible to the user, the method includes modifying the sound cancellation so that the identified external sound is made audible to the user. Various other methods, systems, and computer-readable media are also disclosed.

Abstract: A method for audio beam steering, tracking, and audio effects for an immersive reality application is provided. The method includes receiving, from an immersive reality application, a first audio waveform from a first acoustic source to provide to a user of a headset, identifying a perceived direction for the first acoustic source relative to the headset based on a location of the first acoustic source, and providing, to a first speaker in a client device, an audio signal including the first audio waveform, wherein the audio signal includes a time delay and an amplitude of the first audio waveform based on the perceived direction. A non-transitory, computer-readable medium storing instructions which, when executed by a processor, cause a system to perform the above method, and the system, are also provided.

Abstract: This disclosure describes embodiments of methods, non-transitory computer-readable media, and systems for detecting that a physical space includes a physical object corresponding to an analogous virtual object from an augmented reality experience and rendering (or otherwise modifying) the augment reality experience to integrate the physical object as part of the experience. In particular, the disclosed systems can determine that a physical object within a physical environment corresponds to an analogous virtual object of an augmented reality experience. Based on this correspondence, the disclosed systems can modify one or more of the virtual graphics, sound, or other features corresponding to the augmented reality experience to represent the virtual object using the physical object. For example, the disclosed systems can modify acoustic features of a sound for the augmented reality experience to simulate the sound originating from (or being affected by) the physical object.

Abstract: The disclosed computer-implemented method may include applying, via a sound reproduction system, sound cancellation that reduces an amplitude of various sound signals. The method further includes identifying, among the sound signals, an external sound whose amplitude is to be reduced by the sound cancellation. The method then includes analyzing the identified external sound to determine whether the identified external sound is to be made audible to a user and, upon determining that the external sound is to be made audible to the user, the method includes modifying the sound cancellation so that the identified external sound is made audible to the user. Various other methods, systems, and computer-readable media are also disclosed.

Abstract: An audio system presents enhanced audio content to a user of a headset. The audio system detects sounds from the local area, at least a portion of which originate from a human sound source. The audio system obtains a voice profile of an identifies human sound source that generates at least the portion of the detected sounds. Based in part on the voice profile, the audio system enhances the portion of the detected sounds that are generated by the human sound source to obtain enhanced audio. The audio system presents the enhanced audio to the user.

Abstract: In some embodiments, a method includes detecting an incorporation attribute of audiovisual content; analyzing an audio library to determine an audio file that maps to the incorporation attribute; selecting the audio file from the audio library that maps to the incorporation attribute; incorporating the selected audio file into the audiovisual content to generate egocentric audiovisual content; and providing the egocentric audiovisual content to a user for audiovisual consumption. In some embodiments of the method, the incorporation attribute is at least one of a mood of a target of the audiovisual content, an activity of the target of the audiovisual content, and a background of the audiovisual content.

Abstract: Validation of an activation event of a device based on analyzing audio data is disclosed. The device detects an activation event, e.g., an input to a soft-touch button or a wake word detected by a microphone array. Responsive to detecting the activation event, the device captures, via a microphone array on the device, sound from a local area of the device. The device stores the captured sound as audio data in an audio buffer. The device performs a validation of the activation event by analyzing the captured sound. The device performs an action based on a result of the validation.

Abstract: This disclosure describes embodiments of methods, non-transitory computer-readable media, and systems for detecting that a physical space includes a physical object corresponding to an analogous virtual object from an augmented reality experience and rendering (or otherwise modifying) the augment reality experience to integrate the physical object as part of the experience. In particular, the disclosed systems can determine that a physical object within a physical environment corresponds to an analogous virtual object of an augmented reality experience. Based on this correspondence, the disclosed systems can modify one or more of the virtual graphics, sound, or other features corresponding to the augmented reality experience to represent the virtual object using the physical object. For example, the disclosed systems can modify acoustic features of a sound for the augmented reality experience to simulate the sound originating from (or being affected by) the physical object.

Abstract: A communication system provides audio content to one or more client devices capable of playing spatialized audio content. For example, the communication system receives audio content from a client device and transmits the audio content to other client devices to be played for users. The communication system dynamically modifies audio content transmitted to different client devices based on a payload including audio parameters (e.g., local area acoustic properties, an audiogram for a user, a head related transfer function for a user, etc.) received from a client device.

Abstract: An audio system of a client device applies transformations to audio received over a computer network. The transformations (e.g., HRTFs) effect changes in apparent source positions of the received audio, or of segments thereof. Such transformations may be used to achieve “animation” of audio, in which the source positions of the audio or audio segments appear to change over time (e.g., circling around the listener). Additionally, segmentation of audio into distinct semantic audio segments, and application of separate transformations for each audio segment, can be used to intuitively differentiate the different audio segments by causing them to sound as if they emanated from different positions around the listener.