Abstract: The present disclosure provides a number of techniques for personalization of spatial audio across a plurality of systems. A participant media system participating in a session may generate two outputs: personalized audio data and depersonalized audio data. When a spectator attempts to spectate the session, a media sharing platform may distribute shared depersonalized audio data to the spectator system. Based on this shared depersonalized audio data, the spectator media system can personalized the audio data to cause an endpoint device to render spectator personalized audio based on HRTF data for the spectator. Accordingly, the spectator personalized audio is personalized and allows a rich, immersive spectating experience that overcome drawbacks associated with receiving personalized audio from participant systems.

Abstract: The techniques disclosed herein enable a system to coordinate audio objects that are generated by multiple applications. A system can receive contextual data from several applications and dynamically determine an allocation of a number of audio objects for each application based on the contextual data. The allocation can be based on a status of one or more applications, user interactions with one or more applications, and other factors. Policy data can also cause the system to allocate a number of audio objects to one or more applications based on an application type and other factors. For instance, a policy may cause a system to allocate more audio objects to a game application vs. a communications application. As a user interacts with an application, e.g., moves or resizes a user interface, closes an application, increases or decreases a level of interaction, the system can reallocate audio objects to individual applications.

Abstract: A system for streaming spatial audio and video is provided. In response to a request to share a virtual reality session, a characteristic of a second audio output device and a characteristic of a second video output device can be determined. Further in response to the request, based on the determined characteristic of the second audio output device, spatial audio can be provided to the second audio output device and received virtual reality video can be transcoded based on the determined characteristic of the second video output device. The transcoded virtual reality video can be provided to the second video output device so that other(s) can experience the virtual reality session.

Abstract: A system for progressively streaming spatial audio is provided. The system includes an engine that adaptively selects encoder(s) to stream spatial audio. Selection can be based upon selection metadata which can be based upon bandwidth, time, computing power, trust, cost, audio endpoint configuration, user criteria and the like. In response to detecting or being informed of a change in selection metadata, the engine can select different encoder(s) based upon the changed selection metadata.

Abstract: The present disclosure enables applications of a computing system to coordinate object-based audio resources by the use of a minimum resource working set. The minimum resource working set encourages an application to be fair in its requirements since specifying a large number will most likely result in the application receiving zero resources, or losing all of its resources to another application. A working set, which can include a minimum and a maximum working set, also provides a useful metric for the spatial audio resource manager to use when balancing demand. In addition, a minimum working set provides a performance metric for resource balancing since it exposes what the minimum functional requirement is from the maxim requested resource claim.

Abstract: Participants can control a number of aspects of a virtual reality session. A participant of the session can control the position of an object, such as an avatar. Spectators do not have control over aspects of a session. For instance, spectators cannot control the position of objects or change properties of objects within a virtual environment. In some configurations, the position of a spectator's viewing area is based on the position of an object that is controlled by a participant. In some embodiments, a spectator's viewing area can follow a participant's position but the spectator can look in any direction from that position. By following the participant's position, spectators can follow the action of a session yet have the freedom to control the direction of their viewing area to enhance their viewing experience. Customized spatial audio is also generated for the spectator based on the direction of their viewing area.

Abstract: Methods and devices for correcting warping in spatial audio may include identifying a geometric transform that defines a geometric warping between a first spatial geometric model that represents how sound is produced in a first volumetric space and a second spatial geometric model that represents how sound is produced in a second volumetric space different from the first volumetric space. The methods and devices may include determining an inverse of the geometric transform that compensates for the geometric transform. The methods and devices may include applying the inverse of the geometric transform to a first location in the first spatial geometric model by mapping the first location to a second location in the second spatial geometric model to correct for the geometric warping.

Abstract: The present disclosure enables applications of a computing system to coordinate object-based audio resources by the use of a minimum resource working set. The minimum resource working set encourages an application to be fair in its requirements since specifying a large number will most likely result in the application receiving zero resources, or losing all of its resources to another application. A working set, which can include a minimum and a maximum working set, also provides a useful metric for the spatial audio resource manager to use when balancing demand. In addition, a minimum working set provides a performance metric for resource balancing since it exposes what the minimum functional requirement is from the maxim requested resource claim.

Abstract: The techniques disclosed herein provide apparatuses and related methods for the communication of spatial audio and related metadata. In some implementations, a source provides prerecorded spatial audio that has embedded metadata. A computing device processes the prerecorded spatial audio to generate an audio codec that is segmented to include a first section of audio data and a second section that includes metadata extracted from the prerecorded spatial audio. The generated audio codec may be received by a device that includes an encoder. The encoder may process the generated audio codec to generate audio data that includes the metadata.

Abstract: A system for enabling a shared three-dimensional (“3D”) audio bed available to multiple software applications is provided. The system manages bed metadata defining a number of speaker objects of a 3D audio bed. The bed metadata also associates each speaker object with a location, which in some configurations, is defined by a three-dimensional coordinate system. The bed metadata is communicated to a plurality of applications. The applications can then generate custom 3D audio data that associates individual audio streams with individual speaker objects of the 3D audio bed. The applications can then communicate the custom 3D audio data to a 3D audio bed engine, which causes the processing and rendering of the custom 3D audio data to an output device utilizing a selected spatialization technology. Aspects of the 3D bed can be altered when the spatialization technology or the output device changes.

Abstract: The techniques disclosed herein can enable a system to coordinate the processing of object-based audio and channel-based audio generated by multiple applications. The system determines a spatialization technology to utilize based on contextual data. In some configurations, the contextual data can indicate the capabilities of one or more computing resources. In some configurations, the contextual data can also indicate preferences. The preferences, for example, can indicate user preferences for a type of spatialization technology, e.g., Dolby Atmos, over another type of spatialization technology, e.g., DTSX. Based on the contextual data, the system can select a spatialization technology and a corresponding encoder to process the input signals to generate a spatially encoded stream that appropriately renders the audio of multiple applications to an available output device. The techniques disclosed herein also allow a system to dynamically change the spatialization technologies during use.

Abstract: The techniques disclosed herein enable a system to coordinate audio objects that are generated by multiple applications. A system can receive contextual data from several applications and dynamically determine an allocation of a number of audio objects for each application based on the contextual data. The allocation can be based on a status of one or more applications, user interactions with one or more applications, and other factors. Policy data can also cause the system to allocate a number of audio objects to one or more applications based on an application type and other factors. For instance, a policy may cause a system to allocate more audio objects to a game application vs. a communications application. As a user interacts with an application, e.g., moves or resizes a user interface, closes an application, increases or decreases a level of interaction, the system can reallocate audio objects to individual applications.

Abstract: The techniques disclosed herein provide application programming interfaces (APIs) for enabling a system to select a spatialization technology. The APIs also enable a system to balance resources by allocating audio objects to a number of applications executing on a computer system. The system coordinates the audio objects between applications and each application can control the number of objects they individually generate. In some configurations, the system can also fold audio objects across different applications. Different spatialization technologies can be selected based on an analysis of contextual data and policy data. For instance, when a new headphone system is plugged in, the system may switch from Dolby Atmos to the Microsoft HoloLens HRTF spatialization technology. The system can dynamically control a number of generated audio objects and dynamically change a utilized spatialization technology based on changes to a computing environment.