Abstract: A device includes one or more processors configured to receive, via wireless transmission from a streaming device, encoded ambisonics audio data representing a sound field. The one or more processors are also configured to perform decoding of the ambisonics audio data to generate decoded ambisonics audio data. The decoding of the ambisonics audio data includes base layer decoding of a base layer of the encoded ambisonics audio data and selectively includes enhancement layer decoding in response to an amount of movement of the device. The one or more processors are further configured to adjust the decoded ambisonics audio data to alter the sound field based on data associated with at least one of a translation or an orientation associated with the movement of the device. The one or more processors are also configured to output the adjusted decoded ambisonics audio data to two or more loudspeakers for playback.

Abstract: The present disclosure relates to methods and apparatus for graphics processing. In some aspects, the apparatus may receive, by a first device from a second device, first position information corresponding to a first orientation of the second device. The apparatus can also generate, by the first device, first graphical content based on the first position information. Further, the apparatus can also generate, by the first device, motion information for warping the first graphical content. The apparatus can also encode, by the first device, the first graphical content. Additionally, the apparatus can provide, by the first device to the second device, the motion information and the encoded first graphical content.

Abstract: Techniques and systems are provided for light estimation. In some examples, a system receives a plurality of frames associated with a scene. The plurality of frames includes a first frame and a second frame occurring after the first frame. The system determines, based on image data of the first frame, a first light estimate associated with the scene. The system also determines, based on image data of the second frame, a second light estimate associated with the scene. The system further generates an aggregate light estimate associated with the scene based on combining the second light estimate with at least the first light estimate.

Abstract: Methods and systems for providing software applications on a client device with dynamic control over low-latency mode (LLM) operations of the client device. The client device may monitor downlink data packets of a client software application operating on the client device to detect trigger events. The client device may determine operating parameters of the modem based on a detected trigger event and dynamically adjust the low-latency mode of the modem based on the detect trigger event or the determined operating parameters.

Abstract: A method and a system for warping a rendered frame is disclosed. On a host device of a split-rendering system, the method includes generating the rendered frame based on head tracking information of a user. The method also includes identifying a region of interest (ROI) of the rendered frame. The method also includes generating metadata for a warping operation from the ROI. The method further include transmitting the rendered frame and the metadata for a warping operation of the rendered frame. On a client device of the split-rendering system, the method includes transmitting head tracking information of a user by a client device. The method also includes receiving the rendered frame and metadata. The method further includes warping the rendered frame using the metadata and display pose information. The host device and the client device may be combined into an all-in-one head mounted display.

Abstract: The present disclosure relates to methods and apparatus for graphics processing. The apparatus can determine an eye-buffer including one or more bounding boxes associated with rendered content in a frame. The apparatus can also generate an atlas based on the eye-buffer, the atlas including one or more patches associated with the one or more bounding boxes. Additionally, the apparatus can communicate the atlas including the one or more patches. The apparatus can also calculate an amount of user motion associated with the rendered content in the frame. Further, the apparatus can determine a size of each of the one or more bounding boxes based on the calculated amount of user motion. The apparatus can also determine a size and location of each of the one or more patches in the atlas.

Abstract: The present disclosure relates to methods and apparatus for computer processing. Aspects of the present disclosure can determine at least one of a quality, latency, or capacity of a communication link for communication between a client device and a server. Aspects of the present disclosure can also determine a computational load for an application computation between the client device and the server. Moreover, aspects of the present disclosure can adjust a computational distribution for the application computation between the client device and the server based on at least one of the computational load for the application computation or the at least one of the quality, latency, or capacity of the communication link. Aspects of the present disclosure can also determine a computational capacity of at least one of the client device or the server.

Abstract: A device includes one or more processors configured to receive, via wireless transmission from a playback device, data associated with a pose of the playback device. The one or more processors are also configured to select, based on the data, a particular representation of a sound field from a plurality of representations of the sound field. Each respective representation of the sound field corresponds to a different sector of a set of sectors. A sector represents a range of values associated with movement of the playback device. The one or more processors are further configured to generate audio data corresponding to the selected representation of the sound field. one or more processors are also configured to send, via wireless transmission, the audio data as streaming data to the playback device.

Abstract: A device includes one or more processors configured to obtain sound information from an audio source. The one or more processors are further configured to select, based on a latency criterion associated with a playback device, a compression mode in which a representation of the sound information is compressed prior to transmission to the playback device or a bypass mode in which the representation of the sound information is not compressed prior to transmission to the playback device. The one or more processors are further configured to generate audio data that includes, based on the selected one of the compression mode or the bypass mode, a compressed representation of the sound information or an uncompressed representation of the sound information. The one or more processors are also configured to send the audio data as streaming data, via wireless transmission, to the playback device.

Abstract: A device includes one or more processors configured to receive, via wireless transmission from a streaming device, encoded ambisonics audio data representing a sound field. The one or more processors are also configured to perform decoding of the ambisonics audio data to generate decoded ambisonics audio data. The decoding of the ambisonics audio data includes base layer decoding of a base layer of the encoded ambisonics audio data and selectively includes enhancement layer decoding in response to an amount of movement of the device. The one or more processors are further configured to adjust the decoded ambisonics audio data to alter the sound field based on data associated with at least one of a translation or an orientation associated with the movement of the device. The one or more processors are also configured to output the adjusted decoded ambisonics audio data to two or more loudspeakers for playback.

Abstract: The present disclosure relates to methods and apparatus for graphics processing. The apparatus can determine an eye-buffer including one or more bounding boxes associated with rendered content in a frame. The apparatus can also generate an atlas based on the eye-buffer, the atlas including one or more patches associated with the one or more bounding boxes. Additionally, the apparatus can communicate the atlas including the one or more patches. The apparatus can also calculate an amount of user motion associated with the rendered content in the frame. Further, the apparatus can determine a size of each of the one or more bounding boxes based on the calculated amount of user motion. The apparatus can also determine a size and location of each of the one or more patches in the atlas.

Abstract: The present disclosure relates to methods and apparatus for computer processing. Aspects of the present disclosure can determine at least one of a quality, latency, or capacity of a communication link for communication between a client device and a server. Aspects of the present disclosure can also determine a computational load for an application computation between the client device and the server. Moreover, aspects of the present disclosure can adjust a computational distribution for the application computation between the client device and the server based on at least one of the computational load for the application computation or the at least one of the quality, latency, or capacity of the communication link. Aspects of the present disclosure can also determine a computational capacity of at least one of the client device or the server.

Abstract: A server includes a processor configured to determine, based on reception of information from a client device, synchronization parameters between a server clock rate associated with the server and a client clock rate associated with the client device. The processor is also configured to determine a client-side generation timestamp for an audio frame based on the synchronization parameters and based on a server-side generation timestamp for the audio frame. The processor is further configured to generate the audio frame. The audio frame includes the client-side timestamp.

Abstract: Techniques are provided for adaptively controlling an encoding device to allow dynamic insertion intra-coded video content based on feedback information. For example, at least a portion of a video slice of a video frame in a video bitstream can be determined to be missing or corrupted. Feedback information indicating at least the portion of the video slice is missing or corrupted can be sent to an encoding device. An updated video bitstream can be received from the encoding device in response to the feedback information. The updated video bitstream can include at least one intra-coded video slice having a size that is larger than the missing or corrupted video slice. The size of the at least one intra-coded video slice can be determined to cover the missing or corrupted slice and propagated error in the video frame caused by the missing or corrupted slice.

Abstract: Systems, methods, and computer-readable media are provided for distributed tracking and mapping for extended reality experiences. An example method can include computing, at a device, a pose of the device at a future time, the future time being determined based on a communication latency between the device and a mapping backend system; sending, to the mapping backend system, the pose of the device; receiving, from the mapping backend system, a map slice including map points corresponding to a scene associated with the device, the map slice being generated based on the pose of the device, wherein the map points correspond to the predicted pose; and computing an updated pose of the device based on the map slice.

Abstract: The present disclosure relates to methods and apparatus for graphics processing. Aspects of the present disclosure can render at least one frame including display content at a server. Aspects of the present disclosure can also downscale the at least one frame including the display content, where a downscaling rate of one or more portions of the at least one frame is based on a location of each of the one or more portions. Moreover, aspects of the present disclosure can communicate the downscaled at least one frame including the display content to a client device. Aspects of the present disclosure can also encode the downscaled at least one frame including the display content. Further, aspects of the present disclosure can decode the encoded at least one frame including the display content. Aspects of the present disclosure can also upscale the at least one frame including the display content.

Abstract: A method and a system for warping a rendered frame is disclosed. On a host device of a split-rendering system, the method includes generating the rendered frame based on head tracking information of a user. The method also includes identifying a region of interest (ROI) of the rendered frame. The method also includes generating metadata for a warping operation from the ROI. The method further include transmitting the rendered frame and the metadata for a warping operation of the rendered frame. On a client device of the split-rendering system, the method includes transmitting head tracking information of a user by a client device. The method also includes receiving the rendered frame and metadata. The method further includes warping the rendered frame using the metadata and display pose information. The host device and the client device may be combined into an all-in-one head mounted display.

Abstract: Techniques are provided for adaptively controlling an encoding device to allow dynamic insertion intra-coded video content based on feedback information. For example, at least a portion of a video slice of a video frame in a video bitstream can be determined to be missing or corrupted. Feedback information indicating at least the portion of the video slice is missing or corrupted can be sent to an encoding device. An updated video bitstream can be received from the encoding device in response to the feedback information. The updated video bitstream can include at least one intra-coded video slice having a size that is larger than the missing or corrupted video slice. The size of the at least one intra-coded video slice can be determined to cover the missing or corrupted slice and propagated error in the video frame caused by the missing or corrupted slice.

Abstract: A method and a system for warping a rendered frame is disclosed. On a host device of a split-rendering system, the method includes generating the rendered frame based on head tracking information of a user. The method also includes identifying a region of interest (ROI) of the rendered frame. The method also includes generating metadata for a warping operation from the ROI. The method further include transmitting the rendered frame and the metadata for a warping operation of the rendered frame. On a client device of the split-rendering system, the method includes transmitting head tracking information of a user by a client device. The method also includes receiving the rendered frame and metadata. The method further includes warping the rendered frame using the metadata and display pose information. The host device and the client device may be combined into an all-in-one head mounted display.

Abstract: Systems, methods, and computer-readable media are provided for distributed tracking and mapping for extended reality experiences. An example method can include computing, at a device, a pose of the device at a future time, the future time being determined based on a communication latency between the device and a mapping backend system; sending, to the mapping backend system, the pose of the device; receiving, from the mapping backend system, a map slice including map points corresponding to a scene associated with the device, the map slice being generated based on the pose of the device, wherein the map points correspond to the predicted pose; and computing an updated pose of the device based on the map slice.