Abstract: Systems and techniques are described herein for processing images. The systems and techniques can be implemented by various types of systems, such as by an extended reality (XR) system or device. In some cases, a first processor receives an image of an environment captured by an image sensor, identifies features depicted in the image, and generates descriptors for the features. The first processor sends the descriptors to a second processor, which may be more powerful than the first processor. The second processor receives the descriptors. The second processor associates the plurality of features with a map of the environment based on at least a subset of the plurality of descriptors. For example, the second processor can track at least a subset of the features based on at least a subset of the descriptors and based on feature information from one or more additional images of the environment.

Abstract: Methods and systems for providing software applications on a client device with dynamic control over low-latency mode (LLM) operations of the client device. The client device may monitor downlink data packets of a client software application operating on the client device to detect trigger events. The client device may determine operating parameters of the modem based on a detected trigger event and dynamically adjust the low-latency mode of the modem based on the detect trigger event or the determined operating parameters.

Abstract: The present disclosure relates to methods and apparatus for video processing. In one aspect, the apparatus may configure at least one frame of plurality of frames in a video stream. The apparatus may also divide the at least one frame into one or more slices, each of the one or more slices including one or more data packets. The apparatus may also map each of the one or more slices to one of one or more communication channels. The apparatus may also transmit each of the one or more slices including the one or more data packets via the one of the one or more communication channels. In another aspect, the apparatus may receive each of one or more slices via one of one or more communication channels. The apparatus may also decode each of the one or more slices including one or more data packets.

Abstract: The present disclosure relates to methods and apparatus for graphics processing. The apparatus can determine a combined depth map based on at least one of a 3DRU mesh, a DFS map, and a display pose. The apparatus can also determine one or more re-projected eye and depth buffers based on at least one of one or more eye buffers, one or more depth buffers, one or more bounding box and warp parameters, and the display pose. Further, the apparatus can communicate at least one composited frame based on the combined depth map and the one or more re-projected eye and depth buffers. The apparatus can also determine the at least one composited frame based on the combined depth map and the one or more re-projected eye and depth buffers. The apparatus can also combine the 3DRU mesh, the DFS map, and the display pose.

Abstract: Aspects presented herein relate to methods and apparatus for graphics processing. Aspects presented herein can determine at least one split API for a user device and a server. Further, aspects presented herein can establish a communication interface between the user device and the server based on the split API. Additionally, aspects presented herein can communicate between the user device and the server based on the communication interface and the split API. Some aspects presented herein can implement the at least one split API on at least one of the user device or the server. Aspects presented herein can also divide at least one application workload between the user device and the server based on the at least one split API. Moreover, aspects presented herein can encode or decode application information at the user device or the server.

Abstract: The present disclosure relates to methods and apparatus for computer processing. Aspects of the present disclosure can determine at least one of a quality, latency, or capacity of a communication link for communication between a client device and a server. Aspects of the present disclosure can also determine a computational load for an application computation between the client device and the server. Moreover, aspects of the present disclosure can adjust a computational distribution for the application computation between the client device and the server based on at least one of the computational load for the application computation or the at least one of the quality, latency, or capacity of the communication link. Aspects of the present disclosure can also determine a computational capacity of at least one of the client device or the server.

Abstract: This disclosure provides systems, devices, apparatus and methods, including computer programs encoded on storage media, for multi-layer reprojection techniques for augmented reality. A display processor may obtain a layer of graphics data including a plurality of virtual objects. Each the plurality of virtual objects may be associated with at least one bounding box of a plurality of bounding boxes. The display processor may further obtain metadata indicative of at least one edge of the at least one bounding box of the plurality of bounding boxes, and metadata corresponding to reprojection instructions associated with each of the plurality of bounding boxes. The display processor may reproject the plurality of virtual objects based on the metadata indicative of the at least one edge of the at least one bounding box and the metadata corresponding to the reprojection instructions.

Abstract: The present disclosure relates to methods and apparatus for graphics processing. In some aspects, the apparatus may receive, by a first device from a second device, first position information corresponding to a first orientation of the second device. The apparatus can also generate, by the first device, first graphical content based on the first position information. Further, the apparatus can also generate, by the first device, motion information for warping the first graphical content. The apparatus can also encode, by the first device, the first graphical content. Additionally, the apparatus can provide, by the first device to the second device, the motion information and the encoded first graphical content.

Abstract: Systems and techniques are described herein for processing images. The systems and techniques can be implemented by various types of systems, such as by an extended reality (XR) system or device. In some cases, a first processor receives an image of an environment captured by an image sensor, identifies features depicted in the image, and generates descriptors for the features. The first processor sends the descriptors to a second processor, which may be more powerful than the first processor. The second processor receives the descriptors. The second processor associates the plurality of features with a map of the environment based on at least a subset of the plurality of descriptors. For example, the second processor can track at least a subset of the features based on at least a subset of the descriptors and based on feature information from one or more additional images of the environment.

Abstract: Methods and systems for providing software applications on a client device with dynamic control over low-latency mode (LLM) operations of the client device. The client device may monitor downlink data packets of a client software application operating on the client device to detect trigger events. The client device may determine operating parameters of the modem based on a detected trigger event and dynamically adjust the low-latency mode of the modem based on the detect trigger event or the determined operating parameters.

Abstract: The present disclosure relates to methods and apparatus for graphics processing. The apparatus can determine an eye-buffer including one or more bounding boxes associated with rendered content in a frame. The apparatus can also generate an atlas based on the eye-buffer, the atlas including one or more patches associated with the one or more bounding boxes. Additionally, the apparatus can communicate the atlas including the one or more patches. The apparatus can also calculate an amount of user motion associated with the rendered content in the frame. Further, the apparatus can determine a size of each of the one or more bounding boxes based on the calculated amount of user motion. The apparatus can also determine a size and location of each of the one or more patches in the atlas.

Abstract: A method and a system for warping a rendered frame is disclosed. On a host device of a split-rendering system, the method includes generating the rendered frame based on head tracking information of a user. The method also includes identifying a region of interest (ROI) of the rendered frame. The method also includes generating metadata for a warping operation from the ROI. The method further include transmitting the rendered frame and the metadata for a warping operation of the rendered frame. On a client device of the split-rendering system, the method includes transmitting head tracking information of a user by a client device. The method also includes receiving the rendered frame and metadata. The method further includes warping the rendered frame using the metadata and display pose information. The host device and the client device may be combined into an all-in-one head mounted display.

Abstract: The present disclosure relates to methods and apparatus for computer processing. Aspects of the present disclosure can determine at least one of a quality, latency, or capacity of a communication link for communication between a client device and a server. Aspects of the present disclosure can also determine a computational load for an application computation between the client device and the server. Moreover, aspects of the present disclosure can adjust a computational distribution for the application computation between the client device and the server based on at least one of the computational load for the application computation or the at least one of the quality, latency, or capacity of the communication link. Aspects of the present disclosure can also determine a computational capacity of at least one of the client device or the server.

Abstract: An improved wireless split rendering system for displaying Extended Reality (XR) content is discussed. A rendering server and client head-mounted device (HMD) may communicate over a wireless medium, where communication control is given to a server application layer logic. This allows the server to use request pose information from the HMD only when needed for rendering, while preserving bandwidth on the wireless medium for transmitting frames of the rendered content. This reduces contention and improves channel efficiency.

Abstract: The present disclosure relates to methods and apparatus for graphics processing. The apparatus can determine an eye-buffer including one or more bounding boxes associated with rendered content in a frame. The apparatus can also generate an atlas based on the eye-buffer, the atlas including one or more patches associated with the one or more bounding boxes. Additionally, the apparatus can communicate the atlas including the one or more patches. The apparatus can also calculate an amount of user motion associated with the rendered content in the frame. Further, the apparatus can determine a size of each of the one or more bounding boxes based on the calculated amount of user motion. The apparatus can also determine a size and location of each of the one or more patches in the atlas.

Abstract: An improved wireless split rendering system for displaying Extended Reality (XR) content is discussed. A rendering server and client head-mounted device (HMD) may communicate over a wireless medium, where communication control is given to a server application layer logic. This allows the server to use request pose information from the HMD only when needed for rendering, while preserving bandwidth on the wireless medium for transmitting frames of the rendered content. This reduces contention and improves channel efficiency.

Abstract: The present disclosure relates to methods and apparatus for computer processing. Aspects of the present disclosure can determine at least one of a quality, latency, or capacity of a communication link for communication between a client device and a server. Aspects of the present disclosure can also determine a computational load for an application computation between the client device and the server. Moreover, aspects of the present disclosure can adjust a computational distribution for the application computation between the client device and the server based on at least one of the computational load for the application computation or the at least one of the quality, latency, or capacity of the communication link. Aspects of the present disclosure can also determine a computational capacity of at least one of the client device or the server.

Abstract: A server includes a processor configured to determine, based on reception of information from a client device, synchronization parameters between a server clock rate associated with the server and a client clock rate associated with the client device. The processor is also configured to determine a client-side generation timestamp for an audio frame based on the synchronization parameters and based on a server-side generation timestamp for the audio frame. The processor is further configured to generate the audio frame. The audio frame includes the client-side timestamp.

Abstract: Techniques are provided for adaptively controlling an encoding device to allow dynamic insertion intra-coded video content based on feedback information. For example, at least a portion of a video slice of a video frame in a video bitstream can be determined to be missing or corrupted. Feedback information indicating at least the portion of the video slice is missing or corrupted can be sent to an encoding device. An updated video bitstream can be received from the encoding device in response to the feedback information. The updated video bitstream can include at least one intra-coded video slice having a size that is larger than the missing or corrupted video slice. The size of the at least one intra-coded video slice can be determined to cover the missing or corrupted slice and propagated error in the video frame caused by the missing or corrupted slice.

Abstract: Systems, methods, and computer-readable media are provided for distributed tracking and mapping for extended reality experiences. An example method can include computing, at a device, a pose of the device at a future time, the future time being determined based on a communication latency between the device and a mapping backend system; sending, to the mapping backend system, the pose of the device; receiving, from the mapping backend system, a map slice including map points corresponding to a scene associated with the device, the map slice being generated based on the pose of the device, wherein the map points correspond to the predicted pose; and computing an updated pose of the device based on the map slice.