Abstract: Systems, methods, and computer-readable media are provided for distributed tracking and mapping for extended reality experiences. An example method can include computing, at a device, a pose of the device at a future time, the future time being determined based on a communication latency between the device and a mapping backend system; sending, to the mapping backend system, the pose of the device; receiving, from the mapping backend system, a map slice including map points corresponding to a scene associated with the device, the map slice being generated based on the pose of the device, wherein the map points correspond to the predicted pose; and computing an updated pose of the device based on the map slice.

Abstract: The present disclosure relates to methods and apparatus for graphics processing. Aspects of the present disclosure can render at least one frame including display content at a server. Aspects of the present disclosure can also downscale the at least one frame including the display content, where a downscaling rate of one or more portions of the at least one frame is based on a location of each of the one or more portions. Moreover, aspects of the present disclosure can communicate the downscaled at least one frame including the display content to a client device. Aspects of the present disclosure can also encode the downscaled at least one frame including the display content. Further, aspects of the present disclosure can decode the encoded at least one frame including the display content. Aspects of the present disclosure can also upscale the at least one frame including the display content.

Abstract: On the basis of a bitstream (P), an n-channel audio signal (X) is reconstructed by deriving an m-channel core signal (Y) and multichannel coding parameters (a) from the bitstream, where 1?m<n. Also derived from the bitstream are pre-processing dynamic range control, DRC, parameters (DRC2) quantifying an encoder-side dynamic range limiting of the core signal. The n-channel audio signal is obtained by parametric synthesis in accordance with the multichannel coding parameters and while cancelling any encoder-side dynamic range limiting based on the pre-processing DRC parameters. In particular embodiments, the reconstruction further includes use of compensated post-processing DRC parameters quantifying a potential decoder-side dynamic range compression. Cancellation of an encoder-side range limitation and range compression are preferably performed by different decoder-side components. Cancellation and compression may be coordinated by a DRC pre-processor.

Abstract: A method and a system for warping a rendered frame is disclosed. On a host device of a split-rendering system, the method includes generating the rendered frame based on head tracking information of a user. The method also includes identifying a region of interest (ROI) of the rendered frame. The method also includes generating metadata for a warping operation from the ROI. The method further include transmitting the rendered frame and the metadata for a warping operation of the rendered frame. On a client device of the split-rendering system, the method includes transmitting head tracking information of a user by a client device. The method also includes receiving the rendered frame and metadata. The method further includes warping the rendered frame using the metadata and display pose information. The host device and the client device may be combined into an all-in-one head mounted display.

Abstract: Techniques are provided for adaptively controlling an encoding device to allow dynamic insertion intra-coded video content based on feedback information. For example, at least a portion of a video slice of a video frame in a video bitstream can be determined to be missing or corrupted. Feedback information indicating at least the portion of the video slice is missing or corrupted can be sent to an encoding device. An updated video bitstream can be received from the encoding device in response to the feedback information. The updated video bitstream can include at least one intra-coded video slice having a size that is larger than the missing or corrupted video slice. The size of the at least one intra-coded video slice can be determined to cover the missing or corrupted slice and propagated error in the video frame caused by the missing or corrupted slice.

Abstract: Embodiments are directed to a companding method and system for reducing coding noise in an audio codec. A compression process reduces an original dynamic range of an initial audio signal through a compression process that divides the initial audio signal into a plurality of segments using a defined window shape, calculates a wideband gain in the frequency domain using a non-energy based average of frequency domain samples of the initial audio signal, and applies individual gain values to amplify segments of relatively low intensity and attenuate segments of relatively high intensity. The compressed audio signal is then expanded back to the substantially the original dynamic range that applies inverse gain values to amplify segments of relatively high intensity and attenuating segments of relatively low intensity. A QMF filterbank is used to analyze the initial audio signal to obtain a frequency domain representation.

Abstract: A method and a system for warping a rendered frame is disclosed. On a host device of a split-rendering system, the method includes generating the rendered frame based on head tracking information of a user. The method also includes identifying a region of interest (ROI) of the rendered frame. The method also includes generating metadata for a warping operation from the ROI. The method further include transmitting the rendered frame and the metadata for a warping operation of the rendered frame. On a client device of the split-rendering system, the method includes transmitting head tracking information of a user by a client device. The method also includes receiving the rendered frame and metadata. The method further includes warping the rendered frame using the metadata and display pose information. The host device and the client device may be combined into an all-in-one head mounted display.

Abstract: Systems, methods, and computer-readable media are provided for distributed tracking and mapping for extended reality experiences. An example method can include computing, at a device, a pose of the device at a future time, the future time being determined based on a communication latency between the device and a mapping backend system; sending, to the mapping backend system, the pose of the device; receiving, from the mapping backend system, a map slice including map points corresponding to a scene associated with the device, the map slice being generated based on the pose of the device, wherein the map points correspond to the predicted pose; and computing an updated pose of the device based on the map slice.

Abstract: A server includes a processor configured to determine, based on reception of information from a client device, synchronization parameters between a server clock rate associated with the server and a client clock rate associated with the client device. The processor is also configured to determine a client-side generation timestamp for an audio frame based on the synchronization parameters and based on a server-side generation timestamp for the audio frame. The processor is further configured to generate the audio frame. The audio frame includes the client-side timestamp.

Abstract: Aspects of the present disclosure provide a method for compressing a point cloud. The method includes determining a sub-division technique for the volume based on one or more numbers derived from a distribution of points in the volume, dividing the volume into a number of sub-volumes according to the sub-division technique, and determining whether each sub-volume of the sub-volumes is occupied by at least one point. The method includes generating a bit sequence for the volume comprising a control code that is based on the sub-division technique and an occupancy indicator that indicates whether or not each sub-volume is occupied by at least one point.

Abstract: Example techniques are described for image processing. Processing circuitry may warp image content of a previous frame based on pose information of a device when the device requested image content information of the previous frame and pose information of the device when the device requested image content information of a current frame to generate warped image content, and blend image content from the warped image content with image content of the current frame to generate an error concealed frame. A display screen may display image content based on the error concealed frame.

Abstract: A wearable display device is described that is connected to a host device. The wearable display device includes one or more sensors configured to generate eye pose data indicating a user's field of view, one or more displays, and one or more processors. The one or more processors are configured to output a representation of the eye pose data to the host device and extract one or more depth values for a rendered frame from depth data output by the host device. The rendered frame is generated using the eye pose data. The one or more processors are further configured to modify one or more pixel values of the rendered frame using the one or more depth values to generate a warped rendered frame and output, for display at the one or more displays, the warped rendered frame.

Abstract: The present disclosure relates to methods and apparatus for graphics processing. Aspects of the present disclosure can determine at least one split API for a user device and a server. Further, aspects of the present disclosure can establish a communication interface between the user device and the server based on the split API. Additionally, aspects of the present disclosure can communicate between the user device and the server based on the communication interface and the split API. In some aspects, the present disclosure can implement the at least one split API on at least one of the user device or the server. Aspects of the present disclosure can also divide at least one application workload between the user device and the server based on the at least one split API. Moreover, aspects of the present disclosure can encode or decode application information at the user device or the server.

Abstract: On the basis of a bitstream (P), an n-channel audio signal (X) is reconstructed by deriving an m-channel core signal (Y) and multichannel coding parameters (?) from the bitstream, where 1?m<n. Also derived from the bitstream are pre-processing dynamic range control, DRC, parameters (DRC2) quantifying an encoder-side dynamic range limiting of the core signal. The n-channel audio signal is obtained by parametric synthesis in accordance with the multichannel coding parameters and while cancelling any encoder-side dynamic range limiting based on the pre-processing DRC parameters. In particular embodiments, the reconstruction further includes use of compensated post-processing DRC parameters quantifying a potential decoder-side dynamic range compression. Cancellation of an encoder-side range limitation and range compression are preferably performed by different decoder-side components. Cancellation and compression may be coordinated by a DRC pre-processor.

Abstract: Embodiments are directed to a companding method and system for reducing coding noise in an audio codec. A compression process reduces an original dynamic range of an initial audio signal through a compression process that divides the initial audio signal into a plurality of segments using a defined window shape, calculates a wideband gain in the frequency domain using a non-energy based average of frequency domain samples of the initial audio signal, and applies individual gain values to amplify segments of relatively low intensity and attenuate segments of relatively high intensity. The compressed audio signal is then expanded back to the substantially the original dynamic range that applies inverse gain values to amplify segments of relatively high intensity and attenuating segments of relatively low intensity. A QMF filterbank is used to analyze the initial audio signal to obtain a frequency domain representation.

Abstract: The present disclosure relates to methods and apparatus for graphics processing. In some aspects, the apparatus may receive, by a first device from a second device, first position information corresponding to a first orientation of the second device. The apparatus can also generate, by the first device, first graphical content based on the first position information. Further, the apparatus can also generate, by the first device, motion information for warping the first graphical content. The apparatus can also encode, by the first device, the first graphical content. Additionally, the apparatus can provide, by the first device to the second device, the motion information and the encoded first graphical content.

Abstract: On the basis of a bitstream (P), an n-channel audio signal (X) is reconstructed by deriving an m-channel core signal (Y) and multichannel coding parameters (a) from the bitstream, where 1?m<n. Also derived from the bitstream are pre-processing dynamic range control, DRC, parameters (DRC2) quantifying an encoder-side dynamic range limiting of the core signal. The n-channel audio signal is obtained by parametric synthesis in accordance with the multichannel coding parameters and while cancelling any encoder-side dynamic range limiting based on the pre-processing DRC parameters. In particular embodiments, the reconstruction further includes use of compensated post-processing DRC parameters quantifying a potential decoder-side dynamic range compression. Cancellation of an encoder-side range limitation and range compression are preferably performed by different decoder-side components. Cancellation and compression may be coordinated by a DRC pre-processor.

Abstract: On the basis of a bitstream (P), an n-channel audio signal (X) is reconstructed by deriving an m-channel core signal (Y) and multichannel coding parameters (?) from the bitstream, where 1?m<n. Also derived from the bitstream are pre-processing dynamic range control, DRC, parameters (DRC2) quantifying an encoder-side dynamic range limiting of the core signal. The n-channel audio signal is obtained by parametric synthesis in accordance with the multichannel coding parameters and while cancelling any encoder-side dynamic range limiting based on the pre-processing DRC parameters. In particular embodiments, the reconstruction further includes use of compensated post-processing DRC parameters quantifying a potential decoder-side dynamic range compression. Cancellation of an encoder-side range limitation and range compression are preferably performed by different decoder-side components. Cancellation and compression may be coordinated by a DRC pre-processor.

Abstract: A method and a system for warping a rendered frame is disclosed. On a host device of a split-rendering system, the method includes generating the rendered frame based on head tracking information of a user. The method also includes identifying a region of interest (ROI) of the rendered frame. The method also includes generating metadata for a warping operation from the ROI. The method further include transmitting the rendered frame and the metadata for a warping operation of the rendered frame. On a client device of the split-rendering system, the method includes transmitting head tracking information of a user by a client device. The method also includes receiving the rendered frame and metadata. The method further includes warping the rendered frame using the metadata and display pose information. The host device and the client device may be combined into an all-in-one head mounted display.

Abstract: Aspects of the present disclosure provide a method for compressing a point cloud. The method includes determining a sub-division technique for the volume based on one or more numbers derived from a distribution of points in the volume, dividing the volume into a number of sub-volumes according to the sub-division technique, and determining whether each sub-volume of the sub-volumes is occupied by at least one point. The method includes generating a bit sequence for the volume comprising a control code that is based on the sub-division technique and an occupancy indicator that indicates whether or not each sub-volume is occupied by at least one point.