Abstract: Systems and techniques are provided for imaging. For example, a process can include determining a direction of gaze of a user toward one or more displays, wherein the direction of gaze is based on image data obtained using one or more cameras. The process can include determining a region of the one or more displays corresponding to the direction of gaze of the user. The process can include generating one or more graphical user interface (GUI) control actions indicative of a respective configuration of a GUI associated with the one or more displays, wherein the one or more GUI control actions are based on the determined region. The process can include outputting, using the one or more displays and based on the one or more GUI control actions, the respective configuration of the GUI.

Abstract: Systems and techniques are provided for imaging. For example, a process can include determining estimated transmission power information associated with a channel between the apparatus and a computing device. The process can include obtaining one or more extended reality (XR) application parameters associated with an XR application corresponding to one or more of the apparatus or the computing device. The process can include determining, based on the estimated transmission power information and the one or more XR application parameters, a split perception configuration for balancing a perception workload of the XR application between the apparatus and the computing device.

Abstract: Systems and techniques are described herein for displaying images. For instance, a method for displaying images is provided. The method may include obtaining image data for display at a display of a head-mounted device (HMD); determining a content-based region of interest (ROI) within the image data based on content of the image data; determining a gaze-based ROI of the image data based on gaze information; displaying, at the display, content-based-ROI image data of the content-based ROI at a first resolution; displaying, at the display, gaze-based ROI image data of the gaze-based ROI at a second resolution; and displaying, at the display, peripheral image data outside the content-based ROI and outside the ROI at a third resolution.

Abstract: Systems and techniques are described herein for generating foveated image data. For instance, an apparatus for generating foveated image data is provided. The apparatus may include a memory; and a first processor communicatively connected directly to an image sensor and communicatively connected to the memory, the first processor configured to: determine an orientation of at least one eye of a user; store an indication of the orientation in the memory; and provide the indication of the orientation to the image sensor wherein the image sensor is configured to capture a foveated image based on the indication of the orientation, the foveated image comprising a first portion and a second portion, wherein the first portion is based on the indication of the orientation and has a first resolution, and wherein the second portion has a second resolution that is lower than the first resolution.

Abstract: Systems and techniques are described herein for displaying images. For instance, a method for displaying images is provided. The method may include obtaining image data for display at a display of a head-mounted device (HMD); determining a content-based region of interest (ROI) within the image data based on content of the image data; determining a gaze-based ROI of the image data based on gaze information; displaying, at the display, content-based-ROI image data of the content-based ROI at a first resolution; displaying, at the display, gaze-based ROI image data of the gaze-based ROI at a second resolution; and displaying, at the display, peripheral image data outside the content-based ROI and outside the ROI at a third resolution.

Abstract: This disclosure provides systems, devices, apparatus, and methods, including computer programs encoded on storage media, for sparse composition. A processor obtains at least one frame including at least one component, where the at least one component includes (1) data of the at least one component and (2) metadata of the at least one component, and where the metadata includes enclosing boundary information of the at least one component. The processor predicts, based on the enclosing boundary information and pose information, a set of boundary coordinates of the at least one component. The processor adjusts, based on a display order, the predicted set of boundary coordinates of the at least one component. The processor determines whether the predicted set of boundary coordinates overlaps with at least one component boundary associated with a composition engine. The processor processes, based on the determination, the data.

Abstract: Disclosed are systems, apparatuses, processes, and computer-readable media for generating one or more images. For example, a method includes generating a first frame from sensor data obtained from an image sensor, wherein the first frame has a first resolution; generating a first portion of the first frame from the sensor data based on information corresponding to a first region of interest (ROI); generating a second frame from the sensor data, wherein the second frame has a second resolution that is less than the first resolution; and outputting at least one of the first frame, the first portion of the first frame, or the second frame based on a foveation enable signal.

Abstract: Systems and techniques are described herein for generating foveated image data. For instance, an apparatus for generating foveated image data is provided. The apparatus may include a memory; and a first processor communicatively connected directly to an image sensor and communicatively connected to the memory, the first processor configured to: determine an orientation of at least one eye of a user; store an indication of the orientation in the memory; and provide the indication of the orientation to the image sensor wherein the image sensor is configured to capture a foveated image based on the indication of the orientation, the foveated image comprising a first portion and a second portion, wherein the first portion is based on the indication of the orientation and has a first resolution, and wherein the second portion has a second resolution that is lower than the first resolution.

Abstract: Systems and techniques are provided for foveated sensing. For example, a process can include obtaining, from a first image sensor, a first image of a scene. The first image includes a full region including a fovea region and a peripheral region. The first image sensor is associated with a first spatial resolution. The process can include obtaining, from a second image sensor, a second image of the scene. The second image includes the fovea region. The second image sensor is associated with a second spatial resolution different from the first spatial resolution. The process can include generating a combined image. The combined image includes a first plurality of pixels associated with the fovea region of the scene and a second plurality of pixels associated with associated with a peripheral region of the scene. The second plurality of pixels is generated from pixel values of the first image.

Abstract: Systems and techniques are described herein for processing data. For instance, an apparatus for processing data is provided. The apparatus may include an image signal processor (ISP) configured to: receive image data and an indication of a region of interest (ROI) from an image sensor; determine image-processing settings for processing the image data based on the ROI; and process the image data based on the image-processing settings.

Abstract: Systems and techniques are described for performing foveated sensing. In some aspects, a method (e.g., implemented by an image sensor) can include capturing sensor data for a frame associated with a scene, obtaining information corresponding to a region of interest (ROI) associated with the scene, generating a first portion (having a first resolution) of the frame corresponding to the ROI, generating a second portion of the frame having a second resolution, and outputting (e.g., to an image signal processor (ISP)) the first portion and the second portion. In some aspects, a method (e.g., implemented by an ISP) can receive, from an image sensor, sensor data for a frame associated with a scene, generating a first version (having a first resolution) of the frame based on an ROI associated with the scene, and generating a second version of the frame having a second resolution (lower than the first resolution).

Abstract: Methods, devices, and apparatuses are provided to facilitate a positioning of an item of virtual content in an extended reality environment. For example, a placement position for an item of virtual content can be transmitted to one or more of a first device and a second device. The placement position can be based on correlated map data generated based on first map data obtained from the first device and second map data obtained from the second device. In some examples, the first device can transmit the placement position to the second device.

Abstract: Methods, devices, and apparatuses are provided to facilitate a positioning of an item of virtual content in an extended reality environment. For example, a placement position for an item of virtual content can be transmitted to one or more of a first device and a second device. The placement position can be based on correlated map data generated based on first map data obtained from the first device and second map data obtained from the second device. In some examples, the first device can transmit the placement position to the second device.

Abstract: Certain aspects of the present disclosure relate to a method for compressed sensing (CS). The CS is a signal processing concept wherein significantly fewer sensor measurements than that suggested by Shannon/Nyquist sampling theorem can be used to recover signals with arbitrarily fine resolution. In this disclosure, the CS framework is applied for sensor signal processing in order to support low power robust sensors and reliable communication in Body Area Networks (BANs) for healthcare and fitness applications.

Abstract: Methods, devices, and apparatuses are provided to facilitate a positioning of an item of virtual content in an extended reality environment. For example, a placement position for an item of virtual content can be transmitted to one or more of a first device and a second device. The placement position can be based on correlated map data generated based on first map data obtained from the first device and second map data obtained from the second device. In some examples, the first device can transmit the placement position to the second device.

Abstract: Methods, devices, and apparatuses are provided to facilitate a positioning of an item of virtual content in an extended reality environment. For example, a placement position for an item of virtual content can be transmitted to one or more of a first device and a second device. The placement position can be based on correlated map data generated based on first map data obtained from the first device and second map data obtained from the second device. In some examples, the first device can transmit the placement position to the second device.

Abstract: Techniques and systems are provided for light estimation. In some examples, a system receives a plurality of frames associated with a scene. The plurality of frames includes a first frame and a second frame occurring after the first frame. The system determines, based on image data of the first frame, a first light estimate associated with the scene. The system also determines, based on image data of the second frame, a second light estimate associated with the scene. The system further generates an aggregate light estimate associated with the scene based on combining the second light estimate with at least the first light estimate.

Abstract: Techniques and systems are provided for determining one or more camera settings. For example, an indication of a selection of an image quality metric for adjustment can be received, and a target image quality metric value for the selected image quality metric can be determined. A data point can be determined from a plurality of data points. The data point corresponds to a camera setting having an image quality metric value closest to the target image quality metric value.

Abstract: Systems and techniques are described for data encoding using a machine learning approach to generate a distortion prediction {circumflex over (D)} and a predicted bit rate {circumflex over (R)}, and to use {circumflex over (D)} and {circumflex over (R)} to perform rate-distortion optimization (RDO). For example, a video encoder can generate the distortion prediction {circumflex over (D)} and the bit rate residual prediction based on outputs of the one or more neural networks in response to the one or more neural networks receiving a residual portion of a block of a video frame as input. The video encoder can determine bit rate metadata prediction based on metadata associated with a mode of compression, and determine {circumflex over (R)} to be the sum of and . The video encoder can determine a rate-distortion cost prediction ? as a function of {circumflex over (D)} and {circumflex over (R)}, and can determine a prediction mode for compressing the block based on ?.

Abstract: An example method of capturing a 360° field-of-view image includes capturing, with one or more processors, a first portion of a 360° field-of-view using a first camera module and capturing, with the one or more processors, a second portion of the 360° field-of-view using a second camera module. The method further includes determining, with the one or more processors, a target overlap region based on a disparity in a scene captured by the first portion and the second portion and causing, with the one or more processors, the first camera module, the second camera module, or both the first camera module and the second camera module to reposition to a target camera setup based on the target overlap region. The method further includes capturing, with the one or more processors, the 360° field-of-view image with the first camera and the second camera arranged at the target camera setup.