Abstract: Disclosed are systems, apparatuses, processes, and computer-readable media to capture images with subjects at different depths. A method of processing image data includes obtaining, at an imaging device, a first image of an environment from an image sensor of the imaging device; determining a region of interest of the first image based on features depicted in the first image, wherein the features are associated with the environment; determining a representative luma value associated with the first image based on image data in the region of interest of the first image; determining one or more exposure control parameters based on the representative luma value; and obtaining, at the imaging device, a second image captured based on the one or more exposure control parameters.

Abstract: Systems and techniques are described herein for processing images. The systems and techniques can be implemented by various types of systems, such as by an extended reality (XR) system or device. In some cases, a first processor receives an image of an environment captured by an image sensor, identifies features depicted in the image, and generates descriptors for the features. The first processor sends the descriptors to a second processor, which may be more powerful than the first processor. The second processor receives the descriptors. The second processor associates the plurality of features with a map of the environment based on at least a subset of the plurality of descriptors. For example, the second processor can track at least a subset of the features based on at least a subset of the descriptors and based on feature information from one or more additional images of the environment.

Abstract: Systems and techniques are provided for determining and applying corrected poses in digital content experiences. An example method can include receiving, from one or more sensors associated with an apparatus, inertial measurements and one or more frames of a scene; based on the one or more frames and the inertial measurements, determining, via a first filter, an angular and linear motion of the apparatus and a gravity vector indicating a direction of gravitational force interacting with the apparatus; when a motion of the apparatus is below a threshold, determining, via a second filter, an updated gravity vector indicating a direction of gravitational force interacting with the apparatus; determining, based on the updated gravity vector, parameters for aligning an axis of the scene with a gravity direction in a real-world spatial frame; and aligning, using the parameters, the axis of the scene with the gravity direction in the real-world spatial frame.

Abstract: Methods, devices, and apparatuses are provided to facilitate a positioning of an item of virtual content in an extended reality environment. For example, a placement position for an item of virtual content can be transmitted to one or more of a first device and a second device. The placement position can be based on correlated map data generated based on first map data obtained from the first device and second map data obtained from the second device. In some examples, the first device can transmit the placement position to the second device.

Abstract: Methods, devices, and apparatuses are provided to facilitate a positioning of an item of virtual content in an extended reality environment. For example, a placement position for an item of virtual content can be transmitted to one or more of a first device and a second device. The placement position can be based on correlated map data generated based on first map data obtained from the first device and second map data obtained from the second device. In some examples, the first device can transmit the placement position to the second device.

Abstract: Systems and techniques are described herein for processing frames. The systems and techniques can be implemented by various types of systems, such as by an extended reality (XR) system or device. In some cases, a process can include obtaining feature information associated with a feature in a current frame, wherein the feature information is based on one or more previous frames; determining an estimated pose of the apparatus associated with the current frame; obtaining a distance associated with the feature in the current frame; and determining an estimated scale of the feature in the current frame based on the feature information associated with the feature, the estimated pose, and the distance associated with the feature.

Abstract: Systems and techniques are provided for determining and applying corrected poses in digital content experiences. An example method can include receiving, from one or more sensors associated with an apparatus, inertial measurements and one or more frames of a scene; based on the one or more frames and the inertial measurements, determining, via a first filter, an angular and linear motion of the apparatus and a gravity vector indicating a direction of gravitational force interacting with the apparatus; when a motion of the apparatus is below a threshold, determining, via a second filter, an updated gravity vector indicating a direction of gravitational force interacting with the apparatus; determining, based on the updated gravity vector, parameters for aligning an axis of the scene with a gravity direction in a real-world spatial frame; and aligning, using the parameters, the axis of the scene with the gravity direction in the real-world spatial frame.

Abstract: An example method of capturing a 360° field-of-view image includes capturing, with one or more processors, a first portion of a 360° field-of-view using a first camera module and capturing, with the one or more processors, a second portion of the 360° field-of-view using a second camera module. The method further includes determining, with the one or more processors, a target overlap region based on a disparity in a scene captured by the first portion and the second portion and causing, with the one or more processors, the first camera module, the second camera module, or both the first camera module and the second camera module to reposition to a target camera setup based on the target overlap region. The method further includes capturing, with the one or more processors, the 360° field-of-view image with the first camera and the second camera arranged at the target camera setup.

Abstract: Systems and techniques are described herein for processing images. The systems and techniques can be implemented by various types of systems, such as by an extended reality (XR) system or device. In some cases, a first processor receives an image of an environment captured by an image sensor, identifies features depicted in the image, and generates descriptors for the features. The first processor sends the descriptors to a second processor, which may be more powerful than the first processor. The second processor receives the descriptors. The second processor associates the plurality of features with a map of the environment based on at least a subset of the plurality of descriptors. For example, the second processor can track at least a subset of the features based on at least a subset of the descriptors and based on feature information from one or more additional images of the environment.

Abstract: Methods, devices, and apparatuses are provided to facilitate a positioning of an item of virtual content in an extended reality environment. For example, a placement position for an item of virtual content can be transmitted to one or more of a first device and a second device. The placement position can be based on correlated map data generated based on first map data obtained from the first device and second map data obtained from the second device. In some examples, the first device can transmit the placement position to the second device.

Abstract: Methods, devices, and apparatuses are provided to facilitate a positioning of an item of virtual content in an extended reality environment. For example, a placement position for an item of virtual content can be transmitted to one or more of a first device and a second device. The placement position can be based on correlated map data generated based on first map data obtained from the first device and second map data obtained from the second device. In some examples, the first device can transmit the placement position to the second device.

Abstract: Methods, systems, and devices for deep learning based head motion prediction for extended reality are described. The head pose prediction may involve training one or more layers of a machine learning network based on application data and an estimated head motion range associated with the extended reality system. The network may receive one or more bias corrected inertial measurement unit (IMU) measurements based on a sensor. The network may model a relative head pose of the user as a polynomial of time over a prediction interval based on the bias corrected IMU measurements and the trained one or more layers of the machine learning network. The network may determine a future relative head pose of the user based on the polynomial (e.g., which may be used for virtual object generation, display, etc. within an extended reality system).

Abstract: Systems, methods, and computer-readable media are provided for distributed tracking and mapping for extended reality experiences. An example method can include computing, at a device, a pose of the device at a future time, the future time being determined based on a communication latency between the device and a mapping backend system; sending, to the mapping backend system, the pose of the device; receiving, from the mapping backend system, a map slice including map points corresponding to a scene associated with the device, the map slice being generated based on the pose of the device, wherein the map points correspond to the predicted pose; and computing an updated pose of the device based on the map slice.

Abstract: Methods, devices, and apparatuses are provided to facilitate a positioning of an item of virtual content in an extended reality environment. For example, a placement position for an item of virtual content can be transmitted to one or more of a first device and a second device. The placement position can be based on correlated map data generated based on first map data obtained from the first device and second map data obtained from the second device. In some examples, the first device can transmit the placement position to the second device.

Abstract: Systems, methods, and computer-readable media are provided for distributed tracking and mapping for extended reality experiences. An example method can include computing, at a device, a pose of the device at a future time, the future time being determined based on a communication latency between the device and a mapping backend system; sending, to the mapping backend system, the pose of the device; receiving, from the mapping backend system, a map slice including map points corresponding to a scene associated with the device, the map slice being generated based on the pose of the device, wherein the map points correspond to the predicted pose; and computing an updated pose of the device based on the map slice.

Abstract: Methods, devices, and apparatuses are provided to facilitate a positioning of an item of virtual content in an extended reality environment. For example, a first user may access the extended reality environment through a display of a mobile device, and in some examples, the methods may determine positions and orientations of the first user and a second user within the extended reality environment. The methods may also determine a position for placement of the item of virtual content in the extended reality environment based on the determined positions and orientations of the first user and the second user, and perform operations that insert the item of virtual content into the extended reality environment at the determined placement position.

Abstract: Systems, methods, and computer-readable media are provided for immersive extended reality experiences on mobile platforms. In some examples, a method can include obtaining sensor measurements from one or more sensors on a mobile platform and/or a device associated with a user in the mobile platform, the sensor measurements including motion parameters associated with the mobile platform and the user; identifying features of the mobile platform and an environment outside of the mobile platform; tracking, using the sensor measurements, a first pose of the mobile platform relative to the environment outside of the mobile platform; tracking, using the sensor measurements, a second pose of the user relative to at least one of the features of the mobile platform; and tracking, based on the first pose and the second pose, a third pose of the user relative to at least one of the features of the environment outside of the mobile platform.

Abstract: Systems, methods, and computer-readable media are provided for immersive extended reality experiences on mobile platforms. In some examples, a method can include obtaining sensor measurements from one or more sensors on a mobile platform and/or a device associated with a user in the mobile platform, the sensor measurements including motion parameters associated with the mobile platform and the user; identifying features of the mobile platform and an environment outside of the mobile platform; tracking, using the sensor measurements, a first pose of the mobile platform relative to the environment outside of the mobile platform; tracking, using the sensor measurements, a second pose of the user relative to at least one of the features of the mobile platform; and tracking, based on the first pose and the second pose, a third pose of the user relative to at least one of the features of the environment outside of the mobile platform.

Abstract: Techniques are described for addressing rolling shutter delay and in some cases rolling shutter delay and stabilization. Processing circuits may receive image content in overlapping portions of images, and may adjust the image content until there is overlap in the overlapping portions. Processing circuits may also receive information of deviation of the device from a common reference. Based on the overlapping image content, the deviation of the device from the common reference, and image content in non-overlapping portions, the processing circuits may determine mapping of coordinates to a rectangular mesh for generating an equirectangular image.

Abstract: A method for stitching images by an electronic device is described. The method includes obtaining at least two images. The method also includes selecting a stitching scheme from a set of stitching schemes based on one or more content measures of the at least two images. The set of stitching schemes includes a first stitching scheme, a second stitching scheme, and a third stitching scheme. The method further includes stitching the at least two images based on a selected stitching scheme.