Abstract: Systems and techniques are provided for processing one or more frames. For example, a process can include obtaining a first plurality of frames associated with a first settings domain from an image capture system, wherein the first plurality of frames is captured prior to obtaining a capture input. The process can include obtaining a reference frame associated with a second settings domain from the image capture system, wherein the reference frame is captured proximate to obtaining the capture input. The process can include obtaining a second plurality of frames associated with the second settings domain from the image capture system, wherein the second plurality of frames is captured after the reference frame. The process can include, based on the reference frame, transforming at least a portion of the first plurality of frames to generate a transformed plurality of frames associated with the second settings domain.

Abstract: Embodiments include systems and methods that may be performed by a processor of a computing device. Embodiments may be applied for keypoint detection in an image. In embodiments, the processor of the computing device may apply to an image a first-stage neural network to define and output a plurality of regions, apply to each of the plurality of regions a respective second-stage neural network to output a plurality of keypoints in each of the plurality of regions, and apply to the plurality of keypoints a third-stage neural network to determine a correction for each of the plurality of keypoints to provide corrected keypoints.

Abstract: Methods, systems, and apparatuses are provided to automatically detect objects within images. For example, an image capture device may capture an image, and may apply a trained neural network to the image to generate an object value and a class value for each of a plurality of portions of the image. Further, the image capture device may determine, for each of the plurality of image portions, a confidence value based on the object value and the class value corresponding to each image portion. The image capture device may also detect an object within at least one image portion based on the confidence values. Further, the image capture device may output a bounding box corresponding to the at least one image portion. The bounding box defines an area of the image that includes one or more objects.

Abstract: Methods, systems, and apparatuses are provided to automatically detect objects within images. For example, an image capture device may capture an image, and may apply a trained neural network to the image to generate an object value and a class value for each of a plurality of portions of the image. Further, the image capture device may determine, for each of the plurality of image portions, a confidence value based on the object value and the class value corresponding to each image portion. The image capture device may also detect an object within at least one image portion based on the confidence values. Further, the image capture device may output a bounding box corresponding to the at least one image portion. The bounding box defines an area of the image that includes one or more objects.

Abstract: Systems and techniques are described herein for enabling a multi-user extended reality (XR) experience. In one illustrative example, a user device can receive, associated with a user from a host device, a message comprising a prompt to join an XR room hosted by the host device. The user device can connect to the host device based on the message. The user device can can obtain images of a three-dimensional (3D) scene of a physical environment and can transmit the images to the host device. The user device can receive synthetic content from the host device. A virtual representation of the user can be localized with respect to the XR room based on features in the images matched with features of the 3D scene of the physical environment. The user device can render the synthetic content for the XR room based on a pose of the apparatus.

Abstract: Systems and techniques are described for performing scalable voxel block selection. For example, a computing device can determine a fixed block configuration based on a storage size limitation. The computing device can select a plurality of blocks of the scene based on the fixed block configuration. The computing device can convert indices of the plurality of blocks associated with the fixed block configuration to indices of a plurality of blocks associated with a particular block configuration (of a plurality of block configurations) that corresponds to a particular 3DR application. The particular block configuration is different from the fixed block configuration.

Abstract: Systems, methods, and non-transitory media are provided for predictive camera initialization. An example method can include obtain, from a first image capture device, image data depicting a scene; classify the scene based on the image data; based on the classification of the scene, predict a camera use event; and based on the predicted camera use event, adjust a power mode of at least one of the first image capture device and a second image capture device.

Abstract: Systems, methods, and non-transitory media are provided for predictive camera initialization. An example method can include obtain, from a first image capture device, image data depicting a scene; classify the scene based on the image data; based on the classification of the scene, predict a camera use event; and based on the predicted camera use event, adjust a power mode of at least one of the first image capture device and a second image capture device.

Abstract: Systems and techniques are provided for processing one or more frames. For example, a process can include obtaining a first plurality of frames associated with a first settings domain from an image capture system, wherein the first plurality of frames is captured prior to obtaining a capture input. The process can include obtaining a reference frame associated with a second settings domain from the image capture system, wherein the reference frame is captured proximate to obtaining the capture input. The process can include obtaining a second plurality of frames associated with the second settings domain from the image capture system, wherein the second plurality of frames is captured after the reference frame. The process can include, based on the reference frame, transforming at least a portion of the first plurality of frames to generate a transformed plurality of frames associated with the second settings domain.

Abstract: Systems and techniques are provided for processing one or more frames. For example, a process can include obtaining a first plurality of frames associated with a first settings domain from an image capture system, wherein the first plurality of frames is captured prior to obtaining a capture input. The process can include obtaining a reference frame associated with a second settings domain from the image capture system, wherein the reference frame is captured proximate to obtaining the capture input. The process can include obtaining a second plurality of frames associated with the second settings domain from the image capture system, wherein the second plurality of frames is captured after the reference frame. The process can include, based on the reference frame, transforming at least a portion of the first plurality of frames to generate a transformed plurality of frames associated with the second settings domain.

Abstract: Systems and techniques are provided for registering three-dimensional (3D) images to deformable models.

Abstract: Systems and techniques are provided for modeling three-dimensional (3D) meshes using images. An example method can include receiving, via a neural network system, an image of a target and metadata associated with the image and/or a device that captured the image; determining, based on the image and metadata, first 3D mesh parameters of a first 3D mesh of the target, the first 3D mesh parameters and first 3D mesh corresponding to a first reference frame associated with the image and/or the device; and determining, based on the first 3D mesh parameters, second 3D mesh parameters for a second 3D mesh of the target, the second 3D mesh parameters and second 3D mesh corresponding to a second reference frame, the second reference frame including a 3D coordinate system of a real-world scene where the target is located.

Abstract: A method is presented. The method includes determining a number of landmarks in an image comprising multiple pixels. The method also includes determining a number of channels for the image based on a function of the number of landmarks. The method further includes determining, for each one of the number of channels, a confidence of each pixel of the multiple pixels corresponding to a landmark. The method still further includes identifying the landmark in the image based on the confidence.

Abstract: Methods, systems, and apparatuses are provided to automatically detect objects within images. For example, an image capture device may capture an image, and may apply a trained neural network to the image to generate an object value and a class value for each of a plurality of portions of the image. Further, the image capture device may determine, for each of the plurality of image portions, a confidence value based on the object value and the class value corresponding to each image portion. The image capture device may also detect an object within at least one image portion based on the confidence values. Further, the image capture device may output a bounding box corresponding to the at least one image portion. The bounding box defines an area of the image that includes one or more objects.

Abstract: Systems and techniques are provided for processing one or more frames. For example, a process can include obtaining a first plurality of frames associated with a first settings domain from an image capture system, wherein the first plurality of frames is captured prior to obtaining a capture input. The process can include obtaining a reference frame associated with a second settings domain from the image capture system, wherein the reference frame is captured proximate to obtaining the capture input. The process can include obtaining a second plurality of frames associated with the second settings domain from the image capture system, wherein the second plurality of frames is captured after the reference frame. The process can include, based on the reference frame, transforming at least a portion of the first plurality of frames to generate a transformed plurality of frames associated with the second settings domain.

Abstract: Systems, methods, and non-transitory media are provided for predictive camera initialization. An example method can include obtain, from a first image capture device, image data depicting a scene; classify the scene based on the image data; based on the classification of the scene, predict a camera use event; and based on the predicted camera use event, adjust a power mode of at least one of the first image capture device and a second image capture device.

Abstract: Systems and techniques are provided for modeling three-dimensional (3D) meshes using images. An example method can include receiving, via a neural network system, an image of a target and metadata associated with the image and/or a device that captured the image; determining, based on the image and metadata, first 3D mesh parameters of a first 3D mesh of the target, the first 3D mesh parameters and first 3D mesh corresponding to a first reference frame associated with the image and/or the device; and determining, based on the first 3D mesh parameters, second 3D mesh parameters for a second 3D mesh of the target, the second 3D mesh parameters and second 3D mesh corresponding to a second reference frame, the second reference frame including a 3D coordinate system of a real-world scene where the target is located.

Abstract: Systems and techniques are provided for registering three-dimensional (3D) images to deformable models.

Abstract: Systems, methods, and non-transitory media are provided for capturing a region of interest (ROI) with a multi-camera system. An example method can include initializing image sensors of an electronic device, each image sensor being initialized in a lower-power mode having a lower power consumption than a higher-power mode supported by one or more of the image sensors; obtaining images captured by the image sensors in the lower-power mode; determining, based on the images, that an ROI in a scene is within a field-of-view (FOV) of a first image sensor from the image sensors; based on determining that the ROI is within the FOV of the first image sensor, decreasing the lower-power mode of one or more second image sensors to a power-off mode or an additional lower-power mode having a lower power consumption than the lower-power mode; and capturing, using the first image sensor, an image of the ROI.

Abstract: Embodiments include systems and methods that may be performed by a processor of a computing device. Embodiments may be applied for keypoint detection in an image. In embodiments, the processor of the computing device may apply to an image a first-stage neural network to define and output a plurality of regions, apply to each of the plurality of regions a respective second-stage neural network to output a plurality of keypoints in each of the plurality of regions, and apply to the plurality of keypoints a third-stage neural network to determine a correction for each of the plurality of keypoints to provide corrected keypoints.