Abstract: In some implementations, a first electronic device including a first image sensor uses a processor to perform a method. The method involves obtaining a first set of keyframes based on images of a physical environment captured by the first image sensor. The method generates a mapping defining relative locations of keyframes of the first set of keyframes. The method receives a keyframe corresponding to an image of the physical environment captured at a second, different electronic device and localizes the received keyframe to the mapping. The method then receives an anchor from the second electronic device that defines a position of a virtual object relative to the keyframe. The method displays a CGR environment including the virtual object at a location based on the anchor and the mapping.

Abstract: Methods and apparatus for glint-assisted gaze tracking in a VR/AR head-mounted display (HMD). Images of a user's eyes captured by gaze tracking cameras may be analyzed to detect glints (reflections on the cornea of light sources that illuminate the user's eyes) and the pupil. The glints are matched to particular ones of the light sources. The glint-light source matches are used to determine the cornea center of the eye, and the pupil center is determined. The optical axis of the eye is reconstructed from the cornea center and the pupil center, and the visual axis is then reconstructed from the optical axis and a 3D model of the user's eye. The point of gaze on the display is then determined based on the visual axis and a 3D model of the HMD.

Abstract: In some implementations, a first electronic device including a first image sensor uses a processor to perform a method. The method involves obtaining a first set of keyframes based on images of a physical environment captured by the first image sensor. The method generates a mapping defining relative locations of keyframes of the first set of keyframes. The method receives a keyframe corresponding to an image of the physical environment captured at a second, different electronic device and localizes the received keyframe to the mapping. The method then receives an anchor from the second electronic device that defines a position of a virtual object relative to the keyframe. The method displays a CGR environment including the virtual object at a location based on the anchor and the mapping.

Abstract: Methods and apparatus for glint-assisted gaze tracking in a VR/AR head-mounted display (HMD). Images of a user's eyes captured by gaze tracking cameras may be analyzed to detect glints (reflections on the cornea of light sources that illuminate the user's eyes) and the pupil. The glints are matched to particular ones of the light sources. The glint-light source matches are used to determine the cornea center of the eye, and the pupil center is determined. The optical axis of the eye is reconstructed from the cornea center and the pupil center, and the visual axis is then reconstructed from the optical axis and a 3D model of the user's eye. The point of gaze on the display is then determined based on the visual axis and a 3D model of the HMD.

Abstract: Systems, methods, and computer readable media to track and estimate the accuracy of a visual inertial odometry (VIO) system. Various embodiments are able to receive one or more VIO feature measurements associated with a set of image frames from a VIO system and generate a plurality of feature models to estimate health values for the VIO system. The various embodiments determine a plurality of feature health values with the feature models based on the VIO feature measurements and compare the feature health values with ground truth health scores associated with the set of image frames to determine one or more errors. The feature model parameters are updated based on the comparison with the feature health values with ground truth health scores.

Abstract: A method performed by an electronic device is described. The method includes determining a haziness confidence level based on multiple modalities. The method also includes determining whether to perform an action based on the haziness confidence level. The method may include performing the action, including performing haziness reduction based on the haziness confidence level.

Abstract: A method includes receiving, from an image capture device, a first image frame of a sequence of image frames. The method also includes estimating, at a processor, a camera pose corresponding to the first image frame by comparing the first image frame to a second image frame. The second image frame precedes the first image frame in the sequence of image frames. The method further includes estimating, at the processor, a refined camera pose corresponding to the first image frame by comparing the first image frame to a keyframe. The keyframe corresponds to a particular image frame that precedes the second image frame in the sequence of image frames.

Abstract: A method includes generating, at an electronic device, a three-dimensional model of an object based on a sequence of images captured by an image capture device associated with the electronic device. The method further includes displaying the three-dimensional model via a display device associated with the electronic device. The method also includes, based on detecting that the three-dimensional model includes an anomaly, presenting, via the display device, one or more selectable options to enable correction of the anomaly.

Abstract: A method includes receiving, from an image capture device, a first image frame of a sequence of image frames. The method also includes estimating, at a processor, a camera pose corresponding to the first image frame by comparing the first image frame to a second image frame. The second image frame precedes the first image frame in the sequence of image frames. The method further includes estimating, at the processor, a refined camera pose corresponding to the first image frame by comparing the first image frame to a keyframe. The keyframe corresponds to a particular image frame that precedes the second image frame in the sequence of image frames.

Abstract: A method includes generating, at an electronic device, a three-dimensional model of an object based on a sequence of images captured by an image capture device associated with the electronic device. The method further includes displaying the three-dimensional model via a display device associated with the electronic device. The method also includes, based on detecting that the three-dimensional model includes an anomaly, presenting, via the display device, one or more selectable options to enable correction of the anomaly.

Abstract: A method performed by an electronic device is described. The method includes determining a haziness confidence level based on multiple modalities. The method also includes determining whether to perform an action based on the haziness confidence level. The method may include performing the action, including performing haziness reduction based on the haziness confidence level.