Abstract: Various implementations disclosed herein include devices, systems, and methods that isolates movement of a user from movement of a platform moving with the user. For example, a process may obtain motion sensor data corresponding to an electronic device while the electronic device is located on a moving platform. The motion sensor data includes a measurement representing a combined motion of a user and the moving platform. The process may further extract from the motion sensor data, user motion data representing motion of the user. The process may further allocate the extracted user motion data as input for user motion analysis.

Abstract: Various implementations disclosed herein include devices, systems, and methods that determine a transform relating poses of two mobile electronic devices using range date from a ultra-wideband (UWB) sensor and movement data from captured images. In some implementations, a distance is determined between a first mobile device and a second mobile device based on a UWB signal transmitted between the first mobile device and the second mobile device. In some implementations, first movement data corresponding to movement of the first mobile device and second movement data corresponding to movement of the second mobile device are received. The first movement data based on a camera or a sensor on the first mobile device. The second movement data is based on a camera or a sensor on the second mobile device. Then, a transform is determined that relates a pose of the first mobile device to a pose of the second mobile device based on the distance, the first movement data, and the second movement data.

Abstract: A method for detecting an occlusion of an image sensor includes obtaining, via the image sensor, a plurality of images of a physical environment of the electronic device while the electronic device is moving. The method includes detecting an occlusion of the image sensor based on a repeated occurrence of a static feature across the plurality of images. The method includes modifying a weight associated with the static feature to decrease an impact of the occlusion on a performance of a function.

Abstract: Implementations of the subject technology provide extended reality display devices that can be used on and/or off of a moving platform. Systems and methods are disclosed for separating out the motion of the moving platform from other motions of the device so that virtual content can be displayed without erroneous motions caused by the motion of the moving platform. The subject technology can provide extended reality settings on any suitable moveable platform such as in a car, a watercraft, an aircraft, a train, or any other vehicle.

Abstract: Implementations of the subject technology provide extended reality display devices that can be used on and/or off of a moving platform. Systems and methods are disclosed for separating out the motion of the moving platform from other motions of the device so that virtual content can be displayed without erroneous motions caused by the motion of the moving platform. The subject technology can provide extended reality settings on any suitable moveable platform such as in a car, a watercraft, an aircraft, a train, or any other vehicle.

Abstract: Methods for performing a camera calibration process for outward-facing cameras on devices such as head-mounted display devices are disclosed. Using cameras with overlapping fields of view, relative rotational parameters of the cameras with respect to one another may be determined using an optimization technique such as a two-view bundle adjustment algorithm. A statistical analysis of the relative rotational parameters of the cameras, determined for a plurality of moments in time, may then be made to provide updated relative rotational parameters for recalibration of the cameras. A camera calibration process, such as those disclosed, may not depend on tracking points of interest over multiple moments in time, but rather on a convergence of the relative rotational parameters determined for respective moments in time.

Abstract: Location mapping and navigation user interfaces may be generated and presented via mobile computing devices. A mobile device may detect its location and orientation using internal systems, and may capture image data using a device camera. The mobile device also may retrieve map information from a map server corresponding to the current location of the device. Using the image data captured at the device, the current location data, and the corresponding local map information, the mobile device may determine or update a current orientation reading for the device. Location errors and updated location data also may be determined for the device, and a map user interface may be generated and displayed on the mobile device using the updated device orientation and/or location data.

Abstract: Methods for performing a calibration process for outward-facing cameras on devices such as head-mounted display devices are disclosed. Extrinsic parameters of the camera are first estimated using inputs to the calibration process such as information from an inertial measurement unit and points of interest within images captured by the camera that are tracked with time. Then, extrinsic and intrinsic parameters are concurrently determined in an optimization problem such that updated values of said parameters may be stored and used by applications that run on the device and make use of the camera. The calibration process may be extended to concurrently calibrate multiple cameras based, at least in part, on information from the inertial measurement unit that is local to the camera.

Abstract: Performing a combination localization technique includes determining a target localization parameter, selecting a combination localization technique in accordance with the target localization parameter, including a first localization technique associated with a first power error time profile, and a second localization technique associated with a second power error time profile. A device location is determined using the first localization technique for a first time period in accordance with the first power error time profile, and an updated device location is determined using the second localization technique in response to a triggering condition. A further updated device location is determined using the first localization technique following the first time period based on the updated device location. At least one of a combined energy value and a combined maximum error rate for the combination localization technique satisfies the target localization parameter.

Abstract: Systems, methods, and computer readable media to track and estimate the accuracy of a visual inertial odometry (VIO) system. Various embodiments are able to receive one or more VIO feature measurements associated with a set of image frames from a VIO system and generate a plurality of feature models to estimate health values for the VIO system. The various embodiments determine a plurality of feature health values with the feature models based on the VIO feature measurements and compare the feature health values with ground truth health scores associated with the set of image frames to determine one or more errors. The feature model parameters are updated based on the comparison with the feature health values with ground truth health scores.

Abstract: Location mapping and navigation user interfaces may be generated and presented via mobile computing devices. A mobile device may detect its location and orientation using internal systems, and may capture image data using a device camera. The mobile device also may retrieve map information from a map server corresponding to the current location of the device. Using the image data captured at the device, the current location data, and the corresponding local map information, the mobile device may determine or update a current orientation reading for the device. Location errors and updated location data also may be determined for the device, and a map user interface may be generated and displayed on the mobile device using the updated device orientation and/or location data.

Abstract: Location mapping and navigation user interfaces may be generated and presented via mobile computing devices. A mobile device may detect its location and orientation using internal systems, and may capture image data using a device camera. The mobile device also may retrieve map information from a map server corresponding to the current location of the device. Using the image data captured at the device, the current location data, and the corresponding local map information, the mobile device may determine or update a current orientation reading for the device. Location errors and updated location data also may be determined for the device, and a map user interface may be generated and displayed on the mobile device using the updated device orientation and/or location data.

Abstract: Implementations of the subject technology provide extended reality display devices that can be used on and/or off of a moving platform. Systems and methods are disclosed for separating out the motion of the moving platform from other motions of the device so that virtual content can be displayed without erroneous motions caused by the motion of the moving platform. The subject technology can provide extended reality settings on any suitable moveable platform such as in a car, a watercraft, an aircraft, a train, or any other vehicle.

Abstract: In some implementations, a first electronic device including a first image sensor uses a processor to perform a method. The method involves obtaining a first set of keyframes based on images of a physical environment captured by the first image sensor. The method generates a mapping defining relative locations of keyframes of the first set of keyframes. The method receives a keyframe corresponding to an image of the physical environment captured at a second, different electronic device and localizes the received keyframe to the mapping. The method then receives an anchor from the second electronic device that defines a position of a virtual object relative to the keyframe. The method displays a CGR environment including the virtual object at a location based on the anchor and the mapping.

Abstract: A method of creating a local map includes: receiving, at a mobile electronic data processing apparatus, a request from a server to generate a map of a specified destination; sending to the server a message accepting the request to generate the map responsive to receiving, at a user input of the mobile electronic data processing device, a user command indicating acceptance of the request; generating, using a processor, information related to construction of the map; an transmitting, from the mobile electronic data processing apparatus, the information related to construction of the map.

Abstract: Location mapping and navigation user interfaces may be generated and presented via mobile computing devices. A mobile device may detect its location and orientation using internal systems, and may capture image data using a device camera. The mobile device also may retrieve map information from a map server corresponding to the current location of the device. Using the image data captured at the device, the current location data, and the corresponding local map information, the mobile device may determine or update a current orientation reading for the device. Location errors and updated location data also may be determined for the device, and a map user interface may be generated and displayed on the mobile device using the updated device orientation and/or location data.

Abstract: In some implementations, a first electronic device including a first image sensor uses a processor to perform a method. The method involves obtaining a first set of keyframes based on images of a physical environment captured by the first image sensor. The method generates a mapping defining relative locations of keyframes of the first set of keyframes. The method receives a keyframe corresponding to an image of the physical environment captured at a second, different electronic device and localizes the received keyframe to the mapping. The method then receives an anchor from the second electronic device that defines a position of a virtual object relative to the keyframe. The method displays a CGR environment including the virtual object at a location based on the anchor and the mapping.

Abstract: In some implementations, a first electronic device including a first image sensor uses a processor to perform a method. The method involves obtaining a first set of keyframes based on images of a physical environment captured by the first image sensor. The method generates a mapping defining relative locations of keyframes of the first set of keyframes. The method receives a keyframe corresponding to an image of the physical environment captured at a second, different electronic device and localizes the received keyframe to the mapping. The method then receives an anchor from the second electronic device that defines a position of a virtual object relative to the keyframe. The method displays a CGR environment including the virtual object at a location based on the anchor and the mapping.

Abstract: Systems, methods, and computer readable media to track and estimate the accuracy of a visual inertial odometry (VIO) system. Various embodiments are able to receive one or more VIO feature measurements associated with a set of image frames from a VIO system and generate a plurality of feature models to estimate health values for the VIO system. The various embodiments determine a plurality of feature health values with the feature models based on the VIO feature measurements and compare the feature health values with ground truth health scores associated with the set of image frames to determine one or more errors. The feature model parameters are updated based on the comparison with the feature health values with ground truth health scores.

Abstract: A method includes: receiving sensor measurements from a pre-processing module, in which the sensor measurements include image data and inertial data for a device; transferring, using a processor, information derived from the sensor measurements, from a first set of variables associated with a first window of time to a second set of variables associated with a second window of time, in which the first and second windows consecutively overlap in time; and outputting, to a post-processing module, a state of the device based on the transferred information.