Abstract: A system determines an accuracy of a set containing a plurality of sensor location measurements of a client device and generating a set of sensor location measurements labeled with an associated accuracy estimate, is presented. The system receives a plurality of sensor location measurements of the client device generated by a location sensor. The system may determine the accuracy of a sensor location measurement by comparing the sensor location measurement to a reference location measurement, such as VIO location measurements computed using VIO data. The system computes a first set of location translations for the set of sensor location measurements and a second set of location translations for the set of VIO location measurements. The system may calculate a measurement difference between each corresponding pair of location translations from the first and second set, identify measurement differences that exceed a threshold, and label the corresponding sensor location measurement as inaccurate.

Abstract: The present disclosure describes approaches to camera re-localization that improve the accuracy of re-localization determinations by performing simulated consistency checks for three-dimensional maps. Client devices associated with users of a location-based application transmit image scans to a game server, which divides the received scan data into mapping sets used to generate 3D maps of environments and validation sets used to test the accuracy of the maps. To perform the testing, the game server identifies query scans in the validation set having GPS coordinates within a threshold distance of the mapped location and uses the 3D map of the environment to generate a pose estimate for each frame. The results of the localization queries are analyzed by comparing differences between the localization pose estimates and differences between the poses of independent pairs of frames in the query scan to evaluate the accuracy of the 3D map.

Abstract: A system determines an accuracy of a set containing a plurality of sensor location measurements of a client device and generating a set of sensor location measurements labeled with an associated accuracy estimate, is presented. The system receives a plurality of sensor location measurements of the client device generated by a location sensor. The system may determine the accuracy of a sensor location measurement by comparing the sensor location measurement to a reference location measurement, such as VIO location measurements computed using VIO data. The system computes a first set of location translations for the set of sensor location measurements and a second set of location translations for the set of VIO location measurements. The system may calculate a measurement difference between each corresponding pair of location translations from the first and second set, identify measurement differences that exceed a threshold, and label the corresponding sensor location measurement as inaccurate.

Abstract: An online system uses a pose prior model and a pose objective function to estimate the pose of a client device. A pose prior model is a model for prior information known about client devices and their poses without reference to a particular client device and its pose data. The online system receives pose data from a client device and computes an estimated pose for the client device based on the received pose data, the pose prior model, and a generated initial candidate pose for the client device. The online system uses these as inputs to a pose objective function and optimizes the pose objective function to estimate a pose for the client device. The online system transmits this estimated pose to the client device, and may use the estimated pose as the pose for the client device for the purposes of delivering content to the user.

Abstract: The present disclosure describes a method for estimating a pose of a client device using a magnetic field vector map. The method includes receiving a plurality of magnetic field measurements from a plurality of client devices, each magnetic field measurement describing a magnetic field vector at a geographic location. The method further includes grouping the magnetic field measurements into one or more region groups, aggregating the magnetic field measurements in each region group to generate a probability distribution of magnetic field vectors associated with the geographic region, determining a magnetic field vector within each geographic region, and generating a magnetic field vector map. Based on the magnetic field vector map, the method may include estimating a pose of a client device based on a user location of the client device and received magnetic field vector from the client device.

Abstract: A client device estimates an error in a heading component of a magnetometer measurement based on an error in a vertical component of the magnetometer measurement. The client device receives a three-dimensional measurement from a magnetometer including a heading and a vertical component. The client device identifies an expected vertical component based on a position of the client device and a magnetic map. The client device generates an error estimate based on the vertical component of the three-dimensional measurement and the expected vertical component. The client device compares the error to an error threshold and, in response to the error estimate being less than an error threshold, determines a pose of the client device based on the heading. The client device displays augmented reality content to the user using the pose.

Abstract: An online system uses a pose prior model and a pose objective function to estimate the pose of a client device. A pose prior model is a model for prior information known about client devices and their poses without reference to a particular client device and its pose data. The online system receives pose data from a client device and computes an estimated pose for the client device based on the received pose data, the pose prior model, and a generated initial candidate pose for the client device. The online system uses these as inputs to a pose objective function and optimizes the pose objective function to estimate a pose for the client device. The online system transmits this estimated pose to the client device, and may use the estimated pose as the pose for the client device for the purposes of delivering content to the user.

Abstract: The present disclosure describes a method for estimating a pose of a client device using a magnetic field vector map. The method includes receiving a plurality of magnetic field measurements from a plurality of client devices, each magnetic field measurement describing a magnetic field vector at a geographic location. The method further includes grouping the magnetic field measurements into one or more region groups, aggregating the magnetic field measurements in each region group to generate a probability distribution of magnetic field vectors associated with the geographic region, determining a magnetic field vector within each geographic region, and generating a magnetic field vector map. Based on the magnetic field vector map, the method may include estimating a pose of a client device based on a user location of the client device and received magnetic field vector from the client device.

Abstract: The present disclosure describes approaches to camera re-localization that improve the speed and accuracy with which pose estimates are generated by fusing output of a computer vision algorithm with data from a prior model of a geographic area in which a user is located. For each candidate pose estimate output by the algorithm, a game server maps the estimate to a position on the prior model (e.g., a specific cell on a heatmap-style histogram) and retrieves a probability corresponding to the mapped position. A data fusion module fuses, for each candidate pose estimate, a confidence score generated by the computer vision algorithm with the location probability from the prior model to generate an updated confidence score. If an updated confidence score meets or exceeds a score threshold, a re-localization module initiates a location-based application (e.g., a parallel reality game) based on the associated candidate pose estimate.

Abstract: The present disclosure describes a method for calibrating a magnetic sensor of a client device. The method may include receiving a set of magnetic field measurements, each of which includes a device location, an orientation of the client device, and an observed magnetic field vector measured by the magnetic sensor. The method may include computing a device correction vector for the client device based on the set of magnetic field measurements. For each magnetic field measurement, the method includes determining a world magnetic field vector at the device location of the magnetic field measurement, computing an expected measured magnetic field vector at the device location, accessing an estimated device correction vector for the client device, computing an expected adjusted vector for the client device, comparing the observed magnetic field vector associated with the magnetic field measurement and the expected adjusted vector, and computing the device correction vector based on the comparison.

Abstract: A client device, or an online system, uses structure lines that are generated based on an image to predict a pose of the client device. Structure lines are lines that delineate structures in the physical world depicted in the image. The client device also uses a structure model to predict its pose. A structure model is a model that represents structures in the physical world within an area. The client device predicts its pose based on the structure model and the structure lines by applying an objective function. The client device may then iteratively update the estimated pose and score the updated poses until the client device identifies an estimated pose at which the structure lines sufficiently fit the structure model.

Abstract: The disclosure describes a method for calibrating a magnetic sensor of a client device. The method may include receiving a set of magnetic field measurements, each of which includes a device location, an orientation of the client device, and an observed magnetic field vector measured by the magnetic sensor. The method may include computing a device correction vector for the client device based on the set of magnetic field measurements. For each magnetic field measurement, the method includes determining a world magnetic field vector at the device location of the magnetic field measurement, computing an expected measured magnetic field vector at the device location, accessing an estimated device correction vector for the client device, computing an expected adjusted vector for the client device, comparing the observed magnetic field vector associated with the magnetic field measurement and the expected adjusted vector, and computing the device correction vector based on the comparison.

Abstract: A method, system, and computer-readable storage medium are disclosed for displaying virtual elements (e.g., AR content) in a physical environment by a client device using a virtual camera pose that is different from the pose of the physical camera used to capture images of the physical environment. The client device uses a three-dimensional (3D) map (e.g., a topographical mesh) of the physical environment to determine a pose (a position and orientation) of the camera of the client device. The 3D map can include geometry, colors, textures, or any other suitable information describing the physical environment.

Abstract: The disclosure describes a method for calibrating a magnetic sensor of a client device. The method may include receiving a set of magnetic field measurements, each of which includes a device location, an orientation of the client device, and an observed magnetic field vector measured by the magnetic sensor. The method may include computing a device correction vector for the client device based on the set of magnetic field measurements. For each magnetic field measurement, the method includes determining a world magnetic field vector at the device location of the magnetic field measurement, computing an expected measured magnetic field vector at the device location, accessing an estimated device correction vector for the client device, computing an expected adjusted vector for the client device, comparing the observed magnetic field vector associated with the magnetic field measurement and the expected adjusted vector, and computing the device correction vector based on the comparison.

Abstract: A method of determining a position for a virtual object is described. A location of a client device is determined, and, based on the determined location a set of map segments is retrieved. A virtual object is determined to be displayed on the client device. Relation vectors between the virtual object and each map segment of the retrieved set of map segments are obtained. Each relation vector is weighted based on object parameters of the virtual object. A position to display the virtual object is determined based on the weighted relation vectors. The virtual object is provided for display on the client device as the determined position.

Abstract: An augmented reality (“AR”) device applies smooth correction methods to correct the location of the virtual objects presented to a user. The AR device may apply an angular threshold to determine whether a virtual object can be moved from an original location to a target location. An angular threshold is a maximum angle by which a line from the AR device to the virtual object can change within a timestep. Similarly, the AR device may apply a motion threshold, which is a maximum on the distance that a virtual object's location can be corrected based on the motion of the virtual object. Furthermore, the AR device may apply a pixel threshold to the correction of the virtual object's location. A pixel threshold is a maximum on the distance that a pixel projection of the virtual object can change based on the virtual object's change in location.

Abstract: A method of determining a position for a virtual object is described. A location of a client device is determined, and, based on the determined location a set of map segments is retrieved. A virtual object is determined to be displayed on the client device. Relation vectors between the virtual object and each map segment of the retrieved set of map segments are obtained. Each relation vector is weighted based on object parameters of the virtual object. A position to display the virtual object is determined based on the weighted relation vectors. The virtual object is provided for display on the client device as the determined position.

Abstract: A system determines an accuracy of a set containing a plurality of sensor location measurements of a client device and generating a set of sensor location measurements labeled with an associated accuracy estimate, is presented. The system receives a plurality of sensor location measurements of the client device generated by a location sensor. The system may determine the accuracy of a sensor location measurement by comparing the sensor location measurement to a reference location measurement, such as VIO location measurements computed using VIO data. The system computes a first set of location translations for the set of sensor location measurements and a second set of location translations for the set of VIO location measurements. The system may calculate a measurement difference between each corresponding pair of location translations from the first and second set, identify measurement differences that exceed a threshold, and label the corresponding sensor location measurement as inaccurate.

Abstract: A system determines an accuracy of a set containing a plurality of sensor location measurements of a client device and generating a set of sensor location measurements labeled with an associated accuracy estimate, is presented. The system receives a plurality of sensor location measurements of the client device generated by a location sensor. The system may determine the accuracy of a sensor location measurement by comparing the sensor location measurement to a reference location measurement, such as VIO location measurements computed using VIO data. The system computes a first set of location translations for the set of sensor location measurements and a second set of location translations for the set of VIO location measurements. The system may calculate a measurement difference between each corresponding pair of location translations from the first and second set, identify measurement differences that exceed a threshold, and label the corresponding sensor location measurement as inaccurate.

Abstract: An augmented reality (“AR”) device applies smooth correction methods to correct the location of the virtual objects presented to a user. The AR device may apply an angular threshold to determine whether a virtual object can be moved from an original location to a target location. An angular threshold is a maximum angle by which a line from the AR device to the virtual object can change within a timestep. Similarly, the AR device may apply a motion threshold, which is a maximum on the distance that a virtual object's location can be corrected based on the motion of the virtual object. Furthermore, the AR device may apply a pixel threshold to the correction of the virtual object's location. A pixel threshold is a maximum on the distance that a pixel projection of the virtual object can change based on the virtual object's change in location.