Abstract: A game server provides game data of a location-based game to a plurality of client devices based on a first set of one or more game designs. First activity data associated with the location-based game is received from the plurality of client devices based on the first set of one or more game designs. The game server determines whether the first activity data from the plurality of client devices meets a predetermined starting condition associated with a local virtual event. The local virtual event is started for the plurality of client devices in response to determining that the first activity data meets the predetermined starting condition. The game server provides game data based on the local virtual event to the plurality of client devices. The virtual event has a second game design that is different from the first set of one or more game designs.

Abstract: A parallel-reality application enables users to tag virtual elements in their proximity for later interaction. A geographic location of a user is received and used to identify a virtual location, in a virtual world, that maps to the geographic location of the user. A region of the virtual world is identified based on the identified virtual location and one or more virtual elements within the region of the virtual world are selected. An identifier of the one or more virtual elements is stored in conjunction with an identifier of the user and a list of tagged virtual elements is provided for display to the user. At a later time, the user interacts with a selected one of the tagged virtual elements.

Abstract: A system and method for determining a location of a client device is described herein. In particular, a client device receives images captured by a camera at the client device. The client device identifies features in the images. The features may be line junctions, lines, curves, or any other features found in images. The client device retrieves a 3D map of the environment from the map database and compares the identified features to the 3D map of the environment, which includes map features such as map line junctions, map lines, map curves, and the like. The client device identifies a correspondence between the features identified from the images and the map features and determines a location of the client device in the real world based on the correspondence. The client device may display visual data representing a location in a virtual world corresponding to the location in the real world.

Abstract: A message router partially decodes messages to determine how to route the messages. The message router receives a message and identifies a field of the message as a candidate field for including an envelope identifier that indicates an envelope type of the message. The envelope type of the message indicates where information, such as where to route the message, is stored within the message. The message router attempts to decode the candidate field to determine whether the candidate field includes the envelope identifier, and responsive to the candidate filed including the envelope identifier, the message router determines the envelope type of the message. The message router routes the message according to the envelope type.

Abstract: The disclosure describes a method for calibrating a magnetic sensor of a client device. The method may include receiving a set of magnetic field measurements, each of which includes a device location, an orientation of the client device, and an observed magnetic field vector measured by the magnetic sensor. The method may include computing a device correction vector for the client device based on the set of magnetic field measurements. For each magnetic field measurement, the method includes determining a world magnetic field vector at the device location of the magnetic field measurement, computing an expected measured magnetic field vector at the device location, accessing an estimated device correction vector for the client device, computing an expected adjusted vector for the client device, comparing the observed magnetic field vector associated with the magnetic field measurement and the expected adjusted vector, and computing the device correction vector based on the comparison.

Abstract: An augmented reality system generates computer-mediated reality on a client device. The client device has sensors including a camera configured to capture image data of an environment and a location sensor to capture location data describing a geolocation of the client device. The client device creates a three-dimensional (3-D) map with the image data and the location data for use in generating virtual objects to augment reality. The client device transmits the created 3-D map to an external server that may utilize the 3-D map to update a world map stored on the external server. The external server sends a local portion of the world map to the client device. The client device determines a distance between the client device and a mapping point to generate a computer-mediated reality image at the mapping point to be displayed on the client device.

Abstract: The present disclosure describes approaches to camera re-localization using a graph neural network (GNN). A re-localization model includes encoding an input image into a feature map. The model retrieves reference images from an image database of a previously scanned environment based on the feature map of the image. The model builds a graph based on the image and the reference images, wherein nodes represent the image and the reference images, and edges are defined between the nodes. The model may iteratively refine the graph through auto-aggressive edge-updating and message passing between nodes. With the graph built, the model predicts a pose of the image based on the edges of the graph. The pose may be a relative pose in relation to the reference images, or an absolute pose.

Abstract: A method of determining a position for a virtual object is described. A location of a client device is determined, and, based on the determined location a set of map segments is retrieved. A virtual object is determined to be displayed on the client device. Relation vectors between the virtual object and each map segment of the retrieved set of map segments are obtained. Each relation vector is weighted based on object parameters of the virtual object. A position to display the virtual object is determined based on the weighted relation vectors. The virtual object is provided for display on the client device as the determined position.

Abstract: The present disclosure describes approaches for evaluating interest points for localization uses based on a repeatability of the detection of the interest point in images capturing a scene that includes the interest point. The repeatability of interest points is determined by using a trained repeatability model. The repeatability model is trained by analyzing a time series of images of a scene and determining repeatability functions for each interest point in the scene. The repeatability function is determined by identifying which images in the time series of images allowed for the detection of the interest point by an interest point detection model.

Abstract: A client device associated with a user of a location-based application detects client devices associated with other users which are within proximity of the client device. This detection of other client devices may result in various application actions occurring, such as triggering a multi-user activity between users. Detection of client devices may be performed using personal area network devices of the client devices, such as Bluetooth. Proximity detection can occur when client devices are disconnected from an online system hosting the location-based application with the detection later being reported to the online system by one or both devices.

Abstract: An augmented reality (“AR”) device applies smooth correction methods to correct the location of the virtual objects presented to a user. The AR device may apply an angular threshold to determine whether a virtual object can be moved from an original location to a target location. An angular threshold is a maximum angle by which a line from the AR device to the virtual object can change within a timestep. Similarly, the AR device may apply a motion threshold, which is a maximum on the distance that a virtual object's location can be corrected based on the motion of the virtual object. Furthermore, the AR device may apply a pixel threshold to the correction of the virtual object's location. A pixel threshold is a maximum on the distance that a pixel projection of the virtual object can change based on the virtual object's change in location.

Abstract: Systems and methods for generating and storing metrics are described herein. In particular, a game server may receive game activity data from one or more client devices connected to the game server via a network. The game server sends the activity data to an anticheat server that uses one or more nodes to each calculate a portion of the activity data to generate one or more metrics. The metrics may indicate whether the data is indicative of cheating behavior within a parallel reality game. The nodes add their respective generated metrics to a relational database using prepared insert statements. The anticheat server may instruct the game server to take action with respect to one or more client devices if the metrics indicate that cheating behavior was exhibited within the data.

Abstract: A scene reconstruction model is disclosed that outputs a heightfield for a series of input images. The model, for each input image, predicts a depth map and extracts a feature map. The model builds a 3D model utilizing the predicted depth maps and camera poses for the images. The model raycasts the 3D model to determine a raw heightfield for the scene. The model utilizes the raw heightfield to sample features from the feature maps corresponding to positions on the heightfield. The model aggregates the sampled features into an aggregate feature map. The model regresses a refined heightfield based on the aggregate feature map. The model determines the final heightfield based on a combination of the raw heightfield and the refined heightfield. With the final heightfield, a client device may generate virtual content augmented on real-world images captured by the client device.

Abstract: A client device associated with a player of a location-based game detects client devices associated with other players which are within proximity of the player device. This detection of other player client devices may result in various game actions occurring, such as an exchange of game elements between players, game progress for a player, access to a game feature, or establishing a connection between players. Detection of player devices may be performed using personal area network devices of the client devices, such as Bluetooth. Proximity detection can occur when player client devices are disconnected from an online system hosting the location-based game with the detection later being reported to the online system by one or both devices.

Abstract: A system enables an arbitrary number of items to be indexed in a geographic region that provides a predictable query response time across a sharded database. Items indexed to the geographic region are stored on a single shard and additional items are added to that shard as long as an overflow condition indicative of undesirable query response times is not met. If the overflow condition is met the system expands the storage of items indexed to the geographic region to one or more additional shards in order to maintain predictable query response times. The system may maintain a shard count representing the total number of shards being used to store items corresponding to a geographic region, which can be used to query one or more relevant shards. The system may apply deterministic hashing in order to evenly distribute shards across database nodes of the sharded database.

Abstract: Events in an interactive application (e.g., a location-based parallel reality game) are triggered by a user's real-world activity meeting one or more criteria. A background process executing on the user's client device periodically extracts activity data from an activity monitoring application (e.g., a fitness application) and provides it to a server hosting the interactive application. The server determines whether received activity data triggers an event and, if so, sends a notification of the event to the client device.

Abstract: A multi-frame depth estimation model is disclosed. The model is trained and configured to receive an input image and an additional image. The model outputs a depth map for the input image based on the input image and the additional image. The model may extract a feature map for the input image and an additional feature map for the additional image. For each of a plurality of depth planes, the model warps the feature map to the depth plane based on relative pose between the input image and the additional image, the depth plane, and camera intrinsics. The model builds a cost volume from the warped feature maps for the plurality of depth planes. A decoder of the model inputs the cost volume and the input image to output the depth map.

Abstract: An image matching system for determining visual overlaps between images by using box embeddings is described herein. The system receives two images depicting a 3D surface with different camera poses. The system inputs the images (or a crop of each image) into a machine learning model that outputs a box encoding for the first image and a box encoding for the second image. A box encoding includes parameters defining a box in an embedding space. Then the system determines an asymmetric overlap factor that measures asymmetric surface overlaps between the first image and the second image based on the box encodings. The asymmetric overlap factor includes an enclosure factor indicating how much surface from the first image is visible in the second image and a concentration factor indicating how much surface from the second image is visible in the first image.