Abstract: Systems and methods for linking real world activity, such as real world commercial activity or real world data collection activity, with a location-based parallel reality game are provided. In particular, a game server hosting a parallel reality game can modify, update, or add to game data stored in a game database associated with the parallel reality game to include certain game features in the parallel reality game linked with real world activity in the real world. The game features can be linked with activities in the real world such that player actions associated with the game features in the virtual world can lead to or encourage activity in the real world, such as commercial activity and/or data collection activity in the real world. A one-time password may be used to validate a player's location in the real world and to enable controlled access to game features in the virtual world.

Abstract: A system and method determine the location of a device. The device collects sensor data using one or more sensors. Based on the sensor data, one or more localization models are selected from a plurality of localization models. The selected models are applied to generate one or more candidate locations. The current location of the device is determined based on the one or more candidate locations.

Abstract: Systems and methods for providing a shared augmented reality environment are provided. In particular, the latency of communication is reduced by using a peer-to-peer protocol to determine where to send datagrams. Datagrams describe actions that occur within the shared augmented reality environment, and the processing of datagrams is split between an intermediary node of a communications network (e.g., a cell tower) and a server. As a result, the intermediary node may provide updates to a local state of a client device when a datagram is labelled peer-to-peer, and otherwise provides updates to the master state on the server. This may reduce the latency of communication and allow users of the location-based parallel reality game to see actions occur more quickly in the shared augmented reality environment.

Abstract: A dataflow hierarchy protocol is implemented by one or more devices to optimize how the one or more devices process datagrams for network communications. The dataflow hierarchy considers various available network pathways for dataflow. A device implementing the dataflow hierarchy selects one or more of the available network pathways to provide low latency in data communication with other devices. The device may sample various available network pathways to determine pathway metrics (e.g., latency) and select one or more network pathways based on the metrics. The available network pathways can include pathways through one or more intermediary nodes, such as pathways through a game server, pathways through a cell tower, and pathways through a network.

Abstract: An augmented reality system generates computer-mediated reality on a client device. The client device has sensors including a camera configured to capture image data of an environment and a location sensor to capture location data describing a geolocation of the client device. The client device creates a three-dimensional (3-D) map with the image data and the location data for use in generating virtual objects to augment reality. The client device transmits the created 3-D map to an external server that may utilize the 3-D map to update a world map stored on the external server. The external server sends a local portion of the world map to the client device. The client device determines a distance between the client device and a mapping point to generate a computer-mediated reality image at the mapping point to be displayed on the client device.

Abstract: Processing of actions within a shared augmented reality experience is split between an edge node of a communications network (e.g., a cell tower) and a server. As a result, computation of the current state may be sharded naturally based on real-world location, with state updates generally provided by the edge node and the server providing conflict resolution based on a master state (e.g., where actions connected to different edge nodes potentially interfere with each other). In this way, latency may be reduced as game actions are communicated between clients connected to the same edge node using a peer-to-peer (P2P) protocol without routing the actions via the game server.

Abstract: A client device can display various augmented reality and virtual reality images to a user. For example, the client device captures images of an external environment and identifies objects in the environment. If an object satisfies a virtual environment criterion, the client device displays a window to a virtual environment at the location of the object in the environment. In another example, the client device includes a light assembly that can generate light. If the user turns the light assembly on, virtual objects displayed in the environment will gather to a region highlighted by the light from the light assembly. In another example, the client device estimates depth information of an object in the environment. The client device then displays a virtual object in the environment based on the depth information and may also display the virtual object interacting with an object in the external environment.

Abstract: A method and system for verifying a client device's location in a parallel reality game hosted by a server. The client transmits its location to the server and receives verification instructions comprising a landmark and a verification pathway. The client prompts a player to capture image data of the landmark and, in response, receives a first set of image data of the landmark from an initial perspective. The client determines whether the first set of image data matches to the landmark before prompting the player to move along the verification pathway while capturing image data. The client receives a second set of image data of the landmark from a moving perspective. The client determines whether the second set of image data matches to an expected change in perspective of the landmark. Upon completion of the verification instructions, the client confirms to the server the client's location.

Abstract: A method for training a depth estimation model with depth hints is disclosed. For each image pair: for a first image, a depth prediction is determined by the depth estimation model and a depth hint is obtained; the second image is projected onto the first image once to generate a synthetic frame based on the depth prediction and again to generate a hinted synthetic frame based on the depth hint; a primary loss is calculated with the synthetic frame; a hinted loss is calculated with the hinted synthetic frame; and an overall loss is calculated for the image pair based on a per-pixel determination of whether the primary loss or the hinted loss is smaller, wherein if the hinted loss is smaller than the primary loss, then the overall loss includes the primary loss and a supervised depth loss between depth prediction and depth hint. The depth estimation model is trained by minimizing the overall losses for the image pairs.

Abstract: The virtual location of a player in a location-based game is determined from the real-world location of the player's client device. The location-based game provides the player access to one or more chat room based on their location. To determine the locations of chat room, a server analyzes player locations in a geographic region, clusters player locations to identify centroids, and adjusts the clusters based on constraints. The server selects chat room locations (e.g., at points of interest) to more evenly balance the number of players in each chat room while complying with one or more restraints on the size of the geographic area served by each chat room.

Abstract: A shared virtual experience is provided to players of a location-based game. The virtual location of a player in the game is based on the real world location of the player's client device. If a player's location in the game is within an interaction distance of a virtual element, in indication that a shared virtual experience is available may be provided. Player input is received requesting participation in the shared virtual experience and the player is added to a lobby including a specified start time. At the specified time, the shared virtual experience begins and includes interaction with other players who were in the lobby. An update to game data is sent based on an outcome of the shared virtual experience.

Abstract: Computer-implemented methods and systems for locating virtual elements that can be used or collected by players of a parallel reality game having a virtual world that parallels at least a portion of the real world are provided. In particular, the location of virtual elements in the virtual world is determined based on data associated with one or more real world conditions. Virtual elements can be located in the virtual world at locations corresponding to locations in the real world that encourage safe and effective game play. Locating virtual elements in the virtual world based on data associated with real world conditions improves the link between the parallel virtual world and the real world, enhancing the illusion that the virtual world is another dimension of the real world that the player can interact with through the parallel reality game.

Abstract: A location-based parallel reality game may use regions to determine how to disperse virtual elements, locate virtual experiences, and/or define game objectives. To determine the regions of the geographic area, a server divides the geographic region into cells. The server assigns real-world data to each of the cells and constructs a graph of nodes and edges based on the cells. Each node corresponds to a cell, and each edge connects nodes that correspond to adjacent cells. The server partitions the graph into contiguous segments based on the real-world data. The server defines regions of the geographic area based on the contiguous segments and stores the defined regions in a data store. The server may hierarchically partition the regions into higher order regions until each region meets a set of criteria.

Abstract: An example method includes obtaining, obtaining, from one or more sensors of a computing device, data relating to a feature in an environment. Following, the method includes analyzing the data to identify one or more details of the feature in the environment and determining, based on a comparison of the data to a stored dataset in a database, that the details includes a detail that the stored dataset lacks. The method includes providing one or more game elements for gameplay on an interface of the computing device based on the details including the detail that the stored dataset lacks.

Abstract: Systems and methods are described for predicting depth from colour image data using a statistical model such as a convolutional neural network (CNN), The model is trained on binocular stereo pairs of images, enabling depth data to be predicted from a single source colour image. The model is trained to predict, for each image of an input binocular stereo pair, corresponding disparity values that enable reconstruction of another image when applied, to the image. The model is updated based on a cost function that enforces consistency between the predicted disparity values for each image in the stereo pair.

Abstract: A method for training a depth estimation model and methods for use thereof are described. Images are acquired and input into a depth model to extract a depth map for each of the plurality of images based on parameters of the depth model. The method includes inputting the images into a pose decoder to extract a pose for each image. The method includes generating a plurality of synthetic frames based on the depth map and the pose for each image. The method includes calculating a loss value with an input scale occlusion and motion aware loss function based on a comparison of the synthetic frames and the images. The method includes adjusting the plurality of parameters of the depth model based on the loss value. The trained model can receive an image of a scene and generate a depth map of the scene according to the image.

Abstract: Systems and methods for providing a shared augmented reality environment are provided. In particular, the latency of communication is reduced by using a peer-to-peer protocol to determine where to send datagrams. Datagrams describe actions that occur within the shared augmented reality environment, and the processing of datagrams is split between an intermediary node of a communications network (e.g., a cell tower) and a server. As a result, the intermediary node may provide updates to a local state of a client device when a datagram is labelled peer-to-peer, and otherwise provides updates to the master state on the server. This may reduce the latency of communication and allow users of the location-based parallel reality game to see actions occur more quickly in the shared augmented reality environment.

Abstract: A method for training a depth estimation model with depth hints is disclosed. For each image pair: for a first image, a depth prediction is determined by the depth estimation model and a depth hint is obtained; the second image is projected onto the first image once to generate a synthetic frame based on the depth prediction and again to generate a hinted synthetic frame based on the depth hint; a primary loss is calculated with the synthetic frame; a hinted loss is calculated with the hinted synthetic frame; and an overall loss is calculated for the image pair based on a per-pixel determination of whether the primary loss or the hinted loss is smaller, wherein if the hinted loss is smaller than the primary loss, then the overall loss includes the primary loss and a supervised depth loss between depth prediction and depth hint. The depth estimation model is trained by minimizing the overall losses for the image pairs.

Abstract: Events in an interactive application (e.g., a location-based parallel reality game) are triggered by a user's real-world activity meeting one or more criteria. A background process executing on the user's client device periodically extracts activity data from an activity monitoring application (e.g., a fitness application) and provides it to a server hosting the interactive application. The server determines whether received activity data triggers an event and, if so, sends a notification of the event to the client device.

Abstract: Systems and methods for detecting and preventing cheating in a location-based game are provided. In particular, a server-side statistical cheat detection algorithm can be implemented upon receiving a player request to perform a game action. A given dataset describing the respective times and locations of previous game events performed by the player can be analyzed to determine a probability that such game events represent legitimate gameplay. The determined probability can affect whether the player-requested game action is effectuated or can influence a number of points awarded for completion of the game action. An example system includes a host server and a client device in communication over a network.