Abstract: An augmented reality (AR) game system hosted and executed on a mobile client enables occlusion of AR game elements by objects in the real world. The mobile client captures a camera view of an environment including a real-world object. The AR game system identifies a cluster of feature points associated with the real-world object and generates a mesh from the cluster. The AR game system represents the mesh on a virtual coordinate plane, such that an AR game element can appear to interact with the real-world object. The AR game system renders, on the mobile client, the AR game element to appear as though it is occluded by the real-world object.

Abstract: A video is generated based on a user's interactions with an augmented reality (AR) client in an AR system. The AR system uses machine learning models to determine content within the images of the user's interactions (e.g., gameplay of an AR game) and the quality of the images (e.g., based on features of the images such as brightness, contrast, particular AR objects, behaviors of AR objects, etc.). A first machine learning model is applied to identify content within the images (e.g., the presence of an AR object). The AR system selects a first subset of the images to which the system applies a second machine learning model that classifies a quality score for each image. The AR system generates a video using a second subset of the image frames classified by the second machine learning model (e.g., a gameplay highlight video showing the user interacting with AR objects).

Abstract: An augmented reality (AR) gaming experience is provided to one or more users of a networked gaming system. The gaming infrastructure includes a custom state machine that facilitates execution of a game session by sending session states and the status of AR session objects from the game system to one or more participating client devices, allowing users to view object behavior in real-time as other users interact with the objects. The system utilizes session identifiers from the state machine and the game system to synchronize video recording between all client devices in a given game session. A composite video of the game session is further generated for each participating client device by stitching together a first captured video of the game session with a second captured video of the user's reaction to the game session.