Abstract: An augmented reality (AR) game system hosted and executed on a mobile client enables occlusion of AR game elements by objects in the real world. The mobile client captures a camera view of an environment including a real-world object. The AR game system identifies a cluster of feature points associated with the real-world object and generates a mesh from the cluster. The AR game system represents the mesh on a virtual coordinate plane, such that an AR game element can appear to interact with the real-world object. The AR game system renders, on the mobile client, the AR game element to appear as though it is occluded by the real-world object.

Abstract: A video is generated based on a user's interactions with an augmented reality (AR) client in an AR system. The AR system uses machine learning models to determine content within the images of the user's interactions (e.g., gameplay of an AR game) and the quality of the images (e.g., based on features of the images such as brightness, contrast, particular AR objects, behaviors of AR objects, etc.). A first machine learning model is applied to identify content within the images (e.g., the presence of an AR object). The AR system selects a first subset of the images to which the system applies a second machine learning model that classifies a quality score for each image. The AR system generates a video using a second subset of the image frames classified by the second machine learning model (e.g., a gameplay highlight video showing the user interacting with AR objects).