Abstract: A video is generated based on a user's interactions with an augmented reality (AR) client in an AR system. The AR system uses machine learning models to determine content within the images of the user's interactions (e.g., gameplay of an AR game) and the quality of the images (e.g., based on features of the images such as brightness, contrast, particular AR objects, behaviors of AR objects, etc.). A first machine learning model is applied to identify content within the images (e.g., the presence of an AR object). The AR system selects a first subset of the images to which the system applies a second machine learning model that classifies a quality score for each image. The AR system generates a video using a second subset of the image frames classified by the second machine learning model (e.g., a gameplay highlight video showing the user interacting with AR objects).

Abstract: An augmented reality (AR) system hosted and executed on a mobile client enables surface distinction for AR objects to interact with multiple surfaces captured within a camera view of the mobile client. The AR system receives an image from a camera of a mobile client, where the image depicts planar surfaces. The AR system identifies clusters of feature points corresponding to respective planar surfaces and generates meshes from the identified clusters. The AR system generates a 3D virtual coordinate plane that includes the identified clusters and generated meshes that correspond to respective surfaces. The AR system determines a spatial relationship between identified surfaces (e.g., a height between surfaces). The AR system receives a user interaction based on the identified surfaces and provides an AR object for display at the mobile client based on the received interaction (e.g., an AR object traveling between identified surfaces).

Abstract: A video is generated based on a user's interactions with an augmented reality (AR) client in an AR system. The AR system uses machine learning models to determine content within the images of the user's interactions (e.g., gameplay of an AR game) and the quality of the images (e.g., based on features of the images such as brightness, contrast, particular AR objects, behaviors of AR objects, etc.). A first machine learning model is applied to identify content within the images (e.g., the presence of an AR object). The AR system selects a first subset of the images to which the system applies a second machine learning model that classifies a quality score for each image. The AR system generates a video using a second subset of the image frames classified by the second machine learning model (e.g., a gameplay highlight video showing the user interacting with AR objects).