Abstract: Methods and systems are disclosed for performing operations comprising: receiving an image that includes a depiction of a face of a user; generating a plurality of landmarks of the face based on the received image; removing a set of interfering landmarks from the plurality of landmarks resulting in a remaining set of landmarks of the plurality of landmarks; obtaining a depth map for the face of the user; and computing a real-world scale of the face of the user based on the depth map and the remaining set of landmarks.

Abstract: The present invention relates to improvements to systems and methods for determining a current location of a client device, and for identifying and selecting appropriate geo-fences based on the current location of the client device. An improved geo-fence selection system performs operations that include associating media content with a geo-fence that encompasses a portion of a geographic region, sampling location data from a client device, defining a boundary based on the sampled location data from the client device, detecting an overlap between the boundary and the geo-fence, retrieving the media content associated with the geo-fence, and loading the media content at a memory location of the client device, in response to detecting the overlap.

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for providing augmented reality-based makeup. The program and method provide for receiving a request to present augmented reality content in association with an image captured by a device camera, the image depicting a user's face; accessing an augmented reality content item applying makeup to the face, the augmented reality content configured to generate a mesh for tracking plural regions of the face and to present available makeup products with respect to the plural regions; presenting the augmented reality content item in association with the face depicted in the image; receiving user input selecting a region of the plural regions; determining a set of available makeup products corresponding to the selected region; and updating presentation of the augmented reality content item based on the set of available makeup products.

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for providing augmented reality content corresponding to a translation in association with travel. The program and method provide for receiving, by a messaging application running on a device of a user, a request to scan an image captured by a device camera; obtaining, in response to receiving the request, a travel parameter associated with the request, and an attribute of an object depicted in the image; determining, based on the travel parameter and the attribute, to perform a translation with respect to the object; performing, in response to the determining, the translation with respect to the object; and displaying an augmented reality content item, which includes the translation, with the image.

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for providing augmented reality-based makeup. The program and method provide for receiving, by a messaging application running on a device of a user, a request to present augmented reality content in association with an image captured by a device camera, the image depicting a face of the user; accessing an augmented reality content item configured to generate a plurality of completed looks with respect to applying makeup to the face; presenting the augmented reality content item, including the plurality of completed looks, in association with the face depicted in the image; receiving user input selecting a completed look of the plurality of completed looks; and displaying, in response to receiving the user input, an interface with a set of makeup products associated with the selected completed look.

Abstract: Systems, methods, and computer-readable media for adding beauty products to tutorials are presented. Methods include accessing video data comprising images of a presenter creating a tutorial, the tutorial depicting the presenter applying a beauty product to a body part of the presenter. Methods further include processing the video data to identify changes to the body part of the presenter from an application of the beauty product, and responding to identifying changes to the body part of the presenter from the application of the beauty product by processing the video data to identify the beauty product. Methods further include retrieving information regarding the beauty product and causing presentation of information regarding the beauty product on a display device.

Abstract: The subject technology retrieves, by a client device from a storage device, first image data captured by the client device at a previous time. The subject technology receiving first metadata corresponding to a selected image processing operation. The subject technology generates second image data based on the first metadata and the image processing operation performed on the first image data. The subject technology generates second metadata comprising information related to the image processing operation, the second metadata including a first identifier associated with the first image data, and second identifier associated with the second image data. The subject technology generates a message comprising the second metadata, the second image data, and the first image data.

Abstract: The subject technology causes, at a client device, display of a graphical interface comprising a plurality of selectable graphical items, each selectable graphical item corresponding to a respective content item associated with a different geolocation. The subject technology receives, at the client device, a selection of a first selectable graphical item from the plurality of selectable graphical items, the first selectable graphical item corresponding to a particular geolocation. The subject technology causes display, at the client device, a second plurality of selectable graphical items, each of the second plurality of selectable graphical items corresponding to a particular second geolocation of an activity or place of business within a geographical area associated with the particular geolocation.

Abstract: The subject technology requests a set of augmented reality (AR) content generators based on a group ID, generated by an extension application programming interface (API), using a camera API. The subject technology receives the set of AR content generators. The subject technology provides for display representations of the set of AR content generators in an interface. The subject technology receives a selection of a first AR content generator from the set of AR content generators. The subject technology renders the first AR content generator for display, using the camera API. The subject technology requests metadata for a set of products based on the selected first AR content generator using the extension API. The subject technology receives the metadata from the extension API. The subject technology provides for display the set of representations of products based on the received metadata.

Abstract: Augmented reality content item templates may be used to generate augmented reality content items that are executable to display content corresponding to products offered for purchase to users of a client application. The augmented reality content item templates may include regions that may be populated according to information related to individual products. In this way an augmented reality content item template may be used to generate augmented reality content items that are customized based on the information about the individual products. The augmented reality content item templates may be classified according to a classification system such that augmented reality content item templates corresponding to a respective classification may be used to generate augmented reality content items for products that also correspond to the respective classification.

Abstract: The subject technology selects a set of augmented reality content generators from a plurality of augmented reality content generators. The subject technology causes display, at a client device, of a graphical interface comprising a plurality of selectable graphical items. The subject technology receives, at the client device, a selection of a first selectable graphical item from the plurality of selectable graphical items, the first selectable graphical item comprising a first augmented reality content generator corresponding to a particular geolocation. The subject technology causes display, at the client device, at least one augmented content reality item generated by the first augmented reality content generator.

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for providing augmented reality content corresponding to a translation in association with travel. The program and method provide for receiving, by a messaging application running on a device of a user, a request to scan an image captured by a device camera; obtaining, in response to receiving the request, a travel parameter associated with the request, and an attribute of an object depicted in the image; determining, based on the travel parameter and the attribute, to perform a translation with respect to the object; performing, in response to the determining, the translation with respect to the object; and displaying an augmented reality content item, which includes the translation, with the image.

Abstract: Systems, methods, and computer readable media for augmented reality beauty product tutorials are disclosed. Methods disclose displaying on a first portion of a screen a beauty product tutorial including pre-recorded images of a presenter applying a beauty product to a body part of the presenter and determining first effects indicating changes to make to live images of a user to simulate applying the beauty product to the body part of the user, where the first effects are determined based on second effects indicating changes to the pre-recorded images from applying the beauty product to the body part of the presenter. Methods further disclose processing the live images of the user to add the first effects and indications of motion of the beauty product and displaying the processed live images of the user on a second portion of the screen.

Abstract: The subject technology retrieves, by a client device from a storage device, first image data captured by the client device at a previous time. The subject technology receiving first metadata corresponding to a selected image processing operation. The subject technology generates second image data based on the first metadata and the image processing operation performed on the first image data. The subject technology generates second metadata comprising information related to the image processing operation, the second metadata including a first identifier associated with the first image data, and second identifier associated with the second image data. The subject technology generates a message comprising the second metadata, the second image data, and the first image data.

Abstract: The subject technology causes, at a client device, display of a graphical interface comprising a plurality of selectable graphical items, each selectable graphical item corresponding to a respective content item associated with a different geolocation. The subject technology receives, at the client device, a selection of a first selectable graphical item from the plurality of selectable graphical items, the first selectable graphical item corresponding to a particular geolocation. The subject technology causes display, at the client device, a second plurality of selectable graphical items, each of the second plurality of selectable graphical items corresponding to a particular second geolocation of an activity or place of business within a geographical area associated with the particular geolocation.

Abstract: The subject technology requests a set of augmented reality (AR) content generators based on a group ID, generated by an extension application programming interface (API), using a camera API. The subject technology receives the set of AR content generators. The subject technology provides for display representations of the set of AR content generators in an interface. The subject technology receives a selection of a first AR content generator from the set of AR content generators. The subject technology renders the first AR content generator for display, using the camera API. The subject technology requests metadata for a set of products based on the selected first AR content generator using the extension API. The subject technology receives the metadata from the extension API. The subject technology provides for display the set of representations of products based on the received metadata.

Abstract: Systems, methods, and computer readable media for augmented reality beauty product tutorials are disclosed. Methods disclose displaying on a first portion of a screen a beauty product tutorial including pre-recorded images of a presenter applying a beauty product to a body part of the presenter and determining first effects indicating changes to make to live images of a user to simulate applying the beauty product to the body part of the user, where the first effects are determined based on second effects indicating changes to the pre-recorded images from applying the beauty product to the body part of the presenter. Methods further disclose processing the live images of the user to add the first effects and indications of motion of the beauty product and displaying the processed live images of the user on a second portion of the screen.

Abstract: The subject technology receives, at a client device, a selection of a first selectable graphical item, the first selectable graphical item comprising a first augmented reality content generator corresponding to a particular geolocation. The subject technology causes display, at the client device, of a graphical interface comprising a plurality of selectable augmented reality content items, each selectable augmented reality content item corresponding to a particular activity based at in part on the particular geolocation. The subject technology receives, at the client device, a second selection of a particular selectable augmented reality content item from the plurality of selectable augmented reality content items.

Abstract: The subject technology receives image data including a representation of a face of a user. The subject technology analyzes the image data to determine a set of characteristics of the representation of the face. The subject technology, based at least in part on the determined set of characteristics, selects a particular product and a set of media content associated with the particular product. The subject technology causes display, at a client device, at least one recommendation corresponding to the set of media content associated with the particular product.

Abstract: Systems, methods, and computer readable media for messaging system with augmented reality (AR) makeup are presented. Methods include processing a first image to extract a makeup portion of the first image, the makeup portion representing the makeup from the first image and training a neural network to process images of people to add AR makeup representing the makeup from the first image. The methods may further include receiving, via a messaging application implemented by one or more processors of a user device, input that indicates a selection to add the AR makeup to a second image of a second person. The methods may further include processing the second image with the neural network to add the AR makeup to the second image and causing the second image with the AR makeup to be displayed on a display device of the user device.