Abstract: A method for detecting full-body gestures by a mobile device includes a host mobile device detecting the tracked body of a co-located participant in a multi-party session. When the participant's tracked body provides a full-body gesture, the host's mobile device recognizes that there is a tracked body providing a full-body gesture. The host mobile device iterates through the list of participants in the multi-party session and finds the closest participant mobile device with respect to the screen-space position of the head of the gesturing participant. The host mobile device then obtains the user ID of the closest participant mobile device and broadcasts the recognized full-body gesture event to all co-located participants in the multi-party session, along with the obtained user ID. Each participant's mobile device may then handle the gesture event as appropriate for the multi-party session. For example, a character or costume may be assigned to a gesturing participant.

Abstract: A resource optimized kiosk mode that improves the mobile experience for creators and users of mobile devices such as an augmented reality (AR)-enabled wearable eyewear device. An eyewear device enters a kiosk mode by receiving a kiosk mode request for an application and, in response to the request, determining which services and application programming interfaces (APIs) are required to execute the selected application. An identification of the determined services and APIs required to execute the selected application are stored and the eyewear device is rebooted. After reboot, the selected application is started, and only the identified services and APIs are enabled. To determine which services and APIs are required to execute the selected application, metadata may be associated with the selected application specifying the services and/or APIs that the selected application requires to use when in operation.

Abstract: A method and system for classifying image content in a sequence (SEQ) of image frames. Frame image data (FI), representing an appearance of image content of the image frames (Fm,Fn), is converted into frame event data (FEn). The conversion comprises determining, for each event, a set of event parameters. The event parameters include positional coordinates (x,y) representing a corresponding position of the respective image content in the frame image data (FI) having changed value between different image frames (Fm,Fn) in the sequence (SEQ). The event parameters of the frame event data (FEn) are processed, preferably by clustering, to determine at least one event-based region of interest (R1) in the sequence (SEQ) of image frames. At least one classification (C1) is calculated based on an intersection of the event-based region of interest (R1) with the frame image data (FI) of an image frame (Fn) in the sequence (SEQ).

Abstract: A system to perform operations that include: minting a Non-Fungible Token (NFT) that comprises a media object and mutable metadata; allocating the NFT to a user of a client device; granting the user of the client device a permission to change the mutable metadata of the NFT based on the allocating the NFT to the user of the client device; generating an open-edition of the NFT, the open-edition of the NFT comprising a reference to the mutable metadata; receiving a change to the mutable metadata from the user of the client device; and updating the open-edition of the NFT based on the change.

Abstract: Augmented reality (AR) systems, devices, media, and methods are described for generating AR experiences including interactions with virtual or physical prop objects. The AR experiences are generated by capturing images of a scene with a camera system, identifying an object receiving surface and corresponding surface coordinates within the scene, identifying an AR primary object and a prop object (physical or virtual), establishing a logical connection between the AR primary object and the prop object, generating AR overlays including actions associated with the AR primary object responsive to commands received via a user input system that position the AR primary object adjacent the object receiving surface responsive to the primary object coordinates and the surface coordinates within the scene and that position the AR primary object and the prop object with respect to one another in accordance with the logical connection, and presenting the generated AR overlays with a display system.

Abstract: A communication link is established between a first mobile device and a second mobile device using communication setup information in a machine-readable code that is displayed on a display of the second mobile device. The first mobile device captures and decodes an image of the machine-readable code to extract dynamically-generated communication setup information. A communication link is then established between the two devices using the communication setup information. The machine readable code may also be used as a fiducial marker to establish an initial relative pose between the two devices. Pose updates received from the second mobile device can then be used as user-interface inputs to the first mobile device.

Abstract: The present disclosure relates to methods and systems for providing a touch-based augmented reality (AR) experience. During a capture phase, a first user may grip an object. An intensity of a force applied on the object in the grip and/or a duration of the grip may be recorded. A volumetric representation of the first user holding the object may also be captured. During an experience phase, a second user may touch the object, the object may provide haptic feedback (e.g., a vibration) to the second user at an intensity and a duration corresponding to an intensity of the force applied on the object and a duration of the grip of the object. If a volumetric representation of the first user holding the object is captured, touching the object may also cause a presentation of the first user's volumetric body that holds the object.

Abstract: Systems and methods herein describe generating automatic reactions in an augmented reality messenger system. The claimed systems and methods generate a contextual trigger defining a set of conditions at a first computing device, detect at least one of the set of conditions has been satisfied, cause presentation of an augmented reality content item at a second computing device, generate a user reaction in response to the presentation of the augmented reality content item and transmit the user reaction to a first computing device.

Abstract: Systems and methods to generate and cause display of a contextual navigation menu within a GUI, wherein the contextual navigation menu presents contextually relevant interface elements.

Abstract: A server has a processor and a memory storing a multiple channel message thread module with instructions executed by the processor to identify when participants at client devices are actively viewing a common message thread at the same time to establish a participant viewing state. An alternate channel communication lock prompt is supplied the client devices in response to the participant viewing state. An alternate channel communication is delivered to the client devices in response to activation of the alternate channel communication lock prompt by at least one participant.

Abstract: A text entry process for an Augmented Reality (AR) system. The AR system detects, using one or more cameras of the AR system, a start text entry gesture made by a user of the AR system. During text entry, the AR system detects, using the one or more cameras, a symbol corresponding to a fingerspelling sign made by the user. The AR system generates entered text data based on the symbol and provides text in a text scene component of an AR overlay provided by the AR system to the user based on the entered text data.

Abstract: A method that comprises receiving at a network connected server from a first client terminal, a message comprising, an user application ID of a user selecting a media object using a user interface presented on a display of the first client terminal and the media object, generating a web document which presents a browser user interface and the media object when accessed by a browser, the web document having a network accessible storage address, sending the network accessible storage address from the network connected server to allow a browser installed in a second client terminal to use of the network accessible storage address to display the media object the browser user interface, identifying a usage of the browser user interface for inputting a reaction to the media object by a user of the second client terminal, and forwarding the reaction to the first client terminal using the sender user ID.

Abstract: Systems, devices, media, and methods are presented for transmitting shared visual content between networked devices with a linked source for the visual content by accessing and presenting visual content, receiving a network location for a network resource associated with the visual content, linking the network location to the visual content to generate linked visual content, and cause presentation of the linked visual content in a draft message within a graphical user interface.

Abstract: A messaging system, which hosts a backend service for an associated messaging client application, includes a voice notes system that addresses the technical problem of serving an audio message to the recipient in a manner that permits the recipient to consume the message in a text format as a transcription. A chat user interface (UI) provided with the voice notes system permits a user to play an audio message or request generation of the transcription of the audio message on-demand to prevent unnecessary cluttering of the UI real estate.

Abstract: The subject technology determines a gaze direction in a field of view of a user using an eyewear device. The subject technology generates an anchor point in the field of view based at least in part on the determined gaze direction. The subject technology identifies a surface corresponding to a ground plane in the field of view. The subject technology determines a distance from the identified surface to the anchor point. The subject technology generates AR content based at least in part on the determined distance. The subject technology renders the generated AR content in the field of view for display by the eyewear device.

Abstract: Clustering a plurality of client devices running an application as a function of a data structure such that the plurality of client devices are each assigned a cluster. Client devices having similar performance metrics are assigned the same cluster. An operation of the application is modified as a function of the performance metrics of the client device. The modification of application operation is performed by turning certain features of the application on and off using a rule based on device cluster.

Abstract: Method for generating media content items on demand starts with a processor receiving an animation file including a first metadata based on an animation input. The animation file is associated with a media content identification. The processor generates puppets associated with frames in the animation file using the first metadata. The processor causes a puppet matching interface to be displayed on a client device. The puppet matching interface includes one of the puppets in a first pose. The processor receives a puppet posing input associated with a second pose from the client device. The processor causes the one of the puppets to be displayed in the second pose in the puppet matching interface by the client device. The processor can also generate a second metadata based on the puppet posing input. Other embodiments are disclosed herein.

Abstract: Embodiments enable virtual hair generation. The virtual hair generation can be performed by generating a first image of a face using a GAN model, applying 3D virtual hair on the first image to generate a second image with 3D virtual hair, projecting the second image with 3D virtual hair into a GAN latent space to generate a third image with virtual hair, performing a blend of the virtual hair with the first image of the face to generate a new image with new virtual hair that corresponds to the 3D virtual hair, training a neural network that receives the second image with the 3D virtual hair and provides an output image with virtual hair, and generating using the trained neural network, a particular output image with hair based on a particular input image with 3D virtual hair.

Abstract: Methods and systems are disclosed for generating AR experiences on a messaging platform. The methods and systems perform operations including: detecting a real-world object depicted in a first image captured by a first camera of a client device, the client device comprising a second camera; extracting one or more textures from the real-world object depicted in the first image; selecting a target object depicted in a second image captured by the second camera, the second image being captured by the second camera simultaneously with the first image captured by the first camera; generating an augmented reality (AR) element comprising the target object modified based on the one or more textures extracted from the real-world object depicted in the first image; and causing display of the AR element within the second image.

Abstract: Embodiments of the present disclosure relate generally to systems for presenting content suggestions within a messaging interface. A content suggestion system performs operations that include: receiving inputs that include a text string into a messaging interface; parsing a keyword from the text string of the input in response to receiving the inputs; retrieving message content that includes at least a media item based on the keyword; and presenting a chat suggestion that includes the message content within the messaging interface, according to certain example embodiments.