Abstract: A hand-tracking input pipeline dimming system for an AR system is provided. The AR system deactivates the hand-tracking input pipeline and places a camera component of the hand-tracking input pipeline in a limited operational mode. The AR system uses the camera component to detect initiation of a gesture by a user of the AR system and in response to detecting the initiation of the gesture, the AR system activates the hand-tracking input pipeline and places the camera component in a fully operational mode.

Abstract: A personalized preview system to receive a request to access a collection of media items from a user of a user device. Responsive to receiving the request to access the collection of media items, the personalized preview system accesses user profile data associated with the user, wherein the user profile data includes an image. For example, the image may comprise a depiction of a face, wherein the face comprises a set of facial landmarks. Based on the image, the personalized preview system generates one or more media previews based on corresponding media templates and the image, and displays the one or more media previews within a presentation of the collection of media items at a client device of the user.

Abstract: Methods and systems are disclosed for performing operations comprising: receiving an image that includes a depiction of a person wearing a fashion item; generating a segmentation of the fashion item by the person depicted in the image; receiving voice input associated with the person depicted in the image; in response to receiving the voice input, generating one or more augmented reality elements representing the voice input; and applying the one or more augmented reality elements to the fashion item worn by the person based on the segmentation of the fashion item worn by the person.

Abstract: Input indicative of a selection of volumetric content for presentation is received. The volumetric content comprises a volumetric representation of one or more elements of a real-world three-dimensional space. In response to the input, device state data associated with the volumetric content is accessed. The device state data describes a state of one or more network-connected devices associated with the real-world three-dimensional space. The volumetric content is presented. The presentation of the volumetric content includes presentation of the volumetric representation of the one or more elements overlaid on the real-world three-dimensional space by a display device and configuring the one or more network-connected devices using the device state data.

Abstract: The subject technology receives frames of a source media content. The subject technology detects from the frames of the source media content, a first gesture indicating a cut point at a particular frame of the source media content, the cut point associated with a trimming operation to be performed on the source media content. The subject technology selects a starting frame and an ending frame from the frames based at least in part on the cut point at the particular frame. The subject technology performs the trimming operation based on the starting frame and the ending frame. The subject technology generates a second media content using the third set of frames. The subject technology provides for display at least a portion of the third set of frames of the second media content.

Abstract: Systems, methods, devices, server computers, storage media, and instructions for prioritized device action triggered by device scan data are described. In one embodiment, a mobile device performs a method that involves executing a messaging application with an image capture interface and a scanning input. An associated scanning mode comprises capture of scan data from a plurality of input/output modules of the first client device, analyzes the scan data to identify one or more scan data patterns by matching at least a portion of the scan data against a set of data patterns, and selects a priority system action based on the results of the matching of the portion of the scan data against the set of data patterns. In some embodiments, the priority system action is selected based on a priority ranking for identified scan data types.

Abstract: An occlusion detection system to perform operations that include: capturing image data that depicts an environment at a client device, the environment including a target object at a position within the environment; causing display of a presentation of the environment at the client device, the presentation of the environment including a display of the target object at the position within the environment; detecting a first attribute of the display of the target object at the client device; performing a comparison of the first attribute of the display of the target object and a second attribute associated with the target object; and detecting an occlusion based on the comparison.

Abstract: The present invention relates to improvements to systems and methods for determining a current location of a client device, and for identifying and selecting appropriate geo-fences based on the current location of the client device. An improved geo-fence selection system performs operations that include associating media content with a geo-fence that encompasses a portion of a geographic region, sampling location data from a client device, defining a boundary based on the sampled location data from the client device, detecting an overlap between the boundary and the geo-fence, retrieving the media content associated with the geo-fence, and loading the media content at a memory location of the client device, in response to detecting the overlap.

Abstract: Aspects of the present disclosure involve systems comprising a computer-readable storage medium storing a program and a method for performing operations comprising: receiving, by a messaging application, a first content item generated by a first client device using a third-party resource; generating the first content item for display on a second client device together with an indication of the third-party resource used to generate the first content item; receiving, from the second client device, selection of the indication of the third-party resource; generating, for display on the second client device, a first user interface that includes a plurality of content items associated with the third-party resource and a first option to access the third-party resource, the plurality of content items including the first content item; and enabling the second client device to use the third-party resource in response to receiving selection of the first option to access the third-party resource.

Abstract: There is a remote control device or drone, which has software and a combination of lights or LED on an lighting ring or apparatus that can move independently of the drone; the drone can be programmed or be reactive to sound or other stimulus to create the effect of writing shapes or words in the air and typically at nighttime against a dark sky.

Abstract: When a messaging system generates connection recommendations for a new user, who first registers with the messaging system, the signals available for generation of recommendations may be limited to the user's contact book matches. Using just this limited signal poses a concern associated with leaking information about users represented by the recommendations. The technical problem of generating connection recommendations for a user at registration time in a privacy-safe manner is addressed by a recommendation methodology that obscures the connection source and the connection distance of the recommended profiles.

Abstract: Systems, computer readable medium and methods for unlocking an autonomous drone are disclosed. Example methods include receiving an indication of a selection of a fly instruction, capturing an image using an image capturing device of the autonomous drone, processing the image to determine whether a face is present in the image, and if the face is present in the image, taking off. The face has to be within a predetermined distance of the autonomous drone. This ensures that the face is likely from the person that selected the fly instruction and ensures that the autonomous drone is far enough away from the face that the autonomous drone will not crash into the face on take-off. In some examples, the autonomous drone determines whether the autonomous drone is sitting on a hand before taking off. The autonomous drone uses a position of the face to determine an initial flight plan.

Abstract: Systems and methods herein describe a media capture system that receives sensor data from biometric sensors coupled to a head-wearable apparatus, detects a trigger event corresponding to a user of the head-wearable apparatus based on the sensor data, captures images using a camera coupled to the head-wearable apparatus, and transmits the captured images to a client device.

Abstract: A multimodal video generation framework (MMVID) that benefits from text and images provided jointly or separately as input. Quantized representations of videos are utilized with a bidirectional transformer with multiple modalities as inputs to predict a discrete video representation. A new video token trained with self-learning and an improved mask-prediction algorithm for sampling video tokens is used to improve video quality and consistency. Text augmentation is utilized to improve the robustness of the textual representation and diversity of generated videos. The framework incorporates various visual modalities, such as segmentation masks, drawings, and partially occluded images. In addition, the MMVID extracts visual information as suggested by a textual prompt.

Abstract: An electronic eyewear device includes first and second systems-on-chip (SoCs) having independent time bases. The first and second SoCs are connected by a shared general purpose input/output (GPIO) connection and an inter-SoC interface. The first and second SoCs are synchronized to each other by the first SoC asserting the shared GPIO connection to the second SoC where assertion of the message to the shared GPIO connection triggers an interrupt request (IRQ) at the second SoC. The first SoC records a first timestamp for assertion of the message to the GPIO connection, and the second SoC records a second timestamp of receipt of the IRQ. The first SoC sends a message including the first timestamp to the second SoC over the inter-SoC interface. The second SoC calculates a clock offset between the first and second SoCs as a difference between the first and second timestamps.

Abstract: Aspects of the present disclosure involve providing a platform user notification to users in a chat session. A user device receives from a server chat status message data for a chat session. The user device detects a specified platform being used by a user in the chat session based on the chat status message data. The user device provides a chat session user interface a platform presence icon associated with the user indicating that the user is using the specified platform.

Abstract: A system to generate a graphical user interface (GUI) to display presentations of media collections associated with a user profile are described. Embodiments of the present disclosure relate generally to systems for generating and causing display of a GUI that comprises a presentation of a user profile associated with a user, wherein the presentation includes a user identifier of the user, and a display of one or more media collections that comprises media items generated by the user.

Abstract: A gesture-based text entry user interface for an Augmented Reality (AR) device is provided. The AR system detects a start text entry gesture made by a user of the AR system, generates a virtual keyboard user interface including a virtual keyboard having a plurality of virtual keys, and provides to the user the virtual keyboard user interface. The AR system determines using the one or more cameras, the user's selection of one or more selected virtual keys of the plurality of virtual keys and generates entered text data based on the one or more selected virtual keys. The AR system provides the entered text data to the user using a display of the AR system.

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for providing audio with captured video clips. The program and method provide for displaying, by a messaging application, a capture user interface for capturing video; providing a camera mode selection element which is selectable to switch between a first camera mode for capturing a single video clip and a second camera mode for capturing multiple video clips, to generate a media content item; providing an audio selection element which is selectable to select an audio track for the media content item; receiving, via the camera mode selection element, first user input selecting the second camera mode; receiving, via the audio selection element, second user input selecting the audio track; and providing for capturing multiple video clips in association with the selected audio track for generating the media content item.

Abstract: Eyewear devices that include two SoCs that share processing workload. Instead of using a single SoC located either on the left or right side of the eyewear devices, the two SoCs have different assigned responsibilities to operate different devices and perform different processes to balance workload. In one example, the eyewear device utilizes a first SoC to operate displays, and it performs three-dimensional graphics and compositing. A second SoC operates first and second color cameras, first and second computer vision (CV) cameras, an operating system (OS), CV algorithms, and visual odometry (VIO), and it performs hand gesture tracking of the user and provides depth from stereo images. This configuration provides organized logistics to efficiently operate various features, and balanced power consumption.