Abstract: A panoramic system can receive content (e.g., live video) recorded in a landscape orientation and use a portrait dynamic crop window to display a portrait area of the received content. The panoramic system can further display the content in a landscape dynamic crop window that has a moveable user interface control that can pan the portrait crop window as the content plays in one or both of the dynamic crop windows.

Abstract: Provided are systems and methods for portrait animation. An example method includes receiving, by a computing device, scenario data including information concerning movements of a first head, receiving, by the computing device, a target image including a second head and a background, determining, by the computing device and based on the target image and the information concerning the movements of the first head, two-dimensional (2D) deformations of the second head in the target image, applying, by the computing device, the 2D deformations to the target image to obtain at least one output frame of an output video, the at least one output frame including the second head displaced according to the movements of the first head, and filling, by the computing device and using a background prediction neural network, a portion of the background in gaps between the displaced second head and the background.

Abstract: Method for generating media content items on demand starts with a processor receiving an animation file including a first metadata based on an animation input. The animation file is associated with a media content identification. The processor generates puppets associated with frames in the animation file using the first metadata. The processor causes a puppet matching interface to be displayed on a client device. The puppet matching interface includes one of the puppets in a first pose. The processor receives a puppet posing input associated with a second pose from the client device. The processor causes the one of the puppets to be displayed in the second pose in the puppet matching interface by the client device. The processor can also generate a second metadata based on the puppet posing input. Other embodiments are disclosed herein.

Abstract: Methods and systems are disclosed for generating AR experiences on a messaging platform. The methods and systems perform operations including: detecting a real-world object depicted in a first image captured by a first camera of a client device, the client device comprising a second camera; extracting one or more textures from the real-world object depicted in the first image; selecting a target object depicted in a second image captured by the second camera, the second image being captured by the second camera simultaneously with the first image captured by the first camera; generating an augmented reality (AR) element comprising the target object modified based on the one or more textures extracted from the real-world object depicted in the first image; and causing display of the AR element within the second image.

Abstract: Systems, devices, media and methods are presented for detecting anomalous resources and events in social data. The systems and methods receive a plurality of events associated with a plurality of resources, wherein the plurality of events includes a plurality of features. The systems and methods detect a set of anomalous resources from the plurality of resources and identify a set of anomalous events associated with the set of anomalous resources. The systems and methods cause an interface to be displayed on a computing device, wherein the interface includes the set of anomalous resources and the set of anomalous events.

Abstract: The present application discloses a message based multi-processor system comprising a message exchange network and a plurality of processor clusters to mutually exchange messages. A processor cluster comprises one or more processor cluster elements, and a message generator. The message based multi-processor system is configured as a neural network processor system having a plurality of neural network processing layers each being assigned one or more of the processor clusters. The message generator of a processor cluster comprises a logic module. The logic module, in response to an activation signal of a processor cluster element, selectively generates and transmits a message for each of a set of destination processor clusters in accordance with respective message generation control data for said destination processor clusters stored in a message generator control storage space.

Abstract: A method for generation of a collection of video clips from a plurality of video files includes performing facial recognition on a video file of the plurality of video files to identify a portion of the video file including a face, generating a video clip by trimming the portion of the video file including the face, from the video file, and adding the video clip to the collection of video clips. A video montage may be created by adding a plurality of video clips from the collection of video clips together and adding an audio track to the video montage file.

Abstract: Methods and systems are disclosed for performing generating AR experiences on a messaging platform. The methods and systems perform operations including: receiving, from a client device, a request to access an augmented reality (AR) experience; adding one or more AR elements to a first image captured by the client device, the first image depicting a real-world object; storing data representing a position of the one or more AR elements relative to the real-world object, the data being maintained after the AR experience is terminated; receiving a request to resume the AR experience after the AR experience has been terminated; and in response to receiving the request to resume the AR experience, accessing the data that was stored prior to termination of the AR experience to generate a display of the AR experience that depicts the one or more AR elements at a particular position within a second image.

Abstract: Context-sensitive remote controls for use with electronic devices (e.g., eyewear device). The electronic device is configured to perform activities (e.g., email, painting, navigation, gaming). The context-sensitive remote control includes a display having a display area, a display driver coupled to the display, and a transceiver. The remote control additionally includes memory that stores controller layout configurations for display in the display area of the display by the display driver. A processor in the context-sensitive remote control is configured to establish, via the transceiver, communication with an electronic device, detect an activity currently being performed by the electronic device, select one of the controller layout configurations responsive to the detected activity, and present, via the display driver, the selected controller layout configuration in the display area of the display.

Abstract: Eyewear providing an interactive augmented reality experience to users in a first physical environment viewing objects in a second physical environment (e.g., X-ray effect). The second environment may be a room positioned behind a barrier, such as a wall. The user views the second environment via a sensor system moveable on the wall using a track system. As the user in the first environment moves the eyewear to face the outside surface of the wall along a line-of-sight (LOS) at a location (x, y, z), the sensor system on the track system repositions to the same location (x, y, z) on the inside surface of wall. The image captured by the sensor system in the second environment is wirelessly transmitted to the eyewear for displayed on the eyewear displays, providing the user with an X-ray effect of looking through the wall to see the objects within the other environment.

Abstract: Bending data is used to facilitate tracking operations of an extended reality (XR) device, such as hand tracking or other object tracking operations. The XR device obtains bending data indicative of bending of the XR device to accommodate a body part of a user wearing the XR device. The XR device determines, based on the bending data, whether to use previously identified biometric data in a tracking operation. A mode of the XR device is selected responsive to determining whether to use the previously identified biometric data. The selected mode is used to initialize the tracking operation. The selected mode may be a first mode in which the previously identified biometric data is used in the tracking operation or a second mode in which the previously identified biometric data is not used in the tracking operation.

Abstract: Systems and methods herein describe a multi-modal interaction system. The multi-modal interaction system, receives a selection of an augmented reality (AR) experience within an application on a computer device, displays a set of AR objects associated with the AR experience on a graphical user interface (GUI) of the computer device, display textual cues associated with the set of augmented reality objects on the GUI, receives a hand gesture and a voice command, modifies a subset of augmented reality objects of the set of augmented reality objects based on the hand gesture and the voice command, and displays the modified subset of augmented reality objects on the GUI.

Abstract: Systems and methods are provided for performing operations on an augmented reality (AR) device. The system accesses, by the AR device, movement data comprising inertial measurement data and camera data. The system determines three-dimensional (3D) movement of the AR device based on the movement data. The system presents, by the AR device, an AR object on a real-world environment being viewed using the AR device. The system, in response to determining the 3D movement of the AR device, modifies the AR object by the AR device.

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program, method, and user interfaces to facilitate communication between users via multiple modes of electronic communication. The system displays a conversation view that presents messages exchanged between a first user and at least one other user in an electronic chat conversation. The conversation view includes an interactive element to enable the first user to initiate a video call with the at least one other user. Based on receiving user input corresponding to an interaction with the interactive element, the system initiates the video call with the at least one other user. The system displays an active call view to facilitate the video call with the at least one other user. The system toggles from displaying the active call view to displaying the conversation view in response to further user input.

Abstract: A messaging system performs neural network hair rendering for images provided by users of the messaging system. A method of neural network hair rendering includes processing a three-dimensional (3D) model of fake hair and a first real hair image depicting a first person to generate a fake hair structure, and encoding, using a fake hair encoder neural subnetwork, the fake hair structure to generate a coded fake hair structure. The method further includes processing, using a cross-domain structure embedding neural subnetwork, the coded fake hair structure to generate a fake and real hair structure, and encoding, using an appearance encoder neural subnetwork, a second real hair image depicting a second person having a second head to generate an appearance map. The method further includes processing, using a real appearance renderer neural subnetwork, the appearance map and the fake and real hair structure to generate a synthesized real image.

Abstract: In one or more implementations, user content items generated using a client application may be shared with users that are not contacts of the user within the client application. A user interface that indicates a number of recipients of the user content item may be generated that also includes a first section that displays the user content item and a second section to add text content to the user content item. In various examples, one or more classifications may be associated with the user content item.

Abstract: The subject technology receives, by a client device from a storage device, first image data captured by the client device at a previous time. The subject technology receives first metadata corresponding to at least a first image processing operation and a second image processing operation. The subject technology generates, in a first render pass, second image data based on the first metadata and the image processing operation performed on the first image data. The subject technology generates, in a second render pass, third image data based on the first metadata and the second image processing operation performed on the second image data. The subject technology generates second metadata comprising information corresponding to the third image data. The subject technology generates a composite AR content item comprising the second metadata, the third image data, and the first image data.

Abstract: Eyewear including a voltage controller in the frame that generates dynamic analog control signals to control voltage regulators in the temple. The voltage regulators include a voltage rail for each electronic component in the temple. A separate analog control loop is coupled to each voltage regulator and receives the respective analog control signal. Each voltage regulator generates a rail voltage on the respective voltage rail that is controlled by the respective analog control signal. The analog control loop configures the respective voltage regulator as a voltage follower regulator such that the respective rail voltage follows a voltage of the analog control signal. A power source, such as a battery, is included in the temple and provides the operating power to each electronic component, and power is not communicated across a hinge to the temple electronic components.

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for displaying augmented reality content. The program and method provide for displaying, within a messaging application running on a device, a first user interface for viewing a media content item; receiving user input via the first user interface, the user input corresponding to a predefined touch gesture; and in response to receiving the user input, switching, within the messaging application, from the first user interface to a second user interface in which a camera of the device is activated to capture an image, and displaying, via the second user interface, a resized version of the media content item with the image.

Abstract: An architecture is provided for packaging visual overlay-based user interfaces (UIs) into mobile device applications to work as user interface extensions that allow certain flows and logic to be displayed on an eyewear device when connected to the mobile device application. The extension of the UIs of the mobile device applications to the display of the eyewear device allows for inexpensive experimentation with augmented reality (AR) UIs for eyewear devices and allows for reusing of business logic across mobile devices and associated eyewear devices. For example, a mobile device application for maps or navigation may be extended to show directions on an associated eyewear device once the destination is chosen in the navigation application on the mobile device. In this example, the business logic would still live in the navigation application on the mobile device but the user would see AR directions overlaid on a display of the eyewear device.