Abstract: An eyewear device, method for use with an eyewear device, and a non-transient computer readable medium for detecting blood sugar levels are disclosed. The eyewear device has a frame including a first rim configured to support a first lens, a second rim configured to support a second lens, and a bridge connecting the first rim to the second rim. The bridge is configured to receive a nose of a user when the eyewear device is worn by the user. Blood sugar levels are detected by monitoring behavior of a pupil of an eye of the user with a sensor, comparing the monitored behavior with known blood sugar level behaviors, identifying a match responsive to the comparison between the monitored behavior and one of the known blood sugar level behaviors corresponding to a particular blood sugar level, and selecting the particular blood sugar level as a blood sugar level of the user.

Abstract: Method of generating a contextual action bar starts with processor causing an application icon associated with an application to be displayed by a display screen of a client device. Processor receives selection of application icon from a user and determines a context of client device. Context can comprise identification of application, type associated with application, or type of interface including application icon. Processor generates action bar based on the context of the client device, causes a first portion of display screen to display an application interface associated with the application, and causes a second portion to display the action bar that is associated with a messaging system. Other embodiments are also disclosed herein.

Abstract: Methods and systems are disclosed for performing operations for recoloring a fashion item. The operations include receiving an image that includes a depiction of a person wearing a fashion item. The operations include generating a segmentation of the fashion item worn by the person depicted in the image. The operations include extracting a portion of the image corresponding to the segmentation of the fashion item. The operations include estimating lights and shadows being cast on the fashion item in the portion of the image. The operations include applying one or more augmented reality elements to the fashion item in the image based on the lights and shadows being cast on the fashion item.

Abstract: The subject technology receives frames of a source media content, the frames of the source media content including representations of a head and a face of a source actor. The subject technology generates sets of source pose parameters. The subject technology receives at least one target image, the at least one target image including representations of a target head and a target face of a target entity. The subject technology generates, based at least in part on the sets of source pose parameters, an output media content, each frame of the output media content includes an image of the target face in at least one frame of the output media content. The subject technology provides an online advertisement based at least in part on the output media content for display on a computing device.

Abstract: A path tracing system in which the traversal task is distributed between one global acceleration structure, which is central in the system, and multiple local acceleration structures, distributed among cells, of high locality and of autonomous processing. Accordingly, the centrality of the critical resource of accelerating structure is reduced, lessening bottlenecks, while improving parallelism.

Abstract: Methods and systems are disclosed for performing operations for applying augmented reality elements to a person depicted in an image. The operations include receiving an image that includes data representing a depiction of a person; generating a segmentation of the data representing the person depicted in the image; extracting a portion of the image corresponding to the segmentation of the data representing the person depicted in the image; applying a machine learning model to the portion of the image to predict a surface normal tensor for the data representing the depiction of the person, the surface normal tensor representing surface normals of each pixel within the portion of the image; and applying one or more augmented reality (AR) elements to the image based on the surface normal tensor.

Abstract: Method of generating animation overlays starts with a processor causing a communication interface for a communication session to be displayed at client devices. The processor receives from a first client device an animation sending request and causes an animation sending interface to be displayed in the communication interface of the first client device. The animation sending interface comprises animation overlay icons. Each of the animation overlay icons comprise a first avatar associated with the first user. The processor receives from the first client device a selection of a first animation overlay icon and causes the first animation overlay corresponding to the first animation overlay icon to be displayed as an overlay on the communication interface of each of the plurality of client devices. The first animation overlay is an animation that includes the first avatar. Other embodiments are disclosed.

Abstract: A testing system includes a client library that functions as a proxy for remote procedure calls to allow commands to be issued directly from a test framework including an interface that enables remote procedure call commands to be sent to a device under test from a mobile device and responses from the device under test to be received and forwarded to the mobile device, even where the mobile device and device under test have different operating systems. A client program adapted to run on the device under test sends and receives remote procedure call commands from the device under test to and from a test server that implements the test framework and manages a device state of the device under test. In a pairing test, the remote procedure call commands include commands adapted to emulate a button press for initiating pairing between the mobile device and the device under test.

Abstract: A collaborative session (e.g., a virtual time capsule) in which access to a collaborative object with an associated material and added virtual content is provided to users. In one example of the collaborative session, a user selects the associated material of the collaborative object. Physical characteristics are assigned to the collaborative object as a function of the associated material to be perceived by the participants when the collaborative object is manipulated. In one example, the material associated to the collaborative object is metal, wherein the interaction between the users and the collaborative object generates a response of the collaborative object that is indicative of the physical properties of metal, such as inertial, acoustic, and malleability.

Abstract: Systems and methods for receiving a media content item, causing a media content item interface containing the media content item to be displayed by a display device, and determining that a first contextual element is associated with the media content item displayed in the media content item interface. The systems and methods are include determining that a destructive action is applied to the media content item, and generating an overlay dialog component that is overlaid on top of the media content item in response to determining that the first contextual element is associated with the media content item in the media content item interface and that the destructive action has been applied to the media content item.

Abstract: A method for mitigating motion blur in a visual tracking system is described. In one aspect, a method for selective motion blur mitigation in a visual tracking system includes accessing a first image generated by an optical sensor of the visual tracking system, identifying camera operating parameters of the optical sensor during the optical sensor generating the first image, determining a motion of the optical sensor during the optical sensor generating the first image, determining a motion blur level of the first image based on the camera operating parameters of the optical sensor and the motion of the optical sensor, and determining whether to downscale the first image using a pyramid computation algorithm based on the motion blur level.

Abstract: Described are authoring tools for creating interactive AR experiences. The story-authoring application enables a user with little or no programming skills to create an interactive story that includes recording voice commands for advancing to the next scene, inserting and manipulating virtual objects in a mixed-reality environment, and recording a variety of interactions with connected IoT devices. The story creation interface is presented on the display as a virtual object in an AR environment.

Abstract: An image manipulation system for generating modified images using a generative adversarial network (GAN) trains GANs using domain changes, aligns input images with generated images, classifies and associates target images based on a symmetry, and uses a modified discriminator structure. A method for domain changes includes generating, using a pre-trained GAN trained on a plurality of first target images, a plurality of images, and determining a feature for each of the plurality of images. The method further includes determining the feature for each of a plurality of second target images and matching, based on the feature, second target images of the plurality of second target images with the plurality of images. The method further includes training a discriminator of the pre-trained GAN with the second target images and the plurality of images.

Abstract: Systems and methods are provided for notifying users about videos in a playback sequence. The systems and methods determine that a video that meets a criterion is currently available on a video server associated with a messaging client. In response to determining that the video that meets the criterion is currently available, the messaging client on the client device prefetches a sequence of videos from a recommendation engine that match a profile of a user associated with the messaging client. The recommendation engine is being used to provide sequence of videos to a video playback graphical user interface (GUI) that automatically plays back the videos in the sequence. The systems and methods determine that the video that meets the criterion is in a first position in the sequence of videos and, in response, present a notification that indicates the availability of the video on the video playback GUI.

Abstract: Described herein is a technique for processing a received media content item (e.g., a message), received at a messaging application of a first end-user of a messaging service, to generate a selection of some predetermined number of recommended stickers. The recommended stickers are then presented in a user interface to the first end-user, allowing the first end-user to select a sticker for use in replying to the received media content item. To generate the selection of recommended stickers, in response to receiving the media content item, the messaging application processes the media content item to identify specific attributes and characteristics (e.g., text included with the message, stickers used with the message, and other contextual metadata). The identified attributes and characteristics of the received message are then processed by a scoring model to identify the predetermined number of stickers for presenting in the reply interface as recommended reply stickers.

Abstract: Methods, systems, user interfaces, media, and devices provide a geographically-based graphical user interface (GUI) suited to single-handed operation of a device. The method and system provide for causing display of a GUI including map data displayed in a first view; receiving input comprising at least one input point; determining first and second characteristics of the input; processing, when the first characteristic fulfils a first operational mode criterion, the input according to a first operational mode and causing display of the map data in a second view, the second view being based on the second characteristic of the input; and processing, when the first characteristic of the input fulfils a second operational mode criterion, the input according to a second operational mode, and causing display of the map data in a third view, the third view being based on the second characteristic of the input.

Abstract: Example systems, devices, media, and methods are described for controlling one or more virtual elements on a display in response to hand gestures detected by an eyewear device that is capturing frames of video data with its camera system. An image processing system detects a hand and presents a menu icon on the display in accordance with a detected current hand location. The image processing system detects a series of hand shapes in the captured frames of video data and determines whether the detected hand shapes match any of a plurality of predefined hand gestures stored in a hand gesture library. In response to a match, the method includes executing an action in accordance with the matching hand gesture. In response to an opening gesture, an element animation system presents one or more graphical elements incrementally moving along a path extending away from the menu icon. A closing hand gesture causes the elements to retreat along the path toward the menu icon.

Abstract: Methods, systems, and devices for generating contact suggestions for a user of a social network. A first score is computed for each one of the plurality of users, the first score being computed using an edge-weighted ranking algorithm based on the user graph. A second score is computed, using a machine learning model, for each user of the plurality of users, the second score of each user being, at least partially, based on the first score of said user, with the second score of each user being representative of a probability of a first user sending a connection request to said user. A ranked contact suggestion list of one or more users of the plurality of users is generated, the one or more users being ranked based on their respective second score.

Abstract: Augmented reality (AR) systems, devices, media, and methods are described for sending and receiving AR objects (e.g., customized AR objects) based on/responsive to interactions with the physical world. AR virtual delivery route overlays are generated responsive to selected virtual delivery routes and include the AR object and a delivery mode (air, tunnel, etc.) corresponding to the virtual delivery route. Physical world actions associated with the delivery mode (blowing adjacent an AR device or scratching a surface) result in sending a communication corresponding to the AR object for delivery to a receiver and generating AR sending overlays including the AR object moving in accordance with the delivery mode.

Abstract: A text entry process for an Augmented Reality (AR) system. The AR system detects, using one or more cameras of the AR system, a start text entry gesture made by a user of the AR system. During text entry, the AR system detects, using the one or more cameras, a symbol corresponding to a fingerspelling sign made by the user. The AR system generates entered text data based on the symbol and provides text in a text scene component of an AR overlay provided by the AR system to the user based on the entered text data.