Abstract: Systems, devices, media, and methods are presented for an immersive augmented reality (AR) experience using an eyewear device. A portable eyewear device includes a processor, a memory, and a display projected onto at least one lens assembly. The memory has programming stored therein that, when executed by the processor, captures information depicting an environment surrounding the device and identifies a match between objects in that information and predetermined objects in previously obtained information for the same environment. When the position of the eyewear device reaches a preselected location with respect to the matched objects, a physical output is provided to produce the immersive experience. The physical output changes as the position of the eyewear device moves to maintain the immersive experience.

Abstract: Apparatus and methods are disclosed that relate to contact lenses having multiple power sources. In particular, in one aspect, a contact lens can include a transparent substrate and a circuit. The circuit can include one or more sensors, circuitry, one or more photovoltaic cells disposed on or within at least a portion of the substrate, and a hybrid power component that supplies power to the circuitry that includes solar power.

Abstract: Methods and systems are disclosed for using machine learning models to perform pixel-based deformation of fashion items using N-flows estimation. The methods and systems receive one or more images depicting a first person in a first pose and receive a source image depicting a target fashion item worn on a portion of a body of a second person in a second pose, the target fashion item comprising a plurality of parts. The methods and systems process, using one or more machine learning models, the one or more images together with the source image to generate a set of data comprising a plurality of flow fields each associated with a different one of the plurality of parts of the target fashion item. The methods and systems modify a portion of the one or more images to overlay the target fashion item on the first person in the first pose.

Abstract: A method of providing guidance, such as to the user of a personal mobility device, uses a head-worn device including a camera and one or more display devices. The method includes capturing images using the camera of the head-worn device and identifying an area, represented in the captured images, having a certain permission associated therewith. An augmented reality effect is overlaid on the area, and the captured images with the overlaid augmented reality effect are displayed on the head-worn device.

Abstract: Systems, devices, media, and methods are presented for using a flexible electronic device to selectively interact with an eyewear device. A portable eyewear device includes a processor, a memory, and a display projected onto at least one lens assembly. A flexible electronic device includes an integrated circuit, a plurality of input sensors, and a power system, all mounted on a flexible substrate that is sized and shaped to conform to a graspable object such as a ring. The flexible electronic device operates according to a power budget, operating on a sensor power budget until it detects a first interaction with at least one of the input sensors. If the first interaction exceeds a sensitivity threshold, the flexible electronic device sends a wake signal to a nearby eyewear device. In response to the wake signal, the eyewear device presents a graphical user interface (GUI) on the display.

Abstract: In some implementations, a system may establish a video call between a first device associated with a first user and a second device associated with a second user of a communications platform. The system may present a video interface for the video call, the video interface comprising a first video stream generated by the first device of the first user and a second video stream generated by the second device associated with the second user. The system may present a first set of image augmentations selected by the communications platform in the video interface, the first set of video augmentations being selectable by the first user for augmentation of the first video stream generated by the first user device. The system may identify a second set of image augmentations used by a further set of users of the communications platform.

Abstract: Systems and methods for improving operating characteristics of displays such as liquid crystal on silicon displays.

Abstract: A miniaturized active dual pixel stereo system and method for close range depth extraction includes a projector adapted to project a locally distinct projected pattern onto an image of a scene and a dual pixel sensor including a dual pixel sensor array that generates respective displaced images of the scene. A three-dimensional image is generated from the displaced images of the scene by projecting the locally distinct projected pattern onto the image of the scene, capturing the respective displaced images of the scene using the dual pixel sensor, generating disparity images from the respective displaced images of the scene, determining depth to each pixel of the disparity images, and generating the three-dimensional image from the determined depth to each pixel. A three-dimensional image of a user's hands generated by the active dual pixel stereo system may be processed by gesture recognition software to provide an input to an electronic eyewear device.

Abstract: Eyewear having an image signal processor (ISP) dynamically operable in a camera pipeline for augmented reality (AR) and computer vision (CV) systems. Multi-purpose cameras are used for simultaneous image capture and CV on wearable AR devices. The cameras are coupled to a frame and configured to generate images, wherein the cameras and the ISP operate in a first AR mode and capture images having a first resolution suitable for use in AR, and are configured to operate in a second CV mode to provide the images having a second resolution suitable for use in CV. The first resolution in the AR mode is higher than the second resolution in the CV mode, and the cameras and the ISP consume less power in the second CV mode than the first AR mode. The cameras and the ISP save significant system power by operating in the low power mode CV mode.

Abstract: A head-wearable extended reality (XR) device includes a display arrangement. The display arrangement has a display to display virtual content, and also has one or more optical elements to direct the virtual content along an optical path to an eye of a user of the XR device. The virtual content is presented in a virtual content field of view. The display arrangement further includes an adjustment mechanism to alter the optical path so as to adjust the virtual content field of view between at least two display modes.

Abstract: Systems and methods herein describe pausing operation of an augmented reality device based on user facial movement by monitoring user eye movement of a user of a head-wearable apparatus, determining that the user eye movement is directionally facing away from a display device coupled to the head-wearable apparatus, based on the determination, pausing operation of one or more sensors of the head-wearable apparatus.

Abstract: A decentralized two-phase commit in a datastore or database includes determining pre-validating a transaction, writing first metadata to a leader partition, the first metadata including a state indicating preparing and mutations of the transaction, and writing second metadata of a follower partition to indicate the follower partition is locked by the leader partition. The decentralized two-phase commit further includes updating the state of the first metadata to indicate committed and performing the transaction on the leader partition, performing the transaction on the follower partition and unlocking the follower partition, and updating the first metadata to remove the mutations of the transaction and updating the state of the first metadata to indicate the transaction is completed. The decentralized two-phase commit may further include selecting a leader partition key and updating the state of the first metadata to indicate committed unapplied.

Abstract: Systems and methods are provided for notifying users about videos in a playback sequence. The systems and methods determine that a video that meets a criterion is currently available on a video server associated with a messaging client. In response to determining that the video that meets the criterion is currently available, the messaging client on the client device prefetches a sequence of videos from a recommendation engine that match a profile of a user associated with the messaging client. The recommendation engine is being used to provide sequence of videos to a video playback graphical user interface (GUI) that automatically plays back the videos in the sequence. The systems and methods determine that the video that meets the criterion is in a first position in the sequence of videos and, in response, present a notification that indicates the availability of the video on the video playback GUI.

Abstract: A method for minimizing latency of moving objects in an augmented reality (AR) display device is described. In one aspect, the method includes determining an initial pose of a visual tracking device, identifying an initial location of an object in an image that is generated by an optical sensor of the visual tracking device, the image corresponding to the initial pose of the visual tracking device. rendering virtual content based on the initial pose and the initial location of the object, retrieving an updated pose of the visual tracking device, tracking an updated location of the object in an updated image that corresponds to the updated pose, and applying a time warp transformation to the rendered virtual content based on the updated pose and the updated location of the object to generate transformed virtual content.

Abstract: Methods and systems are disclosed for performing operations for estimating a 3D scene representation from one or multiple 2D images. The operations include: receiving one or multiple two-dimensional (2D) images representing a real-world environment; and generating, by a machine learning model, a three-dimensional (3D) scene representation of the 2D image, which explicitly (separately) defines the a 3D shape and appearance of the background as well as a 3D position, 3D shape and appearance of each object of the scene depicted in the set of images, where the machine learning model has been trained in an unsupervised approach from a dataset of images and their camera poses (e.g. without any manually labelled annotations, such as depth maps, segmentation masks, object poses).

Abstract: Methods, devices, media, and other embodiments are described for a state-space system for pseudorandom animation. In one embodiment animation elements within a computer model are identified, and for each animation element motion patterns and speed harmonics are identified. A set of motion data values comprising a state-space description of the motion patterns and the speed harmonics are generated, and a probability assigned to each value of the set of motion data values for the state-space description. The probability can then be used to select and update a particular motion used in an animation generated from the computer model.

Abstract: Systems and methods for radial gesture navigation are provided. In example embodiments, user input data is received from a user device. The user input data indicates a continuous physical user interaction associated with a display screen of the user device. An initial point and a current point are detected from the user input data. A radius distance for a circle that includes the current point and is centered about the initial point is determined. An action is selected from among multiple actions based on the radius distance being within a particular range among successive ranges along a straight line that starts at the initial point and extends through the circle. Each range among the successive ranges corresponds to a particular action among the multiple actions. The selected action is performed in response to detecting a completion of the continuous physical user interaction.

Abstract: Method of creating shared AR session based on a gesture starts with server receiving observed motion data from first device associated with first user. First device generating observed motion data based on an analysis of data stream comprising images of second user performing a gesture. Second user being associated with second device. Server receiving from second device captured motion data that corresponds to the gesture. Captured motion data being recorded by a sensor included in second device. Server determines whether there is a match between observed motion data from first device and captured motion data from second device. In response to determining that there is the match, server generates shared AR session between first device and second device and causes shared AR session to be displayed by first device and second device. Other embodiments are described herein.

Abstract: Systems and methods are provided for retrieving first query result data associated with a first user account and rendering the first query result data into a first result item, generating a shareable search result stream comprising the first result item associated with the first user account, retrieving second query result data associated with a second user account and rendering the second query result data into a second result item, adding the second result item to the shareable search result stream associated with the first user account, and providing the sharable search result stream comprising the first result item and the second result item to a first computing device associated with the first user account and a second computing device associated with the second user account.

Abstract: The systems and methods display, by at least one processor on a first portion of a real-world environment visible on a display of a user device, one or more virtual objects at a first virtual coordinate in three-dimensional (3D) space. The systems and methods receive a request to move the one or more virtual objects to a second virtual coordinate in 3D space. The systems and methods in response to receiving the request, maintain display of the one or more virtual objects at the second virtual coordinate as a second portion of the real-world environment is visible on the display of the user device.