Abstract: Method of generating customizable avatar outfits and accessories starts with a processor receiving a first identification information associated with a first article of clothing for sale on a first third-party server. The processor generates an outfit update interface comprising an avatar associated with a user of a client device and a first selectable item corresponding to the first article of clothing. The processor receives a selection from the client device of the first selectable item. The processor modifies the avatar to generate a modified avatar that is wearing an avatar outfit that comprises a first item that resembles the first article of clothing. The processor then causes the modified avatar to be displayed on a display screen of the client device. Other embodiments are disclosed.

Abstract: Systems, methods, devices, media, and computer readable instructions are described for local image tagging in a resource constrained environment. One embodiment involves processing image data using a deep convolutional neural network (DCNN) comprising at least a first subgraph and a second subgraph, the first subgraph comprising at least a first layer and a second layer, processing, the image data using at least the first layer of the first subgraph to generate first intermediate output data; processing, by the mobile device, the first intermediate output data using at least the second layer of the first subgraph to generate first subgraph output data, and in response to a determination that each layer reliant on the first intermediate data have completed processing, deleting the first intermediate data from the mobile device. Additional embodiments involve convolving entire pixel resolutions of the image data against kernels in different layers if the DCNN.

Abstract: Aspects of the present disclosure involve a device comprising a computer-readable storage medium storing a program and method for providing contextual information with a keyboard interface. The program and method provide for receiving a media content item provided by a second device; displaying, on a screen of the device, the media content item together with a reply element, the reply element being user-selectable to generate a reply message with respect to the media content item; receiving user selection of the reply element; and in response to receiving the user selection, displaying, in a first preset area of the screen, a keyboard interface for user selection of text to include in the reply message, and displaying, in a second preset area of the screen, a context information interface which is user-selectable to provide information corresponding to at least one type of context associated with the media content item.

Abstract: Disclosed are systems, methods, and non-transitory computer-readable media for misaligned vantage point mitigation for computer stereo vision. A misaligned vantage point mitigation system determines whether vantage points of the optical sensors are misaligned from an expected vantage point and, if so, determines an adjustment variable to mitigate the misalignment based on the location of matching features identified in images captured by both optical sensors.

Abstract: An audio track with vocals is played back using a device with a display screen that displays a video feed from a camera. A location of a mouth depicted in the video feed is detected. A timestamp of playback of the audio track is compared to viseme-timestamp data for the audio track to identify a viseme corresponding to the timestamp of the audio playback. A viseme is positioned at the detected location of the mouth in the video feed.

Abstract: A system comprising a head-wearable apparatus, which includes a plurality of microphones, each microphone of the plurality of microphones being spatially separated from each other microphone of the plurality of microphones, thereby defining a set of spatial separations. The system also includes one or more processors. The system also includes a non-transitory computer readable storage medium including instructions that, when executed by the one or more processors, cause the one or more processors to perform operations including generating, by each microphone of the plurality of microphones, an audio signal corresponding to sound detected by the microphone, thereby generating a plurality of audio signals, and processing the plurality of audio signals, using a trained machine learning model, to generate wearer speech detection data representative of a likelihood that a wearer of the head-wearable apparatus is speaking.

Abstract: A gesture-based wake process for an AR system is described herein. The AR system places a hand-tracking input pipeline of the AR system in a suspended mode. A camera component of the hand-tracking input pipeline detects a possible visual wake command being made by a user of the AR system. On the basis of detecting the possible visual wake command, the AR system wakes the hand-tracking input pipeline and places the camera component in a fully operational mode. If the AR system, using the hand-tracking input pipeline, verifies the possible visual wake command as an actual wake command, the AR system initiates execution of an AR application.

Abstract: A waveguide includes a waveguide body including an optically transmissive material having a refractive index different from a surrounding medium and defining an output surface. The waveguide body is configured to propagate light by total internal reflection in one or more directions substantially tangential to the output surface. The waveguide includes one or more diffractive optical elements (DOEs), each configured to change its diffraction efficiency in response to a respective stimulus, and a DOE driver configured to provide the stimuli to each of the DOEs independently.

Abstract: An antenna system includes a loop conductor and non-loop conductor configured to provide respective loop and dipole antenna functionality. A signal feed mechanism connects to both conductors at a signal feed point for simultaneous signal transceiving. The non-loop conductor comprises two elongate arms with an offset feed point creating shorter and longer arms, where the longer arm has increased width via a loop-shaped portion. When incorporated in eyewear devices, the loop-shaped portion extends circumferentially along a lens holder to retain lenses. The conductors may be configured through a diplexer for different frequency domains, with the non-loop conductor serving as a GPS antenna and the loop conductor as a data communication antenna. The radiation patterns of the conductors may be substantially orthogonal when driven by the signal feed mechanism. The system provides integrated antenna and mechanical functionality in wearable devices.

Abstract: Methods and systems are disclosed for generating 3D assets, such as for an extended reality (XR) experience. The system receives a plurality of textures associated with an object, each texture of the plurality of textures corresponding to a different view of the object and automatically generates an initial three-dimensional (3D) model of the object based on an initial alignment of the plurality of textures to respective portions of the 3D model. The system receives input that adjusts the initial alignment of the plurality of textures to the respective portions of the 3D model and combines the plurality of textures into a single texture based on the input, the single texture defining visual attributes of the object from multiple views. The system stores the 3D model in association with the single texture.

Abstract: Eyewear having an electronic tinting lens for controlling a light transmissive property of an optical assembly. The electronic tinting lens has a substrate and a separator that is integrated with the waveguide to reduce the number of layers within the electronic tinting lens and therefore reduce the weight of the assembly. The upper protective glass layer of the waveguide is used as a substrate for the electronic tinting lens, wherein the electronic tinting lens substrate and the upper waveguide substrate each include an electrode configured to control a tint of the electronic tinting lens. The glass layer additionally encapsulates the waveguide to protect the waveguide against environmental factors, such as moisture.

Abstract: Described are systems and methods for generating personalized videos with customized text messages. An example method includes receiving an input text, a video template including a sequence of frame images, and at least one parameter for animation of the input text across the sequence of frame images, generating, based on the input text and the at least one parameter for animation, a configuration file including a text style for the input text for a frame in the sequence of frame images, and rendering, based on the configuration file, an output frame of an output video, where the output frame includes the frame in the sequence of frame images and a layer, and where the layer includes the input text stylized based on the text style. The method further includes providing an option enabling a user to change the at least one parameter for animation.

Abstract: Systems, devices, media, and methods are presented for using a handheld device such as a ring to manipulate a virtual object being displayed by a wearable device such as eyewear. The path of the virtual object in motion is substantially linked, in time and space, to the course traveled by a hand holding the handheld device. The methods in some implementations include presenting the virtual object on a display at a first location relative to a three-dimensional coordinate system, collecting motion data from an inertial measurement unit on the handheld device, displaying the virtual object at a second location along a path based on the motion data. In some implementations the eyewear includes a projector located and configured to project the display onto a lens assembly.

Abstract: Various examples described herein support or provide generation and management operations of personalized images using machine learning technologies, including receiving portrait images of an entity; using machine learning models to generate an identity that represents the entity; identifying a template that comprises text descriptions of a scene and conditions; and using a machine learning models to generate a personalized image based on the identity and the template.

Abstract: A virtual interface application presented in augmented reality (AR) is described for controlling Internet of Things (IoT) products. The virtual interface application enables a user of a portable electronic device to activate a virtual control interface overlay on a display, receive a selection from the user using her hands or feet, and send a control signal to a nearby IoT product which executes an action in accordance with the selection. The virtual control interface is presented on the display as a virtual object in an AR environment. The virtual interface application includes a foot tracking tool for detecting an intersection between the foot location (in the physical environment) and the virtual surface position (in the AR environment). When an intersection is detected, the virtual interface application sends a control signal with instructions to the IoT product.

Abstract: Eyewear providing an interactive augmented reality experience between two eyewear devices by using alignment between respective 6DOF trajectories, also referred to herein as ego motion alignment. An eyewear device of user A and an eyewear device of user B track the eyewear device of the other user, or an object of the other user, such as on the user's face, to provide the collaborative AR experience. This enables sharing common three-dimensional content between multiple eyewear users without using or aligning the eyewear devices to common image content such as a marker, which is a more lightweight solution with reduced computational burden on a processor. An inertial measurement unit may also be used to align the eyewear devices.

Abstract: Described herein are techniques for generating captions for augmented reality (AR) effects in videos using an adapted multimodal large language model (MLLM). The technique involves sampling frames from base and AR-applied videos, combining them into concatenated frames, and processing them with a hybrid vision encoder. Visual tokens are projected into a language model token space, reshaped, downsampled, and interleaved with text tokens. A fine-tuned large language model processes this input sequence to generate AR effect captions. Optical character recognition extracts text from AR frames, which is combined with generated captions and metadata to produce merged captions and content tags. This approach enables accurate description of temporal AR effects and facilitates downstream applications like search and ranking.

Abstract: Aspects of the present disclosure involve a system for providing a footwear mirror experience. The system accesses a video stream captured by a camera directed at feet of users. The system detects a depiction of one or more feet of a user in the video stream captured by the camera. The system, in response to detecting the one or more feet in the video stream captured by the camera, activates a display screen that is positioned at an eye-level of the user and presents the video stream including the depiction of the one or more feet of the user on the display screen.

Abstract: A waveguide is disclosed for use in an augmented reality or virtual reality display. The waveguide includes a plurality of optical structures exhibiting differences in refractive index from a surrounding waveguide medium. The optical structures are arranged in an array to provide at least two diffractive optical elements overlaid on one another in the waveguide. Each of the two diffractive optical elements is configured to receive light from an input direction and couple it towards the other diffractive optical element which can then act as an output diffractive optical element, providing outcoupled orders towards a viewer. The optical structures have a shape, when viewed in the plane of the waveguide, comprising a plurality of substantially straight sides having respective normal vectors at different angles and this can effectively reduce the amount of light that is coupled out of the waveguide on first interaction with the optical structures.