Abstract: A wearable device includes a frame and a magnetic coupler opening formed in the frame. Wearable device further includes a processor, a memory accessible to the processor, and a very high frequency (VHF) radio transceiver for data transmission and reception and connected to the processor. Wearable device further includes a magnetic coupler connected to the VHF radio transceiver. Magnetic coupler includes a diamagnetic material shaped to form a VHF transmission or reception terminal that partially or fully aligns with the magnetic coupler opening. During transmission, magnetic coupler is configured to radiate transmitted VHF band radio modulated signals into tissue of the user. During reception, magnetic coupler is configured to absorb received VHF band radio modulated signals from the tissue of the user.

Abstract: An AR system includes multiple input-modalities. A hand-tracking pipeline supports Direct Manipulation of Virtual Object (DMVO) and gesture input methodologies. In addition, a voice processing pipeline provides for speech inputs. Direct memory buffer access to preliminary hand-tracking data, such as skeletal models, allows for low latency communication of the data for use by DMVO-based user interfaces. A system framework component routes higher level hand-tracking data, such as gesture identification and symbols generated based on hand positions, via a Snips protocol to gesture-based user interfaces.

Abstract: Optical display engines with reduced size and/or increased efficiency for Augmented Reality (AR) and near-eye devices that incorporate LED, microLED, and/or OLED displays. Example optical systems provide different light paths for polarized (or split unpolarized) light, with recombining oppositely polarized light corresponding to different image portions/channels, in a compact system using polarized beam splitters, quarter wave plates, half wave plates, and reflective elements such as curved mirrors or reflective lenses, resulting in high optical power. Images and/or portions thereof are presented at multiple observation planes to project a more realistic synthetic image, and at different angular resolutions to enable creation of a large composite field of view with high apparent resolution from a single small and efficient display.

Abstract: The systems and techniques described herein relate to predicting user conversions in online advertising. Input data associated with user and advertisement features may be processed through neural networks to generate embedding representations or feature cross representations. A multi-task layer calculates probabilities associated with multiple user actions like clicks, page views, sign-ups, or purchases. Click-through and view-through conversion probabilities may be calculated to generate a score. The systems and techniques described herein perform predictions on multiple types of user actions despite data sparsity and negative transfer challenges, enhancing advertisement targeting and improving conversion metrics.

Abstract: A client device processing received emoji messages using emoji-first messaging. Text messaging is automatically converted to emojis by an emoji-first application so that only emojis are communicated from one client device to another client device. Each client device has a library of emojis that are mapped to words, which libraries are customizable and unique to the users of the client devices, such that the users can communicate secretly in code. Upon receipt of a string of emojis, a user can select the emoji string to convert to text if desired, for a predetermined period of time.

Abstract: Examples in the present disclosure relate to scale estimation for facilitating extended reality (XR) experiences. An image of a hand of a user is captured via one or more optical sensors of an XR device. The image is processed to detect a hand pose relative to the XR device. A hand scale estimate corresponding to the detected hand pose is accessed. The hand scale estimate is one of a plurality of hand scale estimates each uniquely associated with a respective hand pose. The hand scale estimate is applied to generate positional data for one or more features of the hand of the user. The XR device tracks the hand of the user based on the positional data while the user uses the XR device.

Abstract: Methods and systems are disclosed for generating video by applying a template to various content items. The methods and systems select, by an interaction application, a video generation template comprising instructions for combining a set of content items into a video using one or more augmented reality (AR) elements. The methods and systems identify a subset of content items from a plurality of previously captured content items and modify one or more content items of the identified subset of content items based on the AR elements of the video generation template. The methods and systems generate a video comprising a collection of content items including the identified subset of content items and the modified one or more content items based on the instructions of the video generation template.

Abstract: A method and system for augmenting live video feeds with augmented reality (AR) effects. A live video feed comprising a plurality of video frames is received and the format of the video frames is determined. The video frames are converted to a format compatible with an AR software development kit (SDK). One or more AR effects from the AR SDK are applied to the converted frames. This can include detecting depictions of objects in the frames and applying effects to the detected objects. The effects can be selected based on detected object types. The frames are then re-converted back to the original format. If the frame rate differs between the video feed and AR SDK, frame rate conversion is performed before and after applying the AR effects. The augmented video frames including the AR effects are provided as output, such as for broadcast or display.

Abstract: A model generation system captures, using cameras, hand-tracking data of a gesture made by a user demonstrating the gesture. The model generation system generates a three-dimensional model of the gesture using the hand-tracking data and provides a display of the three-dimensional model to the user. The model generation system receives, from the user, model refining data refining the three-dimensional model. The model generation system generates a refined three-dimensional model of the gesture using the model refining data and the three-dimensional model. The refined three-dimensional model is used for detecting the gesture.

Abstract: The subject technology receives information for a product. The subject technology generates a 3D model file of the product in a first format. The subject technology converts the 3D model file to a 3D object file in a second format. The subject technology associates the 3D object file to the product in a product catalog service. The subject technology publishes an augmented reality (AR) content generator corresponding to the product.

Abstract: A method for improving the startup time of a six-degrees of freedom tracking system is described. An augmented reality system receives a device initialization request and activates a first set of sensors in response to the device initialization request. The augmented reality system receives first tracking data from the first set of sensors. The augmented reality system receives an augmented reality experience request and in response to the augmented reality request, causes display of a set of augmented reality content items based on the first tracking data and simultaneously activates a second set of sensors. The augmented reality system receives second tracking data from the activated second set of sensors. The augmented reality system updates the display of the set of augmented reality content items based on the second tracking data.

Abstract: One aspect disclosed is a method including determining a location from a positioning system receiver, determining, using a hardware processor and the location, that the location is approaching a path of direction of visual direction information, displaying the visual direction information on a display of a wearable device in response to the determining, determining, using the positioning system receiver, whether the turn of the visual direction information has been made, determining, by the hardware processor, a first period of time for display of the content data based on whether the turn of the visual direction information has been made, powering on the display and displaying, using the display, content data for the first period of time, turning off the display and the hardware processor following display of the content data.

Abstract: Method for generating haptic feedback responses using haptic augmentations starts with processor receiving a media content item from a first client device and receiving from the first client device a selection of a haptic overlay associated with a haptic feedback response to be applied to the media content item. The processor generates a modified media content item by overlaying the haptic overlay on the media content item. The processor receives from the first client device a selection of a second user associated with the second user and causes the modified media content item to be displayed by a second user interface of the second client device. The processor detects a selection of the haptic overlay from the second client device, and in response to detecting the selection of the haptic overlay, causing the second user interface to generate the haptic feedback response. Other embodiments are described herein.

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing programs and methods for performing operations comprising: receiving a request from a client device of a first user to engage in a shared virtual reality shopping experience with a second user; generating, for display on respective client devices of the first and second users, the shared virtual reality shopping experience comprising a plurality of virtual reality items that represent real-world products; receiving, from the client device of the second user, data indicating a selection of a first virtual reality item of the plurality of virtual reality items made by the second user; and modifying a display attribute of the first virtual item in the display of the shared virtual reality shopping experience on the client device of the first user to indicate the selection of the first virtual reality item made by the second user.

Abstract: The subject technology receives, by one or more hardware processors implementing a local wireless network, a request from a client device to mirror media content displayed on a screen of the client device on a wearable device. In response to the request, the subject technology causes a display of the media content in a mirroring lens of the wearable device. While the media content is being displayed in the mirroring lens of the wearable device, the subject technology tracks hand gestures of a user wearing the wearable device and viewing the media content displayed in the mirroring lens of the wearable device. The subject technology processes navigational or manipulation data based on the tracked hand gestures and sends a navigation or manipulation instruction to the client device or a mirroring lens processor of the wearable device based on the tracked hand gestures.

Abstract: Methods and systems are disclosed for generating 3D assets, such as for an extended reality (XR) experience. The system accesses a three-dimensional (3D) model of an object that has been generated based on at least one texture image of the object. The system processes the at least one texture image of the object by a machine learning model to generate a depth map and selects a portion of the 3D model. The system modifies the selected portion of the 3D model based on the depth map that has been generated by the machine learning model.

Abstract: A system is disclosed, including a processor and a memory. The memory stores instructions that, when executed by the processor, configure the system to perform operations. Autoexposure (AE) primary camera information is obtained, identifying a first camera as an AE primary camera to be used for computing AE settings of the first camera and a second camera. First camera region of interest (ROI) information and second camera ROI information are obtained, representative of a first number of ROIs within a field of view (FOV) of the first camera and a second number of ROIs within a FOV of the second camera. In response to determining that the first number is zero and the second number is greater than zero, the AE primary camera information is updated to identify the second camera as the AE primary camera, and the second camera ROI information is processed to generate the AE settings.

Abstract: Methods and systems are disclosed for detecting whether a wearable device is being worn by a user. The system transmits a radio signal from a first communication device of a wearable device to a second communication device of the wearable device and measures a signal strength associated with the radio signal received by the second communication device. The system compares the signal strength to a threshold value and generates an indication of a wear status associated with the wearable device based on comparing the signal strength to the threshold value.