Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for performing operations comprising: receiving an image that includes a depiction of a real-world environment; processing the image to obtain data indicating presence of a real-world object in the real-world environment; receiving input that selects an AR experience comprising an AR object; determining that the real-world object detected in the real-world environment depicted in the image indicated in the obtained data corresponds to the AR object; applying a machine learning technique to the image to generate a new image that depicts the real-world environment without the real-world object; and applying the AR object to the new image to generate a modified new image that depicts the real-world environment including the AR object in place of the real-world object.

Abstract: A system and method for accessing a media item on a mobile device are described. The mobile device includes a media placement application and a media display application. The media placement application receives a selection of a media item generated by the mobile device. The media placement application generates access conditions for the media item based on geolocation and position information of the mobile device associated with the selected media item. The media display application monitors the geolocation and position of the mobile device and determines whether the geolocation and position of the mobile device meet the access conditions of the selected media item. The media display application generates a notification that the selected media item is available to view in a display of the mobile device in response to determining that the geolocation and position of the mobile device meet the access conditions of the selected media item.

Abstract: Systems and methods for projecting each of a chronology of images as a sequence of images using a shifting element as part of a near-eye display system are provided for use in virtual reality, augmented reality, or mixed reality systems. In some example embodiments, a chronology of images is received by a peripheral sequencing system. The system divides each image into image portions and generates sequences of image portions to recreate the images based on arrangement data. The system then causes a high-speed display of each sequence of images such that they appear simultaneous to a viewer. In some embodiments, the projection is transmitted to a shifting optical element such as a rotating micromirror that propagates a display to a user. In some embodiments, the system further detects and corrects for image and environmental distortions.

Abstract: Eyewear configured to identify movements of a remote physical object to provide a 3D painting interactive augmented reality experience. In an example, the eyewear uses the identified movements of the remote physical object as a virtual brush to create a three-dimensional (3D) virtual object on an eyewear display. The eyewear determines positional information of the remote physical object. The eyewear uses the positional information and responsively displays and edits the virtual object as a function of the relative 3D positioning of the remote physical object.

Abstract: There is disclosed a waveguide for use in an augmented reality or virtual reality display. The waveguide comprises a plurality of optical structures in a photonic crystal. The plurality of optical structures are arranged in an array to provide at least two diffractive optical elements. Each of the two diffractive optical elements is configured to receive light from an input direction and couple it towards the other diffractive optical element which can then act as an output diffractive optical element, providing outcoupled orders towards a viewer. The plurality of optical structures respectively have a shape, when viewed in the plane of the waveguide, comprising twelve substantially straight sides, six of the sides having respective normal vectors at a first angle, and the other six of the sides having respective normal vectors at a second angle which is different to the first angle.

Abstract: A carousel module that enables a user of an application to access related entities which may be friends where the friends are friends to the user within a messaging system. The carousel module accesses profile information of the friends from the messaging system and makes the profile information available within the application. The carousel module provides new functions related to friends for the user within the application. Methods include a carousel module accessing from a messaging system a list of friends of a user of an application, the list of friends indicating friends of a user within the messaging system. The methods include causing a carousel to be displayed for the user on a computing device and causing a status within the application of the friends of the carousel to be displayed on the computing device.

Abstract: Eyewear having a speech to moving lips algorithm that receives and translates speech and utterances of a person viewed through the eyewear, and then displays an overlay of moving lips corresponding to the speech and utterances on a mask of the viewed person. A database having text to moving lips information is utilized to translate the speech and generate the moving lips in near-real time with little latency. This translation provides the deaf/hearing impaired users the ability to understand and communicate with the person viewed through the eyewear when they are wearing a mask. The translation may include automatic speech recognition (ASR) and natural language understanding (NLU) as a sound recognition engine.

Abstract: A method of generating a modified video file via a user interface presented on a mobile device including a display is disclosed. The method comprises displaying media corresponding to a first video file on a display of the mobile video device. In response to receiving an initial user input, generation of a second video file corresponding to the displayed media commences. During the displaying of the media corresponding to the first video file, user inputs to manipulate the first video file are received. The second video file is then generated from the first video file as manipulated by the user inputs. A sequence of versions of the video file may be created in a similar manner, which are selectable via a video version stack.

Abstract: Systems and methods are provided for: receiving an image containing a code that has one or more visual qualities that fail to satisfy respective thresholds; applying a trained machine learning model to find a rough location of the code by generating a bounding box and cropping out the portion of the image; applying another trained machine learning model to the portion of the image to estimate key point locations of the code depicted in the portion of the image, aligning the portion of the image that depicts the code based on the estimated key point locations; and decoding, by the other trained machine learning model, the aligned portion of the image that depicts the code.

Abstract: Systems and methods for varying an intensity of pixels of displays are provided. A display subsystem may include a display driver for receiving image frame data and commands from an image or data source, including pixel intensity values for bit planes of an image frame; a parser for receiving the image frame data and the commands, and configured to determine a drive waveform having an pixel drive value and a pixel drive time interval for each bit plane of the image frame data; a display backplane for receiving the drive waveform, the display backplane including a pixel array comprising an array of pixels, each pixel driven by a pixel circuit; and display driver circuitry for driving the pixels in accordance with the drive waveform; and wherein an intensity of the pixel varies for each bit plane according to the pixel drive value and the pixel drive time interval.

Abstract: Systems and methods are provided for receiving a request to generate a media overlay corresponding to a home of a first user, and for generating the media overlay corresponding to the home of the first user using media content received in the request. The systems and methods further provide for associating, with the media overlay, a location of the home of the first user and a selection of users to grant permission to access the media overlay corresponding to the home of the first user. The systems and methods further provide for determining whether a second user and a location corresponding to the second computing device trigger access to the media overlay and providing the media overlay to the second computing device, based on determining that the second user and location corresponding to the second computing device trigger access to the media overlay.

Abstract: System, method, and non-transitory computer readable medium for presenting images on a mobile device. Images are presented by monitoring the location and the orientation of the mobile device, requesting previously captured images corresponding to where the previously captured images were captured in relation to the location from a server, receiving the requested previously captured images from the server, selecting images from the requested previously captured images responsive to the location and the orientation of the mobile device, generating overlay images from the selected images including image icons associated with the selected images, presenting the overlay images on an optical assembly, receiving an image selection identifying one of the image icons in the presented overlay images, and display the selected image associated with the identified image icon on the viewing area of the optical assembly.

Abstract: A wearable device includes a frame and a magnetic coupler opening formed in the frame. Wearable device further includes a processor, a memory accessible to the processor, and a very high frequency (VHF) radio transceiver for data transmission and reception and connected to the processor. Wearable device further includes a magnetic coupler connected to the VHF radio transceiver. Magnetic coupler includes a diamagnetic material shaped to form a VHF transmission or reception terminal that partially or fully aligns with the magnetic coupler opening. During transmission, magnetic coupler is configured to radiate transmitted VHF band radio modulated signals into tissue of the user. During reception, magnetic coupler is configured to absorb received VHF band radio modulated signals from the tissue of the user.

Abstract: Methods and systems are disclosed for performing operations for providing a shared augmented reality experience in a video chat. A video chat can be established between a plurality of client devices. During the video chat, videos of users associated with the client devices can be displayed. During the video chat, a request from a first client device to activate a first AR experience can be received, and in response, and body parts of users depicted in the videos are modified to include one or more AR elements associated with the first AR experience.

Abstract: A wearable device includes a body having fasteners and a frame coupled between two fasteners. The frame includes first and second sections. A first portion of the body includes the first section of the frame and one fastener and a second portion of the body includes the second section of the frame and the other fastener. A speaker and a microphone are connected to the first portion and another speaker and another microphone are connected to the second portion. The body also includes a processor, memory accessible to the processor, and programming in the memory for configuring the processor to selectively activate the speakers and microphones such that a first speaker emits an output sound signal while a first microphone and a second speaker are deactivated and a second microphone captures an input sound signal during the emission of the output sound signal by the first speaker.

Abstract: A messaging system that extracts accompaniment portions from songs. Methods of accompaniment extraction from songs includes receiving an input song that includes a vocal portion and an accompaniment portion, transforming the input song to an input image, where the input image represents the frequencies and intensities of the input song, processing the input image using a convolutional neural network (CNN) to generate an output image, and transforming the output image to an output accompaniment, where the output accompaniment includes the accompaniment of the input song.

Abstract: A mixed-reality media content system may be configured to perform operations that include: causing display of image data at a client device, the image data comprising a depiction of an object that includes a graphical code at a position upon the object; detecting the graphical code at the position upon the depiction of the object based on the image data; accessing media content within a media repository based on the graphical code scanned by the client device; and causing display of a presentation of the media content at the position of the graphical code upon the depiction of the object at the client device.

Abstract: A case for an eyewear device includes a body defining an opening leading to a storage chamber that is sized for retaining the eyewear device. A cover depends from the body and is movable between an open position, in which the opening is exposed, and a closed position, in which the opening is covered by the cover. A battery is mounted to the body for charging the eyewear device. A detector is positioned on either the body or the cover for detecting when the cover is in the open position or the closed position. A display displays a charge state of the battery when the cover is in the open position.

Abstract: Method starts with a processor causing an electronic program guide (EPG) to be displayed on a first client device that is associated with a first user. Processor detects a selection of a selectable item included in the EPG associated with a media content item by the first user. Processor identifies a second user that is connected to the first user on a communication platform. Processor retrieves a main user identification associated with the first user and a secondary user identification associated with the second user and generates a request for the media content item including media content identification, main user identification, and secondary user identification. Processor then receives a playlist associated with the media content item that comprises media content item segments including first avatar associated with first user and second avatar associated with second user. Other embodiments are disclosed herein.

Abstract: A UAV having a manual gimbal including a camera, and a flight mode selector configured to select both a flight mode and manually establish a camera position as a function of the selected fight mode. A controller responds to a position of the gimbal or selector to establish the flight mode. The flight mode is selected from several available modes, for example, a horizontal flight mode, a 45-degree flight mode, and a vertical (aerial) flight mode. The flight mode selector is mechanically coupled to the gimbal and establishes a pitch angle of the gimbal, and thus the camera angle attached to the gimbal.