Abstract: The subject technology receives, by a client device from a storage device, first image data captured by the client device at a previous time. The subject technology receives first metadata corresponding to at least a first image processing operation and a second image processing operation. The subject technology generates, in a first render pass, second image data based on the first metadata and the image processing operation performed on the first image data. The subject technology generates, in a second render pass, third image data based on the first metadata and the second image processing operation performed on the second image data. The subject technology generates second metadata comprising information corresponding to the third image data. The subject technology generates a composite AR content item comprising the second metadata, the third image data, and the first image data.

Abstract: The present application discloses examples of various apparatuses and systems that can be utilized for augmented reality. According to one example, a wearable device that can optionally comprise: a frame configured for wearing by a user; one or more optical elements mounted on the frame; an array having a plurality of light emitting diodes coupled to the one or more optical elements, wherein the one or more optical elements and the array are mounted within a field of view of the user when the frame is worn by the user; and additional onboard electronic components carried by the frame including at least a battery that is configured to provide for electrically powered operation of the array.

Abstract: The present application discloses examples of various apparatuses and systems that can be utilized for augmented reality. According to one example, a wearable device that can optionally comprise: a frame configured for wearing by a user; one or more optical elements mounted on the frame; an array having a plurality of light emitting diodes coupled to the one or more optical elements, wherein the one or more optical elements and the array are mounted within a field of view of the user when the frame is worn by the user; and additional onboard electronic components carried by the frame including at least a battery that is configured to provide for electrically powered operation of the array.

Abstract: System, method, and non-transitory computer readable medium for presenting images on a mobile device. Images are presented by monitoring one or more physical characteristics surrounding the mobile device using at least one sensor, determining a contextual state of the mobile device based on the monitored one or more physical characteristics, selecting an image from a plurality of related images associated with the determined contextual state, generating at least one overlay image from the selected image, and presenting the at least one overlay image with an optical assembly of the mobile device.

Abstract: An eyewear device presents, via an image display, an initial displayed image. The initial displayed image has an initial field of view corresponding to an initial head direction or an initial eye direction. Eyewear device detects movement of a user of the eyewear device by: (i) tracking, via a head movement tracker, a head movement of a head of the user, or (ii) tracking, via an eye movement tracker, an eye movement of an eye of the user of the eyewear device. Eyewear device determines a field of view adjustment to the initial field of view of the initial displayed image based on the detected movement of the user. Field of view adjustment includes a successive field of view corresponding to a successive head direction or a successive eye direction. Eyewear device generates a successive displayed image based on the field of view adjustment and presents the successive displayed image.

Abstract: System, method, and non-transitory computer readable medium for presenting images on a mobile device. Images are presented by monitoring the location and the orientation of the mobile device, requesting previously captured images corresponding to where the previously captured images were captured in relation to the location from a server, receiving the requested previously captured images from the server, selecting images from the requested previously captured images responsive to the location and the orientation of the mobile device, generating overlay images from the selected images including image icons associated with the selected images, presenting the overlay images on an optical assembly, receiving an image selection identifying one of the image icons in the presented overlay images, and display the selected image associated with the identified image icon on the viewing area of the optical assembly.

Abstract: Systems, devices, media, and methods are presented for detecting and interpreting motion of a device and a remote object to control operations of the device. The systems and methods identify a sensor input within a drone. The sensor input indicates movement of the drone within a three dimensional space. The systems and methods determine one or more movement attributes from the sensor input and, in response to the one or more movement attributes, selects one or more maneuvers corresponding to at least one movement attribute. The system and methods then execute the one or more maneuvers by controlling one or more drone control components to move the drone within the three dimensional space.

Abstract: Systems, methods, and non-transitory computer readable media for augmenting scenes viewed thorough displays of an eyewear devices with audio-related image information. Scenes may be augmented by capturing, via a camera of the eyewear device, initial images of a scene, identifying features within the initial images; receiving audio-related image information (e.g., lyrics and/or images), registering the audio-related image information to the identified features, creating audio-based visual overlays including the audio-related image information registered to the identified features, and displaying the audio-based visual overlays over the scene.

Abstract: System, method, and non-transitory computer readable medium for presenting images on a mobile device. Images are presented by monitoring one or more physical characteristics surrounding the mobile device using at least one sensor, determining a contextual state of the mobile device based on the monitored one or more physical characteristics, selecting an image from a plurality of related images associated with the determined contextual state, generating at least one overlay image from the selected image, and presenting the at least one overlay image with an optical assembly of the mobile device.

Abstract: Apparatuses, systems and methods for wearable devices such as eyewear are described. According to one embodiment, the wearable device includes a body, electronics, and a connector. The body is configured to hold one or more optical elements, the body being disposable between a collapsed condition and a wearable condition in which the device is wearable by a user to hold the one or more optical elements within user view. The electronics are carried by the body. The connector is configured to enable establishment of an electrical and/or electronic connection with the electronics via the connector, the connector being housed by the body such that the connector is substantially obscured from view when the body is in the wearable condition, and such that the connector is exposed for connective access when the body is in the collapsed condition.

Abstract: System, method, and non-transitory computer readable medium for presenting images on a mobile device. Images are presented by monitoring the location and the orientation of the mobile device, requesting previously captured images corresponding to where the previously captured images were captured in relation to the location from a server, receiving the requested previously captured images from the server, selecting images from the requested previously captured images responsive to the location and the orientation of the mobile device, generating overlay images from the selected images including image icons associated with the selected images, presenting the overlay images on an optical assembly, receiving an image selection identifying one of the image icons in the presented overlay images, and display the selected image associated with the identified image icon on the viewing area of the optical assembly.

Abstract: A method for controlling movement of a drone is disclosed. A spatial vector between a flight-capable drone and a reference object is computed. The spatial vector defines a direction and a distance by which the drone is spaced from the reference object. Flightpath attributes based on the computed vector are determined. The flightpath attributes include one or more of a flight direction, a flight distance, and a flight speed. The flight direction is variable as a function of the direction of the spatial vector. The flight distance is variable as a function of the distance of the spatial vector. The flight speed is variable as a function of the distance of the spatial vector.

Abstract: Apparatuses, systems and methods for wearable devices such as eyewear are described. According to one embodiment, the wearable device includes a body, electronics, and a connector. The body is configured to hold one or more optical elements, the body being disposable between a collapsed condition and a wearable condition in which the device is wearable by a user to hold the one or more optical elements within user view. The electronics are carried by the body. The connector is configured to enable establishment of an electrical and/or electronic connection with the electronics via the connector, the connector being housed by the body such that the connector is substantially obscured from view when the body is in the wearable condition, and such that the connector is exposed for connective access when the body is in the collapsed condition.

Abstract: A wearable or a mobile device includes a camera to capture an image of a scene with an unknown object. Execution of programming by a processor configures the device to perform functions, including a function to capture, via the camera, the image of the scene with the unknown object. To create lightweight human-machine user interactions, execution of programming by the processor further configures the device to determine a recognized object-based adjustment; and produce visible output to the user via the graphical user interface presented on the image display of the device based on the recognized object-based adjustment. Examples of recognized object-based adjustments include launch, hide, or display of an application for the user to interact with or utilize; display of a menu of applications related to the recognized object for execution; or enable or disable of a system level feature.

Abstract: A drone system is configured to capture an audio stream that includes voice commands from an operator, to process the audio stream for identification of the voice commands, and to perform operations based on the identified voice commands. The drone system can identify a particular voice stream in the audio stream as an operator voice, and perform the command recognition with respect to the operator voice to the exclusion of other voice streams present in the audio stream. The drone can include a directional camera that is automatically and continuously focused on the operator to capture a video stream usable in disambiguation of different voice streams captured by the drone.

Abstract: In a camera-enabled electronic device, photo capture is triggered by a press-and-hold input only if the holding duration of the press-and-hold input is greater than a predefined threshold duration. A press-and-hold input shorter in duration than the threshold triggers video capture. Thus, a short press triggers video capture, while a long press triggers photo capture.

Abstract: System, method, and non-transitory computer readable medium for presenting audio-based visual overlays on see-through optical assemblies. Overlays are presented by capturing, via a camera of an eyewear device, initial images of a scene, receiving an audio signal, modifying the initial images responsive to the audio signal to create overlay images, and displaying, via a see-through optical assembly of the eyewear device, the overlay images to a wearer of the eyewear device over the scene in a viewing area of the eyewear device.

Abstract: A virtual mailbox is established for a first user based on a virtual mailbox definition specified by the first user. A message directed to the first user is received from a first device associated with a second user. A location within a real-world environment of the first user corresponding to the virtual mailbox is identified. A marker associated with the virtual mailbox is detected within the real-world environment of the first user. Based on detecting the marker, the second device presents the message overlaid on the real-world environment at the location corresponding to the virtual mailbox.

Abstract: A method for controlling movement of a drone is disclosed. A spatial vector between a flight-capable drone and a reference object is computed. The spatial vector defines a direction and a distance by which the drone is spaced from the reference object. Flightpath attributes based on the computed vector are determined. The flightpath attributes include one or more of a flight direction, a flight distance, and a flight speed. The flight direction is variable as a function of the direction of the spatial vector. The flight distance is variable as a function of the distance of the spatial vector. The flight speed is variable as a function of the distance of the spatial vector. In an automated operation, movement of the drone is controlled according to the determined flightpath attributes.