Abstract: In one embodiment, a method includes establishing a video call between a plurality of client systems, wherein access to an assistant system is persistently maintained during the video call, receiving, from a first client system of the plurality of client systems, a request by a first user to be performed by the assistant system during the video call, wherein the request references one or more activities associated with one or more users associated with the plurality of client systems, analyzing, by a context engine of the assistant system, images of a scene of the video call to identify the one or more activities within the scene, instructing the assistant system to execute the request based on the identified one or more activities, and sending, to one or more of the plurality of client systems, a response to the request while maintaining the video call between the plurality of client systems.

Abstract: In one example, an apparatus is provided. The apparatus is part of a mobile device and comprises: an array of pixel cells each configured to perform an image sensing operation, one or more attributes of the image sensing operation being configurable based on an image sensing configuration; a sensor data processor configured to execute a hierarchical set of processing operations; and a controller configured to: receive, from the sensor data processor, a first-level processing output from a first-level processing operation of the hierarchical set of processing operations; and based on the first-level processing output satisfying a condition, control the sensor data processor to execute a second-level processing operation on at least one image frame captured by the array of pixel cells to generate a second-level processing output.

Abstract: A method for providing imagery to a user on a display includes receiving eye tracking data. The method also includes determining a current gaze location and a relative distance between the current gaze location and an edge of the display using the eye tracking data. The method also includes defining a first tile centered at the current gaze location and multiple tiles that surround the first tile using the current gaze location and the relative distance between the current gaze location and the edge of the display. The method includes providing a foveated rendered image using the first tile and the multiple tiles.

Abstract: A method for the delayed processing of image data captured by a wrist-wearable device is provided. The method includes receiving, at a wrist-wearable device, image data having a first resolution. The method includes, responsive to receiving the image data having the first resolution, storing the image data on the wrist-wearable device. The method further includes downscaling at least a portion of the image data to produce downscaled image data having a second resolution and presenting, on the wrist-wearable device, the downscaled image data having the second resolution. The method further includes receiving a request to send a representation of the image data to a device distinct from the wrist-wearable device. Responsive to the request, the method includes causing image-processing algorithms to be applied to the stored image data such that processed image data is generated and made available at the distinct device.

Abstract: The disclosed system may include (1) a processor that generates an artificial environment that includes a virtual object, (2) a display that presents the artificial environment, (3) an input subsystem that detects tracks positioning of a body, and (4) a plurality of haptic actuators that are arranged to apply haptic feedback to a first plurality of locations on the body, where the processor (a) determines, based on the positioning of the body, a virtual contact of one of a second plurality of locations on the body with the virtual object, where the second plurality of locations is different from the first plurality of locations, and (b) activates, in response to the virtual contact, at least one of the plurality of haptic actuators based on a mapping of the second plurality of locations to the plurality of haptic actuators. Various other systems and methods are also disclosed.

Abstract: A projector for illuminating a target area is presented. The projector includes an array of emitters positioned on a substrate according to a distribution. Each emitter in the array of emitters has a non-circular emission area. Operation of at least a portion of the array of emitters is controlled based in part on emission instructions to emit light. The light from the projector is configured to illuminate the target area. The projector can be part of a depth camera assembly for depth sensing of a local area, or part of an eye tracker for determining a gaze direction for an eye.

Abstract: Fabrication of optical lenses in optical assemblies of small form factor cameras and augmented reality (AR)/virtual reality (VR) near-eye display devices is described. An optical lens may be printed using a three-dimensional printing technique over a substrate such as glass following application of an optically clear adhesive (OCA) layer on the substrate. A buffer layer may also be used between the substrate and the OCA layer. Alternatively, a display waveguide may be used as substrate with a low optical index buffer layer between the OCA layer and the waveguide eliminating an air gap between the optical lens and the waveguide.

Abstract: According to examples, a system for implementing structured polarized illumination techniques to determine and reconstruct a shape and a material property of an object is described. The system may include a light source to transmit an original beam of light; a first grating to diffract the original beam of light into a first light beam and a second light beam, a second grating to emit overlapping light beams towards an object, a polarization camera to capture light reflected from the object, and a computer system, comprising a processor and a non-transitory computer-readable storage medium having an executable stored thereon. The processor, when executing the instructions, may cause the system to analyze the light reflected from the object to determine a fringe projection analysis associated with the object, determine a shape of the object; and determine a Mueller matrix to describe material properties of the object.

Abstract: A system, device, and method for a universal intersystem connection are provided. In one aspect, a shielded twisted pair of wires electrically connects a first circuit board and a second circuit board in an apparatus, where the first and second circuit boards may communicate over the shielded twisted pair using differential signalling, and the first circuit board may also be powered by direct current (DC) power over the shielded twisted pair. In an example, the second circuit board is a controller of the first circuit board and may be electrically connected to a battery which powers the first circuit board via the shielded twisted pair. In other examples, a plurality of circuit boards may be electrically connected by the shielded twisted pair to a power source and may communicate with each other via differential signalling over shielded twisted pair.

Abstract: A facial tracking device including an illuminator and a photon detector. The illuminator configured to project a light toward a first portion of a head of a user and the photon detector configured to detect light reflected from a second portion of the head of the user. The facial tracking device further including a processor that is connected to the illuminator and photon detector and configured to cause the illuminator to project the light toward the head of the user, receive information from the photon detector, and determine a facial expression of the user.

Abstract: Three-dimensional (3D) resolution of a single photon avalanche diode (SPAD) based eye tracking system is enhanced through sensor and/or light source steering. The light source may illuminate the whole eye, while the sensor and/or one or more mirrors associated with the sensor may be steered to capture images of different portions of the eye. The images are stitched to achieve a high-resolution, full view of the eye. Alternatively, the sensor may capture a low-resolution image of the whole eye and be steered to capture multiple low-resolution images, which may be used for interpolation and creation of a high-resolution image of the eye. Both the light source and the sensor (and/or the mirrors) may be steered to illuminate and capture images of different portions of the eye, which may then be stitched to achieve a high-resolution, full view of the eye.

Abstract: The disclosed computer-implemented method may include synchronizing a timing scheme between a display and a camera. The method may also include triggering, using a display trigger based on the timing scheme, the display to display a content frame, and triggering, using a capture trigger based on the timing scheme, the camera to capture the displayed content frame. The method may further include building a timeline of frame events for the displayed content frame and the captured content frame, and matching, using the timeline of frame events, the displayed content frame with the captured content frame. Various other methods, systems, and computer-readable media are also disclosed.

Abstract: In one embodiment, a method includes instructing sensors on an AR headset to capture data of a real-world environment, discovering a 3D location where a native application configured to run natively on an operating system is downloadable, generating and displaying a virtual object representing the native application at the 3D location, detecting a user of the AR system interacting with the virtual object, downloading a first executable portion of the native application, executing the first executable portion within a sandbox process having a predetermined set of access permissions, progressively downloading a remaining executable portion of the native application while the user is interacting with the executed first executable portion, and in response to a determination that the native application is completely downloaded, installing the native application on the operating system.

Abstract: An elongated heat pipe or vapor chamber is described. In examples, the heat pipe may include a tubular body comprising a hollow glass tube or fiber. The heat pipe may further include a wick within the tubular body and a working fluid within the tubular body. The wick may be made of glass, polymer, metal, etc. The wick may be an elongated cylinder disposed within the tubular body and, some cases, may be bonded to an inner floor of the tubular body. In examples, the wick may have an outer diameter substantially equal to an inner diameter of the tubular body. In examples, the wick may have a substantially cross-shaped cross section or may have a helical shape.

Abstract: Aspects of the present disclosure are for an interaction mode system that provides multiple interaction modes in an artificial reality environment with automatic, context-specific transitions between interaction modes. An interaction mode can specify how the interaction mode system determines direction indications and movement within an artificial reality environment and interactions for making selections or performing other actions. In some implementations, the interaction mode system can control at least four interaction modes including a no-hands, 3DoF mode, a no-hands, 6DoF mode, a gaze and gesture mode, and a ray casting mode. The interaction mode system can employ a mapping of interaction mode context factors (e.g., which components are enabled, mode settings, lighting or other environment conditions, current body positions, etc.) to interaction modes to control transitioning between particular interaction modes.

Abstract: A method of forming a pixel-aligned volumetric avatar includes receiving multiple two-dimensional images having at least two or more fields of view of a subject. The method also includes extracting multiple image features from the two-dimensional images using a set of learnable weights, projecting the image features along a direction between a three-dimensional model of the subject and a selected observation point for a viewer, and providing, to the viewer, an image of the three-dimensional model of the subject. A system and a non-transitory, computer readable medium storing instructions to perform the above method, are also provided.

Abstract: A wearable device for use in immersive reality applications is provided. The wearable device includes eyepieces to provide a forward-image to a user, a first forward-looking camera mounted on the frame and having a field of view, a processor configured to identify a region of interest within the forward-image, and an interface device to indicate to the user that a field of view of the first forward-looking camera is misaligned with the region of interest. Methods of use of the device, a memory storing instructions and a processor to execute the instructions to cause the device to perform the methods of use, are also provided.

Abstract: Methods, systems, and apparatuses for eye tracking in a near-eye display device are described. In one aspect, an eye tracking system has an array of Vertical Cavity Surface-Emitting Lasers (VCSELs) with different groupings of VCSELs within the array providing structured light with different polarization states such that the structured light projected onto the user's eye has (i) a spatially varying and/or temporally-varying intensity profile and/or (ii) a spatially varying and/or temporally-varying polarization profile. In another aspect, an eye tracking system includes a light source, an image sensor, and a controller which controls the image sensor to capture series of images of light reflections from the eye when the eye is stationary, determines blood flow characteristics using pattern changes in the captured series of images, and performs user authentication and/or liveness detection based on the detected blood flow characteristics.

Abstract: Antenna systems for near-eye devices are described which include multiple resonance sections fed by a single parasitic element. In one example, an antenna has multiple resonance sections which include an open-ended slot antenna section that resonates within a 2.4 GHz frequency band. In one example, the antenna has a monopole section and a dipole section which both resonate within a 5 GHz to 7 GHz frequency band. In one example, a floating dipole is created by locating an open-ended slot antenna parallel to a monopole antenna, where both antennas share the same electrical feed. In one example, multiple resonance sections on one side of a printed circuit board (PCB) are simultaneously electrically fed by a single parasitic element on the other side of the PCB.

Abstract: Systems and methods for performing listen-before-talk (LBT) operations include a first wireless communication device which schedules a data burst comprising a plurality of chirps for transmitting on a channel to a second device. The wireless communication device may perform an LBT operation on the channel between the first device and the second device to determine that the channel is available, prior to transmitting a first chirp of the plurality of chirps. The wireless communication device may transmit the first chirp on the channel responsive to performing the LBT operation.