Abstract: An imaging device for imaging of a local area surrounding the imaging device. The imaging device includes a lens assembly, a filtering element and a detector. The lens assembly is configured to receive light from a local area surrounding the imaging device and to direct at least a portion of the received light to the detector. The filtering element is placed in the imaging device within the lens assembly such that light is incident at a surface of the filtering element within a range of angles determined by a design range of angles at which the filtering element is designed to filter light. The detector is configured to capture image(s) of the local area including the filtered light. The imaging device can be integrated into a depth camera assembly for determining depth information of object(s) in the local area based on the captured image(s).

Abstract: A pick-up head assembly comprises a body of a liquid crystalline elastomer (LCE) that undergoes a reversible expansion when exposed to a first frequency of light and contracts when exposed to a second frequency of light. Selective portions of the LCE in the pick-up head assembly are irradiated with the first frequency to cause an expansion in the selective portions. The adhesive forces of the expanded portions of the LCE are used to pick-up semiconductor devices from a first substrate. The semiconductor devices are placed on a second substrate by exposing the expanded portions of the LCE to the second frequency of light, causing the expanded portions to contract.

Abstract: An optical element includes a first boundary layer and a second boundary layer. A solution is disposed between the first boundary layer and the second boundary layer. The solution includes liquid crystals co-mingled with oblong photochromic dye molecules.

Abstract: A head mounted display (HMD) includes an electronic display configured to display a virtual scene to a user, an optics block, an eye tracking system, and a varifocal actuation that mechanically changes a distance between the optics block and the electronic display. The varifocal actuation block is configured to change a location of an image plane of the HMD and includes a motor, a power screw coupled to the actuating motor configured to turn responsive to actuation of the motor, and a nut sled on the power screw that is coupled to the electronic display. The nut sled is configured to move back and forth along a length of the power screw responsive to the power screw being turned by the motor that results in movement of the electronic display relative to the optics block.

Abstract: In one embodiment, a method includes capturing, using one or more cameras implemented in a wearable device worn by a user, a first image depicting at least a part of a hand of the user holding a controller in an environment, identifying one or more features from the first image to estimate a pose of the hand of the user, estimating a first pose of the controller based on the pose of the hand of the user and an estimated grip that defines a relative pose between the hand of the user and the controller, receiving IMU data of the controller, and estimating a second pose of the controller by updating the first pose of the controller using the IMU data of the controller. The method utilizes multiple data sources to track the controller under various conditions of the environment to provide an accurate controller tracking consistently.

Abstract: A haptic device including a magnet, a rigid and conductive material arranged with respect to the magnet so as to create a gap between the magnet and the rigid and conductive material is disclosed. The haptic device is configured such that a magnetic field is generated in the gap by the magnet. The haptic device also includes a plurality of coils each connected to a respective one of a plurality of protruding elements where each of the plurality of coils are configured to be positioned within the gap. Each of the plurality of protruding elements are configured to produce a respective contact with an object when the respective coil is energized. In some embodiments, the object is a hand of a user.

Abstract: The disclosed radar system may include a radar mechanism comprising a transmitter and at least one receiver. The radar system may also include a signal generator that generates a frequency-modulated radar signal. In addition, the radar system may include a delay mechanism that (1) receives the frequency-modulated radar signal from the signal generator and (2) after a certain period of delay, passes the frequency-modulated radar signal to the transmitter to be transmitted to a transponder located on a wearable artificial reality device. The radar system may also include a processing device that (1) receives the frequency-modulated radar signal from the signal generator, (2) detects a signal returned to the receiver from the transponder, and (3) calculates a distance between the transponder and the receiver based at least in part on an analysis of the signal returned from the transponder and the frequency-modulated radar signal received from the signal generator.

Abstract: In one embodiment, a method for eye-tracking comprises capturing images of a user using one or more cameras, the captured images of the user depicting at least an eye of the user, storing the captured images of the user in a storage device, reading, from the storage device, a down-sampled version of the captured images of the user, detecting one or more first segments in the down-sampled version of the captured images by processing the down-sampled version of the captured images using a machine-learning model, the one or more first segments comprising features of the eye of the user, reading, from the storage device, one or more second segments in the captured images corresponding to the one or more first segments in the down-sampled version of the captured images, and computing a gaze of the user based on the one or more second segments in the captured images.

Abstract: Embodiments relate to a display device including an active display area with pixels arranged in rows and columns, where a focus area of the active display area is operated in a progressive scanning manner and a non-focus area of the active display area is operated in an interlaced scanning manner. The active display area is driven by a gate driver circuit that supplies gate signals the pixels. First stages of the gate driver circuit are coupled to first rows of the pixels that are in the focus area and output first gate signals in the progressive scanning manner. Second stages of the gate driver circuit are coupled to second rows of the pixels that are in the non-focus area and output second gate signals in the interlaced scanning manner.

Abstract: Aspects of the present disclosure are for an interaction mode system that provides multiple interaction modes in an artificial reality environment with automatic, context-specific transitions between interaction modes. An interaction mode can specify how the interaction mode system determines direction indications and movement within an artificial reality environment and interactions for making selections or performing other actions. In some implementations, the interaction mode system can control at least four interaction modes including a no-hands, 3DoF mode, a no-hands, 6DoF mode, a gaze and gesture mode, and a ray casting mode. The interaction mode system can employ a mapping of interaction mode context factors (e.g., which components are enabled, mode settings, lighting or other environment conditions, current body positions, etc.) to interaction modes to control transitioning between particular interaction modes.

Abstract: A system can include a device and a graphics processing unit (GPU). The device can be configured to receive a first image from one or more cameras corresponding to a first view and a second image from the one or more cameras corresponding to a second view. The GPU can include a motion estimator and configured to receive the first image and the second image and be configured to receive the first image and the second image. The motion estimator can be configured to determine first disparity offsets for the first image and second disparity offsets for the second image. The device can be configured to generate, for rendering 3D image using the first image and the second image, a first depth buffer for the first image derived from the first disparity offsets and a second depth buffer for the second image derived from the second disparity offsets.

Abstract: An audio system for providing content to a user. The system includes a first and a second transducer assembly of a plurality of transducer assemblies, an acoustic sensor, and a controller. The first transducer assembly couples to a portion of an auricle of the user's ear and vibrates over a first range of frequencies based on a first set of audio instructions. The vibration causes the portion of the ear to create a first range of acoustic pressure waves. The second transducer assembly is configured to vibrate over a second range of frequencies to produce a second range of acoustic pressure waves based on a second set of audio instructions. The acoustic sensor detects acoustic pressure waves at an entrance of the ear. The controller generates the audio instructions based on audio content to be provided to the user and the detected acoustic pressure waves from the acoustic sensor.

Abstract: A device includes one or more light guides. The device also includes a first in-coupling element configured to couple a first light having a first input field of view (“FOV”) into a first light guide, and a second in-coupling element configured to couple a second light having a second input FOV into a second light guide. The device also includes a first out-coupling element configured to couple the first light out of the first light guide as a first output light having a first output FOV, and a second out-coupling element configured to couple the second light out of the second light guide as a second output light having a second output FOV substantially non-overlapping with the first output FOV. A combination of the first output FOV and the second output FOV is larger than at least one of the first output FOV or the second output FOV.

Abstract: In one embodiment, a method includes determining a first time interval for rendering pixel values for a first portion of a display, where the first portion of the display is one of a plurality of portions of the display, accessing a representation of virtual objects in a virtual scene, and determining that the first time interval is insufficient for rendering pixel values for the first portion of the display based on the representation of virtual objects. The method further includes, for each of the portions of the display, determining a current viewpoint of a user, rendering, based on the current viewpoint of the user, pixel values for that portion of the display, and sending the rendered pixel values to be output by the display, where the rendered pixel values for the first portion of the display are rendered based on a simplified representation of virtual objects in the virtual scene.

Abstract: The technology disclosed relates to a machine learning based speech-to-text transcription intermediary which, from among multiple speech recognition engines, selects a speech recognition engine for accurately transcribing an audio channel based on sound and speech characteristics of the audio channel.

Abstract: In one embodiment, a method includes creating, by a computing device, an inter process communication (IPC) channel for communication from a producer process to a consumer process. The IPC channel includes a message buffer mapped to a first virtual address space of a kernel of an operating system and to a second virtual address space of the consumer process in a user space of the operating system. A sender handle for the message buffer is sent to the producer process. The kernel receives a request, which includes the sender handle, to send a message from the producer process to the consumer process through the IPC channel. Responsive to a determination that the request is permitted, the kernel writes the message to the message buffer using the sender handle, such that message written in the message buffer is accessible to the consumer process through the second address space.

Abstract: The disclosed computer-implemented method may include applying, via a sound reproduction system, sound cancellation that reduces an amplitude of various sound signals. The method further includes identifying, among the sound signals, an external sound whose amplitude is to be reduced by the sound cancellation. The method then includes analyzing the identified external sound to determine whether the identified external sound is to be made audible to a user and, upon determining that the external sound is to be made audible to the user, the method includes modifying the sound cancellation so that the identified external sound is made audible to the user. Various other methods, systems, and computer-readable media are also disclosed.

Abstract: In one embodiment, a method includes receiving a user input comprising one or more words at a client system, wherein each word comprises one or more characters, inputting the words to a convolutional neural network (CNN) model stored on the client system, accessing a plurality of character-embeddings for a plurality of characters, respectively, from a data store of the client system, generating one or more word-embeddings for the one or more words, respectively, based on the accessed character-embeddings by processing the accessed character-embeddings with one or more convolutional layers and one or more gated linear units of the CNN model, determining one or more tasks corresponding to the user input for execution based on an analysis of the one or more word-embeddings by the CNN model, and providing an output responsive to the user input based on the execution of the one or more tasks at the client system.

Abstract: A sensor assembly for determining one or more features of a local area is presented herein. The sensor assembly includes a plurality of stacked sensor layers. A first sensor layer of the plurality of stacked sensor layers located on top of the sensor assembly includes an array of pixels. The top sensor layer can be configured to capture one or more images of light reflected from one or more objects in the local area. The sensor assembly further includes one or more sensor layers located beneath the top sensor layer. The one or more sensor layers can be configured to process data related to the captured one or more images. A plurality of sensor assemblies can be integrated into an artificial reality system, e.g., a head-mounted display.

Abstract: Systems and methods for beamforming include a device including at least one of a head wearable display (HWD) or a console. The device establishes a first connection between an active HWD radio-frequency integrated circuit (RFIC) and an active console RFIC. The device compares a modulation and coding scheme (MCS) of the first connection to an MCS threshold. The device performs MCS measurements for a second connection of at least one of an idle HWD RFIC or an idle console RFIC, while the first connection is maintained, in response to the MCS not satisfying the MCS threshold. The device compares the MCS measurements of the second connection to the MCS threshold. The device switches to the second connection when at least one of the one or more MCS measurements satisfies the MCS threshold and/or above the MCS of the first connection.