Abstract: An artificial reality system is described that renders, presents, and controls user interface elements within an artificial reality environment, and performs actions in response to one or more detected gestures of the user. In one example, an artificial reality system comprises an image capture device configured to capture image data representative of a physical environment; a head-mounted display (HMD) configured to output artificial reality content; a gesture detector configured to identify, from the image data, a gesture comprising a motion of two fingers from a hand to form a pinching configuration and a subsequent pulling motion while in the pinching configuration; a user interface (UI) engine configured to generate a UI input element in response to identifying the gesture; and a rendering engine configured to render the UI input element as an overlay to at least some of the artificial reality content.

Abstract: In one embodiment, a computing system may determine a quantization range having a first quantization endpoint and a second quantization endpoint. While fixing the second quantization endpoint to an initial value determined based on the color range, one of a plurality of first candidate values for the first quantization endpoint is selected based on a plurality of corresponding first quantization errors. While fixing the first quantization endpoint to the selected first candidate value, one of a plurality of second candidate values for the second quantization endpoint is selected based on a plurality of corresponding second quantization errors. The computing system may define quantization levels corresponding to the bit depth using the quantization range defined by the first quantization endpoint and the second quantization endpoint, and then encode the one or more color components of the pixel region using the quantization levels.

Abstract: In one embodiment, a method includes generating a visual interaction tool that moves and extends in a three-dimensional artificial-reality environment according to hand and arm movements of a user. It may be detected that the visual interaction tool intersects a predefined region associated with a virtual item of a first type in the AR environment. The visual interaction tool may attach to the first virtual item. A first operating mode for the visual interaction tool may be selected based on the first type of the first virtual item. The first operating mode may be selected from multiple operating modes for the visual interaction tool. A first input from the user may be received while the visual interaction tool is attached to the first virtual item. First operations with the first virtual item may be performed according to the first operating mode and the first input.

Abstract: In one embodiment, a method includes receiving playback statuses from one or more second client computing devices that are in a synchronous presentation session with a first client computing device. Content presented on the first client computing device and the one or more second client computing devices may be determined to be out of synchronization. A plurality of synchronization options for synchronization of the content may be displayed on the first client computing device. The plurality of synchronization options may be based on the playback statuses associated with the first client computing device and the one or more second client computing devices. A particular synchronization option may be selected from the plurality of synchronization options. The content on the first client computing device may be synchronized with the one or more second client computing devices based on the particular synchronization option.

Abstract: The disclosed systems and methods are generally directed to interpreting neuromuscular signals. The system includes (A) a plurality of neuromuscular sensors that detect a plurality of neuromuscular signals from a user and (B) at least one multiplexer in communication with the plurality of neuromuscular sensors and that is capable of dynamically adjusting neuromuscular sensor processing based on neuromuscular signal characteristics. A computer processor is programmed to (i) receive a set of neuromuscular signals from the plurality of neuromuscular sensors, (ii) determine, via a real-time system, at least one signal characteristic included in a neuromuscular signal, where the neuromuscular signal is associated with a first neuromuscular sensor included in the plurality of neuromuscular sensors; and (iii) dynamically reconfigure the processing of neuromuscular signals from the plurality of neuromuscular sensors based on an output from the multiplexer.

Abstract: An imaging device for imaging of a local area surrounding the imaging device. The imaging device includes a lens assembly, a filtering element and a detector. The lens assembly is configured to receive light from a local area surrounding the imaging device and to direct at least a portion of the received light to the detector. The filtering element is placed in the imaging device within the lens assembly such that light is incident at a surface of the filtering element within a range of angles determined by a design range of angles at which the filtering element is designed to filter light. The detector is configured to capture image(s) of the local area including the filtered light. The imaging device can be integrated into a depth camera assembly for determining depth information of object(s) in the local area based on the captured image(s).

Abstract: A housing assembly for mounting a first lens and a second lens includes a first lens holder. The first lens holder includes a ring-shaped structure configured to mount the first lens. The housing assembly also includes a second lens holder including a cup-shaped structure. The cup-shaped structure includes an upper portion configured to mount the first lens holder, and a lower portion configured to mount the second lens.

Abstract: Disclosed herein are related to systems and methods for providing inputs through a virtual keyboard with an adaptive language model. In one approach, one or more processors determine whether a user intended to provide semantically meaningful characters or not, when providing a hand motion or a hand pose with respect to a virtual keyboard. The virtual keyboard may be located on a surface without physical keys. In one approach, the one or more processors determine an input to the virtual keyboard based on the hand motion or the hand pose. In one approach, the one or more processors determine weight of a language model according to the determined user intention. In one approach, the one or more processors modify the detected input according to the determined weight of the language model.

Abstract: Disclosed herein are related to a system and a method of reducing contention in accessing a wireless link. In one aspect, a first device includes a transceiver configured to communicate with a second device via a wireless link. In one aspect, the first device includes a processor configured to determine that a type of content to be transferred between the first device and the second device through the wireless link. In one aspect, the processor is configured to modify one or more parameters for the first device to access the wireless link, from a first set of values to a second set of values, in response to determining that the type of content is artificial reality. In one aspect, the processor is configured to cause the transceiver to access the wireless link using the second set of values.

Abstract: The disclosed method may include receiving neuromuscular activity data over a first time series from a first sensor on a wearable device donned by a user receiving ground truth data over a second time series from a second sensor that indicates a body part state of a body part of the user, generating one or more training datasets by time-shifting at least a portion of the neuromuscular activity data over the first time series relative to the second time series, to associate the neuromuscular activity data with at least a portion of the ground truth data, and training one or more inferential models based on the one or more training datasets. Various other related methods and systems are also disclosed.

Abstract: A display device including a laser-based light engine is disclosed. Optical interference effects due to the coherent nature of a laser light source are mitigated by shortening a coherence length of the laser source. The coherence length shortening is achieved by at least one of the following: providing a multiple longitudinal mode laser source, pulsing a laser source to achieve spectral broadening, or providing multi-emitter laser source(s) with emission wavelength varying from emitter to emitter.

Abstract: Techniques disclosed herein relate to modifying refractive index modulation in a holographic optical element, such as a holographic grating. According to certain embodiments, a holographic optical element or apodized grating includes a polymer layer comprising a first region characterized by a first refractive index and a second region characterized by a second refractive index. The holographic optical element or apodized grating includes a plurality of nanoparticles dispersed in the polymer layer. The nanoparticles have a higher concentration in either the first region or the second region. In some embodiments, the nanoparticles may be configured to increase the refractive index modulation. In some embodiments, the nanoparticles may be configured to apodize the grating by decreasing the refractive index modulation proximate to sides of the grating. The refractive index may be modulated by applying a monomer reservoir buffer layer to the polymer layer, either before or after hologram fabrication.

Abstract: A near eye display (NED) includes multiple PBP optical elements combined with one or more C-plates to improve optical angular performance. The PBP optical elements may be configured for beam steering or for focusing light to a point. A C-plate may reduce or eliminate an undesirable polarization phase shift introduced by the PBP optical elements to angular, off-axis light. Birefringence of the PBP optical elements produces such a polarization phase shift. A C-plate provides an additional polarization phase shift that is opposite to the extra polarization phase shift by the PBP optical elements. Thus, the additional polarization phase shift by the C-plate at least partially reduces the phase shift by the PBP element.

Abstract: A head mounted display (HMD) includes a field curvature corrected (FC) display to mitigate field curvature in an image that is output to a user's eyes. The FC display includes elements that generate the image light and elements to mitigate field curvature from the image light. The FC display may include a display panel with lenses, a display panel with a reflective polarizer and reflective surface, or other optical elements. The FC display may include a pancake lens configuration including a polarized display with a quarter wave plate, a reflective mirror, and a polarization reflective mirror.

Abstract: A method for classifying a gesture made in proximity to a touch interface. A system receives data related to the position and/or movement of hand. The data is delimited by identifying a variable length window of touch frames. The variable length window of touch frames is selected to include touch frames indicative of feature data. The variable length window of touch frames is classified based upon classifications learned by the classifying module to identify gestures.

Abstract: A pick-up head assembly comprises a body of a liquid crystalline elastomer (LCE) that undergoes a reversible expansion when exposed to a first frequency of light and contracts when exposed to a second frequency of light. Selective portions of the LCE in the pick-up head assembly are irradiated with the first frequency to cause an expansion in the selective portions. The adhesive forces of the expanded portions of the LCE are used to pick-up semiconductor devices from a first substrate. The semiconductor devices are placed on a second substrate by exposing the expanded portions of the LCE to the second frequency of light, causing the expanded portions to contract.

Abstract: Embodiments relate to a display device including an active display area with pixels arranged in rows and columns, where a focus area of the active display area is operated in a progressive scanning manner and a non-focus area of the active display area is operated in an interlaced scanning manner. The active display area is driven by a gate driver circuit that supplies gate signals the pixels. First stages of the gate driver circuit are coupled to first rows of the pixels that are in the focus area and output first gate signals in the progressive scanning manner. Second stages of the gate driver circuit are coupled to second rows of the pixels that are in the non-focus area and output second gate signals in the interlaced scanning manner.

Abstract: An optical element includes a first boundary layer and a second boundary layer. A solution is disposed between the first boundary layer and the second boundary layer. The solution includes liquid crystals co-mingled with oblong photochromic dye molecules.

Abstract: A head mounted display (HMD) includes an electronic display configured to display a virtual scene to a user, an optics block, an eye tracking system, and a varifocal actuation that mechanically changes a distance between the optics block and the electronic display. The varifocal actuation block is configured to change a location of an image plane of the HMD and includes a motor, a power screw coupled to the actuating motor configured to turn responsive to actuation of the motor, and a nut sled on the power screw that is coupled to the electronic display. The nut sled is configured to move back and forth along a length of the power screw responsive to the power screw being turned by the motor that results in movement of the electronic display relative to the optics block.

Abstract: In one embodiment, a method for eye-tracking comprises capturing images of a user using one or more cameras, the captured images of the user depicting at least an eye of the user, storing the captured images of the user in a storage device, reading, from the storage device, a down-sampled version of the captured images of the user, detecting one or more first segments in the down-sampled version of the captured images by processing the down-sampled version of the captured images using a machine-learning model, the one or more first segments comprising features of the eye of the user, reading, from the storage device, one or more second segments in the captured images corresponding to the one or more first segments in the down-sampled version of the captured images, and computing a gaze of the user based on the one or more second segments in the captured images.