Abstract: Optimizations are provided for reconstructing geometric surfaces for an environment that includes moving objects. Multiple depth maps for the environment are created, where some of the depth maps correspond to different perspectives of the environment. A motion state identifier is assigned to at least some pixels in at least some of the depth maps corresponding to moving objects in the environment. A composite 3D mesh is built using at least some of the multiple depth maps, by incorporating pixel information from the depth maps, while omitting pixel information identified by the motion state identifiers as being associated with moving objects.

Abstract: Optimizations are provided to control access to virtual content included within a three-dimensional (3D) mesh. Specifically, after the 3D mesh is accessed, then objects represented by the 3D mesh are segmented so that they are distinguishable from one another. Once segmented, then a permissions is assigned to each object or even to groups of objects. For instance, all of the objects that are associated with a particular sub-space (e.g., a bedroom or a living room) may be assigned the same permissions. By assigning permissions to objects, it is possible to control which requesting entities will have access to the objects as well as how much access each of those requesting entities is afforded.

Abstract: A head-mounted device (HMD) is configured to perform depth detection in conjunction with movement tracking. The HMD includes a stereo camera pair comprising a first camera and a second camera, both of which are mounted on the HMD. The fields of view for both of the cameras overlap to form an overlapping field of view. These cameras are configured to detect both visible light and infrared (IR) light. The HMD also includes an IR dot-pattern illuminator that is configured to emit an IR dot-pattern illumination. The HMD uses the IR dot-pattern illumination to determine an object's depth. The HMD also includes one or more flood IR light illuminators that emit a flood of IR light. The HMD uses the flood of IR light to track at least its own movements, and sometimes even hand movements, in various environments, even low light environments.

Abstract: Optimizations are provided to control access to virtual content included within a three-dimensional (3D) mesh. Specifically, after the 3D mesh is accessed, then objects represented by the 3D mesh are segmented so that they are distinguishable from one another. Once segmented, then a permissions is assigned to each object or even to groups of objects. For instance, all of the objects that are associated with a particular sub-space (e.g., a bedroom or a living room) may be assigned the same permissions. By assigning permissions to objects, it is possible to control which requesting entities will have access to the objects as well as how much access each of those requesting entities is afforded.

Abstract: An improved eye tracking illumination system is disclosed. The system includes (i) an RGB laser device that is associated with a first collimating optic and (ii) an IR illumination device that is associated with a second collimating optic. The system also includes a DMA that has a MEMS mirror system. The DMA optically combines IR light and RGB light to generate combined light. The combined light is then directed towards a user's eye via a transport medium (e.g., a waveguide). One or more photodetector(s) are positioned to capture reflected light that is reflected off of the user's eye. The photodetectors include an IR detector configured to detect reflected IR light off of the user's eye in order to perform eye tracking.

Abstract: A head-mounted device (HMD) is configured to perform head tracking, even in low light environments. The HMD includes a stereo camera pair that includes a first camera and a second camera having overlapping fields of view. Both cameras are mounted on the HMD and are configured to detect both visible light and infrared (IR) light. The HMD also includes a flood IR light illuminator that is configured to emit a flood of IR light that spans an illumination area that overlaps with the cameras' fields of view. The intensity of the IR light is sometimes modified to accommodate low light environmental conditions. The cameras obtain images of reflected IR light. These images are then used to track movements of the HMD, even in low light environments.

Abstract: An illumination source is operated to illuminate an operating environment and an optical sensor is periodically operated for a detection period to detect illumination reflected from one or more subjects within the operating environment. Upon recognizing a source of interfering pulsed illumination within the operating environment, the timing of a subsequent detection period may be varied. In this way, sensing of the interfering pulsed illumination may be averted.

Abstract: Techniques are provided to reduce power consumption when performing surface reconstruction. A surface mesh of an environment is generated, where the surface mesh is generated from multiple depth maps that are obtained of the environment. After the surface mesh is generated, a change detection image of that environment is captured while refraining from obtaining a new depth map of the environment. The change detection image is compared to the surface mesh. If a difference between the change detection image and the surface mesh is detected and if that difference satisfies a pre-determined difference threshold, then a new depth map of the environment is obtained. The surface mesh is then updated using the new depth map.

Abstract: Optimizations are provided for reconstructing geometric surfaces for an environment that includes moving objects. Multiple depth maps for the environment are created, where some of the depth maps correspond to different perspectives of the environment. A motion state identifier is assigned to at least some pixels in at least some of the depth maps corresponding to moving objects in the environment. A composite 3D mesh is built using at least some of the multiple depth maps, by incorporating pixel information from the depth maps, while omitting pixel information identified by the motion state identifiers as being associated with moving objects.

Abstract: A modulated light source comprises a laser diode and a drive circuit coupled operatively to the laser diode. The laser diode is configured to lase upon passing an above-threshold current for an accumulation period. The drive circuit is configured to draw a priming current through the laser diode over a priming period, the priming current being insufficient to cause the laser diode to lase during the priming period, but sufficient to shorten the accumulation period. The drive circuit is further configured to draw the above-threshold current through the laser diode after the priming period, thereby triggering emission from the laser diode following a shortened accumulation period.

Abstract: A method for three-dimensional imaging includes emitting an output light with a structured light illuminator in a structured light pattern, receiving a trigger command, changing a field of illumination of the illuminator, and changing a field of view of an imaging sensor. The field of view and the field of illumination are linked, such that the field of view of the imaging sensor is the same as the field of illumination of the illuminator at a short throw field of view and a long throw field of view. The method further includes detecting a reflected light with the imaging sensor and measuring a depth value by calculating a distortion of the structured light pattern.

Abstract: Disclosed embodiments include methods and systems for utilizing a structured projection pattern to perform depth detection. In some instances, the structured projection pattern forms a dot pattern, which is projected by an infrared (IR) illuminator, wherein the dot pattern includes a replicated sub-pattern having a predefined height and width. The sub-pattern is replicated in at least one direction such that the dot pattern comprises a plurality of replicated sub-patterns that are adjacently positioned.

Abstract: A stereoscopic 3D imaging system includes multiple imaging sensors with adjustable optics. The adjustable optics are variable to alter the FOV of each of the multiple imaging sensors to improve angular resolution of the imaging system.

Abstract: A light sensing circuit includes a photomultiplier in electrical communication with an array of capacitors or resistors. Each capacitor or resistor in the array having an associated switch and having a capacitance or resistance different from every other capacitor or resistor in the array. Each switch has an open state and a closed state, thus enabling each capacitor or resistor to be placed in electrical communication with the photomultiplier or be isolated from the photomultiplier. The switchable array may be in electrical communication with an analog to digital converter (ADC) or a transimpedance amplifier (TIA). The switchable array allows the ADC or TIA to be sensitive to low value signals and operate at a large dynamic range and operate at a fast rate.

Abstract: A method including receiving an image of a scene illuminated by both a predetermined structured light pattern and a flood fill illumination, generating an active brightness image of the scene based on the received image of the scene including detecting a plurality of dots of the predetermined structured light pattern, and removing the plurality of dots of the predetermined structured light pattern from the active brightness image, and generating a depth map of the scene based on the received image and the active brightness image.

Abstract: Eye tracking system for use in a visible light display device incorporates and/or use one or more silicon photomultiplier (SiPM) sensor and an infrared module of a microelectromechanical (MEMs)-based scanner. The infrared module emits a beam of photons, where at least some of the photons are directed towards a user's eye while the eye tracking system is being used. The SiPM sensor(s) capture a reflection that emanates off of the user's eye.

Abstract: A device and method use multiple light emitters with a single, multi-spectrum imaging sensor to perform multi-modal infrared light based depth sensing and visible light based Simultaneous Localization and Mapping (SLAM). The multi-modal infrared based depth sensing may include, for example, any combination of infrared-based spatial mapping, infrared based hand tracking and/or infrared based semantic labeling. The visible light based SLAM may include head tracking, for example.

Abstract: Systems and methods are utilized for performing geometric multiplexing in MEMS display systems that utilize RGB laser diodes and MEMS mirrors to compensate for angular separation between the RGB light that results from passing the RGB light emitted from the RGB laser diodes through a single collimating lens shared by the RGB laser diodes, as opposed to utilizing a separate collimating lens for each corresponding laser diode. Spatial offsets between the RGB light at the target display, resulting from the angular separation, are compensated for by applying temporal buffers to the pulsing of the RGB laser sources so that the RGB light is horizontally and vertically aligned at the appropriate pixels of the target display during scanning by the MEMS mirrors system.

Abstract: A head-mounted device (“HMD”) is configured to perform intrinsic and/or extrinsic calibration of the HMD's camera system by exploiting a displayed electronic image rendered on a separate display screen. A series of images are captured using one or more of the HMD's cameras. The displayed image is a known image that includes markers with known characteristics to the HMD. The known characteristics include known marker shapes and a number of coded or un-coded markers. Each image in the series captures the displayed image at a different angle or distance relative to another image in the series. The HMD then identifies, from within the series of images, two-dimensional image positions of the markers. The HMD uses the two-dimensional image positions and a determined three-dimensional position of the markers to perform a bundle adjustment used to subsequently determine a position and angular alignment of the separate display screen relative to the HMD.

Abstract: A virtual-reality (“VR”) renders a virtual anchor object within the VR environment that correlates to a real-world anchor object. The anchor object's real-world location relative to a computer system is determined and rendered at a location within the VR environment in such a manner that the virtual anchor object is world-locked relative to the real-world environment, as opposed to being world-locked relative to the VR environment. In response to movements of the computer system, the virtual anchor object's location is updated in order to maintain the real-world world-locked relationship. Objects having known properties can also be used as a comparison to captured images to determine relative positioning of the VR device.