Abstract: Apparatus for generating a dynamic structured light pattern for optical tracking in three-dimensional space, comprises an array of lasers, such as a VCSEL laser array, to project light in a pattern into a three-dimensional space; and an optical element or elements arranged in cells. The cells are aligned with subsets of the laser array, and each cell individually applies a modulation, in particular an intensity modulation, to light from the laser or lasers of the subset, to provide a distinguishable and separately controllable part of the dynamic structured light pattern. A method of generating a structured light pattern is disclosed, in which light is provided from an array of lasers, and light is individually projected from subsets of the array of lasers to provide differentiated parts of the structured light pattern.

Abstract: A depth camera assembly (DCA) determines depth information for a local area. The DCA includes a camera assembly and at least one illuminator. The DCA may select a subset of the VCSELs to provide illumination at any given time. The illuminator may comprise near-field VCSELs configured to generate a structured light (SL) pattern for depth sensing in the near-field and far-field VCSELs configured to generate a SL pattern for depth sensing in the far-field. The near-field VCSELs may comprise a linear emission region which is shorter than a linear emission region of the far-field VCSELs. The DCA may generate and phase shift a quasi-sinusoidal SL pattern. The DCA may phase shift the quasi-sinusoidal SL pattern by alternating which traces on the illuminator are active.

Abstract: In one embodiment, a system includes at least one projector configured to project a plurality of projected patterns, where a projected lighting characteristic of each of the projected patterns varies over a time period in accordance with an associated predetermined temporal lighting-characteristic pattern, a camera configured to capture images of detected patterns during the time period, and one or more processors configured to: determine, for each detected pattern, a detected temporal lighting-characteristic pattern based on variations in a detected lighting characteristic of the detected pattern, identify a detected pattern that corresponds to one of the projected patterns by comparing at least one of the detected temporal lighting-characteristic patterns to at least one of the temporal lighting-characteristic patterns, and compute a depth associated with the detected patterns based on the one or more of the projected patterns, the detected pattern, and a relative position between the camera and the project

Abstract: An audio system in a local area providing an audio signal to a headset of a remote user is presented herein. The audio system identifies sounds from a human sound source in the local area, based in part on sounds detected within the local area. The audio system generates an audio signal for presentation to a remote user within a virtual representation of the local area based in part on a location of the remote user within the virtual representation of the local area relative to a virtual representation of the human sound source within the virtual representation of the local area. The audio system provides the audio signal to a headset of the remote user, wherein the headset presents the audio signal as part of the virtual representation of the local area to the remote user.

Abstract: In one embodiment, a system includes at least one projector comprising a plurality of light emitters, where the projector is configured to project a projected pattern comprising a plurality of projected features having different locations; a camera configured to capture an image comprising a detected pattern corresponding to a reflection of the projected pattern; and one or more processors configured to: identify at least one detected feature of the detected pattern, wherein the detected feature corresponds to at least one reflection of the projected features; and activate or deactivate one or more of the light emitters based on the detected feature. The light emitters may be activated or deactivated by determining a detected feature measurement based on the detected feature, and activating or deactivating one or more of the light emitters when the detected feature measurement satisfies a threshold feature measurement condition.

Abstract: An audio system in a local area providing an audio signal to a headset of a remote user is presented herein. The audio system identifies sounds from a human sound source in the local area, based in part on sounds detected within the local area. The audio system generates an audio signal for presentation to a remote user within a virtual representation of the local area based in part on a location of the remote user within the virtual representation of the local area relative to a virtual representation of the human sound source within the virtual representation of the local area. The audio system provides the audio signal to a headset of the remote user, wherein the headset presents the audio signal as part of the virtual representation of the local area to the remote user.

Abstract: A method and a system for reconstructing obstructed face portions are provided herein. The method may include the following steps: obtaining off-line 3D data, being 3D data of a head of a person not wearing a face-obstructing object, being an object which obstructs a portion of the face of the person; obtaining in real time, real-time 3D data, being 3D data of said head, wherein said person wears said face-obstructing object; applying a 3D transformation to at least a portion of the off-line 3D data, based on the real-time 3D data, to yield reconstructed real time 3D data, being real-time 3D data related to the obstructed face portions; and merging the reconstructed real time 3D data into the real-time 3D data. The system may implement the aforementioned steps over a computer processor.

Abstract: In one embodiment, a system includes at least one projector configured to project a plurality of projected patterns, where a projected lighting characteristic of each of the projected patterns varies over a time period in accordance with an associated predetermined temporal lighting-characteristic pattern, a camera configured to capture images of detected patterns during the time period, and one or more processors configured to: determine, for each detected pattern, a detected temporal lighting-characteristic pattern based on variations in a detected lighting characteristic of the detected pattern, identify a detected pattern that corresponds to one of the projected patterns by comparing at least one of the detected temporal lighting-characteristic patterns to at least one of the temporal lighting-characteristic patterns, and compute a depth associated with the detected patterns based on the one or more of the projected patterns, the detected pattern, and a relative position between the camera and the project

Abstract: A depth camera assembly (DCA) determines depth information for a local area. The DCA includes a camera assembly and at least one illuminator. The DCA may select a subset of the VCSELs to provide illumination at any given time. The illuminator may comprise near-field VCSELs configured to generate a structured light (SL) pattern for depth sensing in the near-field and far-field VCSELs configured to generate a SL pattern for depth sensing in the far-field. The near-field VCSELs may comprise a linear emission region which is shorter than a linear emission region of the far-field VCSELs. The DCA may generate and phase shift a quasi-sinusoidal SL pattern. The DCA may phase shift the quasi-sinusoidal SL pattern by alternating which traces on the illuminator are active.

Abstract: A depth camera assembly (DCA) determines depth information for a local area. The DCA includes a plurality of cameras and at least one illuminator. The DCA dynamically determines depth sensing modes (e.g., passive stereo, active stereo, structured stereo) based in part on the surrounding environment and/or user activity. The DCA uses the depth information to update a depth model describing the local area. The DCA may determine that a portion of the depth information associated with some of portion of the local area is not accurate. The DCA may then select a different depth sensing mode for the portion of the local area and update the depth model with the additional depth information. In some embodiments, the DCA may update the depth model by utilizing a machine learning model to generate a refined depth model.

Abstract: In one embodiment, a system includes at least one projector configured to project a plurality of projected patterns, where a projected lighting characteristic of each of the projected patterns varies over a time period in accordance with an associated predetermined temporal lighting-characteristic pattern, a camera configured to capture images of detected patterns, and one or more processors configured to: determine, for each detected pattern, a detected temporal lighting-characteristic pattern based on variations in a detected lighting characteristic of the detected pattern, identify a detected pattern that corresponds to one of the projected patterns by comparing at least one of the detected temporal lighting-characteristic patterns to at least one of the temporal lighting-characteristic patterns, and compute a depth associated with the detected patterns based on the one or more of the projected patterns, the detected pattern, and a relative position between the camera and the projector.

Abstract: Apparatus for generating a dynamic structured light pattern for optical tracking in three-dimensional space, comprises an array of lasers, such as a VCSEL laser array, to project light in a pattern into a three-dimensional space; and an optical element or elements arranged in cells. The cells are aligned with subsets of the laser array, and each cell individually applies a modulation, in particular an intensity modulation, to light from the laser or lasers of the subset, to provide a distinguishable and separately controllable part of the dynamic structured light pattern. A method of generating a structured light pattern is disclosed, in which light is provided from an array of lasers, and light is individually projected from subsets of the array of lasers to provide differentiated parts of the structured light pattern.

Abstract: Apparatus for generating a dynamic structured light pattern for optical tracking in three-dimensional space, comprises an array of lasers, such as a VCSEL laser array, to project light in a pattern into a three-dimensional space; and an optical element or elements arranged in cells. The cells are aligned with subsets of the laser array, and each cell individually applies a modulation, in particular an intensity modulation, to light from the laser or lasers of the subset, to provide a distinguishable and separately controllable part of the dynamic structured light pattern. A method of generating a structured light pattern is disclosed, in which light is provided from an array of lasers, and light is individually projected from subsets of the array of lasers to provide differentiated parts of the structured light pattern.

Abstract: In one embodiment, a system includes at least one projector comprising a plurality of light emitters, where the projector is configured to project a projected pattern comprising a plurality of projected features having different locations; a camera configured to capture an image comprising a detected pattern corresponding to a reflection of the projected pattern; and one or more processors configured to: identify at least one detected feature of the detected pattern, wherein the detected feature corresponds to at least one reflection of the projected features; and activate or deactivate one or more of the light emitters based on the detected feature. The light emitters may be activated or deactivated by determining a detected feature measurement based on the detected feature, and activating or deactivating one or more of the light emitters when the detected feature measurement satisfies a threshold feature measurement condition.

Abstract: A depth camera assembly (DCA) configured to determine distances between the headset and one or more objects in an area surrounding the headset. The DCA includes an imaging device, an illumination source, and a controller. The controller identifies objects in a portion of the local area, determines a depth zone for each object and corresponding structured light (SL) illumination parameters including a SL pattern for each object based on the depth zone, instructs the illumination source to illuminate a scene comprising the one or more objects with the determined SL pattern, and instructs the imaging device to capture images of the illuminated objects. The controller determines the depth information for the illuminated objects and updates the depth information associated with the objects.

Abstract: In one embodiment, a system includes at least one projector comprising a plurality of light emitters, where the projector is configured to project a projected pattern comprising a plurality of projected features having different locations; a camera configured to capture an image comprising a detected pattern corresponding to a reflection of the projected pattern; and one or more processors configured to: identify at least one detected feature of the detected pattern, wherein the detected feature corresponds to at least one reflection of the projected features; and activate or deactivate one or more of the light emitters based on the detected feature. The light emitters may be activated or deactivated by determining a detected feature measurement based on the detected feature, and activating or deactivating one or more of the light emitters when the detected feature measurement satisfies a threshold feature measurement condition.

Abstract: Disclosed herein are techniques for structured light pattern generation. A method for generating a one-dimensional structured light pattern in a first direction and with a desired intensity pattern includes generating a plurality of one-dimensional light patterns using a two-dimensional diffractive optical element with different periods in two orthogonal directions, and combining the plurality of one-dimensional light patterns to form the one-dimensional structured light pattern with the desired intensity pattern. Each of the one-dimensional light patterns includes a one-dimensional light pattern in the first direction. The plurality of one-dimensional light patterns is distributed in a second direction different from the first direction. A separation angle between each pair of adjacent one-dimensional light patterns of the plurality of one-dimensional light patterns in the second direction and with respect to the two-dimensional diffractive optical element is less than a threshold value.

Abstract: In one embodiment, a system includes at least one projector configured to project a plurality of projected patterns onto a scene, the projected patterns including a first projected pattern that includes a plurality of first projected features, a camera configured to capture a plurality of images including a first detected pattern corresponding to a reflection of the first projected pattern, and one or more processors configured to: compute a depth map of the scene based on the first projected pattern, the first detected pattern, and relative positions of the camera and the at least one projector, project, using the projector, a second projected pattern comprising a plurality of second projected features onto a portion of the scene, where the second projected pattern is more sparse than the first projected pattern, and capture, using the camera, a second detected pattern corresponding to a reflection of the second projected pattern.

Abstract: In one embodiment, a computing system may access a first image and a second image of at least a common portion of an environment while a light emission with a predetermined emission pattern is projected by a projector. The first and second images are respectively captured by a first and a second detector that are respectively separated from the projector by a first and a second distance. The system may determine that a first portion of the first image corresponds to a second portion of the second image. The system may compute, using triangulation, a first depth value associated with the first portion and a second depth value associated with the second portion. The system may determine that the first and second depth values match in accordance with one or more predetermined criteria, and generate a depth map of the environment based on at least one of the depth values.

Abstract: In one embodiment, a system includes a first projector configured to project a first projected pattern having one or more first projected lighting characteristics, a second projector configured to project a second projected pattern having one or more second projected lighting characteristics, a camera configured to capture an image comprising first and second detected patterns corresponding to reflections of the first and second projected patterns, respectively, and one or more processors configured to: identify a detected point in the image that corresponds to a projected point in at least one of the first and second projected patterns by comparing detected lighting characteristics of the first and second detected patterns with the first and second projected lighting characteristics, and compute a depth associated with the detected point based on the projected point, the detected point, and a relative position between the camera and at least one of the projectors.