Abstract: A system for low compute high-resolution depth map generation using low-resolution cameras is configured to obtain a stereo pair of images and generate a depth map by performing stereo matching on the stereo pair of images. The system is also configured to obtain a first image comprising first texture information for the environment that has a first image resolution that is higher than an image resolution of images of the stereo pair of images. The system is further configured to generate a reprojected first image by reprojecting the first image to correspond to an image capture perspective associated with the depth map. The reprojection of the first image is based on depth information from the depth map and includes reprojected first texture information for the environment. The system is also configured to generate an upsampled depth map based on the depth map.

Abstract: A system for updating continuous image alignment of separate cameras identifies a previous alignment matrix associated with a previous frame pair captured at one or more previous timepoints by a reference camera and a match camera. The previous alignment matrix is based on visual correspondences in the previous frame pair. The system also identifies a current matrix associated with a current frame pair captured at one or more current timepoints by the reference camera and the match camera. The current matrix is based on visual correspondences in the current frame pair. The system also identifies a difference value associated with the reference camera or the match camera relative to the one or more previous timepoints and the one or more current timepoints. The system also generates an updated alignment matrix by using the previous alignment matrix, the current matrix, and the difference value as inputs.

Abstract: Techniques to temporally filter images via a filtering weight computation are disclosed. A first image having a first timestamp and a second image having a second timestamp are acquired. These images are generated by a camera, and the first timestamp is before the second timestamp. A motion compensation (MC) operation is performed on the first image to produce an MC image. A difference image is generated using the MC image and the second image. The difference image reflects differences in intensities that exist between the two images. A local weight map is generated based on those differences. A global weight map is generated based on certain IMU data. A final weight map is generated based on a combination of the local weight map and the global weight map. The final weight map is used to generate a filtered image.

Abstract: Techniques for aligning images generated by two cameras are disclosed. This alignment is performed by computing a relative 3D orientation between the two cameras. A first gravity vector for a first camera and a second gravity vector for a second camera are determined. A first camera image is obtained from the first camera, and a second camera image is obtained from the second camera. A first alignment process is performed to partially align the first camera's orientation with the second camera's orientation. This process is performed by aligning the gravity vectors, thereby resulting in two degrees of freedom of the relative 3D orientation being eliminated. Visual correspondences between the two images are identified. A second alignment process is performed to fully align the orientations. This process is performed by using the identified visual correspondences to identify and eliminate a third degree of freedom of the relative 3D orientation.

Abstract: One method comprises receiving a hit signal from a device worn by a first player, receiving a position of the device, receiving an orientation of a launch axis of a virtual-projectile launcher, receiving a position of a second player, and outputting a hit assignment on determining, pursuant to receiving the hit signal, that a recognized object and the second player are coincident at an indicated launch of a virtual projectile. Another method comprises receiving an indication of launch of a virtual projectile by a virtual-projectile launcher of a first player, receiving an image aligned to a launch axis of the virtual-projectile launcher, outputting a hit signal to a server on determining, pursuant to receiving the indication of launch, that a recognized object is imaged in a projectile-delivery area of the image, and outputting a position of the device and an orientation of the launch axis to the server.

Abstract: Techniques for aligning images generated by two cameras are disclosed. This alignment is performed by computing a relative 3D orientation between the two cameras. A first gravity vector for a first camera and a second gravity vector for a second camera are determined. A first camera image is obtained from the first camera, and a second camera image is obtained from the second camera. A first alignment process is performed to partially align the first camera's orientation with the second camera's orientation. This process is performed by aligning the gravity vectors, thereby resulting in two degrees of freedom of the relative 3D orientation being eliminated. Visual correspondences between the two images are identified. A second alignment process is performed to fully align the orientations. This process is performed by using the identified visual correspondences to identify and eliminate a third degree of freedom of the relative 3D orientation.

Abstract: Techniques for performing a hardware-based image alignment process are disclosed. A relative position and orientation between a system camera and a detached external camera are determined. This process is performed using 6 degree of freedom (DOF) trackers on the system camera and on the external camera. A depth measurement, which indicates a distance between the external camera and a scene where the external camera is aimed, is obtained. The system camera generates a system camera image, and the external camera generates an image. An overlaid image is generated by using the relative position and orientation and the depth measurement to reproject the second content onto the first content.

Abstract: A system for obtaining dark current images includes one or more processors and one or more hardware storage devices storing instructions that are executable by the one or more processors to configure the system to perform various acts. The acts include obtaining a first image frame, generating a first low-pass filtered image by applying a low-pass filter to the first image frame, and generating a first estimated dark current image by subtracting the first low-pass filtered image from the first image frame.

Abstract: A system for selectively modifying gating rate in a single photon avalanche diode (SPAD) is configurable to access first frame metadata associated with a first image frame. The first image frame is captured by performing a first plurality of gate operations to configure the SPAD array to enable photon detection over a frame capture time period. The first plurality of gate operations is performed at a first gating rate such that the first plurality of gate operations comprises a first quantity of gate operations performed over the frame capture time period. The system is further configurable to define a second gating rate based on the first frame metadata and capture a second image frame by performing a second plurality of gate operations to configure the SPAD array to enable photon detection at the second gating rate.

Abstract: Techniques for correcting an overlay misalignment between an external camera image and a system camera image are disclosed. A first system camera image and a first external camera image are acquired. A first visual alignment is performed between those two images to produce an overlaid image. Some of the content in the overlaid image is surrounded by a bounding element. A position of the bounding element is modified based on movements of the system camera and/or the external camera. In response to performing a second visual alignment using new images, an update vector is computed. Relative movement between the two cameras is determined. Based on the movement and based on the update vector, the bounding element is progressively transitioned to a corrected position in the overlaid image. A speed by which the bounding element is progressively transitioned is proportional to the amount of movement.

Abstract: Techniques for evaluating multiple images, which originate from multiple different sources, and for selecting specific images to generate an overlaid image are disclosed. A first set of system camera images (e.g., based on a first FPS rate) and a second set of external camera images (e.g., based on a second FPS rate) are obtained. A set of rules are accessed in order to govern a selection process for selecting a specific system camera image and a specific external camera image. The selected images are designated for use in generating an overlaid image. The selection process is performed using the accessed set of rules. The overlaid image is generated by overlaying and aligning content obtained from the selected images.

Abstract: An illumination system having a reduced z-dimensional profile, which is achieved by reflecting light out of plane relative to a light source that generated the light, is disclosed herein. This illumination system includes an IR illumination device, a collimating optic, a turning optic, and a waveguide. The turning optic is specially configured to receive IR light from the IR illumination device and to reflect the IR light out of plane relative to the emission orientation of the IR illumination device. The reflected IR light is reflected towards the collimating optic. The waveguide is positioned in a fixed position relative to the collimating optic and includes an input port or grating to receive the collimated IR light. By reflecting light out of the plane, the size of the illumination system can be beneficially reduced in the z-direction.

Abstract: Techniques for de-aliasing depth ambiguities included within infrared phase depth images are described herein. An illuminator emits reference light towards a target object. Some of this light is reflected back and detected. A phase image is generated based on phase differences between the reference light and the reflected light. The phase differences represent changes in depth within overlapping sinusoidal periods of the reference and reflected light. The phase image also includes ambiguities because multiple different depths within the phase image share the same phase difference value, even though these depths actually correspond to different real-world depths. The phase image is fed as input to a machine learning (“ML”) component, which is configured to de-alias the ambiguities by determining, for each pixel in the phase image, a corresponding de-aliasing interval. A depth map is generated based on the phase image and any de-aliasing intervals generated by the ML component.

Abstract: An image capture module configured for improved heat dissipation includes an image sensor, a first heat spreading element positioned to direct heat from the image sensor along a first heat dissipation path toward a first portion of the image capture module, a processing board in data communication with the image sensor, and a second heat spreading element positioned to dissipate heat from the processing board along a second heat dissipation path toward a second portion of the image capture module. Thermal isolation is used to isolate the different heat paths. The first heat dissipation path does not overlap the second heat dissipation path, the first portion of the image capture module is separate from the second portion of the image capture module.

Abstract: Techniques for generating a high resolution full color output image from lower resolution sparse color input images are disclosed. A camera generates images. The camera's sensor has a sparse Bayer pattern. While the camera is generating the images, IMU data for each image is acquired. The IMU data indicates a corresponding pose the camera was in while the camera generated each image. The images and IMU data are fed into a motion model, which performs temporal filtering on the images and uses the IMU data to generate a red-only image, a green-only image, a blue-only image, and a monochrome image. The color images are up-sampled to match the resolution of the monochrome image. A high resolution output color image is generated by combining the up-sampled images and the monochrome image.

Abstract: A method for estimating human body temperature includes receiving, via a thermal camera, a thermal image captured of a real-world environment, the thermal image including thermal intensity values for each of a plurality of pixels of the thermal image. A position of a human face is identified within the thermal image, the human face corresponding to a human subject. An indication of a distance between the human subject and the thermal camera is received. Based on the distance, a distance correction factor is applied to one or more thermal intensity values of one or more pixels corresponding to the human face to give one or more distance-corrected thermal intensity values. Based on the one or more distance-corrected thermal intensity values an indication of a body temperature of the human subject is reported.

Abstract: Techniques for updating a position of overlaid image content using IMU data to reflect subsequent changes in camera positions to minimize latency effects are disclosed. A “system camera” refers to an integrated camera that is a part of an HMD. An “external camera” is a camera that is separated from the HMD. The system camera and the external camera generate images. These images are overlaid on one another and aligned to form an overlaid image. Content from the external camera image is surrounded by a bounding element in the overlaid image. IMU data associated with both the system camera and the external camera is obtained. Based on that IMU data, an amount of movement that the system camera and/or the external camera have moved since the images were originally generated is determined. Based on that movement, the bounding element is shifted to a new position in the overlaid image.

Abstract: A method for estimating human body temperature includes receiving, via a thermal camera, one or more thermal images captured of a real-world environment, the one or more thermal images including thermal intensity values for each of a plurality of pixels. Positions of a plurality of human faces are identified in the one or more thermal images. A distribution of thermal intensity values of the plurality of human faces is determined. A position of a test human face of a test human subject is identified within a subsequent thermal image. One or more test thermal intensity values of one or more pixels corresponding to the test human face are identified. An indication of a body temperature of the test human subject is reported based on a comparison of the one or more test thermal intensity values and the distribution of thermal intensity values of the plurality of human faces.

Abstract: A head-mounted device (HMD) is configured to perform depth detection with a stereo camera pair comprising a first camera and a second camera, both of which are configured to detect/capture visible light and IR light. The fields of view for both of the cameras overlap to form an overlapping field of view. The HMD also includes an IR dot-pattern illuminator that is mounted on the HMD with the cameras and that is configured to emit an IR dot-pattern illumination. The IR dot-pattern illuminator emits a dot-pattern illumination that spans at least a part of the overlapping field of view. The IR dot-pattern illumination adds texture to objects in the environment and enables the HMD to determine depth for those objects, even if they have textureless/smooth surfaces.

Abstract: Disclosed herein are techniques for providing an illumination system that emits illumination into an environment while also enabling that system to be undetectable to certain types of external light detection systems. The system includes a single photon avalanche diode (SPAD) low light (LL) detection device and a light emitting device. The light emitting device provides illumination having a wavelength of at least 950 nanometers (nm). An intensity of the illumination is set to a level that causes the illumination to be undetectable from a determined distance away based on the roll off rate of the light. While the light emitting device is providing the illumination, the SPAD LL detection device generates an image of an environment in which the illumination is being provided.