Abstract: A system for updating continuous image alignment of separate cameras identifies a previous alignment matrix associated with a previous frame pair captured at one or more previous timepoints by a reference camera and a match camera. The previous alignment matrix is based on visual correspondences in the previous frame pair. The system also identifies a current matrix associated with a current frame pair captured at one or more current timepoints by the reference camera and the match camera. The current matrix is based on visual correspondences in the current frame pair. The system also identifies a difference value associated with the reference camera or the match camera relative to the one or more previous timepoints and the one or more current timepoints. The system also generates an updated alignment matrix by using the previous alignment matrix, the current matrix, and the difference value as inputs.

Abstract: Techniques to temporally filter images via a filtering weight computation are disclosed. A first image having a first timestamp and a second image having a second timestamp are acquired. These images are generated by a camera, and the first timestamp is before the second timestamp. A motion compensation (MC) operation is performed on the first image to produce an MC image. A difference image is generated using the MC image and the second image. The difference image reflects differences in intensities that exist between the two images. A local weight map is generated based on those differences. A global weight map is generated based on certain IMU data. A final weight map is generated based on a combination of the local weight map and the global weight map. The final weight map is used to generate a filtered image.

Abstract: Techniques for aligning images generated by two cameras are disclosed. This alignment is performed by computing a relative 3D orientation between the two cameras. A first gravity vector for a first camera and a second gravity vector for a second camera are determined. A first camera image is obtained from the first camera, and a second camera image is obtained from the second camera. A first alignment process is performed to partially align the first camera's orientation with the second camera's orientation. This process is performed by aligning the gravity vectors, thereby resulting in two degrees of freedom of the relative 3D orientation being eliminated. Visual correspondences between the two images are identified. A second alignment process is performed to fully align the orientations. This process is performed by using the identified visual correspondences to identify and eliminate a third degree of freedom of the relative 3D orientation.

Abstract: One method comprises receiving a hit signal from a device worn by a first player, receiving a position of the device, receiving an orientation of a launch axis of a virtual-projectile launcher, receiving a position of a second player, and outputting a hit assignment on determining, pursuant to receiving the hit signal, that a recognized object and the second player are coincident at an indicated launch of a virtual projectile. Another method comprises receiving an indication of launch of a virtual projectile by a virtual-projectile launcher of a first player, receiving an image aligned to a launch axis of the virtual-projectile launcher, outputting a hit signal to a server on determining, pursuant to receiving the indication of launch, that a recognized object is imaged in a projectile-delivery area of the image, and outputting a position of the device and an orientation of the launch axis to the server.

Abstract: Techniques for aligning images generated by two cameras are disclosed. This alignment is performed by computing a relative 3D orientation between the two cameras. A first gravity vector for a first camera and a second gravity vector for a second camera are determined. A first camera image is obtained from the first camera, and a second camera image is obtained from the second camera. A first alignment process is performed to partially align the first camera's orientation with the second camera's orientation. This process is performed by aligning the gravity vectors, thereby resulting in two degrees of freedom of the relative 3D orientation being eliminated. Visual correspondences between the two images are identified. A second alignment process is performed to fully align the orientations. This process is performed by using the identified visual correspondences to identify and eliminate a third degree of freedom of the relative 3D orientation.

Abstract: A system for selectively modifying gating rate in a single photon avalanche diode (SPAD) is configurable to access first frame metadata associated with a first image frame. The first image frame is captured by performing a first plurality of gate operations to configure the SPAD array to enable photon detection over a frame capture time period. The first plurality of gate operations is performed at a first gating rate such that the first plurality of gate operations comprises a first quantity of gate operations performed over the frame capture time period. The system is further configurable to define a second gating rate based on the first frame metadata and capture a second image frame by performing a second plurality of gate operations to configure the SPAD array to enable photon detection at the second gating rate.

Abstract: A system for obtaining dark current images includes one or more processors and one or more hardware storage devices storing instructions that are executable by the one or more processors to configure the system to perform various acts. The acts include obtaining a first image frame, generating a first low-pass filtered image by applying a low-pass filter to the first image frame, and generating a first estimated dark current image by subtracting the first low-pass filtered image from the first image frame.

Abstract: Techniques for performing a hardware-based image alignment process are disclosed. A relative position and orientation between a system camera and a detached external camera are determined. This process is performed using 6 degree of freedom (DOF) trackers on the system camera and on the external camera. A depth measurement, which indicates a distance between the external camera and a scene where the external camera is aimed, is obtained. The system camera generates a system camera image, and the external camera generates an image. An overlaid image is generated by using the relative position and orientation and the depth measurement to reproject the second content onto the first content.

Abstract: Techniques for correcting an overlay misalignment between an external camera image and a system camera image are disclosed. A first system camera image and a first external camera image are acquired. A first visual alignment is performed between those two images to produce an overlaid image. Some of the content in the overlaid image is surrounded by a bounding element. A position of the bounding element is modified based on movements of the system camera and/or the external camera. In response to performing a second visual alignment using new images, an update vector is computed. Relative movement between the two cameras is determined. Based on the movement and based on the update vector, the bounding element is progressively transitioned to a corrected position in the overlaid image. A speed by which the bounding element is progressively transitioned is proportional to the amount of movement.

Abstract: Techniques for evaluating multiple images, which originate from multiple different sources, and for selecting specific images to generate an overlaid image are disclosed. A first set of system camera images (e.g., based on a first FPS rate) and a second set of external camera images (e.g., based on a second FPS rate) are obtained. A set of rules are accessed in order to govern a selection process for selecting a specific system camera image and a specific external camera image. The selected images are designated for use in generating an overlaid image. The selection process is performed using the accessed set of rules. The overlaid image is generated by overlaying and aligning content obtained from the selected images.

Abstract: Techniques for de-aliasing depth ambiguities included within infrared phase depth images are described herein. An illuminator emits reference light towards a target object. Some of this light is reflected back and detected. A phase image is generated based on phase differences between the reference light and the reflected light. The phase differences represent changes in depth within overlapping sinusoidal periods of the reference and reflected light. The phase image also includes ambiguities because multiple different depths within the phase image share the same phase difference value, even though these depths actually correspond to different real-world depths. The phase image is fed as input to a machine learning (“ML”) component, which is configured to de-alias the ambiguities by determining, for each pixel in the phase image, a corresponding de-aliasing interval. A depth map is generated based on the phase image and any de-aliasing intervals generated by the ML component.

Abstract: An image capture module configured for improved heat dissipation includes an image sensor, a first heat spreading element positioned to direct heat from the image sensor along a first heat dissipation path toward a first portion of the image capture module, a processing board in data communication with the image sensor, and a second heat spreading element positioned to dissipate heat from the processing board along a second heat dissipation path toward a second portion of the image capture module. Thermal isolation is used to isolate the different heat paths. The first heat dissipation path does not overlap the second heat dissipation path, the first portion of the image capture module is separate from the second portion of the image capture module.

Abstract: Techniques for generating a high resolution full color output image from lower resolution sparse color input images are disclosed. A camera generates images. The camera's sensor has a sparse Bayer pattern. While the camera is generating the images, IMU data for each image is acquired. The IMU data indicates a corresponding pose the camera was in while the camera generated each image. The images and IMU data are fed into a motion model, which performs temporal filtering on the images and uses the IMU data to generate a red-only image, a green-only image, a blue-only image, and a monochrome image. The color images are up-sampled to match the resolution of the monochrome image. A high resolution output color image is generated by combining the up-sampled images and the monochrome image.

Abstract: A method for estimating human body temperature includes receiving, via a thermal camera, a thermal image captured of a real-world environment, the thermal image including thermal intensity values for each of a plurality of pixels of the thermal image. A position of a human face is identified within the thermal image, the human face corresponding to a human subject. An indication of a distance between the human subject and the thermal camera is received. Based on the distance, a distance correction factor is applied to one or more thermal intensity values of one or more pixels corresponding to the human face to give one or more distance-corrected thermal intensity values. Based on the one or more distance-corrected thermal intensity values an indication of a body temperature of the human subject is reported.

Abstract: Techniques for updating a position of overlaid image content using IMU data to reflect subsequent changes in camera positions to minimize latency effects are disclosed. A “system camera” refers to an integrated camera that is a part of an HMD. An “external camera” is a camera that is separated from the HMD. The system camera and the external camera generate images. These images are overlaid on one another and aligned to form an overlaid image. Content from the external camera image is surrounded by a bounding element in the overlaid image. IMU data associated with both the system camera and the external camera is obtained. Based on that IMU data, an amount of movement that the system camera and/or the external camera have moved since the images were originally generated is determined. Based on that movement, the bounding element is shifted to a new position in the overlaid image.

Abstract: A method for estimating human body temperature includes receiving, via a thermal camera, one or more thermal images captured of a real-world environment, the one or more thermal images including thermal intensity values for each of a plurality of pixels. Positions of a plurality of human faces are identified in the one or more thermal images. A distribution of thermal intensity values of the plurality of human faces is determined. A position of a test human face of a test human subject is identified within a subsequent thermal image. One or more test thermal intensity values of one or more pixels corresponding to the test human face are identified. An indication of a body temperature of the test human subject is reported based on a comparison of the one or more test thermal intensity values and the distribution of thermal intensity values of the plurality of human faces.

Abstract: Disclosed herein are techniques for providing an illumination system that emits illumination into an environment while also enabling that system to be undetectable to certain types of external light detection systems. The system includes a single photon avalanche diode (SPAD) low light (LL) detection device and a light emitting device. The light emitting device provides illumination having a wavelength of at least 950 nanometers (nm). An intensity of the illumination is set to a level that causes the illumination to be undetectable from a determined distance away based on the roll off rate of the light. While the light emitting device is providing the illumination, the SPAD LL detection device generates an image of an environment in which the illumination is being provided.

Abstract: Examples are disclosed that relate to motion compensation on a single photon avalanche detector (SPAD) array camera. One example provides a method enacted on an imaging device comprising a SPAD array camera and a motion sensor, the SPAD array camera comprising a plurality of pixels. The method comprises acquiring a plurality of subframes of image data. Each subframe of image data comprises a binary value for each pixel. Based upon motion data from the motion sensor, the method further comprises determining a change in pose of the imaging device between adjacent subframes, applying a positional offset to a current subframe based upon the motion data to align a location of a stationary imaged feature in the current subframe with a location of the stationary imaged feature in a prior subframe to create aligned subframes, summing the aligned subframes to form an image, and outputting the image.

Abstract: Systems and methods are provided for facilitating computer vision tasks (e.g., simultaneous location and mapping) and pass-through imaging include a head-mounted display (HMD) that includes a first set of one or more cameras configured for performing computer vision tasks and a second set of one or more cameras configured for capturing image data of an environment for projection to a user of the HMD. The first set of one or more cameras is configured to detect at least a visible spectrum light and at least a particular band of wavelengths of infrared (IR) light. The second set of one or more cameras includes one or more detachable IR filters configured to attenuate IR light, including at least a portion of the particular band of wavelengths of IR light.

Abstract: A system for dense depth computation aided by sparse feature matching generates a first image using a first camera, a second image using a second camera, and a third image using a third camera. The system generates a sparse disparity map using the first image and the third image by (1) identifying a set of feature points within the first image and a set of corresponding feature points within the third image, and (2) identifying feature disparity values based on the set of feature points and the set of corresponding feature points. The system also applies the first image, the second image, and the sparse disparity map as inputs for generating a dense disparity map.