Abstract: A computer-implemented method of field calibrating a device is disclosed. In one example, a reference image is acquired, via an infrared camera, while the device is in a factory-calibrated state. The reference image measures a factory-calibrated spatial relationship of one or more infrared-visible fiducial markers of the device relative to the infrared camera while the device is in the factory-calibrated state. Later, a field image is acquired, via the infrared camera. The field image measures an updated spatial relationship of the one or more infrared-visible fiducial markers relative to the infrared camera. The device is field calibrated based on the reference image and the field image.

Abstract: The disclosed techniques provide enhanced eye tracking systems utilizing joint estimation of biological parameters and hardware parameters. By the use of joint estimation of biological parameters, e.g., direction and position of an eye, with concurrent estimation of hardware parameters, e.g., camera position or camera direction, a system can self-calibrate and provide eye tracking estimations that can allow a system to accommodate for deformations and other changes of a device. The disclosed techniques include a method to model changes of a device, as well as detect and compensate for them while the eye-tracking device is in normal use, thereby preserving the performance of the system without requiring a factory-calibration procedure to be repeated.

Abstract: A computer-implemented method of field calibrating a device is disclosed. In one example, a reference image is acquired, via an infrared camera, while the device is in a factory-calibrated state. The reference image measures a factory-calibrated spatial relationship of one or more infrared-visible fiducial markers of the device relative to the infrared camera while the device is in the factory-calibrated state. Later, a field image is acquired, via the infrared camera. The field image measures an updated spatial relationship of the one or more infrared-visible fiducial markers relative to the infrared camera. The device is field calibrated based on the reference image and the field image.

Abstract: In various examples there is an apparatus for computing an image depicting a face of a wearer of a head mounted display (HMD), as if the wearer was not wearing the HMD. An input image depicts a partial view of the wearer's face captured from at least one face facing capture device in the HMD. A machine learning apparatus is available which has been trained to compute expression parameters from the input image. A 3D face model that has expressions parameters is accessible as well as a photorealiser being a machine learning model trained to map images rendered from the 3D face model to photorealistic images. The apparatus computes expression parameter values from the image using the machine learning apparatus. The apparatus drives the 3D face model with the expression parameter values to produce a 3D model of the face of the wearer and then renders the 3D model from a specified viewpoint to compute a rendered image. The rendered image is upgraded to a photorealistic image using the photorealiser.

Abstract: Disclosed are an apparatus and a method of low-latency, low-power eye tracking. In some embodiments, the eye tracking method operates a first sensor having a first level of power consumption that tracks positions of an eye of a user. In response to detection that the eye does not change position for a time period, the method stops operation of the first sensor and instead operates a second sensor that detects a change of position of the eye. The second sensor has a level of power consumption lower than the level of power consumption of the first sensor. Once the eye position changes, the second sensor resumes operation.

Abstract: A method of detecting eye location for a head-mounted display system includes directing positioning light to an eye of a user and detecting the positioning light reflected from the eye of the user. The method further includes determining a distance between the eye and a near-eye optic of the head-mounted display system based on attributes of the detected positioning light, and providing feedback for adjusting the distance between the eye and the near-eye optic.

Abstract: Via a near-eye display, one or more pre-saccade image frames are displayed to a user eye. Based on a detected movement of the user eye, the user eye is determined to be performing a saccade. One or more saccade-contemporaneous image frames are displayed with a temporary saccade-specific image effect not applied to the pre-saccade image frames.

Abstract: Examples are disclosed herein that relate to user authentication. One example provides a biometric identification system comprising an iris illuminator, an image sensor configured to capture light reflected from irises of a user as a result of those irises being illuminated by the iris illuminator, a drive circuit configured to drive the iris illuminator in a first mode and a second mode that each cause the irises to be illuminated differently, the first and second modes thereby yielding a first mode output at the image sensor and a second mode output at the image sensor, respectively, and a processor configured to process at least one of the first mode output and the second mode output and, in response to such processing, select one of the first mode and the second mode for use in performing an iris authentication on the user.

Abstract: Disclosed are an apparatus and a method of low-latency, low-power eye tracking. In some embodiments, the eye tracking method operates a first sensor having a first level of power consumption that tracks positions of an eye of a user. In response to detection that the eye does not change position for a time period, the method stops operation of the first sensor and instead operates a second sensor that detects a change of position of the eye. The second sensor has a level of power consumption lower than the level of power consumption of the first sensor. Once the eye position changes, the second sensor resumes operation.

Abstract: Embodiments that relate to determining an estimated pupil region of an eye are disclosed. In one embodiment a method includes receiving an image of an eye, with the image comprising a plurality of pixels. A rough pupil region is generated using at least a subset of the plurality of pixels. A plurality of pupil boundary point candidates are extracted from the rough pupil region, with each of the candidates weighted based on color values of at least two neighbor pixels. A parametric curve may be fitted to the weighted pupil boundary point candidates to determine the estimated pupil region of the eye of the user.

Abstract: Examples are disclosed herein that relate to user authentication. One example provides a biometric identification system comprising an iris illuminator, an image sensor configured to capture light reflected from irises of a user as a result of those irises being illuminated by the iris illuminator, a drive circuit configured to drive the iris illuminator in a first mode and a second mode that each cause the irises to be illuminated differently, the first and second modes thereby yielding a first mode output at the image sensor and a second mode output at the image sensor, respectively, and a processor configured to process at least one of the first mode output and the second mode output and, in response to such processing, select one of the first mode and the second mode for use in performing an iris authentication on the user.

Abstract: Embodiments relating to determining characteristics of eyeglass lenses are disclosed. A head-mounted display device comprises a camera communicatively coupled to a computing device and including an optical axis having a center point. Light sources are configured to emit light rays toward the lens to produce lens glints. The light sources are in a light source plane that is spaced from a lens plane by an offset distance of between 8 mm and 12 mm. The light sources are either spaced vertically from a line perpendicular to the light source plane and extending through the center point by a distance between 13 mm and 53 mm, or spaced horizontally from the line by a distance of between 13 mm and 80 mm. Lens characterization program logic identifies an image location of each lens glint, and outputs an estimated lens shape model comprising the one or more lens characteristics.

Abstract: Embodiments that relate to determining an estimated pupil region of an eye are disclosed. In one embodiment a method includes receiving an image of an eye, with the image comprising a plurality of pixels. A rough pupil region is generated using at least a subset of the plurality of pixels. A plurality of pupil boundary point candidates are extracted from the rough pupil region, with each of the candidates weighted based on color values of at least two neighbor pixels. A parametric curve may be fitted to the weighted pupil boundary point candidates to determine the estimated pupil region of the eye of the user.

Abstract: Embodiments that relate to determining an estimated pupil region of an eye are disclosed. In one embodiment a method includes receiving an image of an eye, with the image comprising a plurality of pixels. A rough pupil region is generated using at least a subset of the plurality of pixels. A plurality of pupil boundary point candidates are extracted from the rough pupil region, with each of the candidates weighted based on color values of at least two neighbor pixels. A parametric curve may be fitted to the weighted pupil boundary point candidates to determine the estimated pupil region of the eye of the user.

Abstract: Embodiments relating to determining characteristics of eyeglass lenses are disclosed. A head-mounted display device comprises a camera communicatively coupled to a computing device and including an optical axis having a center point. Light sources are configured to emit light rays toward the lens to produce lens glints. The light sources are in a light source plane that is spaced from a lens plane by an offset distance of between 8 mm and 12 mm. The light sources are either spaced vertically from a line perpendicular to the light source plane and extending through the center point by a distance between 13 mm and 53 mm, or spaced horizontally from the line by a distance of between 13 mm and 80 mm. Lens characterization program logic identifies an image location of each lens glint, and outputs an estimated lens shape model comprising the one or more lens characteristics.

Abstract: Embodiments that relate to determining an estimated pupil region of an eye are disclosed. In one embodiment a method includes receiving an image of an eye, with the image comprising a plurality of pixels. A rough pupil region is generated using at least a subset of the plurality of pixels. A plurality of pupil boundary point candidates are extracted from the rough pupil region, with each of the candidates weighted based on color values of at least two neighbor pixels. A parametric curve may be fitted to the weighted pupil boundary point candidates to determine the estimated pupil region of the eye of the user.

Abstract: Embodiments are disclosed for adjusting alignment of a near-eye optic of a see-through head-mounted display system. In one embodiment, a method of detecting eye location for a head-mounted display system includes directing positioning light to an eye of a user and detecting the positioning light reflected from the eye of the user. The method further includes determining a distance between the eye and a near-eye optic of the head-mounted display system based on attributes of the detected positioning light, and providing feedback for adjusting the distance between the eye and the near-eye optic.