Abstract: An optical combiner in a display system of a mixed-reality head-mounted display (HMD) device comprises a lens of birefringent material and a ferroelectric liquid crystal (FLC) modulator that are adapted for use with a reflective waveguide to provide multiple different focal planes on which holograms of virtual-world objects (i.e., virtual images) are displayed. The birefringent lens has two orthogonal refractive indices, ordinary and extraordinary, depending on the polarization state of the incident light. Depending on the rotation of the polarization axis by the FLC modulator, the incoming light to the birefringent lens is focused either at a distance corresponding to the ordinary refractive index or the extraordinary refractive index. Virtual image light leaving the birefringent lens is in-coupled to a see-through reflective waveguide which is configured to form an exit pupil for the optical combiner to enable an HMD device user to view the virtual images from the source.

Abstract: Examples are disclosed that relate to calibration of a stereoscopic display system of an HMD via an optical calibration system comprising a waveguide combiner. One example provides an HMD device comprising a first image projector and a second image projector configured to project a stereoscopic image pair, and an optical calibration system. The optical calibration system comprises a first optical path indicative of an alignment of the first image projector, a second optical path indicative of an alignment of the second image projector, a waveguide combiner in which the first and second optical paths combine into a shared optical path, and one or more boresight sensors configured to detect calibration image light traveling along one or more of the first optical or the second optical path.

Abstract: Examples are disclosed that relate to display devices having a common light path region. One example provides a display device comprising a light source configured to emit illumination light along an illumination path, and a spatial light modulator configured to modulate the illumination light and emit the modulated illumination light as image light along an imaging path, wherein at least a portion of the illumination path and at least a portion of the imaging path extend through a common light path region. The display device further comprises one or more optical elements positioned within the common light path region, at least one optical element being configured to guide the illumination light as the illumination light travels through the common light path region toward the spatial light modulator, and shape the image light as the image light travels through the common light path region.

Abstract: Eye and hand tracking systems in head-mounted display (HMD) devices are arranged with lensless camera systems using optical masks as encoding elements that apply convolutions to optical images of body parts (e.g., eyes or hands) of HMD device users. The convolved body images are scrambled or coded representations that are captured by a sensor in the system, but are not human-recognizable. A machine learning system such as a neural network is configured to extract body features directly from the coded representation without performance of deconvolutions conventionally utilized to reconstruct the original body images in human-recognizable form. The extracted body features are utilized by the respective eye or hand tracking systems to output relevant tracking data for the user's eyes or hands which may be utilized by the HMD device to support various applications and user experiences. The lensless camera and machine learning system are jointly optimizable on an end-to-end basis.

Abstract: Examples are disclosed that relate to calibration of a stereoscopic display system of an HMD via an optical calibration system comprising a waveguide combiner. One example provides an HMD device comprising a first image projector and a second image projector configured to project a stereoscopic image pair, and an optical calibration system. The optical calibration system comprises a first optical path indicative of an alignment of the first image projector, a second optical path indicative of an alignment of the second image projector, a waveguide combiner in which the first and second optical paths combine into a shared optical path, and one or more boresight sensors configured to detect calibration image light traveling along one or more of the first optical or the second optical path.

Abstract: Variable-focus lenses are arranged as a lens pair that work on opposite sides of a see-through optical combiner used in a mixed-reality head-mounted display (HMD) device. An eye-side variable-focus lens is configured as a negative lens over an eyebox of the see-through optical combiner to enable virtual-world objects to be set at a close distance. The negative lens is compensated by its conjugate using a real-world-side variable-focus lens configured as a positive lens to provide an unperturbed see-through experience. For non-presbyopes, the powers of the lenses are perfectly offset. For presbyopes, the lens powers is mismatched at times to provide simultaneous views of both virtual-world and real-world objects on the display in sharp focus. Responsively an eye tracker indicating that the user is engaged in close viewing, optical power is added to the real-world-side lens to push close real-world objects optically farther away into sharp focus for the presbyopic user.

Abstract: Eye and hand tracking systems in head-mounted display (HMD) devices are arranged with lensless camera systems using optical masks as encoding elements that apply convolutions to optical images of body parts (e.g., eyes or hands) of HMD device users. The convolved body images are scrambled or coded representations that are captured by a sensor in the system, but are not human-recognizable. A machine learning system such as a neural network is configured to extract body features directly from the coded representation without performance of deconvolutions conventionally utilized to reconstruct the original body images in human-recognizable form. The extracted body features are utilized by the respective eye or hand tracking systems to output relevant tracking data for the user's eyes or hands which may be utilized by the HMD device to support various applications and user experiences. The lensless camera and machine learning system are jointly optimizable on an end-to-end basis.

Abstract: A system is presented for a display engine. An optical imaging pathway comprises at least a selectively reflective image forming device. An illumination beam pathway comprises an optical source cluster including one or more optical sources, optical componentry configured to generate uniform illumination of the selectively reflective image forming device, and one or more photodiodes positioned to capture light reflected off the selectively reflective image forming device. A controller is configured to command the selectively reflective image forming device to operate with a predetermined reflectivity. While the selectively reflective image forming device is operating with the predetermined reflectivity, the optical source is commanded to emit a pulse of light and the one or more photodiodes are read out. A performance profile of one or more of the optical sources and the selectively reflective image forming device is adjusted based on the photodiode readout.

Abstract: A display engine adapted for use in a head-mounted display (HMD) device includes a reflective liquid crystal on silicon (LCoS) spatial light modulator (SLM) that is illuminated using a backlight illumination module and a pair of optical prisms providing a total internal reflection (TIR) function. In an illustrative mixed-reality embodiment, the TIR prism pair guides light to the LCoS SLM from the backlight illumination module and projects virtual images reflected from the LCoS SLM, through projection optics, to a diffractive waveguide combiner for viewing by an HMD user.

Abstract: Variable-focus lenses are arranged as a lens pair that work on opposite sides of a see-through optical combiner used in a mixed-reality head-mounted display (HMD) device. An eye-side variable-focus lens is configured as a negative lens over an eyebox of the see-through optical combiner to enable virtual-world objects to be set at a close distance. The negative lens is compensated by its conjugate using a real-world-side variable-focus lens configured as a positive lens to provide for an unperturbed see-through experience. For non-presbyopes, the powers of the lenses are perfectly offset. For presbyopes, the lens powers may be mismatched at times to provide simultaneous views of both virtual-world and real-world objects on the display in sharp focus. Responsively an eye tracker indicating that the user is engaged in close viewing, optical power is added to the real-world-side lens to push close real-world objects optically farther away and into sharp focus for the presbyopic user.

Abstract: A near-eye display device comprises right and left display projectors, expansion optics, and inertial measurement units (IMUs), in addition to a plurality of angle-sensitive pixel (ASP) elements and a computer. The right and left expansion optics are configured to receive respective display images from the right and left display projectors and to release expanded forms of the display images. The right IMU is fixedly coupled to the right display projector, and the left IMU is fixedly coupled to the left display projector. Each ASP element is responsive to an angle of light of one of the respective display images as received into the right or left expansion optic. The computer is configured to receive output from the right IMU, the left IMU and the plurality of ASP elements, and render display data for the right and left display projectors based in part on the output.

Abstract: A projection system includes an illumination light source configured to emit an illumination light beam, a monitor light source configured to emit a monitor light beam, and a projector configured to project both the illumination light beam and the monitor light beam into a projected combined light beam. A first portion of the projected combined light beam is propagated over a first beam path in a first direction, causing an eye of a user to see a display image. A second portion of the projected combined light beam is propagated over a second beam path in a second direction, causing a monitor camera to capture a monitor image. The monitor image is analyzed to determine an orientation or a position of the monitor image. In response to determining that the monitor image is not properly oriented or positioned, an orientation or position of the projector or the illumination image is adjusted.

Abstract: Examples are disclosed that relate to calibration of a stereoscopic display system of an HMD via an optical calibration system comprising a waveguide combiner. One example provides an HMD device comprising a first image projector and a second image projector configured to project a stereoscopic image pair, and an optical calibration system. The optical calibration system comprises a first optical path indicative of an alignment of the first image projector, a second optical path indicative of an alignment of the second image projector, a waveguide combiner in which the first and second optical paths combine into a shared optical path, and one or more boresight sensors configured to detect calibration image light traveling along one or more of the first optical or the second optical path.

Abstract: Examples are disclosed that relate to display devices having a common light path region. One example provides a display device comprising a light source configured to emit illumination light along an illumination path, and a spatial light modulator configured to modulate the illumination light and emit the modulated illumination light as image light along an imaging path, wherein at least a portion of the illumination path and at least a portion of the imaging path extend through a common light path region. The display device further comprises one or more optical elements positioned within the common light path region, at least one optical element being configured to guide the illumination light as the illumination light travels through the common light path region toward the spatial light modulator, and shape the image light as the image light travels through the common light path region.

Abstract: A display engine adapted for use in a head-mounted display (HMD) device includes a reflective liquid crystal on silicon (LCoS) spatial light modulator (SLM) that is illuminated using a backlight illumination module and a pair of optical prisms providing a total internal reflection (TIR) function. In an illustrative mixed-reality embodiment, the TIR prism pair guides light to the LCoS SLM from the backlight illumination module and projects virtual images reflected from the LCoS SLM, through projection optics, to a diffractive waveguide combiner for viewing by an HMD user.

Abstract: In a see-through waveguide-based HMD device configured to display holographic virtual images within a field of view (FOV) of the device user, a single pixel or group of pixels are lit to supply illumination at known locations on the display that is reflected from the user's eyes and captured by one or more sensors in an eye tracker. The eye tracker may apply real-time image analysis to the captured reflected light, called “glints,” to extract features of the user's eyes to determine where the HMD device user is looking—the gaze point—and calculate eye movement, location, and orientation. A negative lens functionality utilized in the HMD device to provide a fixed focal depth for the virtual images enables the lit pixels to function as virtual glint sources for the eye tracker sensor that are located at the fixed focal depth and from multiple illumination positions within the user's FOV.

Abstract: An optical combiner in a display system of a mixed-reality head-mounted display (HMD) device comprises a lens of birefringent material and a ferroelectric liquid crystal (FLC) modulator that are adapted for use with a reflective waveguide to provide multiple different focal planes on which holograms of virtual-world objects (i.e., virtual images) are displayed. The birefringent lens has two orthogonal refractive indices, ordinary and extraordinary, depending on the polarization state of the incident light. Depending on the rotation of the polarization axis by the FLC modulator, the incoming light to the birefringent lens is focused either at a distance corresponding to the ordinary refractive index or the extraordinary refractive index. Virtual image light leaving the birefringent lens is in-coupled to a see-through reflective waveguide which is configured to form an exit pupil for the optical combiner to enable an HMD device user to view the virtual images from the source.

Abstract: Variable-focus lenses are arranged as a lens pair that work on opposite sides of a see-through optical combiner used in a mixed-reality head-mounted display (HMD) device. An eye-side variable-focus lens is configured as a negative lens over an eyebox of the see-through optical combiner to enable virtual-world objects to be set at a close distance. The negative lens is compensated by its conjugate using a real-world-side variable-focus lens configured as a positive lens to provide for an unperturbed see-through experience. For non-presbyopes, the powers of the lenses are perfectly offset. For presbyopes, the lens powers may be mismatched at times to provide simultaneous views of both virtual-world and real-world objects on the display in sharp focus. Responsively an eye tracker indicating that the user is engaged in close viewing, optical power is added to the real-world-side lens to push close real-world objects optically farther away and into sharp focus for the presbyopic user.

Abstract: A near-eye display device comprises right and left display projectors, expansion optics, and inertial measurement units (IMUs), in addition to a plurality of angle-sensitive pixel (ASP) elements and a computer. The right and left expansion optics are configured to receive respective display images from the right and left display projectors and to release expanded forms of the display images. The right IMU is fixedly coupled to the right display projector, and the left IMU is fixedly coupled to the left display projector. Each ASP element is responsive to an angle of light of one of the respective display images as received into the right or left expansion optic. The computer is configured to receive output from the right IMU, the left IMU and the plurality of ASP elements, and render display data for the right and left display projectors based in part on the output.

Abstract: In a see-through waveguide-based HMD device configured to display holographic virtual images within a field of view (FOV) of the device user, a single pixel or group of pixels are lit to supply illumination at known locations on the display that is reflected from the user's eyes and captured by one or more sensors in an eye tracker. The eye tracker may apply real-time image analysis to the captured reflected light, called “glints,” to extract features of the user's eyes to determine where the HMD device user is looking—the gaze point—and calculate eye movement, location, and orientation. A negative lens functionality utilized in the HMD device to provide a fixed focal depth for the virtual images enables the lit pixels to function as virtual glint sources for the eye tracker sensor that are located at the fixed focal depth and from multiple illumination positions within the user's FOV.