Two-dimensional exit-pupil expansion
A near-eye display system includes an image former and first and second series of mutually parallel beamsplitters. The image former is configured to form a display image and to release the display image through an exit pupil. The first series of mutually parallel beamsplitters is arranged to receive the display image from the image former. The second series of mutually parallel beamsplitters is arranged to receive the display image from the first series of beamsplitters, and to release the display image through an exit pupil longer and wider than that of the image former. The second series of beamsplitters has a different alignment and a different orientation than the first series of beamsplitters.
Latest Microsoft Patents:
Near-eye display technology may be used to present video or computer-display imagery with utmost privacy and mobility. Such technology may be incorporated into a head-mounted display (HMD) device in the form of eyeglasses, goggles, a helmet, a visor, or other eyewear. In a typical near-eye display approach, a small-format display image is received into suitable optics and re-directed toward a wearer's eye. One challenge in this area is to present the display image over a sufficiently expansive exit pupil, but without resorting to large, unwieldy optics that the wearer may find objectionable.
SUMMARYThis disclosure describes a near-eye display system including an image former and first and second series of mutually parallel beamsplitters. The image former is configured to form a display image and to release the display image through an exit pupil. The first series of mutually parallel beamsplitters is arranged to receive the display image from the image former. The second series of mutually parallel beamsplitters is arranged to receive the display image from the first series of beamsplitters, and to release the display image through an exit pupil longer and wider than that of the image former. The second series of beamsplitters has a different alignment and a different orientation than the first series of beamsplitters.
This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Furthermore, the claimed subject matter is not limited to implementations that solve any or all disadvantages noted in any part of this disclosure.
Aspects of this disclosure will now be described by example and with reference to the illustrated embodiments listed above. Components that may be substantially the same in one or more embodiments are identified coordinately and are described with minimal repetition. It will be noted, however, that elements identified coordinately may also differ to some degree. It will be further noted that the drawing figures included in this disclosure are schematic and generally not drawn to scale. Rather, the various drawing scales, aspect ratios, and numbers of components shown in the figures may be purposely distorted to make certain features or relationships easier to see.
Sensors 18 may be arranged in any suitable location in HMD device 10. They may include a gyroscope or other inertial sensor, a global-positioning system (GPS) receiver, and/or a barometric pressure sensor configured for altimetry. These sensors may provide data on the wearer's location or orientation. From the integrated responses of the sensors, controller 16 may track the movement of the HMD device within the wearer's environment.
In one embodiment, sensors 18 may include an eye-tracker—i.e., a sensor configured to detect an ocular state of the wearer of HMD device 10. The eye tracker may locate a line of sight of the wearer, measure an extent of iris closure, etc. If two eye trackers are included, one for each eye, then the two may be used together to determine the wearer's focal plane based on the point of convergence of the lines of sight of the wearer's left and right eyes. This information may be used by controller 16 for placement of a computer-generated display image, for example.
In the illustrated embodiment, each near-eye display system 14 is at least partly transparent, to provide a substantially unobstructed field of view in which the wearer can directly observe his physical surroundings. Each near-eye display system is configured to present, in the same field of view, a computer-generated display image. Controller 16 may control the internal componentry of near-eye display systems 14A and 14B in order to form the desired display images. In one embodiment, controller 16 may cause near-eye display systems 14A and 14B to display the same image concurrently, so that the wearer's right and left eyes receive the same image at the same time. In another embodiment, the near-eye display systems may project somewhat different images concurrently, so that the wearer perceives a stereoscopic, i.e., three-dimensional image. In one scenario, the computer-generated display image and various real images of objects sighted through a near-eye display system may occupy different focal planes. Accordingly, the wearer observing a real-world object may have to shift his or her corneal focus in order to resolve the display image. In other scenarios, the display image and at least one real image may share a common focal plane.
In the HMD devices disclosed herein, near-eye display system 14 may also be configured to acquire video of the surroundings sighted by the wearer. The video may include depth video. It may be used to establish the wearer's location, what the wearer sees, etc. The video acquired by the near-eye display system may be received in controller 16, and the controller may be configured to process the video received. To this end, near-eye display system 14 may include a camera. The optical axis of the camera may be aligned parallel to a line of sight of the wearer of the HMD device, such that the camera acquires video of the external imagery sighted by the wearer. As the HMD device may include two near-eye display systems—one for each eye—it may also include two cameras. More generally, the nature and number of the cameras may differ in the various embodiments of this disclosure. One or more cameras may be configured to provide video from which a time-resolved sequence of three-dimensional depth maps is obtained via downstream processing.
No aspect of
The HMD devices disclosed herein may be used to support a virtual-reality (VR) or augmented-reality (AR) environment for one or more participants. A realistic AR experience may be achieved with each AR participant viewing his environment naturally, through passive optics of the HMD device. Computer-generated imagery, meanwhile, may be projected into the same field of view in which the real-world imagery is received. Imagery from both sources may appear to share the same physical space.
The controller in the HMD device may be configured to run one or more computer programs that support the VR or AR environment. In some embodiments, some computer programs may run on an HMD device, and others may run on an external computer accessible to the HMD device via one or more wired or wireless communication links. Accordingly, the HMD device may include suitable wireless componentry, such as Wi-Fi.
In other embodiments, the image former may be a reflective liquid-crystal-on-silicon (LCOS) or digital micromirror display (DMD) device. In these embodiments, a transparent illuminator 24 may be arranged on the opposite side of the image former. In another embodiment, illuminator 24 may comprise one or more modulated lasers, and image former 26 may be a rastering optic. The image former may be configured to raster the emission of each laser in synchronicity with its modulation, to form the display image. In yet another embodiment, image former 26 may comprise a rectangular array of color LEDs (e.g., organic LEDs) arranged to form the display image. As each color LED array emits its own light, illuminator 24 may be omitted from this embodiment. The various active components of near-eye display system 14—e.g., image former 26 and illuminator 24, if included—may be operatively coupled to controller 16. The controller may provide suitable control signals that, when received by the image former, cause the desired display image to be formed.
In
Waveguide 32 may be substantially transparent to external imagery received normal to its front surface 34. Thus, the waveguide may be positioned in front of the eye of the HMD-device wearer without obstructing the wearer's view of the external imagery. In the embodiment shown in
Continuing in
To this end, waveguide 32 includes a series of transparent sections 40 (40A, 40B, etc.) arranged end-to-end, with beamsplitters 42 (42A, 42B, etc.) arranged between adjacent pairs of transparent sections. In some embodiments, each beamsplitter may be formed as a coating supported on its respective transparent section. In the embodiment shown in
Returning now to
It will be understood that the terms ‘horizontal’, ‘vertical’, ‘width’, and ‘height’ are used primarily to establish relative orientations in the illustrated embodiments, for ease of description. These terms may be intuitive for one usage scenario—e.g., when the wearer of the near-eye display device is upright and forward-facing—and less intuitive for other usage scenarios. Nevertheless, the listed terms should not be construed to limit the scope of the configurations and usage scenarios contemplated herein. For instance, a horizontal or vertical orientation may be aligned with any arbitrary axis of a user's eye and/or HMD device without departing from the scope of this disclosure.
Referring again to
Near-eye display configurations as described above offer the desirable properties of display-image shifting and exit-pupil expansion, but are limited in the degree to which the exit pupil can be expanded in the vertical direction—i.e., orthogonal to the direction of propagation through the waveguide. This is because the vertical expansion is done solely by collimating lens 44.
To form a relatively small exit pupil, or one that transmits the display image over a narrow field of view, a compact collimating lens may suffice. However, for larger exit pupils and larger fields of view, a proportionately larger and thicker lens is needed. For the HMD devices envisaged herein, it is desirable for the near-eye display system to present an exit pupil at least 12 mm high, that supports field of view of 36 to 48°. Ray-trace analysis reveals that a collimating lens 64 to 70 mm in height may be required to present such a pupil. However, a lens of these dimensions may be too large for use in an HMD device—particularly one designed to resemble ordinary eyewear. Accordingly, this disclosure describes an approach in which the exit pupil is expanded in both the horizontal and vertical directions using waveguide-embedded beamsplitters. In this approach, the collimating lens is used only to couple the display image into the waveguide structure, not to expand the exit pupil. This feature greatly reduces the required size of the collimating lens, which is a great advantage in the design of compact HMD devices.
System 14′ also includes a first series 54 of mutually parallel beamsplitters. The first series of beamsplitters are arranged to receive the display image from the image former, and to release the display image through an exit pupil 50A, which is longer than that of the image former. Accordingly, the exit pupil of the first series of beamsplitters is expanded in a vertical direction relative to the exit pupil of the image former. The mode of exit-pupil expansion may be substantially the same as described above, in the context of waveguide 32.
However, system 14′ also includes a second series 56 of mutually parallel beamsplitters. The second series of beamsplitters are arranged to receive the display image from the first series of beamsplitters, and to release the display image through an exit pupil 50B, which is both longer and wider than that of the image former. Thus, the exit pupil of the second series of beamsplitters is expanded in a horizontal direction relative to the exit pupil of the image former and to that of the first series of beamsplitters.
In general, the second series 56 of beamsplitters will have a different alignment and a different orientation than the first series 54. In some embodiments, the second series of beamsplitters will also have a different structure. In the embodiment of
The beamsplitters of the current embodiment may be at least somewhat similar to those described hereinabove. For instance, each beamsplitter may comprise a coating supported on a transparent section. The coating may exhibit an incidence-angle dependent reflectance which is lowest in a notch-shaped region between normal and grazing incidence, as shown in
One potential disadvantage of the RA waveguide is non-uniformity—e.g., banding—in display-image illumination. Another disadvantage is manufacturing complexity, which may make it somewhat less desirable for use in low-cost HMD devices. On the other hand, SRG and VH structures may exhibit a strong wavelength dependence, thereby requiring a plurality of gratings to span the visible wavelength range. One attractive VH variant is the switchable Bragg grating, which, being an active optical component, offers a work-around for the wavelength-dependence issue. In particular, three stacked gratings may be used, with one grating configured for red light, another for green light, and another for blue light. In this embodiment, the image former may be operated in color-sequential mode, synchronized to the activation of the switchable Bragg gratings.
In one embodiment, the transmittance of each successive beamsplitter in the first and/or second series of beamsplitters may decrease in a direction of propagation of display light through that series of beamsplitters. This feature may be used to compensate for the stepwise reduction in the intensity of the display light as it propagates through the beamsplitters, which otherwise could result in the display image being brighter on one side than the other—e.g., brighter on the top than the bottom, or brighter on the left than the right. Accordingly, if beamsplitters in the series are reflective, then the reflectance of each successive beamsplitter may increase in the direction of propagation of display light through the series. In one particular embodiment, a final beamsplitter in the series may be substantially fully reflective. This aspect may be advantageous primarily for the first series 54 of beamsplitters—the vertical series in the illustrated embodiments—which can be located outside of the wearer's field of view.
In contrast, the beamsplitters of second series 56 are arranged directly in front of the wearer's eye. These beamsplitters may be configured with a more uniform transmittance across the series, so that the external imagery received through the waveguide does not appear abnormally dark at one end the field of view. Without being tied to any particular theory, it is believed that the waveguide arrangements disclosed herein exhibit a periscope effect whereby some external-image light from the entry end of a waveguide (the left end in the drawings) propagates through the waveguide and is discharged toward the opposite end. This effect partially compensates for the reduction of external image brightness that could result from a decrease in transmittance across the series of beamsplitters in the waveguide, so that a given reduction in transmittance causes less brightness reduction than would otherwise be expected.
In near-eye display system 14′, the first series 54 of beamsplitters are arranged in first waveguide 32A. The second series 56 of beamsplitters are arranged in a second waveguide 32B, which is materially separate from the first waveguide. The configuration of the beamsplitters within their respective waveguides may be substantially as described above, for waveguide 32. As shown in
In near-eye display system 14″, the second series 56 of beamsplitters may be aligned and oriented substantially as described above. The first series 54, however, is aligned differently. In particular, a longitudinal edge of each beamsplitter of the first series is oblique to the axis of alignment of the beamsplitters, so that each beamsplitter is set on a diagonal with respect to the direction of propagation of display light. This feature causes the display image propagating down the first series to be turned 90° into the second series of beamsplitters. In this manner, a single waveguide provides both horizontal and vertical exit pupil expansion. In one embodiment, the beamsplitters of the first series may have a partially diffractive structure, which may be less costly to fabricate than equivalently oriented partially reflective arrays. In some embodiments, different beamsplitter technologies may be incorporated in a single waveguide. For instance, the SRG-RA structure may be fabricated on the same waveguide substrate—or on different substrates, as shown in
Naturally, the particular waveguide structures in the foregoing drawings should not be understood in a limiting sense, for numerous other structures are contemplated as well. For instance,
In this embodiment, display light couples into waveguide 32′ as described hereinabove for waveguide 32. There, it propagates by TIR from the front and back surfaces of the waveguide. On each reflection from front surface 34 to back surface 38, the display light interacts with VH structure 58, resulting in a portion of that light being reflected out of the waveguide. Accordingly, a single VH structure may provide multiple beamsplitting interactions for each ray of display light propagating through the waveguide. In this manner, the VH structure may embody an entire series of discrete beamsplitter structures such as described in the foregoing embodiments. As in the foregoing embodiments, the multiple beamsplitting interactions provide exit-pupil expansion in one dimension.
It will be noted that the overall structure in
Finally, it will be understood that the articles, systems, and methods described hereinabove are embodiments of this disclosure—non-limiting examples for which numerous variations and extensions are contemplated as well. Accordingly, this disclosure includes all novel and non-obvious combinations and sub-combinations of the articles, systems, and methods disclosed herein, as well as any and all equivalents thereof.
Claims
1. A near-eye display system comprising:
- an image former configured to form a display image and to release the display image through an exit pupil;
- a first series of mutually parallel beamsplitters arranged to receive the display image from the image former; and
- a second series of mutually parallel beamsplitters arranged to receive the display image from the first series of beamsplitters, and to release the display image through an exit pupil longer and wider than that of the image former, the second series of beamsplitters having a different structure, a different alignment, and a different orientation than the first series of beamsplitters.
2. The system of claim 1 wherein each beamsplitter of the first series comprises a partially reflective array.
3. The system of claim 1 wherein each beamsplitter of the first series comprises a surface-relief diffraction grating.
4. The system of claim 1 wherein each beamsplitter of the first series comprises a hologram.
5. The system of claim 1 wherein each beamsplitter of the first series comprises a switchable Bragg grating.
6. The system of claim 1 wherein each beamsplitter of the second series comprises a partially reflective array, a surface-relief diffraction grating, a hologram, or a switchable Bragg grating.
7. The system of claim 1 wherein the beamsplitters of the first series are aligned along a first axis, and wherein the beamsplitters of the second series are aligned along a second axis orthogonal to the first axis.
8. The system of claim 1 wherein each beamsplitter comprises a coating supported on a transparent substrate, and wherein the coating exhibits an incidence-angle dependent reflectance which is lowest in a notch-shaped region between normal and grazing incidence.
9. The system of claim 1 wherein the image former is a liquid-crystal display array with an exit pupil five millimeters or less in length and five millimeters or less in width.
10. A near-eye display system comprising:
- an image former configured to form a display image and to release the display image through an exit pupil;
- a lens configured to receive the display image from the image former and to collimate the display image;
- a first series of mutually parallel beamsplitters arranged to receive the display image from the lens; and
- a second series of partially reflective, mutually parallel beamsplitters arranged to receive the display image from the first series of beamsplitters, and to release the display image through an exit pupil longer and wider than that of the image former, the second series of beamsplitters having a different alignment and a different orientation than the first series of beamsplitters.
11. The system of claim 10 wherein the beamsplitters of the first series are aligned along a first axis, and wherein the beamsplitters of the second series are aligned along a second axis orthogonal to the first axis.
12. The system of claim 11 wherein the first axis is a vertical axis and the second axis is a horizontal axis, wherein the exit pupil of the first series of beamsplitters is expanded in a vertical direction relative to the exit pupil of the image former, and wherein the exit pupil of the second series of beamsplitters is expanded in a horizontal direction relative to the exit pupil of the image former and that of the first series of beamsplitters.
13. The system of claim 10 wherein the first series of beamsplitters are arranged in a first waveguide and configured to release the display image through an exit pupil longer than that of the image former, and wherein the second series of beamsplitters are arranged in a second waveguide materially separate from the first waveguide.
14. The system of claim 13 wherein the first and second waveguides are separated by an air gap or by a material of lower refractive index than that of the first and second waveguides.
15. The system of claim 13 wherein for each series of beamsplitters, a longitudinal edge of each beamsplitter is perpendicular to the axis of alignment of the series.
16. The system of claim 10 wherein the first and second series of beamsplitters are arranged in the same waveguide.
17. The system of claim 16 wherein a longitudinal edge of each beamsplitter of the first series is oblique to the axis of alignment of the beamsplitter, and wherein a longitudinal edge of each beamsplitter of the second series is perpendicular to the axis of alignment of the beamsplitter.
18. The system of claim 10 wherein a transmittance of each successive beamsplitter in the first and/or second series of beamsplitters decreases in a direction of propagation of display light through that series of beamsplitters.
19. A near-eye display system comprising:
- an image former configured to form a display image and to release the display image through an exit pupil;
- a first series of partially diffractive, mutually parallel beamsplitters aligned along a first axis and arranged to receive the display image from the image former; and
- a second series of partially reflective, mutually parallel beamsplitters aligned along a second axis, orthogonal to the first axis, and arranged to receive the display image from the first series of beamsplitters and to release the display image through an exit pupil longer and wider than that of the image former, the second series of beamsplitters having a different alignment and a different orientation than the first series of beamsplitters and being at least partially transparent to imagery external to the near-eye display system;
- a controller configured to provide control signals to the image former to cause the display image to be formed; and
- a wearable mount configured to support the image former and the controller, and to hold the first and second series of beamsplitters directly in front of a wearer's eye.
20. The system of claim 19 wherein a reflectance of each successive beamsplitter in the first series of beamsplitters increases in a direction of propagation of display light through the first series of beamsplitters, and wherein a final beamsplitter in the series is substantially fully reflective.
5610936 | March 11, 1997 | Cantoni |
6049641 | April 11, 2000 | Deacon et al. |
6162655 | December 19, 2000 | Johnson et al. |
6185045 | February 6, 2001 | Hanano |
6289153 | September 11, 2001 | Bergmann |
7262918 | August 28, 2007 | Yoo et al. |
7269317 | September 11, 2007 | Blauvelt et al. |
7778305 | August 17, 2010 | Parriaux et al. |
20100103078 | April 29, 2010 | Mukawa et al. |
- Cameron, Alex., “The Application of Holographic Optical Waveguide Technology to Q-Sight™ Family of Helmet Mounted Displays”, Retrieved at <<http://144.206.159.178/ft/CONF/16431969/16431986.pdf>>, Proceedings of the SPIE, vol. 7326, 2009, pp. 11.
- Mukawa, et al., “A full-color eyewear display using planar waveguides with reflection volume holograms”, Journal of the SID, 2009, pp. 185-193.
- Eisenberg, Hagai “Nonlinear Effects in Waveguide Arrays”, Retrieved at <<http://www.phys.huji.ac.il/˜hagaie/files/HagaiElectronicThesis.pdf>> Nov. 2002, pp. 75.
Type: Grant
Filed: Mar 21, 2012
Date of Patent: May 27, 2014
Patent Publication Number: 20130250431
Assignee: Microsoft Corporation (Redmond, WA)
Inventors: Steve Robbins (Bellevue, WA), David D. Bohn (Fort Collins, CO)
Primary Examiner: Scott J Sugarman
Assistant Examiner: Daniele Manikeu
Application Number: 13/426,385
International Classification: G02B 27/14 (20060101); G02B 27/01 (20060101);