# OPTICAL SEE-THROUGH FREE-FORM HEAD-MOUNTED DISPLAY

A see-through free-form head-mounted display including a wedge-shaped prism-lens having free-form surfaces and low F-number is provided.

## Description

#### RELATED APPLICATIONS

This application claims the benefit of priority of U.S. Provisional Application No. 61/214,117, filed on Apr. 20, 2009, the entire contents of which application are incorporated herein by reference.

#### GOVERNMENT RIGHTS

This invention was made with government supports under contract numbers 0644446 awarded by the U.S. National Science Foundation, 60827003 awarded by the National Natural Science Foundation of China, and 2009AA01Z308 awarded by the Hi-Tech Research and Development Program of China. The U.S. and Chinese governments have certain rights in the invention.

#### FIELD OF THE INVENTION

The present invention relates generally to a see-through free-form head-mounted display, and more particularly, but not exclusively to a wedge-shaped prism-lens having free-form surfaces configured to provide a low F-number heretofore unachieved.

#### BACKGROUND

Optical see-through head-mounted displays (OST-HMD) find myriads of applications from scientific visualization to defense applications, from medical visualization to engineering processes, and from training to entertainment. In mixed or augmented reality systems, OST-HMDs have been one of the basic vehicles for combining computer-generated virtual scene with the views of a real-world scene. Typically through an optical combiner, an OST-HMD maintains a direct view of the physical world and optically superimposes computer-generated images onto the real scene. Compared with a video see-though approach where the real-world views are captured through cameras, it has the advantage of introducing minimal degradation to the real world scene. Therefore an OST-HMD is preferred for applications where a non-blocked real-world view is critical.

On the other hand, designing a wide field of view (FOV), low F-number, compact, and nonintrusive OST-HMD has been a great challenge, especially difficult for a non-pupil forming system. The typical eyepiece structure using rotationally symmetric components has limitations in achieving low F-number, large eye relief, and wide FOV. Many methods have been explored to achieve an HMD optical system which fulfills the above mentioned requirements. These methods include applying catadioptric techniques, introducing new elements such as aspherical surfaces, holographic and diffractive optical components, exploring new design principles such as using projection optics to replace an eyepiece or microscope type lens system in a conventional HMD design, and introducing tilt and decenter or even free-form surfaces. (H. Hoshi, et .al, “Off-axial HMD optical system consisting of aspherical surfaces without rotational symmetry,” SPIE Vol. 2653, 234 (1996). S. Yamazaki, et al., “Thin wide-field-of-view HMD with free-form-surface prism and applications,” Proc. SPIE, Vol. 3639, 453 (1999).)

Among the different methods mentioned above, free-form surfaces demonstrate great promise in designing compact HMD systems. In particular, a wedge-shaped free-form prism, introduced by Morishima et al. (Morishima et al., “The design of off-axial optical system consisting of aspherical mirrors without rotational symmetry,” 20th Optical Symposium, Extended Abstracts, 21, pp. 53-56 (1995)), takes the advantage of total internal reflection (TIR), which helps minimize light loss and improve the brightness and contrast of the displayed images when compared with designs using half mirrors. It is challenging, however, to design a free-form prism based OST-HMD offering a wide FOV, low F-number, and sufficient eye relief.

The concept of free-form HMD designs with a wedge-shaped prism was first presented by Morishima et al. in 1995, and the fabrication and evaluation method were explored by Inoguchi et al. (“Fabrication and evaluation of HMD optical system consisting of aspherical mirrors without rotation symmetry,” *Japan Optics '*95, *Extended Abstracts, *20*pB*06, pp. 19-20, 1995). Following these pioneering efforts, many attempts have been made to design HMDs using free-form surfaces, particularly designs based on a wedge-shaped prism (U.S. Pat. Nos. 5,699,194, 5,701,202, 5,706,136. D. Cheng, et al., “Design of a lightweight and wide field-of-view HMD system with free form surface prism,” Infrared and Laser Engineering, Vol. 36, 3 (2007).). For instance, Hoshi et al. presented an FFS prism offering an FOV of 34° and a thickness of 15 mm; Yamazaki et al. described a 51° OST-HMD design consisting of a FFS prism and an auxiliary lens attached to the FFS prism; and more recently Cakmakci et al. designed a 20° HMD system with one free-form reflecting surface which was based on rational radial basis function and a diffractive lens. (“Optimal local shape description for rotationally non-symmetric optical surface design and analysis,” Opt. Express 16, 1583-1589 (2008)). There are also several commercially available HMD products based on the FFS prism concept. For instance, Olympus released their Eye-Trek series of HMDs based on free-form prisms. Emagin carried Z800 with the optical module WFO5, Daeyang carried i-Visor FX series (GEOMC module, A3 prism) products; Rockwell Collins announced the ProView SL40 using the prism technology of OEM display optics.

Existing FFS-based designs have an exit pupil diameter that is typically from 4 to 8 mm with a FOV typically around 40 degrees or less. In most of the existing designs, the size of the microdisplays is in the range of 1 to 1.3 inches, which affords a focal length of 35˜45 mm for a typical 40-degree FOV. Even with an exit pupil up to 8 mm, the F/# remains fairly high (greater than 4) and eases the optical design challenge. A large size microdisplay, however, offsets the advantage of compactness using a free-form prism. In the more recent designs, smaller microdisplays, typically around 0.6″, were adopted, which requires a focal length of ˜21 mm to achieve a 40-degree FOV. The reduced focal length makes it very challenging to design a system with a large exit pupil. As a result, most of the designs compromise the exit pupil diameter. Thus, commercially available products on average reduce the pupil diameter to about 3˜5 mm to maintain an F/# greater than 4. There are a few designs that achieve a larger pupil by introducing additional free-form elements or diffractive optical elements. For instance, Droessler and Fritz described the design of a high brightness see-through head-mounted system with an F/# as low as 1.7 by using two extra decentered lenses and applying one diffractive surface. (U.S. Pat. No. 6,147,807). The existing work shows that it is extremely difficult to achieve a very fast (low F/#) and wide field of view HMD design with a single wedge-shaped free-form surface prism.

Accordingly, it would be an advance in the field of optical see-through head-mounted displays to provide a head-mounted display which has a wide field of view and low F/#, while also providing a compact, light-weight, and nonintrusive form factor.

#### SUMMARY OF THE DISCLOSURE

In one of its aspects, the present invention provides a free-form prism-lens for use in an optical see-through head-mounted display. The prism-lens may include a first free-form surface configured to receive light from a micro-display and configured to transmit the received light into the body of the prism-lens, and a second free-form surface configured to receive the light transmitted into the body of the prism-lens from the first free-form surface and configured to totally internally reflect the received light at the second surface. In addition, prism-lens may also include a third free-form surface configured to receive the light reflected by the second free-form surface and configured to reflect the light out of the prism-lens and may have an f-number less than 3.5. The prism-lens may optionally include an auxiliary lens disposed proximate the third free-form surface. The auxiliary lens may be configured to minimize the shift and distortion of rays from a real-world scene by the second and third surfaces of the prism-lens.

#### BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing summary and the following detailed description of the preferred embodiments of the present invention will be best understood when read in conjunction with the appended drawings, in which:

**1** and **1**′ depending on their field and pupil positions and controlled to satisfy TIR conditions and avoid stray light;

**1**′ as the ray pupil position varies from the bottom to the top, **1**′ as the field of the ray changes from the lowermost to the uppermost in the meridian plane, **1** as the ray pupil position varies from the bottom to the top, and **1** as the field of the ray changes from the lowermost to the uppermost position in the tangential plane;

#### DETAILED DESCRIPTION

The desire to achieve an optical see-through head-mounted display having a compact, light-weight, and nonintrusive form factor argues for a design having as few optical elements as possible. Accordingly, exemplary designs of the present invention provide a single-element prism-lens **110**, **710** which has sufficient optical power on its own to deliver light from a micro-display **130** to a user, **6**. However, providing a single optical element, such as the prism-lens **710**, in which all the optical power resides can lead to greatly increased aberrations with accompanying loss in resolution and image quality, especially for low F/# systems. Despite these challenges, as a result of the lens design procedures and work described below, the present invention provides a single-element prism-lens **710** based on a 0.61″ microdisplay 130, which offers a diagonal FOV of 53.5°, an F/# of 1.875, an exit pupil diameter of 8 mm, and an eye relief of 18.25 mm. In addition, in order to maintain a non-distorted see-through view of a real-world scene, a cemented auxiliary lens **120**, **720** may be provided for use in conjunction with the prism-lens **110**, **710**.

#### Display System Specifications

Turning first to the design of the wedge-shaped free-form prism-lens **110**, design began with development of the display system specifications. An optical see-through HMD **100** typically consists of an optical path for viewing a displayed virtual image and a path for viewing a real-world scene directly. As shown in **100** of our OST-HMD design may include a wedge-shaped free-form prism-lens **110** cemented to an auxiliary free-form lens **120**. The prism-lens **110** serves as the near-eye viewing optics that magnifies the image displayed through a microdisplay **130** while the auxiliary free-form lens **120** is an auxiliary element attached to the prism-lens **110** in order to maintain a non-distorted see-through view of a real-world scene.

As shown in **110** may include three surfaces labeled as **1**, **2**, and **3**, respectively. For the sake of convenience, the surface adjacent to the exit pupil is labeled as **1** in the refraction path and as **1**′ in the reflection path. We set the center of the exit pupil as the origin of the global coordinate system and the rest of the surfaces were specified with respect to this global reference. We further adopted the convention of tracing the system backward, namely from the eye position to the microdisplay **130**.

The overall system was set to be symmetric about the YOZ plane, but not the XOZ plane. A ray emitted from a point on the microdisplay **130** is first refracted by the surface **3** next to the microdisplay **130**. After two consecutive reflections by the surfaces **1**′ and **2**, the ray is transmitted through the surface **1** and reaches the exit pupil of the system **100**. The first surface (i.e., **1** and **1**′) of the prism-lens **110** is required to satisfy the condition of total internal reflection for rays reflected by this surface **1**′. The rear surface **2** of the prism-lens **110** is coated as a half mirror in order to facilitate the optical see-through capability. The rays from the microdisplay **130** will be reflected by the rear surface **2** while the rays from a real-world scene will be transmitted. An auxiliary lens **120** may be cemented to the wedge-shaped prism-lens **110** in order to counteract the ray shift and distortion caused by the prism-lens **110**. The front surface of the auxiliary free-form lens **120** may match the shape of the rear surface **2** of the prism-lens **110**. The back surface **4** of the auxiliary free-form lens **120** may be optimized to minimize the shift and distortion introduced to the rays from a real-world scene when the auxiliary free-form lens **120** is combined with the prism-lens **110**.

The overall specifications of the system are summarized in Table 1. Our goal was to achieve a very compact, lightweight, and wide FOV design using a wedge-shaped free-form prism-lens **110**. A small size microdisplay **130** with high resolution was thus preferred. Based on the size, resolution, availability and cost, a pair of 0.61-inch Emagin OLED displays were selected, with a resolution of 800×600 pixels and a 15 μm pixel size. We further targeted an HMD system **100** with a diagonal full FOV of at least 50°, which corresponds to a focal length no more than 16.6 mm. A 15 mm focal length was selected, which offers a reasonable balance between FOV (53.5° diagonally) and angular resolution (3.2 arc minutes per pixel). In the design of visual instruments, especially binocular HMDs, a large exit pupil is typically preferred to account for the swiveling of the eyes in their sockets without causing vignetting or loss of image. A large pupil offers better tolerance of the interpupilary distances (IPD) among different users without the need to mechanically adjust the IPD of the binocular optics. A large pupil, however, often not only compromises the compactness and weight of the optical system **100**, but also imposes limitations on the FOV due to the dramatically increased challenge of designing low F/# systems. Taking into account these factors, we set the exit pupil diameter to be 8 mm, which leads to a system **100** with a F/# of 1.875. In designing HMD systems, a large eye relief is desired to accommodate users wearing eyeglasses, but it affects the compactness of the viewing optics. A minimum of a 18 mm eye relief was set to accommodate users wearing low-profile eyeglasses. Balancing between image uniformity and system compactness, we set the limit of the vignetting to be less than 15% at the top and bottom of the visual fields.

Among the aberrations of an optical system, distortion causes the warping of the displayed image without reducing image sharpness, which allows computational or electronic correction. In designing conventional HMDs it is common to optimize the system **100** to minimize the optical aberrations that reduce image quality and cannot be compensated electronically or computationally. In a free-form optical system **100**, however, the distortion can be very large and irregular if it is left without any constraints. We thus set a distortion limit of 12% at the maximum field angle and planned to correct the residual distortion using computational methods. In terms of other types of aberrations, the modulation transfer function (MTF) was selected to evaluate the overall image sharpness and was set to be no less than 10% across the entire visual field at a spatial frequency of 30 lps/mm. With the specifications established, development continued with design of the free-form elements **110**, **120**.

#### Design of Free-Form Elements

Free-form optical surfaces offer more degrees of freedom to optical designers than conventional rotationally symmetric optical surfaces, such as a spherical or aspherical surface, and achieve usually lower wavefront errors and distortion than that achievable with the same number of rotationally symmetric surfaces. A significant benefit in our OST-HMD design lies in its ability to yield display optics with an eyeglass-like form factor. An optical design using free-form surfaces, however, may cause a dramatic increase in the complexity of the design and optimization process. An inadequate method of representing and optimizing a free-form surface may lead to discouraging and unpredictable results. Key issues in the process of designing a FFS HMD include 1) a free-form surface representation and design strategy; 2) total internal reflection condition; and 3) structure constraints to form a valid prism-lens **110**.

Free-Form Surface Representation and Design Strategy

Selecting a suitable method for a free-form surface representation is very important. Different representation methods not only have different impacts on the ray tracing speed and the convergence of optimization, but also offer different degrees of design freedom. A suitable representation method shall 1) provide adequate degrees of freedom; 2) require a reasonable amount of ray tracing time; and 3) offer reliable convergence in the optimization process. Ray tracing speed is a particular concern in designing a free-form prism-lens **110**, as a larger number of fields need to be sampled when optimizing a free-form optical system than need to be sampled in a rotationally symmetrical optical system. Speed becomes a more serious problem when a global optimization is necessary. Although most of the commercially available optical design software, such as CODE V® (Optical Research Associates, Pasadena, Calif.), offers the ability to model free-form surfaces in user-defined methods, the ray tracing speed of user-defined representations typically is much slower than the standard methods available in the software packages.

By taking into account the speed and convergence factors, the following design strategy was adopted in our design process. In the case when we lacked a starting point for an FFS surface, we started to optimize the surface with a spherical type to obtain the correct first-order parameters. The spherical surface was then converted to an aspheric type by adding a conic constant and a 4^{th }order or higher aspheric coefficients. Following an intermediate state of optimization, the ASP-type surface was then converted to an AAS-type surface for better correction by directly adding asymmetric coefficients up to the 10^{th }order. To avoid loss of information, use of aspheric terms higher than the 10^{th }order was not pursued, because the AAS surface has only up to the 10^{th }order of rotationally symmetric coefficients in CODE V®. Optimization with the AAS type surface helped to create a good starting point. The AAS surface was then converted to the XYP-type through a fitting algorithm (e.g., a least square fitting method) for final stage of optimization. High precision was required for the fitting algorithm to avoid a significant deviation from the starting design produced by the AAS surface type.

Total Internal Reflection Constraint

As mentioned above, all the rays striking the first surface **1**′ of the prism-lens **110** from inside should be totally reflected off. The first surface **1**′ cannot be coated with a reflective film, because it is shared by both a refractive and reflective path of the same rays. Therefore, the incident angles of all the rays striking the first surface **1**′ from the microdisplay **130** should be larger than the critical angle, θ_{c}, set by the TIR condition

θ_{c}=arcsin(1/*n*) (1)

where n is the refractive index of the material for the FFS prism-lens **110**. For example, if the index of the material is equal to 1.5, all the incident angles should be larger than 41.82°. Rays incident on the first surface **1**′ of the prism-lens **110** at a smaller angle may be transmitted through the prism-lens **110** without the benefit of reflection off the rear surface **2** (and subsequent refraction at the first surface **1**) and may directly enter the eye, which leads to stray light and a reduction in the image contrast observed by the user. If the TIR condition is met, however, after two consecutive reflections by the front and rear surfaces **1**′ and **2**, respectively, the same ray is returned back and to be transmitted through the front surface **1**. To ensure transmission of the ray after the two consecutive reflections, the incident angle of the ray should be smaller than the critical angle set by Eqn. (1) to avoid the TIR effect.

It was impractical to constrain the incident angle of every ray incident on the surface of interest during the optimization process. An adequate and practical control method was required. Without loss of generality, we made two assumptions: (1) the local departure of the surface **1**′ from a spherical surface was sufficiently small compared to the primary radius of curvature of the surface so that the surface normal of every point on surface **1**′ could be adequately approximated by a line passing through to the center of the primary curvature of the surface (as shown in **1** is concave, as shown in _{1u}, which corresponds to the ray from the maximum object field in the positive Y-direction (i.e. P_{1}) passing through the top edge of the pupil, had the smallest incident angle among all the rays striking the surface **1**′ from the microdisplay **130** side. As shown in **1**′ increased gradually as the ray from the same object field shifted from the top to the bottom of the pupil (e.g. from R_{1u }to R_{1b}); the angle also increased as the ray intersecting the same pupil position shifted from the top to the bottom of the object fields (e.g., from R_{1u }to R_{2u}). Therefore, the constraint on the incident angle was written as

θ_{1b1′}>arcsin(1/*n*) (2)

where θ_{1b1′} is the incident angle of the top marginal ray, R_{1u}, on surface **1**′ from the maximum object field in tangential plane of the microdisplay **130**.

We could further prove that after the two consecutive reflections the top marginal ray, R_{2u}, of the maximum object field in the negative Y-direction (i.e. P_{2}) had the largest incident angle on the surface **1** when the surface **1** was tilted counterclockwise about the X-axis (i.e., the tilt angle, θ_{1}>0); otherwise the bottom marginal ray R_{1b}, of the maximum object field in the positive Y-direction (P_{1}) has the largest incident angle when the surface **1** was tilted clockwise. Therefore, the constraint used to avoid TIR condition on surface **1** was written as:

where θ_{1b1 }is the incident angle of the bottom marginal ray, R_{1b}, striking the surface **1**; and θ_{2u1 }is the incident angle of the top marginal ray, R_{2u}, on surface **1**, and θ_{1 }is the tilt angle of surface **1** about the X-axis.

The simplified constraints in Eqns. (2) and (3) were important in making the optimization practical in designing the FFS prism-lens **110**. Increasing the refractive index of the material could help to relax the ray angle constraints and ease the design task. However, high refractive index materials can increase the color aberrations (due to lower Abbe number) and fabrication cost. Furthermore, our goal in this design was to achieve light weight by using plastic materials, which usually have a moderately low range of refractive indices.

Structure Constraints

Designing the wedge-shaped free-form prism-lens **110** required optimizing the shapes of individual surfaces to minimize wavefront errors under the ray angle constraints set by Eqns. 2 and 3. It further required additional structure constraints in order to ensure that the three surfaces together formed a valid prism-lens shape, that all the rays across the fields could be traced without obstruction or early escaping from a surface, and that the prism-lens **110** maintained desirable center and edge thickness. _{1u}, of the maximum field in the positive Y-direction and the bottom marginal ray, R_{2b}, of the maximum field in the negative Y-direction. As shown in _{a}, P_{a}′, P_{a}″ and P_{b}′ denote the intersection points of the ray R_{2b }with surfaces **1**, **2**, **1**′ and **3**, respectively; and P_{b}, P_{c }and P_{c′} label the intersection points of the ray R_{1u }with surfaces **2**, **1**′ and **3**, respectively. The coordinates of these ray-surface intersections were then used to define the constraints for optimizing the FFS HMD prism-lens **110**. Based on the requirements of the physical structure, the constraints were defined as

where all the Y, Z coordinates in the equations are referenced to the global coordinate system with the origin located at the center of the exit pupil.

Here by constraining the Y coordinates of the points P_{a}, P_{a}′, and P_{a}″, Eqn. (4) ensured that the surfaces **1** and **2** intersected properly so that the bottom marginal ray could be traced through the prism-lens **110** without obstruction. Equation 4 further set the upper and lower limits (e.g. 2 and 0.5 mm, respectively) on the edge thickness of the prism-lens **110** by constraining the Z coordinates of the points P_{a }and P_{a}′. By constraining the Y and Z coordinates of the points P_{b }and P_{b}′, Eqn. (5) avoided the escape of the top marginal ray after reflection by the surface **1** and helped to control the thickness of the prism-lens **110**. By controlling the Y and Z coordinates of the points Pc and Pc′, Eqn. (6) ensured that the surfaces **1** and **3** intersected properly so that the top marginal ray could be traced through the prism-lens **110** without obstruction or escaping from the prism-lens **110**. It further helped control the height of the prism-lens **110**. Eqns. 4 through 6 together ensured the three surfaces formed a valid prism-lens shape. These relationships further set limits on the tilt angles of the surfaces **1** and **2**, which helped to limit the off axis aberrations. By limiting the Z coordinates of the points P_{a }and P_{c}, Eqn. (7) set the minimal value for the eye clearance distance.

#### Optimization of the Free-Form Prism

We selected a patented design by Takahashi (U.S. Pat. No. 5,959,780) as a starting point. The original prism design of Takahashi included two free-form surfaces **501**, **502** and one planar surface **503**. Based on a 1.3 inch microdisplay **530**, the Takahashi design offered a full FOV of the system **500** of 57.8°×34.6°, with an exit pupil diameter of 4 mm and effective focal length of about 27.4 mm. The F/# of the system **500** was only 6.85. To meet our specifications, we scaled the effective focal length to 15 mm, reduced the horizontal FOV to 45°, and increased the exit pupil diameter from 4 mm to 8 mm, yielding a system **500** with an F/# of 1.875. In the scaled system **500**, the eye relief was reduced to 15.5 mm. The significantly reduced F/# imposed a critical challenge on system performance and invalidated several critical conditions of the prism-lens structure.

For instance, the incident angles of the rays on the TIR surface **501** were far smaller than the critical angle and a part of the rays from the top and bottom fields escaped from the prism **510** before completing their paths. We thus had to set considerably large vignetting for the top and bottom fields to obtain a valid starting design. **500**, respectively. The MTF of the starting design was evaluated at an exit pupil diameter of 8 mm with vignetting, was no higher than 0.1 at a spatial frequency of 10 lps/mm across the entire visual field. The rayfan plots were evaluated at a 3 mm pupil, shown in

The system of **530** in CODE V®. During the optimization process, four representative wavelengths, 486.1, 546.1, 587.6, and 656.3 nm, were set with the weights of 1, 1, 2, and 1, respectively. The TIR constraints and structural constraints as well as the basic optical definitions, such as the effective focal length, were always applied. The effective focal lengths in both tangential and sagittal planes were constrained to be 15 mm. We further set the following parameters as variables: all the primary curvatures of all surfaces **501**, **502**, **503** in both tangential and sagittal planes, aspherical coefficients, decenter in both Y and Z directions, and tilt about the X axis. Although we did not directly set a constraint on distortion, we limited the height of the rays striking the image plane to avoid large and irregular distortion.

Due to its single-plane symmetry, the free-form prism-lens design had to be optimized over half of the full FOV sampled in a rectangular grid, as opposed to a linear sample in the radial direction in a rotationally symmetric system. It was difficult, however, to start the optimization across the entire FOV in a densely-sampled grid given the low performance of the starting point. Instead, we adopted a progressive optimization strategy by gradually increasing field samples as the system performance improved during the optimization process. The weighting factors of the sampled fields were inversely proportional to their distance from the center of the field. The decenter and tilt parameters were set as variables during the entire optimization process. *a*), we sampled five fields along the vertical direction with the sagittal field angle being zero. It was important to optimize the system to meet the physical requirements such as eye clearance and TIR condition in this stage. During the optimization, we set the curvatures of surfaces **601** and **602** and the aspherical coefficients on rear surface **602** of the prism-lens **610** as variables. We also added curvature to surface **603** (which was a flat surface **503** in the starting Takahashi system **500**) as a variable, and this surface **603** was later turned into a free-form surface to help limit the ray heights of the marginal fields with respect to the center field and improve the overall optical performance. The surface layout of the optimized system **600** on the XZ plane is shown in *a*). After the first stage of optimization, we expanded the field samples by adding a fraction of the field angles along the sagittal direction. This stage of optimization was done by converting the surface type to AAS-type from ASP-type, and then the curvatures of the three surfaces in XZ plane were set as variables. The layout of the optimized system on the XZ plane is shown in *b*). We continued to expand the field horizontally until the maximum field met our specification, and optimized the system repetitively by gradually adding the asymmetric coefficients as variables. A good starting point was finally achieved after the re-optimization. *c*) shows the layout of the system on the XZ plane. It is worth pointing out that this optimization strategy can reduce the dependence on the performance of the initial starting point.

Following the design strategy above, we optimized the free-form surfaces using aspherical-type representations during the above steps for obtaining a good starting point. We then furthered the optimization by converting the ASP-type surfaces to AAS-type of surfaces and adding asymmetric coefficients up to the 10^{th }order as variables. To further optimize the system **600**, we converted the AAS-type surfaces to XYP representations through a least-square fitting algorithm and carried out a global optimization. We found that this step of optimization was very effective in optimizing the FFS prism-lens system **700**. The layout of the final FFS prism-lens design **700** is shown in **712** to the top **714** of the left edge of the prism-lens **710** was 22 mm, the width along X-direction was 25 mm, the thickness along Z-axis was 12 mm, and the weight was 5 grams. The optical material of the prism-lens **710** was PMMA having a refractive index of 1.492 and Abbe number of 57.2. The locations, and effective areas, of the surfaces **701**, **702**, **703** relative to the global coordinate system having its origin at the exit pupil are given in Tables 2 and 3, respectively, below. The SPS XYP surfaces **701**, **702**, **703** were 10^{th}-order polynomial surfaces added to a base conic. The polynomial was expanded into monomials of x^{m}y^{n}, where m+n≦10. The equation used was:

where z is the sag of the surface along the local z-axis, x and y are the coordinates in the local coordinate system, c is the vertex curvature (CUY), k is the conic constant, and C, is the coefficient for x^{m}y^{n}. **701**, **702**, **703** to show the local x-, y-, and z-axes and tilt, θ (the x-axis is perpendicular to the plane of the figure). The FFS polynomial coefficients are provided in Table 8 at the end of the Detailed Description.

_{1}(x

_{1}, y

_{1}, z

_{1})

_{1}

_{2 }(x

_{2}, y

_{2}, z

_{2})

_{2}

_{3 }(x

_{3}, y

_{3}, z

_{3})

_{3}

_{Im}

_{Im}, y

_{Im}, z

_{Im})

_{Im}

The optical performance of the optimized system **700** was assessed at the following representative field angles for the four design wavelengths: (0°, 0°), (0°, ±8°), (7°, 0°), (14°, 0°), (0°±16°), (22.5°, 0°), (22.5°, ±16°). **702**, which is very difficult to correct, as well as a small amount of barrel distortion. By fitting with the distortion grid, we chose to pre-warp the image displayed on the microdisplay **730** to balance the distortion of the virtual image. The polychromatic MTF plots shown in **730**. The MTF was 0.7 for the central (0°, 0°) field, 0.2 for the (0°, ±16°) field, and above 0.1 for the (22.5°, ±16°) field. The rayfan plots of the system **100** were evaluated at a 3 mm pupil, shown in

To demonstrate the effectiveness of the TIR constraints, **701**′ as a function of the pupil position and field position of the rays, respectively. As the pupil position of the rays is shifted from bottom to top, the incident angle on surface **701**′ decreased from 59.13° to 42.98° for the top field, from 61.66° to 42.2° for the center field and from 65.02° to 46.70° for the bottom field. Given the refractive index of the material in our final design was 1.492, all these angles were well controlled to satisfy the Eqn. (2). **701** as a function of the pupil position and field position of the rays, respectively. As the pupil position of the rays was shifted from bottom to top, the incident angle on surface **701** decreased from 7.8° to 7.0° for the top field, from 3.15° to 2.84° for the center field and from 13.7° to 13.6° for the bottom field. All these angles were far smaller than the critical angle, so they were well controlled to satisfy the Eqn. (3).

#### Design of the Auxiliary Free-Form Lens

The free-form prism-lens **710** with curved surfaces produced optical power in the optical see-through path, causing a significant viewing axis deviation and undesirable distortion as well as other off-axis aberrations to the view of the real world scene. **710**. An auxiliary lens **720** was desired, to not only cancel the optical power in the see-through path, but also to correct the deviation of the optical axis and the off-axis aberrations introduced by the FFS prism-lens **710**.

We chose to trace rays from the real-world scene to the eye space, as shown in **710** (**710**. The reflective mode of the concave mirror surface **702** was changed to refractive mode. We then inserted a plastic auxiliary lens **720** to the left of the prism-lens **710**, and the lens surface adjacent to the prism-lens **710** was matched to the concave surface **702** of the prism-lens **710**, which ensured that the auxiliary lens **720** and prism-lens **710** could be cemented accurately which simplified the design of the auxiliary free-form lens **720**. As a result, the front surface **722** of the auxiliary lens **720** only needed to compensate for the optical power introduced by surface **701** of the prism-lens **710** of the FFS prism-lens **710**. Although we could start the optimization of the lens with a planar front surface **722**, a good approximation is to initialize the front surface **722** with the same shape as surface **701** of the prism-lens **710**. The combination of the auxiliary lens and the prism-lens **710** should ideally form an afocal system **700** for a real-world scene at optical infinity as the object distance is considerably larger than the EFL of the system **700**. Therefore, we inserted an ideal lens at the eye position with an effective focal length equivalent to the human eye to focus the collimated rays. During the optimization process, we only set the curvature and polynomial coefficients of the front surface **722** of the auxiliary lens **720** as variables. We set constraints on the distortion and aberrations. The specification for the front surface **722** of the auxiliary lens **720** is provided as “Surface **4**” in Table 8 below.

**710** is as high as 10%, as shown in **720** effectively corrected the viewing axis deviation and the distortion.

The final design of the auxiliary lens **720** combined with the FFS prism-lens **710** is shown in **700** is approximately the same as the prism-lens **710** alone. The optical material of the auxiliary lens **720** was PMMA having a refractive index of 1.492 and Abbe number of 57.2. The locations, and effective areas, of the surfaces relative to the global coordinate system having its origin at the exit pupil are give in Tables 4 and 5, respectively, below.

_{1}(x

_{1}, y

_{1}, z

_{1})

_{1}

_{2 }(x

_{2}, y

_{2}, z

_{2})

_{2}

_{3 }(x

_{3}, y

_{3}, z

_{3})

_{3}

_{4 }(x

_{4}, y

_{4}, z

_{4})

_{4}

_{Im}

_{Im}, y

_{Im}, z

_{Im})

_{Im}

#### Prototype and Experimental Results

The FFS prism-lens **710** was fabricated through a molding approach. **700** to demonstrate the image quality of the microdisplay viewing optics. The distortion was noticeable and irregular. The upper portion of the displayed image seems nearer than the lower portion of the image. The distortion correction method of a free-form surface system **700** is different from rotationally symmetric systems where distortion can be corrected with sufficient accuracy by 3 radial and 2 tangential coefficients. To correct the distortion in the free-form system **700**, a more complex model with more coefficients would be required. Alternatively, we calculated the mapping from the undistorted image to a distorted image using the distortion plot in **730**.

#### Further Design Example

The techniques described above where employed to provide a second exemplary design. Again, the optical material of the prism-lens was PMMA having a refractive index of 1.492 and Abbe number of 57.2. The locations, and effective areas, of the surfaces relative to the global coordinate system having its origin at the exit pupil are give in Tables 6 and 7, respectively, below. The FFS polynomial coefficients are provided in Table 9 below.

_{1}(x

_{1}, y

_{1}, z

_{1})

_{1}

_{2 }(x

_{2}, y

_{2}, z

_{2})

_{2}

_{3 }(x

_{3}, y

_{3}, z

_{3})

_{3}

_{4 }(x

_{4}, y

_{4}, z

_{4})

_{4}

_{Im}

_{Im}, y

_{Im}, z

_{Im})

_{Im}

These and other advantages of the present invention will be apparent to those skilled in the art from the foregoing specification. Accordingly, it will be recognized by those skilled in the art that changes or modifications may be made to the above-described embodiments without departing from the broad inventive concepts of the invention. For instance, other shapes of free-form surfaces may be utilized in the designs of the present invention. By way of example, if one wanted to vary the surface curvature independently in the x and y directions, the surface could be represented by

where z is the sag along the local z-axis, x and y are the coordinates in the local coordinate system, k is the conic constant, c_{x }is radius of curvature of surface in sagittal direction, c_{y }is radius of curvature of surface in tangential direction, and C_{j }is the coefficient for x^{2m}y^{n}. It should therefore be understood that this invention is not limited to the particular embodiments described herein, but is intended to include all changes and modifications that are within the scope and spirit of the invention as set forth in the claims.

^{2}

^{2}

^{3}

^{2}y

^{2}

^{3}

^{4}

^{3}y

^{2}y

^{2}

^{3}

^{4}

^{5}

^{4}y

^{3}y

^{2}

^{2}y

^{3}

^{4}

^{5}

^{6}

^{5}y

^{4}y

^{2}

^{3}y

^{3}

^{2}y

^{4}

^{5}

^{6}

^{7}

^{6}y

^{5}y

^{2}

^{4}y

^{3}

^{3}y

^{4}

^{2}y

^{5}

^{6}

^{7}

^{8}

^{7}y

^{6}y

^{2}

^{5}y

^{3}

^{4}y

^{4}

^{3}y

^{5}

^{2}y

^{6}

^{7}

^{8}

^{9}

^{8}y

^{7}y

^{2}

^{6}y

^{3}

^{5}y

^{4}

^{4}y

^{5}

^{3}y

^{6}

^{2}y

^{7}

^{8}

^{9}

^{10}

^{9}y

^{8}y

^{2}

^{7}y

^{3}

^{6}y

^{4}

^{5}y

^{5}

^{4}y

^{6}

^{3}y

^{7}

^{2}y

^{8}

^{9}

^{10}

^{2}

^{2}

^{3}

^{2}y

^{2}

^{3}

^{4}

^{3}y

^{2}y

^{2}

^{3}

^{4}

^{5}

^{4}y

^{3}y

^{2}

^{2}y

^{3}

^{4}

^{5}

^{6}

^{5}y

^{4}y

^{2}

^{3}y

^{3}

^{2}y

^{4}

^{5}

^{6}

^{7}

^{6}y

^{5}y

^{2}

^{4}y

^{3}

^{3}y

^{4}

^{2}y

^{5}

^{6}

^{7}

^{8}

^{7}y

^{6}y

^{2}

^{5}y

^{3}

^{4}y

^{4}

^{3}y

^{5}

^{2}y

^{6}

^{7}

^{8}

^{9}

^{8}y

^{7}y

^{2}

^{6}y

^{3}

^{5}y

^{4}

^{4}y

^{5}

^{3}y

^{6}

^{2}y

^{7}

^{8}

^{9}

^{10}

^{9}y

^{8}y

^{2}

^{7}y

^{3}

^{6}y

^{4}

^{5}y

^{5}

^{4}y

^{6}

^{3}y

^{7}

^{2}y

^{8}

^{9}

^{10}

^{2}

^{2}

^{3}

^{2}y

^{2}

^{3}

^{4}

^{3}y

^{2}y

^{2}

^{3}

^{4}

^{5}

^{4}y

^{3}y

^{2}

^{2}y

^{3}

^{4}

^{5}

^{6}

^{5}y

^{4}y

^{2}

^{3}y

^{3}

^{2}y

^{4}

^{5}

^{6}

^{7}

^{6}y

^{5}y

^{2}

^{4}y

^{3}

^{3}y

^{4}

^{2}y

^{5}

^{6}

^{7}

^{8}

^{7}y

^{6}y

^{2}

^{5}y

^{3}

^{4}y

^{4}

^{3}y

^{5}

^{2}y

^{6}

^{7}

^{8}

^{9}

^{8}y

^{7}y

^{2}

^{6}y

^{3}

^{5}y

^{4}

^{4}y

^{5}

^{3}y

^{6}

^{2}y

^{7}

^{8}

^{9}

^{10}

^{9}y

^{8}y

^{2}

^{7}y

^{3}

^{6}y

^{4}

^{5}y

^{5}

^{4}y

^{6}

^{3}y

^{7}

^{2}y

^{8}

^{9}

^{10}

^{2}

^{2}

^{3}

^{2}y

^{2}

^{3}

^{4}

^{3}y

^{2}y

^{2}

^{3}

^{4}

^{5}

^{4}y

^{3}y

^{2}

^{2}y

^{3}

^{4}

^{5}

^{6}

^{5}y

^{4}y

^{2}

^{3}y

^{3}

^{2}y

^{4}

^{5}

^{6}

^{7}

^{6}y

^{5}y

^{2}

^{4}y

^{3}

^{3}y

^{4}

^{2}y

^{5}

^{6}

^{7}

^{8}

^{7}y

^{6}y

^{2}

^{5}y

^{3}

^{4}y

^{4}

^{3}y

^{5}

^{2}y

^{6}

^{7}

^{8}

^{9}

^{8}y

^{7}y

^{2}

^{6}y

^{3}

^{5}y

^{4}

^{4}y

^{5}

^{3}y

^{6}

^{2}y

^{7}

^{8}

^{9}

^{10}

^{9}y

^{8}y

^{2}

^{7}y

^{3}

^{6}y

^{4}

^{5}y

^{5}

^{4}y

^{6}

^{3}y

^{7}

^{2}y

^{8}

^{9}

^{10}

## Claims

1. A free-form prism-lens for use in an optical see-through head-mounted display, comprising:

- a first free-form surface configured to receive light from a micro-display and configured to transmit the received light into the body of the prism-lens;

- a second free-form surface configured to receive the light transmitted into the body of the prism-lens from the first free-form surface and configured to totally internally reflect the received light at the second surface; and

- a third free-form surface configured to receive the light reflected by the second free-form surface and configured to reflect the light out of the prism-lens,

- wherein the prism-lens has an f-number less than 3.5.

2. The free-form prism-lens according to claim 1, wherein the first free-form surface is described by z = c ( x 2 + y 2 ) 1 + sqrt ( 1 - ( 1 + k ) c 2 ( x 2 + y 2 ) ) + ∑ j = 2 66 C j x m y n, j = [ ( m + n ) 2 + m + 3 n ] / 2 + 1,

- where the z is the sag of the first free-form surface measured along the z-axis of a local x, y, z coordinate system, c is the vertex curvature (CUY), k is the conic constant, and Cj is the coefficient for xmyn.

3. The free-form prism-lens according to any one of the preceding claims, wherein the second free-form surface is described by z = c ( x 2 + y 2 ) 1 + sqrt ( 1 - ( 1 + k ) c 2 ( x 2 + y 2 ) ) + ∑ j = 2 66 C j x m y n, j = [ ( m + n ) 2 + m + 3 n ] / 2 + 1,

- where the z is the sag of the first free-form surface measured along the z-axis of a local x, y, z coordinate system, c is the vertex curvature (CUY), k is the conic constant, and Cj is the coefficient for xmyn.

4. The free-form prism-lens according to claim 3, wherein the third free-form surface is described by z = c ( x 2 + y 2 ) 1 + sqrt ( 1 - ( 1 + k ) c 2 ( x 2 + y 2 ) ) + ∑ j = 2 66 C j x m y n, j = [ ( m + n ) 2 + m + 3 n ] / 2 + 1,

- where the z is the sag of the first free-form surface measured along the z-axis of a local x, y, z coordinate system, c is the vertex curvature (CUY), k is the conic constant, and Cj is the coefficient for xmyn.

5. The free-form prism-lens according to claim 1, wherein the third free-form surface is partially mirrored to permit the internally reflected light to be reflected by the second free-form surface and to permit light from a real-world view to be transmitted through the third free-form surface to the exit pupil.

6. The free-form prism-lens according to claim 1, wherein second and third free-form surfaces are configured to provide a wedge-shaped prism lens.

7. The free-form prism-lens according to claim 1, wherein the z-axis is parallel to the optical axis at the exit pupil, and the prism lens is symmetric about the y-z plane and asymmetric about the x-z plane.

8. The free-form prism-lens according to claim 1, wherein the diagonal field of view is at least 40 degrees.

9. The free-form prism-lens according to claim 1, wherein the exit pupil diameter is at least 6 mm.

10. The free-form prism-lens according to claim 1, wherein the modulation transfer function is at least 10%×30 lps/mm.

11. The free-form prism-lens according to claim 1, wherein the eye clearance is at least 16 mm.

12. The free-form prism-lens according to claim 1, comprising an auxiliary lens disposed proximate the third free-form surface, the auxiliary lens configured to minimize the shift and distortion of rays from a real-world scene by the second and third surfaces of the prism-lens.

13. The free-form prism-lens according to claim 12, wherein the auxiliary lens has a surface with the same shape as the third free-form surface of the prism-lens and is disposed in optical contact with the third free-form surface of the prism-lens.

14. The free-form prism-lens according to claim 12 or 13, wherein at least one surface of the auxiliary lens is described by z = c ( x 2 + y 2 ) 1 + sqrt ( 1 - ( 1 + k ) c 2 ( x 2 + y 2 ) ) + ∑ j = 2 66 C j x m y n, j = [ ( m + n ) 2 + m + 3 n ] / 2 + 1,

## Patent History

**Publication number**: 20120081800

**Type:**Application

**Filed**: Apr 20, 2010

**Publication Date**: Apr 5, 2012

**Inventors**: Dewen Cheng (Beijing), Hong Hua (Beijing), Yongtian Wang (Beijing)

**Application Number**: 13/318,864

## Classifications

**Current U.S. Class**:

**Asymmetric (e.g., Prismatic Or Eccentric, Etc.) (359/720)**

**International Classification**: G02B 17/08 (20060101);