Augmented Reality Optics

For augmented reality to work, an optical system must project an image on a transparent display in front of your eyes to overlay the current environment.  This may be in the form of a head-mounted display, handheld display (such as a digital tablet), or a mounted display (such as a windshield).

AR Headsets or Mounted Displays

Typically, an optical system for augmented reality includes light sources (display), receivers (eyes), and optical elements (lenses).

• The light sources for the augmented reality are microdisplays, such as organic light emitting diodes (OLED) or liquid crystal displays (LCD). A binocular HMD typically has two displays that provide separate images for each eye and generate 3D perception through stereoscopy. In a holographic HMD, the light source is modulated coherent light from a spatial light modulator (SLM). The light sources for the real world are light scattered off or emitted by objects within field of view. 
• The receivers are simply the eyes of the user.
• The optical elements combine light from the microdisplays and light from the real world and project augmented information (from the microdisplays) on the real world. An example is shown below in Figure 1, where the micro-display is imaged a distance from the AR glasses through the beam-shaping lens, in-coupling prism, prescription lens, and a free-from image combiner. The real scene and the virtual image (augmented information) are imaged to the eye through the prescription lens.  

Design Considerations

The effects of aberrations to image quality in HMDs are similar to that in other optical systems. Aberrations such as axial chromatic aberration, spherical aberration, coma, astigmatism, and field curvature introduce blur. Aberrations such as distortion, coma, and lateral chromatic aberration induce warping. Aberration control is important in the design of AR HMD optics. An example all-reflective freeform design is shown in Figure 2, where (a) displays the system design, (b) shows the test object, (c) provides the image simulation results of an initial system, and (d) demonstrates the image after optimization. In this example, we can appreciate the power of optimization in reducing aberrations and distortions.  

As It is extremely challenging to match both the FOV and the resolution of human eyes, tradeoffs are often made for specific tasks among FOV, weight (volume, number of optical element), resolution, pupil size (eye box), eye clearance, and the size of the micro-displays. Some of the tradeoffs can be addressed with improved technology. A few potential solutions are listed as below: the tradeoffs between FOV and resolution can be addressed with high-resolution insets, partial binocular overlap, spatial tiling, temporal division, and diffraction-order tiling. The tradeoffs between FOV and pupil size can be addressed by duplicating the exit pupil into an array and employing eye-tracking devices. 

Synopsys provides a complete set of tools to study AR/VR devices. There are multiple software needs for the design of AR optical systems. The optical engineer needs software to create and optimize the imaging system, analyze straylight in the optical path, and designing diffractive optical elements. The mechanical engineer needs a CAD package to draw the system layout and accomplish thermal and structural analysis. The AR system may also require electrical engineer to track the eye motion and send the signal to the optical system. 

A Head’s Up Display (HUD) augments a driver’s field of view with an image from a display. Software is needed to model the rays traveling through the windshield, and also to evaluate the quality of the projected image.

The key difference from virtual reality is that the former simulates the entire visual environment, while the optics for augmented reality captures the real environment and maps the simulated environment to the real environment, presenting them together using a visual display.

In augmented reality, the display is often see-through to combine the simulated environment with the real environment. In virtual reality (VR), the display only needs to output the simulated environment. 

There are a few differences between AR and VR optics.

• First, AR requires high luminance displays especially for bright environment such as outdoors and surgery rooms.
• Second, the dipervergence should be less than 1 to 3 arc minutes for see-through HMDs for AR.
• Lastly, the see-through HMDs often follow a folded design to enable a wide FOV and compact form factor. See-through HMDs must integrate an optical combiner to combine reflected light from the virtual scene and the transmitted light from real-world objects. For prototyping, a beam splitter is often used as a combiner. To reduce the form factor of the combiner, HOEs can be used as they are thin and flat and can function like a beam splitter for a specific wavelength.

• HUD for driving. 
• Surgery for superimposing help and procedure instructions. 
• Combat aid.
• In engineering to aid 3D design of architectures and products.
• Social. For example, interacting with real and virtual audience at the same time.
• Entertainment, including gaming that involves real environment and tourism activities that explain the historical scene or time-evolving changes.
• Education. For example, textbooks that super-impose explanations on real environments.