What are Virtual Reality Optics and How Does it Work? | Synopsys
Table of contents

Definition

Virtual reality (VR) uses technology to create a completely simulated environment in which a user can experience and interact with that environment [1]. The hardware for virtual reality typically includes a computer capable of real-time scene simulation; wearable devices (e.g., haptic gloves) that sense and respond to motions of the user; a display for visual output; devices for audio feedback; and trackers for body, head, and eye.

Virtual reality optics include the cameras that capture raw data for scene simulation; fiber optics used in gloves and clothing to send and receive data; head-mounted displays (HMDs) that generate 3D perception; immersive and semi-immersive projection displays; and sensors that track the motion of the user and their eyes. Currently, the virtual reality optics of most interest are head-mounted displays (HMDs), which are also known as near-eye displays.

Virtual Reality Optics | Synopsys

How do virtual reality optics work?

For virtual reality to work, there must be an optical system in a head-mounted display that will project an image on a display in front of your eyes.

In this optical system, an HMD includes light sources (display), receivers (eyes), and optical elements (lenses).

  • The light sources in an HMD are microdisplays, such as organic light emitting diodes (OLED) or liquid crystal displays (LCD). A binocular HMD typically has two displays that provide separate images for each eye and generate 3D perception through stereoscopy. In a holographic HMD, the light source is modulated coherent light from a spatial light modulator (SLM).
  • The receivers in HMDs are the eyes of the user.
  • The optical elements collect light from the source and generate renderings of a 3D virtual world. An ideal VR HMD must be able to provide a high-resolution image within a large field of view (FOV) while supporting accommodation cues for 3D perception and have a large eyebox (exit pupil) with a compact form factor [2].
Structure of VR HMD (Image credit: SK Hynix) | Synopsys

One of the most important requirements for HMDs is good ergonomic design, i.e., the headset is comfortable to wear and view for prolonged usage. To be comfortable to wear, the headset should be compact and lightweight. Ideally, the weight and size should be no more than a pair of eyeglasses. To be comfortable to view, the headset should provide appropriate viewpoints based on the user’s head position and gaze point. The headsets should also have adequate eye clearance, a large-enough pupil size to allow for natural eye movement, appropriate interpupillary distance (IPD), and low divergence and dipvergence. 

The key optical design constraints for HMDs are pupil (eye-box) size, eye clearance, divergence, dipvergence, and IPD (See Figure 2).

  • The pupil diameter of human eyes ranges from 2 to 8 mm depending on the level of brightness. Ideally, a pupil size of 15 to 18 mm would allow natural eye movement. This requirement may be reduced with eye tracking or balanced to satisfy other system requirements.
  • Eye clearance is believed to be an important factor for comfort. The smallest value of eye clearance for standard eyeglasses is 17 mm, while 23 mm eye clearance is recommended to accommodate most eyeglasses.
  • Eye divergence and dipvergence are two factors that may cause discomfort when they exceed optimal values. As shown in Figure 2, divergence is the act of forcing the eyes simultaneously outwards to focus the images. Dipvergence is the act of forcing the eyes to move at different elevations. Divergence should be less than a few degrees. Dipvergence should be less than 5 to 10 arc minutes for VR HMDs.
  • IPD is critical to visual comfort and depth perception. IPD is the distance between the centers of the pupils of the eyes. IPD varies among the population with a range of about 55 to 75 mm. A mean value of 64 to 65 mm is often considered in engineering investigations.
Biocular Parallax | Synopsys

Figure 2: Schematic diagram of biocular parallax.
(a) No biocular parallax; (b) convergence; (c) divergence; and (d) dipvergence. [4]

Field of Vision (FOV) is Important

An important design goal for VR HMDs is to match the image characteristics of the human-visual system. The FOV of the human eye is roughly 120 degrees vertically and almost 360 degrees horizontally considering eye rotation and head movements. The binocular FOV within which an object is visible to both eyes is about 114 degrees [5].

Field of Vision (Credit: B. Kress) | Synopsys

Considering Aberrations in the Design

The effects of aberrations to image quality in HMDs are similar to that in other optical systems. Aberrations such as axial chromatic aberration, spherical aberration, coma, astigmatism, and field curvature introduce blur. Aberrations such as distortion, coma, and lateral chromatic aberration induce warping. Aberration control is important in the design of VR HMD optics.

Other Factors in HMD Design

Advances in HMD design take advantage of aspheric surfaces, diffractive optical elements (DOEs), holographic optical elements (HOEs), tunable lens and plastic optics.

  • Aspheric surfaces help to control lens aberrations and reduce the number of elements.
  • DOEs provide interesting dispersion properties with negative chromatic dispersion in positive lenses.
  • HOEs have a small form factor and can function like a beam splitter.
  • Tunable lens can be used to extend the depth range.
  • Plastic components have the benefits of low cost and light weight.
  • Although the VR device's eye-box size and FOV can be improved through applying these advanced technologies, it often compromises the form factor. To alleviate this problem, new directions for development include eye-tracking integrated HMDs, multi/varifocal displays, occlusion displays, holographic displays, and light field displays.
Synopsys LightTools Simulation of diffractive optical elements | Synopsys

What is needed to design virtual reality optics?

Optical design software is an important tool for designing VR optics. There are multiple software needs for the design of VR optical systems. The optical engineer needs software to create and optimize the imaging system, analyze straylight in the optical path, and design diffractive optical elements. The mechanical engineer needs a CAD package to draw the system layout and accomplish thermal and structural analysis. The VR system may also require electrical engineer to track the eye motion and send the signal to the optical system. Synopsys provides a complete set of tools to simulate AR/VR devices.

Synopsys Optical Design Tools for Augmented/Virtual Reality Applications

The workflow: 

  • Design optical system using optical design software:
    • CODE V optical design software can be used to trace rays through the optical system, optimize the system to reduce aberration, decrease distortion, and increase resolution. 

    • LightTools illumination design software can model illumination, stray light and ghost images. LightTools can also be used to optimize illumination uniformity. Stray light may cause artifacts of the image and bright spots. 
       
  • Design gratings using the RSoft Photonic Device Tools

    Diffractive gratings couple light into the waveguide plate and couple the light out of the plate into the eyes. Gratings must be designed properly so that the optical system produces good images.  For the design and optimization of gratings, gratings can be optimized based on diffraction angle, efficiencies, etc. of any order or combination of orders.

    • DiffractMOD RCWA is a very efficient tool to rigorously calculate diffraction properties of transversely periodic devices.

    • FullWAVE FDTD is another powerful tool to rigorously calculate diffraction properties of transversely periodic devices when it is necessary. 

    • MOST optimization in the RSoft CAD Environment provides a convenient method to optimize gratings with either FullWAVE or DiffractMOD.

Once the gratings have been built, the Bidirectional Scattering Distribution Function (BSDF) information and layout files can be exported directly to LightTools to define a surface property.  All diffractive properties are included in the RSoft BSDF files, which contain information about how a surface (thin film, patterns, etc.) scatters light. 

Design gratings using the RSoft Photonic Device Tools | Synopsys

What is the difference between virtual reality and augmented reality optics?

In VR, the display only needs to output the simulated environment. In augmented reality (AR) (this will link to AR glossary), the display is often see-through to combine the simulated environment with the real environment.

There are a few differences between VR and AR optics.

  • First, AR requires high luminance displays especially for bright environment such as outdoors and surgery rooms.
  • Second, the dipervergence should be less than 1 to 3 arc minutes for see-through HMDs for AR.
  • Lastly, the see-through HMDs often follow a folded design to enable a wide FOV and compact form factor. See-through HMDs must integrate an optical combiner to combine reflected light from the virtual scene and the transmitted light from real-world objects. For prototyping, a beam splitter is often used as a combiner. To reduce the form factor of the combiner, HOEs can be used as they are thin and flat and can function like a beam splitter for a specific wavelength.
Differences in augmented, mixed, and virtual reality applications | Synopsys

What are some real-world applications for virtual reality optics?

  • Education and training
    For example, military training for flight simulation and battlefield combat; medical training for surgery and emergency; patient education to help patients experience what a procedure is like and learn what to expect; remote learning to recreate a classroom environment or historical scene; museum to provide an immersive experience.

  • Engineering
    For example, aiding in 3D design and virtual prototyping.

  • Social and commerce
    For example, virtual interactions with colleagues or customers; display rooms for online shopping; and 3D experience for real estate tours.

  • Entertainment
    For example, gaming and tourism.

  • Medical rehabilitation and remote surgery
    For example, virtual reality psychology exposure therapy and rehabilitation for Alzheimer’s disease can be carried out with VR.
Augmented reality optics | Synopsys
Virtual Reality Applications | Synopsys

References:

[1] Steuer, Jonathan. "Defining virtual reality: Dimensions determining telepresence." Journal of communication 42.4 (1992): 73-93.

[2] Chang, Chenliang, et al. "Toward the next-generation VR/AR optics: a review of holographic near-eye displays from a human-centric perspective." Optica 7.11 (2020): 1563-1578.

[3] "Virtual Reality in Glasses." Used with permission.

[4] Zhao, Z.; Cheng, D.; et al. “Design and evaluation of a biocular system, ” Applied Optics, Vol. 58, Issue 28, pp. 7851-7857 (2019), https://doi.org/10.1364/AO.58.007851.  Used with permission. © The Optical Society. 

[5] Rolland, Jannick P., and Hong Hua."Head-mounted display systems." Encyclopedia of Optical Engineering 2 (2005).

[6]  Kress, B. "Meeting the optical design challenges of mixed reality." Used with permission.