Mastering AI Chip Complexity
Explore challenges and solutions in AI chip development
Download
Table of Contents
Definition
Augmented reality (AR) uses technology to combine a simulated environment with a real environment. AR relies on optics to create a simulated environment that annotates or enhances the real environment so that the user can experience them as one environment. The hardware for augmented reality typically includes a computer capable of real-time simulation that synchronizes and maps the simulated to the real environment using a visual output display.
Currently, the AR optics of most interest are see-through head-mounted displays (HMDs), which are also known as see-through near-eye displays or head-up displays (HUDs). Like virtual reality, wearable devices (e.g., haptic gloves) that sense and respond to motions of the user; devices for audio feedback; and trackers for body, head, and eye may be used to interact with the simulation. However, in augmented reality, the user also interacts with objects in the real world.
How Do Augmented Reality Optics Work?
To enable augmented reality (AR), an optical system projects digital images onto a transparent display positioned in front of the user's eyes, overlaying virtual information onto the real environment. This can be achieved through different types of displays, such as head-mounted displays (HMDs), handheld devices like tablets, or mounted displays like windshields.
AR Headsets and Mounted Displays
A typical AR optical system consists of three main components:
- Light Sources: Microdisplays such as organic light emitting diodes (OLEDs) or liquid crystal displays (LCDs) generate the augmented images. Binocular HMDs use two displays, one for each eye, to create a 3D effect through stereoscopy. Holographic HMDs use spatial light modulators (SLMs) to produce modulated coherent light for advanced image projection.
- Receivers: The user's eyes receive both the real-world and augmented images.
- Optical Elements: Lenses and combiners blend light from the microdisplays with light from the real environment and project the combined image to the user's eyes. In AR glasses, for example, a microdisplay image passes through a series of optical components, including beam-shaping lenses, prisms, and prescription lenses, before being merged with the real scene for the viewer.
Figure 1: Schematic diagram of Prescription AR:
(a) The side view and the beam path of the AR image of the proposed system. The prescription lens works both for vision-correction and for wave-guide of the AR image. Light rays from a micro display refracted by a beam shaping lens enter to the prescription lens through an in-coupling prism and create a magnified virtual image located a distance from the lens.
(b) The detailed diagram for geometric parameters in the Prescription AR.
(c) The 3D diagram of optical components.
Reprinted with permission. © The Optical Society.
Design Considerations
Image quality in AR HMDs can be affected by optical aberrations, similar to other optical systems. Aberrations such as chromatic aberration, spherical aberration, coma, astigmatism, field curvature, and distortion can cause blur or warping in the projected images. Careful design and optimization are essential to minimize these aberrations and improve visual clarity.
Matching both the field of view (FOV) and resolution of the human eye is challenging, so tradeoffs are often made based on the intended use. Design choices may involve balancing FOV, device weight and size, resolution, pupil size (eye box), eye clearance, and microdisplay dimensions. Technological solutions to address these tradeoffs include using high-resolution insets and spatial or temporal tiling to balance FOV and resolution, employing partial binocular overlap and diffraction-order tiling, and duplicating the exit pupil with arrays along with integrating eye-tracking devices to optimize FOV and pupil size.
These ongoing innovations help AR optics deliver more immersive and practical experiences across various applications.
Figure 2: Simulating an all-reflective freeform design
What is Needed to Design Augmented Reality Optics?
Designing augmented reality (AR) optics requires a combination of specialized software tools and expertise from multiple engineering disciplines. Optical engineers use software to create and optimize imaging systems, analyze stray light, and design diffractive optical elements. Mechanical engineers need CAD tools to lay out the system and perform thermal and structural analysis. Electrical engineers may also be involved to implement eye-tracking and manage signals sent to the optical system.
The workflow:
- Optical Systems:
- CODE V optical design software can be used to trace rays through the optical system, optimize the system to reduce aberration, decrease distortion, and increase resolution as shown in a head-mounted display. Augmented reality optics for automotive head-up displays (HUDs) can also be modeled in CODE V (see next section). The geometry can then be exported to LightTools.
- LightTools illumination design software can model illumination, stray light and ghost images. LightTools can also be used to optimize illumination uniformity.
- Design gratings using the RSoft Photonic Device Tools:
- Diffractive gratings couple light into the waveguide plate and couple the light out of the plate into the eyes. Gratings must be designed properly so that the optical system produces good images. For the design and optimization of gratings, gratings can be optimized based on diffraction angle, efficiencies, etc. of any order or combination of orders.
- DiffractMOD RCWA is a very efficient tool to rigorously calculate diffraction properties of transversely periodic devices.
- FullWAVE FDTD is another powerful tool to rigorously calculate diffraction properties of transversely periodic devices when it is necessary.
- MOST optimization in the RSoft CAD Environment provides a convenient method to optimize gratings with either FullWAVE or DiffractMOD.
- Diffractive gratings couple light into the waveguide plate and couple the light out of the plate into the eyes. Gratings must be designed properly so that the optical system produces good images. For the design and optimization of gratings, gratings can be optimized based on diffraction angle, efficiencies, etc. of any order or combination of orders.
Once the gratings have been built, the Bidirectional Scattering Distribution Function (BSDF) information and layout files can be exported directly to LightTools to define a surface property. All diffractive properties are included in the RSoft BSDF files, which contain information about how a surface (thin film, patterns, etc.) scatters light.
How Do You Design Augmented Reality Optics for an Automotive Application?
A Head’s Up Display (HUD) augments a driver’s field of view with an image from a display. Software is needed to model the rays traveling through the windshield, and also to evaluate the quality of the projected image.
CODE V has powerful features for applications in the HUD design space when tackling a wide range of opto-mechanical systems design challenges. Engineers can use this optical design software for CAD visualization and import for ray tracing. CODE V can accommodate new freeform surfaces with flexibility for design degrees of freedom for compact designs. when modeling windshields as the combiner. New freeform surface types allow for enhanced aberration control as display resolution increases at the viewer’s eye (higher density display pixels) combined with more compact form factors.
After design completion, it is important to check the final system performance not only against nominal criteria but also for actual system as-built performance. For this, LightTools is the logical next step for viewer simulation. A reverse ray trace from spectral color objects representing a display image in LightTools shows the projected HUD image for a viewer (onto a model scene). A LightTools simulation can also help uncover any unforeseen issues with stray images or reflections in the system. Also, engineers can use LightTools CAD import and measurement tools to determine:
Eyebox to windshield distance
Approximate angle of incidence on windshield
Windshield to dashboard distance
LightTools simulation of real world optical system performance is an excellent strength for Synopsys Optical Solutions product users in their engineering and design work.
What Is the Difference Between Augmented Reality and Virtual Reality Optics?
The main difference between augmented reality (AR) and virtual reality (VR) optics is how they interact with the real environment. VR optics simulate the entire visual environment, immersing users in a completely virtual world. In contrast, AR optics capture the real environment and overlay digital information onto it, blending simulated and real elements through a visual display.
In AR, the display is typically see-through, allowing users to view both the real world and digital enhancements simultaneously. VR displays, on the other hand, only present the simulated environment, blocking out the real world.
Key differences between AR and VR optics include:
- Display Requirements: AR needs high luminance displays to ensure visibility in bright environments like outdoors or operating rooms.
- Optical Precision: For see-through AR head-mounted displays (HMDs), dipervergence (the difference in angle between real and virtual images) should be less than 1 to 3 arc minutes for optimal clarity.
- Design Considerations: AR HMDs often use a folded optical design to achieve a wide field of view (FOV) while maintaining a compact form factor. These devices must integrate an optical combiner that merges reflected light from virtual scenes with transmitted light from the real world. Beam splitters are commonly used for prototyping, while holographic optical elements (HOEs) offer a thinner, flatter solution for specific wavelengths.
What Are Some Real-World Applications for Augmented Reality Optics?
These examples illustrate how augmented reality optics are transforming a wide range of industries by enhancing situational awareness, improving decision-making, and creating more interactive experiences.
- Heads-Up Display (HUD) for Driving: Provides real-time information on the windshield to assist drivers.
- Surgical Assistance: Superimposes procedural instructions and helpful data during medical operations.
- Combat Aid: Offers enhanced situational awareness and targeting information for military personnel.
- Engineering and Design: Assists in visualizing and modifying 3D models of buildings and products.
- Social Interaction: Enables communication with both real and virtual audiences simultaneously.
- Entertainment and Tourism: Enhances gaming and tourism experiences by overlaying historical scenes or dynamic information onto the environment.
- Education: Allows textbooks and learning materials to add interactive explanations directly onto real-world objects and settings.
Also see these resources:
- Meet Your Augmented and Virtual Reality Challenges Head-On: Design Your Next System with 2D-Q Freeforms in CODE V
- Sunglass-like displays become a reality with free-form optical technology
- Head-Worn Displays: The Future Through New Eyes
- Automotive HUD case study on SolvNetPlus (PDF, SolvNetPlus accont required)
Blog:
References:
[1] Jui-Yi Wu and Jonghyun Kim, "Prescription AR: a fully-customized prescription-embedded augmented reality display," Opt. Express 28, 6225-6241 (2020).
Continue Reading
Blog
Virtual Prototyping of Optical Systems: Leveraging ImSym and Optical Scattering Data
Blog