Computational Imaging Holographic AR Glasses with Metasurface Waveguides

Conventional AR glasses use amplitude SLMs, such as organic light-emitting diodes or micro light-emitting diodes, which require a projector-based light engine that is typically at least as thick as the focal length of the projection lens. The design of our holographic AR glasses uses a phase-only SLM that can be mounted very close to the in-coupling grating, thereby minimizing the device form factor. Additionally, unlike conventional AR glasses, our holographic design can provide full 3D depth cues for virtual content.

The geometry of our waveguide is designed to accommodate the different propagation angles for red, green, and blue wavefronts. Specifically, we constrain our geometry to maintain the independent degrees-of-freedom needed for full-colour 3D holograms by coupling out a single copy of the propagating wavefronts for all three colors.

Our metasurface gratings have optimized nanoscale features, shown on the left, to efficiently diffract light into and out of our waveguide. The period (Λ) and height (H) of the metasurfaces are 384 nm and 220 nm respectively. The normalized magnetic field maps, shown on the right, illustrate the diffraction by metasurface gratings at our out-coupler for red (638 nm), green (521 nm) and blue (445 nm) wavelengths. The black arrows illustrate the wave vectors of the incident and diffracted light. Scale bar: 400 nm.

Scanning electron microscope images demonstrate the successful fabrication of our metasurface design. Scale bars: 2 μm (left), 200 nm (right).

We construct a learned physical waveguide model to accurately predict the output of our system by combining physical aspects of the waveguide (highlighted in green) with artificial-intelligence components that are learned from camera feedback (highlighted in orange). Our analytically-derived physical terms include the converging illumination that illuminates the SLM and frequency-dependent coefficients to model the propagation within the waveguide. Our learned components include parameterized spatial maps to characterize any variation in the diffraction efficiency across our fabricated metasurface gratings and CNNs for modelling content-dependent non-idealities such as the spatially-varying non-linear response of the SLM. The model is fully differentiable, enabling simple gradient descent CGH algorithms to compute phase patterns for arbitrary target scenes at runtime.