Gaze-Tracking Head-mounted Display

In this project we present an affordable hard- and software solution for eye-tracking and user-friendly lens calibration within an HMD. The use of dichroic mirrors leads to a lean design that provides the full field-of-view (FOV) while using commodity cameras for eye tracking. Our prototype supports personalizable lens positioning to accommodate for different interocular distances. On the software side, a model-based calibration procedure adjusts the eye tracking system and gaze estimation to varying lens positions. We also present example applications for gaze-contigent VR: Gaze map estimation, foveated rendering for depth of field, gaze-contingent level-of-detail, and gaze control of virtual avatars.

Most of this work has been created by my colleague Steve Grogorick and me, Michael Stengel. Additionally we thank our colleagues Martin Eisemann from FH Köln and Elmar Eisemann from TU Delft as well as the leader of our Institute for Computer Graphics in Braunschweig, Prof. Marcus Magnor. Acknowledgements go also to friends and colleagues Pablo Bauszat, Lorenz Branz, Felix Klose, Christian Lipski, Manuela Rolletschek, Günther Stengel and Daniel Ratz for supporting us.

Project Motivation

Everybody is talking about VR and the topic is powered very much by the media and different hardware vendors. It is said that Virtual Reality will manifest as the most innovative and immersive medium of the very near future. This hype is primarily caused by advances in head-mounted displays, which currently get affordable and therefore more popular.

What‘s currently missing in state-of-the-art head-mounted displays is an affordable solution for tracking the eyes. However, gaze is a key component for many tasks in reality and it should be the same with Virtual Reality. Many applications would benefit from gaze-tracking or are not even possible without knowledge about gaze, for example foveated rendering.

Our ETHMD Prototype

We (the Computer Graphics Lab of TU Braunschweig, Germany) therefor provide an HMD design with integrated binocular gaze tracking. We avoid using expensive hardware parts, so it costs less than 400 dollars to create it. Eye-tracking is performed very fast with a latency of less than 20 milliseconds. We also propose efficient user calibration, which can be a very tedious task otherwise. And importantly we do not reduce the available field of view
since we are tracking the eyes virtually from the front view which also results in a quite uniform tracking accuracy.

In this post I will write about current HMD technology in the context of gaze-continguency for perceptually convincing VR experiences. I will also explain how to recreate our eye-tracking HMD for those who are willing to experience gaze-contingent VR or who also want to develop novel VR applications. Later I will explain how the calibration and the gaze tracking works. In the end I will post infos about some novel VR applications.

  1. HMD Technology – Today.
  2. Gaze-Tracking Concept
  3. Assembly
  4. Hardware Calibration
  5. Gaze-Tracking Calibration
  6. Tracking Performance
  7. Applications / Future Work
  8. Publication

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s