Decoder-Assisted Typing using an HMD and a Physical Keyboard

James Walker, Scott Kuhl, Keith Vertanen

CHI '16: Extended Abstracts of the ACM International Conference on Human Factors in Computing Systems (workshop), 2016.

In recent years, head-mounted displays (HMDs) have become increasingly affordable and popular. HMDs pose new challenges for text entry because they typically immerse people in a virtual world while preventing them from seeing the real world. As HMDs become widespread, we expect that some people will want to perform traditional computing tasks with these new devices, such as writing emails or browsing the web. This work explores new ways to support efficient text entry in HMDs by leveraging decoding techniques and ubiquitous physical keyboards. We describe an experiment where we measured the performance of our system on touch-typists who typed a series of short messages on a physical keyboard in a (1) keyboard visible condition, (2) occluded keyboard condition, and (3) headmounted display condition. The results of our pilot study showed that users' speed and accuracy was considerably worse in the occluded and HMD conditions, that our decoder was able to correct a large number of errors, and that increasing compute time for the decoder increased the number of corrected errors.