Efficient Typing on a Visually Occluded Physical Keyboard

James Walker, Bochao Li, Keith Vertanen, Scott Kuhl

CHI '17: Proceedings of the ACM Conference on Human Factors in Computing Systems, 2017.

The rise of affordable head-mounted displays (HMDs) has raised questions about how to best design user interfaces for this technology. This paper focuses on the use of HMDs for home and office applications that require substantial text input. A physical keyboard is a familiar and effective text input device in normal desktop computing. But without additional camera technology, an HMD occludes all visual feedback about a user's hand position over the keyboard. We describe a system that assists HMD users in typing on a physical keyboard. Our system has a virtual keyboard assistant that provides visual feedback inside the HMD about a user's actions on the physical keyboard. It also provides powerful automatic correction of typing errors by extending a state-of-the-art touchscreen decoder. In a study with 24 participants, we found our virtual keyboard assistant enabled users to type more accurately on a visually-occluded keyboard. We found users wearing an HMD could type at over 40 words-per-minute while obtaining an error rate of less than 5%.