Mouse-Based Hand Gesture Interaction in Virtual Reality

Virtual reality requires new forms of interaction and input in three-dimensional environments that isolate the user from the physical world. However, existing input methods, such as handheld controllers or hand gestures, are not yet mature or coherent enough to be easily used by the average user. To address this, a universal set of gestures which are easy to use, remember, and expressive is needed. Based on the observation that the mouse in desktop computing can control different applications adequately, four conceptual input types are proposed: primary and secondary clicks, directional input, and a modifier functionality. These provide flexibility without being overwhelming. To find a promising set of gestures, requirements are formulated based on existing research and observations. Three sets of appropriate gesture combinations are carefully selected and proposed, and a user study is conducted to allow a comparison between them. The study involves a virtual testing environment with tasks to be completed using the gestures. Results show positive responses to all proposed gestures, with one set outperforming the other two in most areas. Based on this and participants’ comments, a combination of four top-ranking gestures is proposed as a universal means for virtual reality interaction.