Mapping contextual menus to controllers

User goals

  • Help the user understand how a menu was triggered so it can easily be closed again
  • Provide contextual options without disrupting or obscuring the main VR experience


Sometimes it’s necessary to call up a menu during a VR experience. One method is to pause the experience and present a large, perhaps even full-screen menu in the central field of view.
This is somewhat disruptive and breaks presence. Another approach is to open a smaller contextual menu within the experience, without pausing the main content or obstructing the user’s view.

However this presents the problem of controlling where the menu appears (i.e. can the user see it, is it obstructing something else the user is trying to look at?) and being able to easily close it.
We find in usability studies that users accidentally trigger menus and then forget which button they pressed or can’t figure out how to close the menu.

By physically attaching the menu to the controller input that triggers and closes the menu in the virtual world, we can solve both these problems:

  1. Show contextual menu attached to the right or left hand, depending which controller triggered the menu
  2. Ideally map the same button to toggle the menu ‘open’ and ‘close’
  3. Visualise the menu control button, so the user understands how to close the menu
  4. The contextual menu moves and orientates with the hand. It follows the hand as if it is attached to it. This allows the user to control where on the screen it’s located, and also not to lose it as they move and orientate in the 3D space (e.g. the common issue where the user does not realise the menu they need has appeared behind them)

Linked to: Motion controller visualisation  and Mapping input prompts to controllers


  • This allows the user to more easily close the menu when not needed or accidentally triggered. The menu also does not stop the game or experience but can be used in parallel in real time.
  • Another positive is that the menu can be moved out of the way just by moving the hand, which is pretty effortless, but the other side of this is that small contextual menus may be overlooked or not noticed if the hands are outside the visible area.


Whereas the mapping to the left or right hand is useful, if it’s not clear which controller button toggles the menu and there are several on that controller, the user may still struggle.


Raw Data, HTC Vive

Raw Data (both hands), HTC Vive

Google Block, Oculus