Design patterns for Immersive Tech

Direct manipulation grab and move

User goals

  • Move a 3D object to another location
  • Pick something up to look at it
  • Rotate a 3D object the way you would naturally, in your hand

Interaction

The Hololens 2 has introduced the ability of directly interacting with 3D objects using just the user’s hands, all thanks to the introduction of fully articulated hand tracking. Previously it was possible only in 6DoF VR systems like Oculus Rift or HTC Vive to grab and object and move it around in your hand using 6DoF controllers. Later the Leap Motion add-on allowed this to be done directly with the hands. It’s now possible to do in mixed reality without any additional hardware with Hololens 2, which is a huge game changer for this platform.

For the 1 handed interaction the user:

  • Brings their preferred hand towards the object
  • The object may show a bounding box or become highlighted to show the interaction is possible
  • The user closes their handed to grab the object, which now becomes attached to the hand
  • The user can move the object with their hand
  • They can rotate the object by rotating their wrist
  • They can also pass the object from one hand to the other, by grabbing it with the second hand and releasing it with the original hand
  • When ready, the user releases their grip to release the object.

Linked to: Movable Menu Panel (VR)

This solves learning issues around correct use of hand held controllers or needing access menu options or interact with UI widgets to launch the ‘move object’ mode. Specifically with Hololens it has overcome the need to use a gaze cursor combined with the air tap gesture, which as a complicated way to initiate this type of interaction.

Examples

Hololens 2: Grab object to move it

Hololens 2: Grab object to rotate it and inspect it