Design patterns for Immersive Tech

Hand ray object manipulation

User goals

  • Manipulate object from a distance
  • Use natural hand gestures and motions
  • Resize, rotate or move an object that is beyond reach of your hands

Interaction

The Hololens 2 has evolved the ray-cast interaction to better support more dexterity and control compared to previous versions. The hand ray links the user’s hands to an object that is beyond reach, allowing the user to control the object with their hands remotely.

Elements to the interaction:

  • A flexible, elastic line connects the hand to the object selected for control
  • The line can be aimed to connect to specific anchor points on the object’s bounding box
  • Attaching to anchors midway between opposite corners along an edge of the bounding box allows the user to rotate the object along a vertical or horizontal axis
  • Attaching to anchors in the corner of the bounding box can be used to resize an object by pinching and dragging
  • Attaching to objects without the bounding box or without the anchors on the bounding box can lead to different interactive behaviours (see example videos below)
  • Twisting and rotating the hand distorts the shape of the line, creating the digital equivalent of tension and providing a natural and tangible feeling of transfer of kinetic energy through the connecting line
  • Twisting will result in spinning or rotating the object
  • It can also be used as a more traditional ray-cast to simply move the object in space without rotating
  • A two-handed version utilising 2 hand rays can be used to resize or spin an object

Essentially the user has Jedi-like powers to move and control objects from a distance and the hand ray is the visual indicator, showing which object is locked in for interaction and providing clues as to how the object can be controlled.

Linked to: Direct manipulation resize, Ray-cast

Examples

Hololens 2: Hand ray used to rotate object using anchor points

Hololens 2: Hand ray used to resize object using anchor points

Hololens 2: Hand ray to move and rotate object (no anchor points)

Hololens 2: Two handed hand ray

Hololens 2: Hand was appear automatically during direct manipulation

This examples shows how an interaction can transition from direct manipulation to hand ray interaction. As the user resizes the object, their hands move away from the edges of the object, but the interaction remains locked in. Hand rays appear, to show the hands are still connected to the object (the pinch grip has not been released by the user).