Design patterns for Immersive Tech

Direct manipulation AR buttons

User goals

  • Input an action or select an option using natural physical world interaction

Interaction

The Hololens 2 allows users to directly interact with virtual buttons as if they were physical objects with physical button like properties. What is the advantage you may ask? Well for starters it removes the concept of a cursor as your hand and its position IS the cursor. No more gaze point or hand hand controller with ray-cast. The holographic UI essentially becomes a virtual touch screen on the one hand or a physical interface on the other.

For most digitally savvy users clicking or tapping an icon is not really an issue, so the advantage of simulating a physical button interaction is one that can be debated, but certainly this increases affordances of pressing for less tech savvy users or users who are unfamiliar with AR and need things to be incredibly obvious. It’s also in a way satisfying as it provides feedback that feels embedded and physical rather than a digital response.

Here we look specifically at button like interactions.

  • Smaller buttons invite air tapping with fingers, enabled by Hololens 2 fully articulated hand tracking
  • Larger buttons afford being pushed with the whole palm
  • A designer should consider the size, height, distance from user and all the physical and ergonomically factors that apply to the design of physical machines.
  • A designer should also consider the animation and ‘feel’ of the button press: the animation showing the press, whether the button comes back up when released or stays pressed, whether there is any colour change, what audio feedback would help. On the one hand this generates the satisfaction of it feeling like a real button and on the other it reinforces that the action has been carried out, and perhaps the clarity and power of this may be enhanced by going beyond the behaviour or real world buttons, particularly when there is no haptic feedback.

Linked to: AR Hand Menus

Examples

Hololens 2: Finger buttons on a UI panel

Hololens 2: Big buttons for pushing