Hand tracking in Wolvic

Human interaction in XR environments is still an active area of research. While the actions provided by physical controllers are more or less standarized nowadays, interacting with the bare hands is still pretty much open to experimentation by the different vendors and application developers, and thus far from consolidated. Despite that, some patterns have already emerged and are being adopted.

In Wolvic, we support a number of common gestures when an user is interacting with its hands only, on devices that support the hand-tracking feature. It is inspired by existing best practices, as seen in popular environments like the Oculus Quest.

We keep researching and developing new and better ways of interacting with Wolvic in general, and with hands in particular. If as a user you have some feedback, criticism or ideas, please don’t hestitate to contact us.

Currently supported gestures

1. Select action

The Select action is the most useful and common action. This action signals the intention of selecting the element being pointed at, when interacting with Wolvic’s user interface. If the element has a URL, selecting will cause Wolvic to navigate to that URL. It is equivalent to the trigger button on devices that support controllers, and typically the left click on a mouse.

A Select action is performed by a pinch gesture with the index and the thumb fingers of either hand. That is, when the tips of the fingers are brought together until they touch each other, and then separated again.

To provide visual feedback for the gesture, as the pinch is performed, the pointer target (the white circle over the Wolvic user interface elements) decreases size until the finger tips touch and the circle background changes to a blue color. When the pinch is released — that is, the finger tips separate from each other — the pointer target returns to its normal size and color.

2. Scroll action

The Scroll action is very similar to the Select action, except that the pinch gesture is kept closed and the user moves their hand either horizontally or vertically to move (scroll) content inside the viewport currently pointed at. It is similar to the tap+drag gesture on touch devices, or the scroll wheel on a mouse.

The Scroll action is stopped by releasing the pinch gesture; that is, separating the finger tips from each other.

3. Back action

The Back action is also very important. It is used to return to the previous place within the navigation history while browsing Web content, similar to the back button on the left controller (if device supports it), or the “Navigate Back” button commonly seen in desktop and mobile browsers.

Similarly, it is used to signal cancellation of an ongoing task within the Wolvic user interface, and during immersive mode it allows the user to exit that mode and return to Wolvic browser interface.

Together with the Select action, the Back action thus allow the user to effectively interact with and navigate both the browser interface and the Web content.

To perform a back navigation, the user opens their left hand and turns it so that the palm of the hand is facing the headset. While in this configuration, a floating button will appear between the tips of the index and the thumb fingers. Then the user performs a pinch gesture as described for the Select action above. In that pose, multiple such pinch gestures will cause one back action each.