Precise hand tracking without a glove

Posted by Emma Woollacott

Researchers at Newcastle University and Microsoft Research Cambridge (MSR) have developed a wrist controller which allows the user to remotely control any device without any extra hardware.

Mapping finger movement and orientation, it includes an infrared camera, IR laser line generator, IR diffuse illuminator and an inertial-measurement unit (IMU) track - and that's it.

The device fits on the wrist, leaving more freedom of movement than data gloves, optically imaging the whole of the user's hand.

"The Digits sensor doesn’t rely on any external infrastructure so it is completely mobile," says David Kim, a PhD student at Newcastle University.

“This means users are not bound to a fixed space. They can interact while moving from room to room or even running down the street. What Digits does is finally take 3D interaction outside the living room."

Unlike existing motion sensors, the Digits system can 'understand' the human hand, from wrist orientation to the angle of each finger joint, so that interaction isn't limited to 3D points in space. It can even understand what the hand is trying to express while inside a pocket, says the team.

"We needed a system that enabled natural 3-D interactions with bare hands, but with as much flexibility and accuracy as data gloves," says Kim.

"We wanted users to be able to interact spontaneously with their electronic devices using simple gestures without even having to reach for them.  Can you imagine how much easier it would be if you could answer your mobile phone while it’s still in your pocket or buried at the bottom of your bag?"