Gloves promise gesture-based computing on the cheap

Posted by Staff writer

They may look like something a clown would wear, but a pair of multi-coloured gloves could soon be de rigeur for gamers.

MIT researchers say the gloves - which could cost just a dollar to make - could make gestural interfaces much more practical.

Other prototypes of low-cost gestural interfaces have used reflective or colored tape attached to the fingertips, but "that’s 2-D information," says Robert Wang of the Computer Science and Artificial Intelligence Laboratory.

"You’re only getting the fingertips; you don’t even know which fingertip [the tape] is corresponding to."

The MIT gloves, co-created by Jovan Popović, an associate professor of electrical engineering and computer science, can translate gestures made with a gloved hand into the corresponding gestures of a 3-D model of the hand on screen, with almost no lag time.

"This actually gets the 3-D configuration of your hand and your fingers," Wang says. "We get how your fingers are flexing."

The most obvious application of the technology is in video games, allowing players to pick up and wield objects simply by using hand gestures.

But Wang also envisages its use by engineers and designers, to  manipulate 3-D models of commercial products or large civic structures.

The design wasn't produced for its beauty. The number of colors had to be restricted so that the system could reliably distinguish them under a range of different lighting conditions.

The arrangement and shapes of the patches were chosen so that the front and back of the hand would be distinct, and also so that collisions of similar-colored patches would be rare.

For instance, the colors on the tips of the fingers could be repeated on the back of the hand, but not on the front, since the fingers would frequently be flexing and closing in front of the palm.

The other ingredient of the system is a new algorithm for rapidly looking up visual data in a database. Once a webcam has captured an image of the glove, Wang’s software crops out the background, so that the glove alone is superimposed upon a white background.

Then the software drastically reduces the resolution of the cropped image, to only 40 pixels by 40 pixels. Finally, it searches through a database containing myriad 40-by-40 digital models of a hand, clad in the distinctive glove, in a range of different positions.

Once it’s found a match, it simply looks up the corresponding hand position. Since the system doesn’t have to calculate the relative positions of the fingers, palm, and back of the hand on the fly, it’s able to provide an answer in a fraction of a second.

Wang next plans to produce a shirt using the same system.