September 5, 2008

Apple Multitouch breakthrough may soon add voice, gaze, biometrics and Facial expression to new devices

by mohsin

If you though Apple’s multitouch screen was cool, perhaps it’s time you think again, with its competitors struggling to catch up with multi-touch technology introduced last year as part of the iPhone, Apple is already conceptualizing new versions of the technology that would fuse a variety of secondary inputs with today’s touch-based gestures to produce more efficient data input operations. According to a new patent filed by Apple, it seems that the company is keen on integrating the multitouch with Voice, Finger Identification / Biometrics, Gaze Vector and Facial Expression.

The patent application gives examples of how each of these could be used in conjunction with multi-touch to provide a better user experience. A few highlights of this patent
Voice – Some tasks are described as being better suited for either voice recognition vs. multi-touch. For example, if a user’s task is to resize, rotate and change the color of an object on the screen, multi-touch would be best suited for the resize/rotation tasks but changing of color (or inserting text) may be better suited to a voice command.
Finger identification – the article suggests that using built in cameras (such as the iSight) with a swing mirror could provide an over the keyboard view of multi-touch gestures. This video information could be used to better distinguish which fingers are being used in which position. In conjunction with the touch input, this could be used to create more specific and accurate gestures.
Gaze – tracking where the user is looking could help pick windows or objects on the screen. Rather than resorting to moving a mouse pointer to the proper window, a user could simply direct his gaze at the particular window and then invoke a touch gesture.
Facial expression – detecting frustration on the face of a user could provide help prompts or even alter input behavior. The example given is if a user is incorrectly trying to scroll a window using 3 fingers, instead of 2, the computer may be able to detect the frustration and either accept the faulty input or prompt the user.
Many of these technologies are likely years away from the market, but continue to provide an interesting peek into Apple’s future.

Source