If you are in doubt whether you should think of multitouch technology as Very Exciting, a Major User Interface improvement, or even a Much Faster Way To Operate Your Computer, read on.

I’ve had some interesting discussions with hard-core gamers about multitouch technology. I learned that they were indeed doubting most of the above. To a large degree I see their point; multitouch is not a proper technology replacement for a mouse or keyboard in the strictest sense. Some technologies overlap existing technologies but don’t replace them - instead, they open a new field of interaction possibilities that were unattainable with the older technologies. In case the overlap is incomplete, certain benefits of the older technology are thus not replicated in the newer.

Put in simple terms: existing applications are designed for mouse and keyboard input. Attempting to operate them with a multitouch screen will likely add very little benefit. In many cases, usability will actually get worse (e.g. Excel, etc.) However, multitouch screens will allow you to do things that you can’t do now. We’ve already seen some examples:

• Two-finger rotating/scaling/moving, as demonstrated for example in the ubiquitous “Multitouch Lightbox” examples (photo albums)
• Three-finger 3D-rotation, as demonstrated in Jeff Han’s Google Earth demonstration
• True on-screen keyboard (allows you to press multiple keys at the same time)

There are also some issues with multitouch screens, which are discussed at length in this paper: Experiences with and Observations of Direct-Touch Tabletops. Some of these issues are typical touch-screen issues (such as occlusion of graphics by fingers/hands/arms), which can be resolved using multitouch gestures (e.g. Digital Tweezers).

We must escape the paradigms of traditional interaction.

Zion control - Matrix Revolutions

Minority Report

The problem is that futuristic presentations of User Interfaces like the above are intended to woo us visually - not necessary convince us of their superior usability.

Multitouch interaction is to a large degree a solution looking for a problem. If we limit ourselves by looking at existing applications, trying to replace series of singletouch operations with multitouch gestures, this technology will never rise to its potential. The trick is to shake off all the UI concepts imposed by Mac, Windows and the like, and to start with a blank screen.

Some benefits were already heralded by Jeff Han, for instance in his presentation at the TED conference:

• Resolution independence - the work surface can be scaled and moved around as easily as the objects on top of it. Specific issues include misplacing objects and having to scale down to find them
• Multi-user - more than one user can operate the application(s) at one time. Specific issues include identifying gestures with specific individuals and detecting how many distinct users are active

To get an idea of which type of current applications might get immediate benefit from multitouch technology and which not, I created the following overview:

Suitable

• Strategy games such as Command & Conquer, Warcraft
• Board games such as Chess, Checkers, Go, Solitaire
• Music- and Sound editing and production
• Computer Aided Design, 3D design
• Graphics and Video editing
• Charting / Mindmapping

Not so suitable

• First-person 3D games (shooters, flight simulators, racing games)
• Office applications (Excel, Word, Outlook)
• Web browsing
• Application development
• Data entry