The Frontiers of Natural User Interface Design

According to Wikipedia, a Natural User Interface is when a UI is effectively invisible, or rather becomes invisible with successive learned interactions. These interactions are intended to feel natural as users operate the user interface design. Natural User Interface represent the third major paradigm in user interface design. The first user interface designs featured Command Line Interfaces which are codified and strictly text-based UIs typified by operating systems such as MS-DOS. Today Command Line Interfaces have mostly given way to Graphical User Interfaces but can still be accessed for scripting or other advanced purposes. Graphical User Interfaces use visual indicators as metaphors rather than text lines and are the way we interact with most modern operating systems today. Graphical User Interfaces allowed users to access computing without the steep learning curve of Command Line Interfaces as users could explore through trial and error. With Natural User Interfaces the learnability of a user interface design is further enhanced by using a UI in ways that feel natural. In the rest of this article I shall talk about the frontiers of Natural User Interface design.

Using Touch/ Gestures to Control a User Interface Design

The iPhone is perhaps the best proponent of using our sense of touch to control a user interface design. Unlike the indirect touch of a mouse using our own fingers in gestures was one of the standout features of the iPhone when it debuted. Pinch-to-zoom and swiping have now become such a common way to manipulate the user interface design of touchscreen devices that it is hard to imagine they didn't exist before the very first iPhone. Using touch to control user interface designs has even graduated from smartphones and tablets to desktop operating systems such as the latest OS X, which implements iOS gestures via a trackpad. The Microsoft Surface is effectively a table-sized tablet that follows the same paradigm.

Using Audio to Control a User Interface Design

Using audio as a way of controlling a user interface design is nothing new. For example, the aforementioned iPhone has a voice control option for accessibility purposes. The Dragon speech recognition software package is perhaps the vanguard of using audio to control user interface designs by allowing users to transcribe speech into text, search or input commands into a user interface design. With the use of audio Dragon software features minimalistic user interface designs as less actual text is needed to express the range of options available to users. The main challenge with audio as a frontier in Natural User Interface design is not the recording of the audio but in making sense of the audio so that users can speak as if they were talking to another person. This is still a very big challenge for NUIs.

Using Visual & Location Information to Control a User Interface Design

Using visual input is another way of creating Natural User Interface designs. Google Goggles is a great example of this whereby users can use a picture as a search entry. Wondering what the fancy building is? Much easier to take a picture and hit search rather than muddling through with various text inputs. Augmented reality apps also make use of all the visual information available around us to provide contextual information. Thanks to GPS technology this can be further enhanced by incorporating location. An upcoming standard feature of the upcoming iOS is the Reminders app which allows users to set reminders not only in a chronological concept but also in terms of location. This means that a reminder can fire off when a user arrives someplace, such as being reminded to go shopping as you leave work.

Natural User Interface designs are here to stay. Using a combination of the aforementioned input types has already bred success in the gaming industry. The Nintendo Wii sold like hot cakes by inviting users to play games using real-life gestures such as swinging a racket or slashing a sword. Microsoft's Kinect took this concept even further by incorporating visual and audio input. The Kinect effectively turns the human body into a controller as users can shout commands and gesture around. Due to its direct and intuitive input nature it is indicative of what can be achieved with Natural User Interfaces. The changes from Command Line Interfaces to Natural User Interfaces represent a shift from us having to be aware of the nature of computers to our computers being more aware of us.