Surface types
Surface computing employs the use of two broad categories of surface types, flat and non-flat. The distinction is made not only due to the physical dimensions of the surfaces, but also the methods of interaction.
Flat
Flat surface types refer to two-dimensional surfaces such as tabletops. This is the most common form of surface computing in the commercial space as seen by products like Microsoft's PixelSense and iTable. The aforementioned commercial products utilize a multi-touch LCD screen as a display, but other implementations use projectors. Part of the appeal of two-dimensional surface computing is the ease and reliability of interaction. Since the advent of tablet computing, a set of intuitive gestural interactions have been developed to complement two-dimensional surfaces. However, the two-dimensional plane limits the range of interactions a user is able to perform. Furthermore, interactions are only detected when making direct contact with the surface. In order to afford the user a wider range of interaction, research has been done to augment the interaction schemes for two-dimensional surfaces. This research involves using the space above the screen as another dimension for interaction, so, for example, the height of a user's hands above the surface becomes a meaningful distinction for interaction. This particular system would qualify as a hybrid that uses a flat surface, but a three-dimensional space for interaction. [7]
Non-flat
While most work with surface computing has been done with flat surfaces, non-flat surfaces have become an interest with researchers. The eventual goal of surface computing itself is tied to the notion of ubiquitous computing "where everyday surfaces in our environment are made interactive". [8] These everyday surfaces are often non-flat, so researchers have begun exploring curved and three-dimensional modes. Some of these include spherical, cylindrical and parabolic surfaces. Including a third dimension to surface computing presents both benefits and challenges. One of these benefits is an extra dimension of interaction. Unlike flat surfaces, three dimensional surfaces allow for a sense of depth and are thus classified as "depth-aware" surfaces. This allows for more diverse gestural interactions. However, one of the main challenges is designing intuitive gestural actions to facilitate interaction with these non-flat surfaces. Furthermore, three-dimensional shapes such as spheres and cylinders require viewing from all angles, also known as omnidirectional displays. Designing compelling views from every angle is a difficult task, as is designing applications that make sense for these display types. [8]
Interaction methods
Various methods of interaction exist in surface computing. The most common method of which is touch based, this includes single and multi-touch interactions. Other interactions exist such as freehand 3D interactions that depth-aware cameras can sense.
•Two Dimensional Typically, traditional surface types are two-dimensional and only require two-dimensional touch interactions. Depending on the system, multi-touch gestures, such as pinch to zoom, are supported. These gestures allow the user to manipulate what they see on the surface by physically touching it and moving their fingers across the surface. For sufficiently large surfaces, multi-touch gestures can extend to both hands, and even multiple sets of hands in multi-user applications.
•Three Dimensional Using depth aware cameras it is possible to make three dimensional gestures. Such gestures allow the user to move in three dimensions of space without having to come into contact with the surface itself, such as the methods used in Depth perception. [8] DepthTouch makes use of a depth-sensing camera, a projector, desktop computer, and a vertical screen for the user to interact with. Instead of physically touching the screen, the user can manipulate the objects he or she sees displayed onto it by making freehand gestures in mid-air. The depth-aware camera can then detect the user's gestures and the computer processes them to show what the user is doing on the display.
A touchscreen or touch screen is the assembly of both an input and output ('display') device. The touch panel is normally layered on the top of an electronic visual display of an information processing system. The display is often an LCD, AMOLED or OLED display while the system is usually a laptop, tablet, or smartphone. A user can give input or control the information processing system through simple or multi-touch gestures by touching the screen with a special stylus or one or more fingers. Some touchscreens use ordinary or specially coated gloves to work while others may only work using a special stylus or pen. The user can use the touchscreen to react to what is displayed and, if the software allows, to control how it is displayed; for example, zooming to increase the text size.
A handheld projector is an image projector in a handheld device. It was developed as a computer display device for compact portable devices such as mobile phones, personal digital assistants, and digital cameras, which have sufficient storage capacity to handle presentation materials but are too small to accommodate a display screen that an audience can see easily. Handheld projectors involve miniaturized hardware, and software that can project digital images onto a nearby viewing surface.
Gesture recognition is a topic in computer science and language technology with the goal of interpreting human gestures via mathematical algorithms. It is a subdiscipline of computer vision. Gestures can originate from any bodily motion or state, but commonly originate from the face or hand. Current focuses in the field include emotion recognition from face and hand gesture recognition. Users can use simple gestures to control or interact with devices without physically touching them. Many approaches have been made using cameras and computer vision algorithms to interpret sign language. However, the identification and recognition of posture, gait, proxemics, and human behaviors is also the subject of gesture recognition techniques. Gesture recognition can be seen as a way for computers to begin to understand human body language, thus building a better bridge between machines and humans than older text user interfaces or even GUIs, which still limit the majority of input to keyboard and mouse and interact naturally without any mechanical devices.
A tangible user interface (TUI) is a user interface in which a person interacts with digital information through the physical environment. The initial name was Graspable User Interface, which is no longer used. The purpose of TUI development is to empower collaboration, learning, and design by giving physical forms to digital information, thus taking advantage of the human ability to grasp and manipulate physical objects and materials.
An interactive whiteboard (IWB), also known as interactive board or smart board, is a large interactive display board in the form factor of a whiteboard. It can either be a standalone touchscreen computer used independently to perform tasks and operations, or a connectable apparatus used as a touchpad to control computers from a projector. They are used in a variety of settings, including classrooms at all levels of education, in corporate board rooms and work groups, in training rooms for professional sports coaching, in broadcasting studios, and others.
In computing, multi-touch is technology that enables a surface to recognize the presence of more than one point of contact with the surface at the same time. The origins of multitouch began at CERN, MIT, University of Toronto, Carnegie Mellon University and Bell Labs in the 1970s. CERN started using multi-touch screens as early as 1976 for the controls of the Super Proton Synchrotron. Capacitive multi-touch displays were popularized by Apple's iPhone in 2007. Plural-point awareness may be used to implement additional functionality, such as pinch to zoom or to activate certain subroutines attached to predefined gestures.
Pen computing refers to any computer user-interface using a pen or stylus and tablet, over input devices such as a keyboard or a mouse.
Microsoft PixelSense was an interactive surface computing platform that allowed one or more people to use and touch real-world objects, and share digital content at the same time. The PixelSense platform consists of software and hardware products that combine vision based multitouch PC hardware, 360-degree multiuser application design, and Windows software to create a natural user interface (NUI).
A surface computer is a computer that interacts with the user through the surface of an ordinary object, rather than through a monitor, keyboard, mouse, or other physical hardware.
In human–computer interaction, an organic user interface (OUI) is defined as a user interface with a non-flat display. After Engelbart and Sutherland's graphical user interface (GUI), which was based on the cathode ray tube (CRT), and Kay and Weiser's ubiquitous computing, which is based on the flat panel liquid-crystal display (LCD), OUI represents one possible third wave of display interaction paradigms, pertaining to multi-shaped and flexible displays. In an OUI, the display surface is always the focus of interaction, and may actively or passively change shape upon analog inputs. These inputs are provided through direct physical gestures, rather than through indirect point-and-click control. Note that the term "Organic" in OUI was derived from organic architecture, referring to the adoption of natural form to design a better fit with human ecology. The term also alludes to the use of organic electronics for this purpose.
Hands-on computing is a branch of human-computer interaction research which focuses on computer interfaces that respond to human touch or expression, allowing the machine and the user to interact physically. Hands-on computing can make complicated computer tasks more natural to users by attempting to respond to motions and interactions that are natural to human behavior. Thus hands-on computing is a component of user-centered design, focusing on how users physically respond to virtual environments.
In computing, 3D interaction is a form of human-machine interaction where users are able to move and perform interaction in 3D space. Both human and machine process information where the physical position of elements in the 3D space is relevant.
In computing, a natural user interface (NUI) or natural interface is a user interface that is effectively invisible, and remains invisible as the user continuously learns increasingly complex interactions. The word "natural" is used because most computer interfaces use artificial control devices whose operation has to be learned. Examples include voice assistants, such as Alexa and Siri, touch and multitouch interactions on today's mobile phones and tablets, but also touch interfaces invisibly integrated into the textiles furnitures.
In computing, an input device is a piece of equipment used to provide data and control signals to an information processing system, such as a computer or information appliance. Examples of input devices include keyboards, mouse, scanners, cameras, joysticks, and microphones.
The DiamondTouch table is a multi-touch, interactive PC interface product from Circle Twelve Inc. It is a human interface device that has the capability of allowing multiple people to interact simultaneously while identifying which person is touching where. The technology was originally developed at Mitsubishi Electric Research Laboratories (MERL) in 2001 and later licensed to Circle Twelve Inc in 2008. The DiamondTouch table is used to facilitate face-to-face collaboration, brainstorming, and decision-making, and users include construction management company Parsons Brinckerhoff, the Methodist Hospital, and the US National Geospatial-Intelligence Agency (NGA).
A holographic screen is a two-dimensional display technology that uses coated glass media for the projection surface of a video projector. "Holographic" refers not to a stereoscopic effect, but to the coating that bundles light using formed microlenses. The lens design and attributes match the holographic area. The lenses may appear similar to the fresnel lenses used in overhead projectors. The resulting effect is that of a free-space display, because the image carrier appears very transparent. Additionally, the beam manipulation by the lenses can be used to make the image appear to be floating in front of or behind the glass, rather than directly on it. However, this display is only two-dimensional and not true three-dimensional. It is unclear if such a technology will be able to provide acceptable three-dimensional images in the future.
A virtual touch screen (VTS) is a user interface system that augments virtual objects into reality either through a projector or optical display using sensors to track a person's interaction with the object. For instance, using a display and a rear projector system a person could create images that look three-dimensional and appear to float in midair. Some systems utilize an optical head-mounted display to augment the virtual objects onto the transparent display utilizing sensors to determine visual and physical interactions with the virtual objects projected.
PrimeSense was an Israeli 3D sensing company based in Tel Aviv. PrimeSense had offices in Israel, North America, Japan, Singapore, Korea, China and Taiwan. PrimeSense was bought by Apple Inc. for $360 million on November 24, 2013.
Microsoft Tablet PC is a term coined by Microsoft for tablet computers conforming to a set of specifications announced in 2001 by Microsoft, for a pen-enabled personal computer, conforming to hardware specifications devised by Microsoft and running a licensed copy of Windows XP Tablet PC Edition operating system or a derivative thereof.
Project Digits is a Microsoft Research Project under Microsoft's computer science laboratory at the University of Cambridge; researchers from Newcastle University and University of Crete are also involved in this project. Project is led by David Kim a Microsoft Research PhD and also a PhD Student in computer science at Newcastle University. Digits is an input device which can be mounted on the wrist of human hand and it captures and displays a complete 3D graphical representation of the user's hand on screen without using any external sensing device or hand covering material like data gloves. This project aims to make gesture controlled interfaces completely hands free with greater mobility and accuracy. It allows user to interact with whatever hardware while moving from room to room or walking down the street without any line of sight connection with the hardware.
