Pen computing

Last updated
Samsung Galaxy Note phablets include a stylus, called "S-Pen". Samsung Galaxy Note 10 (S Pen).jpg
Samsung Galaxy Note phablets include a stylus, called "S-Pen".

Pen computing refers to any computer user-interface using a pen or stylus and tablet, over input devices such as a keyboard or a mouse.

Contents

Pen computing is also used to refer to the usage of mobile devices such as tablet computers, PDAs and GPS receivers. The term has been used to refer to the usage of any product allowing for mobile communication. An indication of such a device is a stylus or digital pen, generally used to press upon a graphics tablet or touchscreen, as opposed to using a more traditional interface such as a keyboard, keypad, mouse or touchpad.

Historically, pen computing (defined as a computer system employing a user-interface using a pointing device plus handwriting recognition as the primary means for interactive user input) predates the use of a mouse and graphical display by at least two decades, starting with the Stylator [1] and RAND Tablet [2] systems of the 1950s and early 1960s.

General techniques

User interfaces for pen computing can be implemented in several ways. Actual systems generally employ a combination of these techniques.

Pointing/locator input

The tablet and stylus are used as pointing devices, such as to replace a mouse. While a mouse is a relative pointing device (one uses the mouse to "push the cursor around" on a screen), a tablet is an absolute pointing device (one places the stylus where the cursor is to appear).

There are a number of human factors to be considered when actually substituting a stylus and tablet for a mouse. For example, it is much harder to target or tap the same exact position twice with a stylus, so "double-tap" operations with a stylus are harder to perform if the system is expecting "double-click" input from a mouse.

A finger can be used as the stylus on a touch-sensitive tablet surface, such as with a touchscreen.

Handwriting recognition

The tablet and stylus can be used to replace a keyboard, or both a mouse and a keyboard, by using the tablet and stylus in two modes:

Different systems switch between the modes (pointing vs. handwriting recognition) by different means, e.g.

The term "on-line handwriting recognition" is used to distinguish recognition of handwriting using a real-time digitizing tablet for input, as contrasted to "off-line handwriting recognition", which is optical character recognition of static handwritten symbols from paper.

Direct manipulation

The stylus is used to touch, press, and drag on simulated objects directly. The Wang Freestyle system [3] is one example. Freestyle worked entirely by direct manipulation, with the addition of electronic "ink" for adding handwritten notes.

Gesture recognition

This is the technique of recognizing certain special shapes not as handwriting input, but as an indicator of a special command.

For example, a "pig-tail" shape (used often as a proofreader's mark) would indicate a "delete" operation. Depending on the implementation, what is deleted might be the object or text where the mark was made, or the stylus can be used as a pointing device to select what it is that should be deleted. With Apple's Newton OS, text could be deleted by scratching in a zig-zag pattern over it.

Recent systems have used digitizers which can recognize more than one "stylus" (usually a finger) at a time, and make use of Multi-touch gestures.

The PenPoint OS was a special operating system which incorporated gesture recognition and handwriting input at all levels of the operating system. Prior systems which employed gesture recognition only did so within special applications, such as CAD/CAM applications [4] [5] or text processing. [6]

History

Pen computing has very deep historical roots. For example, the first patent for an electronic device used for handwriting, the telautograph, was granted in 1888. [7] What is probably the first patent for a system that recognized handwritten characters by analyzing the handwriting motion was granted in 1915. [8] Around 1954 Douglas T Ross, working on the Whirlwind computer at MIT, wrote the "first hand-drawn graphics input program to a computer". [9] The first publicly demonstrated system using a tablet and handwriting text recognition instead of a keyboard for working with a modern digital computer dates to 1956. [10]

In addition to many academic and research systems, there were several companies with commercial products in the 1980s: Pencept, Communications Intelligence Corporation, and Linus were among the best known of a crowded field. Later, GO Corp. brought out the PenPoint OS operating system for a tablet PC product: one of the patents from GO corporation was the subject of recent infringement lawsuit concerning the Tablet PC operating system. [11]

The following timeline list gives some of the highlights of this history:

See also

Related Research Articles

<span class="mw-page-title-main">Pointing device gesture</span>

In computing, a pointing device gesture or mouse gesture is a way of combining pointing device or finger movements and clicks that the software recognizes as a specific computer event and responds to accordingly. They can be useful for people who have difficulties typing on a keyboard. For example, in a web browser, a user can navigate to the previously viewed page by pressing the right pointing device button, moving the pointing device briefly to the left, then releasing the button.

<span class="mw-page-title-main">Pointing device</span> Human interface device for computers

A pointing device is a human interface device that allows a user to input spatial data to a computer. CAD systems and graphical user interfaces (GUI) allow the user to control and provide data to the computer using physical gestures by moving a hand-held mouse or similar device across the surface of the physical desktop and activating switches on the mouse. Movements of the pointing device are echoed on the screen by movements of the pointer and other visual changes. Common gestures are point and click and drag and drop.

<span class="mw-page-title-main">Graphics tablet</span> Computer input device

A graphics tablet is a computer input device that enables a user to hand-draw images, animations and graphics, with a special pen-like stylus, similar to the way a person draws images with a pencil and paper. These tablets may also be used to capture data or handwritten signatures. It can also be used to trace an image from a piece of paper that is taped or otherwise secured to the tablet surface. Capturing data in this way, by tracing or entering the corners of linear polylines or shapes, is called digitizing.

<span class="mw-page-title-main">Handwriting recognition</span> Ability of a computer to receive and interpret intelligible handwritten input

Handwriting recognition (HWR), also known as handwritten text recognition (HTR), is the ability of a computer to receive and interpret intelligible handwritten input from sources such as paper documents, photographs, touch-screens and other devices. The image of the written text may be sensed "off line" from a piece of paper by optical scanning or intelligent word recognition. Alternatively, the movements of the pen tip may be sensed "on line", for example by a pen-based computer screen surface, a generally easier task as there are more clues available. A handwriting recognition system handles formatting, performs correct segmentation into characters, and finds the most possible words.

The PenPoint OS was a product of GO Corporation and was one of the earliest operating systems written specifically for graphical tablets and personal digital assistants. It ran on AT&T Corporation's EO Personal Communicator as well as a number of Intel x86 powered tablet PCs including IBM's ThinkPad 700T series, NCR's 3125, 3130 and some of GRiD Systems' pen-based portables. It was never widely adopted.

<span class="mw-page-title-main">Windows for Pen Computing</span> Software suite for Windows 3.1 for pen compuing

Windows for Pen Computing is a software suite for Windows 3.1x, that Microsoft designed to incorporate pen computing capabilities into the Windows operating environment. Windows for Pen Computing was the second major pen computing platform for x86 tablet PCs; GO Corporation released their operating system, PenPoint OS, shortly before Microsoft published Windows for Pen Computing 1.0 in 1992.

<span class="mw-page-title-main">Digital pen</span> Input device

A digital pen is an input device which captures the handwriting or brush strokes of a user and converts handwritten analog information created using "pen and paper" into digital data, enabling the data to be utilized in various applications. This type of pen is usually used in conjunction with a digital notebook, although the data can also be used for different applications or simply as a graphic.

<span class="mw-page-title-main">Tablet computer</span> Mobile computer with integrated display, circuitry and battery

A tablet computer, commonly shortened to tablet, is a mobile device, typically with a mobile operating system and touchscreen display processing circuitry, and a rechargeable battery in a single, thin and flat package. Tablets, being computers, have similar capabilities, but lack some input/output (I/O) abilities that others have. Modern tablets largely resemble modern smartphones, the only differences being that tablets are relatively larger than smartphones, with screens 7 inches (18 cm) or larger, measured diagonally, and may not support access to a cellular network. Unlike laptops, tablets usually run mobile operating systems, alongside smartphones.

<span class="mw-page-title-main">Ultra-mobile PC</span> Obsolete type of handheld computer

An ultra-mobile PC, or ultra-mobile personal computer (UMPC), is a miniature version of a pen computer, a class of laptop whose specifications were launched by Microsoft and Intel in Spring 2006. Sony had already made a first attempt in this direction in 2004 with its Vaio U series, which was only sold in Asia. UMPCs are generally smaller than subnotebooks, have a TFT display measuring (diagonally) about 12.7 to 17.8 centimetres, are operated like tablet PCs using a touchscreen or a stylus, and can also have a physical keyboard. There is no clear boundary between subnotebooks and ultra-mobile PCs, but UMPCs commonly have major features not found in the common clamshell laptop design, such as small keys on either side of the screen, or a slide-out keyboard.

The term PenPad was used as a product name for a number of Pen computing products by different companies in the 1980s and 1990s. The earliest was the Penpad series of products by Pencept, such as the PenPad M200 handwriting terminal, and the PenPad M320 handwriting/gesture recognition tablet for MS-DOS and other personal computers.

<span class="mw-page-title-main">Multi-touch</span> Technology

In computing, multi-touch is technology that enables a surface to recognize the presence of more than one point of contact with the surface at the same time. The origins of multitouch began at CERN, MIT, University of Toronto, Carnegie Mellon University and Bell Labs in the 1970s. CERN started using multi-touch screens as early as 1976 for the controls of the Super Proton Synchrotron. A form of gesture recognition, capacitive multi-touch displays were popularized by Apple's iPhone in 2007. Plural-point awareness may be used to implement additional functionality, such as pinch to zoom or to activate certain subroutines attached to predefined gestures.

A text entry interface or text entry device is an interface that is used to enter text information in an electronic device. A commonly used device is a mechanical computer keyboard. Most laptop computers have an integrated mechanical keyboard, and desktop computers are usually operated primarily using a keyboard and mouse. Devices such as smartphones and tablets mean that interfaces such as virtual keyboards and voice recognition are becoming more popular as text entry systems.

<span class="mw-page-title-main">Input device</span> Device that provides data and signals to a computer

In computing, an input device is a piece of equipment used to provide data and control signals to an information processing system, such as a computer or information appliance. Examples of input devices include keyboards, mouse, scanners, cameras, joysticks, and microphones.

<span class="mw-page-title-main">Stylus (computing)</span> Pen-shaped instrument used as a human-computer interface

In computing, a stylus is a small pen-shaped instrument whose tip position on a computer monitor can be detected. It is used to draw, or make selections by tapping. While devices with touchscreens such as newer computers, mobile devices, game consoles, and graphics tablets can usually be operated with a fingertip, a stylus provides more accurate and controllable input. The stylus has the same function as a mouse or touchpad as a pointing device; its use is commonly called pen computing.

The history of tablet computers and the associated special operating software is an example of pen computing technology, and thus the development of tablets has deep historical roots. The first patent for a system that recognized handwritten characters by analyzing the handwriting motion was granted in 1914. The first publicly demonstrated system using a tablet and handwriting recognition instead of a keyboard for working with a modern digital computer dates to 1956.

<span class="mw-page-title-main">Microsoft Tablet PC</span> Microsoft

Microsoft Tablet PC is a term coined by Microsoft for tablet computers conforming to a set of specifications announced in 2001 by Microsoft, for a pen-enabled personal computer, conforming to hardware specifications devised by Microsoft and running a licensed copy of Windows XP Tablet PC Edition operating system or a derivative thereof.

<span class="mw-page-title-main">2-in-1 laptop</span> Mobile device combining laptop and tablet characteristics

A 2-in-1 laptop, also known as 2-in-1 PC, 2-in-1 tablet, laplet, tabtop, laptop tablet, or simply 2-in-1, is a portable computer that has features of both tablets and laptops.

<span class="mw-page-title-main">Samsung Galaxy Note (2011 smartphone)</span> Android smartphone by Samsung

The Samsung Galaxy Note is an Android smartphone produced by Samsung Electronics. Unveiled at IFA Berlin 2011, it was first released in Germany in late October 2011, with other countries following afterwards. The Galaxy Note was distinguished by its unusually large form factor—later referred to using the term "phablet"—which straddled the size of the average smartphone at the time, and that of a small tablet: it features a 5.3-inch display, and is bundled with a stylus branded as the "S Pen", which can be used to navigate the device's user interface, and write or draw in supported apps.

<span class="mw-page-title-main">S Pen</span> Line of stylus pens designed by Samsung Electronics

S Pen (Korean: S펜) is a wireless digital pen stylus designed and developed by Samsung Electronics which uses Wacom pen technology. It is made for use with supported Galaxy mobile devices like smartphones and tablets, as well as selected Samsung Notebook, Samsung Galaxy Book, and Chromebook notebooks. It was first released with the Samsung Galaxy Note in 2011, becoming the core of the Note line of products.

<span class="mw-page-title-main">Active pen</span> Type of input device

An active pen is an input device that includes electronic components and allows users to write directly onto the display of a computing device such as a smartphone, tablet computer or ultrabook. The active pen marketplace has long been dominated by N-trig and Wacom, but newer firms Atmel and Synaptics also offer active pen designs.

References

  1. Dimond, T.L. (1957-12-01), Devices for reading handwritten characters, Proceedings of Eastern Joint Computer Conference, pp. 232–237
  2. Groner, G.F. (August 1966), Real-Time Recognition of Handprinted Text, Memorandum RM-5016-ARPA, RAND Corporation
  3. WANG Freestyle demo, Wang Laboratories, 1989, retrieved 2008-09-22
  4. Computerized Graphic Processing System: System User's Manual, Applicon Incorporated, 1973-09-01
  5. Newman, W.M. (1973-09-01), The Ledeen Character Recognizer, Principles of Interactive Computer Graphics, McGraw-Hill, pp. 575–582
  6. Coleman, Michael L. (1969), Text editing on a graphic display device using hand-drawn proofreader's symbols, from Pertinent Concepts in Computer Graphics: Proceedings of the 2nd University of Illinois Conference on Computer Graphics, University of Illinois Press
  7. 1 2 Gray, Elisha (1888-07-31), Telautograph (PDF), United States Patent 386,815 (full image)
  8. 1 2 Goldberg, H.E. (1915-12-28), Controller (PDF), United States Patent 1,117,184 (full image)
  9. Ross, Doug (1989), Retrospectives 1: The early years in computer graphics (PDF), SIGGRAPH 89 Proceedings, pp. 27–28
  10. 1 2 Dimond, Tom (1957-12-01), Devices for reading handwritten characters, Proceedings of Eastern Joint Computer Conference, pp. 232–237, retrieved 2008-08-23
  11. 1 2 Mintz, Jessica (2008-04-04), Microsoft to Appeal $367M Patent Ruling, The Associated Press, retrieved 2008-09-04
  12. Gray (1888-07-31), Telautograph, United States Patent 386,815
  13. Goldberg, H.E. (1915-12-28), Controller, United States Patent 1,117,184
  14. Moodey, H.C. (1942-12-27), Telautograph System, United States Patent 2,269,599
  15. Moodey, H.C. (1942-12-27), Telautograph System (PDF), United States Patent 2,269,599 (full image)
  16. Bush, Vannevar (1945-07-15), As We May Think, The Atlantic Monthly
  17. RAND Tablet, 1961-09-01
  18. 50 Years of Looking Forward, RAND Corporation, 1998-09-01, archived from the original on 2009-05-07
  19. Fryberger, D. (March 1972), An Innovation in Control Panels for Large Computer Control Systems, Proc. 4th IEEE Particle Accelerator Conference, Chicago IL USA, March 1..3, 1971, pp. 414–417
  20. Pencept Penpad (TM) 200 Product Literature, Pencept, Inc., 1982-08-15
  21. Inforite Hand Character Recognition Terminal, Cadre Systems Limited, England, 1982-08-15
  22. Users Manual for Penpad 320, Pencept, Inc., 1984-06-15
  23. Software Control at the Stroke of a Pen, Pencept, Inc., 1985, retrieved 2009-05-21
  24. Handwriter (R) GrafText (TM) System Model GT-5000, Communication Intelligence Corporation, 1985-01-15
  25. The BYTE Awards: GRiD System's GRiDPad, BYTE Magazine, Vol 15. No 1, 1990-01-12, p. 285
  26. Wang Laboratories (1989), The Wang Freestyle System, WANG Laboratories
  27. Lempesis, Bill (May 1990), What's New in Laptops and Pen Computing, Flat Panel Display News
  28. "Momenta Corporation 1/40 Pentop Computer - RICM". sites.google.com. Retrieved 2020-06-04.
  29. Agulnick, Todd (1994-09-13), Control of a computer through a position-sensed stylus, United States Patent 5,347,295
  30. Agulnick, Todd (1994-09-13), Control of a computer through a position-sensed stylus (PDF), United States Patent 5,347,295 (full image)
  31. NCR announces pen-based computer press release, archived from the original on 2008-05-02, retrieved 2007-04-20
  32. Penpoint OS shipping press release, archived from the original on 2007-08-30, retrieved 2007-04-20
  33. "The history of ThinkPad". Archived from the original on 2007-04-29.
  34. Amstrad PenPad PDA600. Computing History (2011-10-21). Retrieved on 2013-12-09.
  35. Trends at COMDEX Event 1999 , retrieved 2008-08-11
  36. Microsoft (2005), Windows XP Tablet PC Edition 2005 Hardware Requirements, www.microsoft.com, retrieved 2009-03-14
  37. "Annual Report 2009" (PDF). Wacom.
  38. Fingerworks, Inc. (2003), iGesture Game Mode Guide, www.fingerworks.com, retrieved 2009-04-30
  39. MacEssentials (2007-08-02), Rubrik Apple: Das Lexikon der Fingersprache, www.mac-essentials.de, retrieved 2009-05-16
  40. Druin, Allison (2009-03-16). Mobile Technology for Children: Designing for Interaction and Learning. Morgan Kaufmann. p. 177. ISBN   978-0-08-095409-7.
  41. Fried, Ina (July 6, 2005). "Go files antitrust suit against Microsoft". CNET.
  42. A New Antitrust Lawsuit - Go Corp. v. Microsoft. Groklaw (2005-07-04). Retrieved on 2013-12-09.
  43. HP TouchSmart tx2z, HP, retrieved 2008-11-28