Optical mouse

Last updated
A Microsoft wireless optical mouse Microsoft-wireless-mouse.jpg
A Microsoft wireless optical mouse

An optical mouse is a computer mouse which uses a miniature camera and digital image processing to detect movement relative to a surface. Variations of the optical mouse have largely replaced the older mechanical mouse and its need for frequent cleaning.

Contents

The earliest optical mice detected movement on prepared surfaces, however they never gained wide acceptance. The modern optical mouse which uses digital image correlation and which works on almost any surface was invented in 2000 by Gary Gordon, Derek Knee, Rajeev Badyal and Jason Hartlove, and awarded US Patent 6,433,780. [1] Its technology is explained in an interview with one of its inventors made by the Computer History Museum.

Mechanical mice

Though not commonly referred to as optical mice, nearly all mechanical mice tracked movement using LEDs and photodiodes to detect when beams of infrared light did and didn't pass through holes in a pair of incremental rotary encoder wheels (one for left/right, another for forward/back), driven by a rubberized ball. Thus, the primary distinction of “optical mice” is not their use of optics, but their complete lack of moving parts to track mouse movement, instead employing an entirely solid-state system.

Early optical mice

An early Xerox optical mouse chip, before the development of the inverted packaging design of Williams and Cherry Xerox Optical Mouse Chip.jpg
An early Xerox optical mouse chip, before the development of the inverted packaging design of Williams and Cherry

The first two optical mice, first demonstrated by two independent inventors in December 1980, had different basic designs: [2] [3] [4] One of these, invented by Steve Kirsch of MIT and Mouse Systems Corporation, [5] [6] used an infrared LED and a four-quadrant infrared sensor to detect grid lines printed with infrared absorbing ink on a special metallic surface. Predictive algorithms in the CPU of the mouse calculated the speed and direction over the grid. The other type, invented by Richard F. Lyon of Xerox, used a 16-pixel visible-light image sensor with integrated motion detection on the same ntype (5 μm) MOS integrated circuit chip, [7] [8] and tracked the motion of light dots in a dark field of a printed paper or similar mouse pad. [9] The Kirsch and Lyon mouse types had very different behaviors, as the Kirsch mouse used an x-y coordinate system embedded in the pad, and would not work correctly when the pad was rotated, while the Lyon mouse used the x-y coordinate system of the mouse body, as mechanical mice do.

OEM-branded Mouse Systems (Kirsch) optical mouse showing underside and mousemat pattern Old optical mouse.jpeg
OEM-branded Mouse Systems (Kirsch) optical mouse showing underside and mousemat pattern

The optical mouse ultimately sold with the Xerox STAR office computer used an inverted sensor chip packaging approach patented by Lisa M. Williams and Robert S. Cherry of the Xerox Microelectronics Center. [10]

The Mouse Systems (Kirsch) design was commercialised and sold in PC compatible form by the company itself [11] alongside variants rebranded for OEM use with Sun Microsystems workstations [12] and by Data General. [13]

Modern optical mice

Optical sensor

Microscope photograph of the IntelliMouse Explorer sensor silicon die ST Microelectronics OSMLT04 H mouse sensor chip with vertical and horizontal illumination.jpg
Microscope photograph of the IntelliMouse Explorer sensor silicon die
The optical sensor from a Microsoft Wireless IntelliMouse Explorer (v. 1.0A) Opto mouse sensor.jpg
The optical sensor from a Microsoft Wireless IntelliMouse Explorer (v. 1.0A)

Modern surface-independent optical mice work by using an optoelectronic sensor (essentially, a tiny low-resolution video camera) to take successive images of the surface on which the mouse operates. As computing power grew cheaper, it became possible to embed more powerful special-purpose image-processing chips in the mouse itself. This advance enabled the mouse to detect relative motion on a wide variety of surfaces, translating the movement of the mouse into the movement of the cursor and eliminating the need for a special mouse-pad. A surface-independent coherent light optical mouse design was patented by Stephen B. Jackson at Xerox in 1988, despite never being shown to work. [14]

Xerox's inventions were never massively commercially exploited, however, and optical mice would remain elusive in the personal computer market until Microsoft released the IntelliMouse with IntelliEye and IntelliMouse Explorer in 1999. [15] [16] These mice used technology developed by Hewlett-Packard under their Agilent Technologies subsidiary (see below). These mice worked on almost any surface, and represented a welcome improvement over mechanical mice, which would pick up dirt, track capriciously, invite rough handling, and need to be taken apart and cleaned frequently. Other manufacturers soon followed Microsoft's lead, including Apple for their Pro Mouse, [15] using components manufactured by Agilent (once they spun off from HP), and over the next several years mechanical mice became obsolete.

S5085 optical sensor IC die (CMOS sensor + driver) S5085 optical mouse IC.jpg
S5085 optical sensor IC die (CMOS sensor + driver)

The technology underlying the modern optical computer mouse is known as digital image correlation, a technology pioneered by the defense industry for tracking military targets. A simple binary-image version of digital image correlation was used in the 1980 Lyon optical mouse. Optical mice use image sensors to image naturally occurring texture in materials such as wood, cloth, mouse pads and Formica. These surfaces, when lit at a grazing angle by a light emitting diode, cast distinct shadows that resemble a hilly terrain lit at sunset. Images of these surfaces are captured in continuous succession and compared with each other to determine how far the mouse has moved.

Principle of operation

To understand how optical flow is used in optical mice, imagine two photographs of the same object except slightly offset from each other. Place both photographs on a light table to make them transparent, and slide one across the other until their images line up. The amount that the edges of one photograph overhang the other represents the offset between the images, and in the case of an optical computer mouse the distance it has moved.

Optical mice capture one thousand successive images or more per second. Depending on how fast the mouse is moving, each image will be offset from the previous one by a fraction of a pixel or as many as several pixels. Optical mice mathematically process these images using cross correlation to calculate how much each successive image is offset from the previous one.[ citation needed ] The output of the optical sensor is usually delta X, Y coordinates. Some optical ICs allow to get image data as well. Mice usually embeds some kind of Image Acquisition System and DSP processors for fast data processing.

An optical mouse might use an image sensor having an 18 × 18 pixel array of monochromatic pixels. Its sensor would normally share the same ASIC as that used for storing and processing the images. One refinement would be accelerating the correlation process by using information from previous motions, and another refinement would be preventing deadbands when moving slowly by adding interpolation or frame-skipping.[ citation needed ]

The development of the modern optical mouse at Hewlett-Packard Co. was supported by a succession of related projects during the 1990s at HP Laboratories. In 1992 William Holland was awarded US Patent 5,089,712 and John Ertel, William Holland, Kent Vincent, Rueiming Jamp, and Richard Baldwin were awarded US Patent 5,149,980 for measuring linear paper advance in a printer by correlating images of paper fibers. Ross R. Allen, David Beard, Mark T. Smith, and Barclay J. Tullis were awarded US Patents 5,578,813 (1996) and 5,644,139 (1997) for 2-dimensional optical navigational (i.e., position measurement) principles based on detecting and correlating microscopic, inherent features of the surface over which the navigation sensor travelled, and using position measurements of each end of a linear (document) image sensor to reconstruct an image of the document. This is the freehand scanning concept used in the HP CapShare 920 handheld scanner. By describing an optical means that explicitly overcame the limitations of wheels, balls, and rollers used in contemporary computer mice, the optical mouse was anticipated. These patents formed the basis for US Patent 5,729,008 (1998) awarded to Travis N. Blalock, Richard A. Baumgartner, Thomas Hornak, Mark T. Smith, and Barclay J. Tullis, where surface feature image sensing, image processing, and image correlation was realized by an integrated circuit to produce a position measurement. Improved precision of 2D optical navigation, needed for application of optical navigation to precise 2D measurement of media (paper) advance in HP DesignJet large format printers, was further refined in US Patent 6,195,475 awarded in 2001 to Raymond G. Beausoleil, Jr., and Ross R. Allen.

While the reconstruction of the image in the document scanning application (Allen et al.) required resolution by the optical navigators on the order of 1/600th of an inch, implementation of optical position measurement in computer mice not only benefit from the cost reductions inherent in navigating at lower resolution, but also enjoy the advantage of visual feedback to the user of the cursor position on the computer display. In 2002, Gary Gordon, Derek Knee, Rajeev Badyal and Jason Hartlove were awarded US Patent 6,433,780 [1] for an optical computer mouse that measured position using image correlation. Some small trackpads (such as those on Blackberry smartphones) work like an optical mouse.

Light source

LED mice

The blue-LED-based V-Mouse VM-101 VMouseBlueLightSource.jpg
The blue-LED-based V-Mouse VM-101

Optical mice often used light-emitting diodes (LEDs) for illumination when first popularized. The color of the optical mouse's LEDs can vary, but red is most common, as red diodes are inexpensive and silicon photodetectors are very sensitive to red light. IR LEDs are also widely used. [17] Other colors are sometimes used, such as the blue LED of the V-Mouse VM-101 illustrated at right.

Laser mice

Although invisible to the naked eye, the light produced by this laser mouse is captured as the color purple because CCDs are sensitive to a broader light wavelength range than the human eye. Captured infrared light from a laser mouse.jpg
Although invisible to the naked eye, the light produced by this laser mouse is captured as the color purple because CCDs are sensitive to a broader light wavelength range than the human eye.

A laser mouse uses an infrared laser diode instead of an LED to illuminate the surface beneath their sensor. As early as 1998, Sun Microsystems provided a laser mouse with their Sun SPARCstation servers and workstations. [18] However, laser mice did not enter the mainstream consumer market until 2004, following the development by a team at Agilent Laboratories, Palo Alto, led by Doug Baney of a laser-based mouse based on a 850 nm VCSEL that offered a 20X improvement in tracking performance. Tong Xie, Marshall T. Depue, and Douglas M. Baney were awarded US patents 7,116,427 and 7,321,359 for their work on low power consumption broad navigability VCSEL-based consumer mice. Paul Machin at Logitech, in partnership with Agilent Technologies introduced the new technology as the MX 1000 laser mouse. This mouse uses a small infrared laser (VCSEL) instead of an LED and significantly increased the resolution of the image taken by the mouse. The laser illumination enabled superior surface tracking compared to LED-illuminated optical mice. [19]

In 2008, Avago Technologies introduced laser navigation sensors whose emitter was integrated into the IC using VCSEL technology. [20]

In August 2009, Logitech introduced mice with two lasers, to track better on glass and glossy surfaces; they dubbed them a "Darkfield" laser sensor. This however has little to do with dark field illumination as used by microscopes; according to Logitech's description [21] it uses a second laser from a different direction for better illumination if it detects low contrast.

Power

Manufacturers often engineer their optical mice—especially battery-powered wireless models—to save power when possible. To do this, the mouse dims or blinks the laser or LED when in standby mode (each mouse has a different standby time). A typical implementation (by Logitech) has four power states, where the sensor is pulsed at different rates per second:[ citation needed ]

Movement can be detected in any of these states; some mice turn the sensor fully off in the sleep state, requiring a button click to wake.

Optical mice utilizing infrared elements (LEDs or lasers) offer substantial increases in battery life over visible spectrum illumination. Some mice, such as the Logitech V450 848 nm laser mouse, are capable of functioning on two AA batteries for a full year, due to the low power requirements of the infrared laser.[ clarification needed ]

Mice designed for use where low latency and high responsiveness are important, such as in playing video games, may omit power-saving features and require a wired connection to improve performance. Examples of mice which sacrifice power-saving in favor of performance are the Logitech G5 and the Razer Copperhead.

Optical versus mechanical mice

The Logitech iFeel optical mouse uses a red LED to project light onto the tracking surface. Optical mouse shining.jpg
The Logitech iFeel optical mouse uses a red LED to project light onto the tracking surface.

Unlike mechanical mice, whose tracking mechanisms can become clogged with lint, optical mice have no moving parts (besides buttons and scroll wheels); therefore, they do not require maintenance other than removing debris that might collect under the light emitter. However, they generally cannot track on glossy and transparent surfaces, including some mouse-pads, causing the cursor to drift unpredictably during operation. Mice with less image-processing power also have problems tracking fast movement, whereas some high-quality mice can track faster than 2 m/s.

Some models of laser mouse can track on glossy and transparent surfaces, and have a much higher sensitivity.

As of 2006 mechanical mice had lower average power requirements than their optical counterparts; the power used by mice is relatively small, and only an important consideration when the power is derived from batteries, with their limited capacity.

Optical models outperform mechanical mice on uneven, slick, soft, sticky, or loose surfaces, and generally in mobile situations lacking mouse pads. Because optical mice render movement based on an image which the LED (or infrared diode) illuminates, use with multicolored mouse pads may result in unreliable performance; however, laser mice do not suffer these problems and will track on such surfaces.

See also

Related Research Articles

<span class="mw-page-title-main">Computer mouse</span> Pointing device used to control a computer

A computer mouse is a hand-held pointing device that detects two-dimensional motion relative to a surface. This motion is typically translated into the motion of the pointer on a display, which allows a smooth control of the graphical user interface of a computer.

<span class="mw-page-title-main">Keyboard technology</span> Hardware technology of keyboards

The technology of computer keyboards includes many elements. Many different keyboard technologies have been developed for consumer demands and optimized for industrial applications. The standard full-size (100%) computer alphanumeric keyboard typically uses 101 to 105 keys; keyboards integrated in laptop computers are typically less comprehensive.

<span class="mw-page-title-main">Pointing device</span> Human interface device for computers

A pointing device is a human interface device that allows a user to input spatial data to a computer. Graphical user interfaces (GUI) and CAD systems allow the user to control and provide data to the computer using physical gestures by moving a hand-held mouse or similar device across the surface of the physical desktop and activating switches on the mouse. Movements of the pointing device are echoed on the screen by movements of the pointer and other visual changes. Common gestures are point and click and drag and drop.

<span class="mw-page-title-main">Trackball</span> Pointing device

A trackball is a pointing device consisting of a ball held by a socket containing sensors to detect a rotation of the ball about two axes—like an upside-down ball mouse with an exposed protruding ball. Users roll the ball to position the on-screen pointer, using their thumb, fingers, or the palm of the hand, while using the fingertips to press the buttons.

<span class="mw-page-title-main">Photodiode</span> Converts light into current

A photodiode is a semiconductor diode sensitive to photon radiation, such as visible light, infrared or ultraviolet radiation, X-rays and gamma rays. It produces an electrical current when it absorbs photons. This can be used for detection and measurement applications, or for the generation of electrical power in solar cells. Photodiodes are used in a wide range of applications throughout the electromagnetic spectrum from visible light photocells to gamma ray spectrometers.

<span class="mw-page-title-main">Laser diode</span> Semiconductor laser

A laser diode is a semiconductor device similar to a light-emitting diode in which a diode pumped directly with electrical current can create lasing conditions at the diode's junction.

<span class="mw-page-title-main">Photonics</span> Technical applications of optics

Photonics is a branch of optics that involves the application of generation, detection, and manipulation of light in form of photons through emission, transmission, modulation, signal processing, switching, amplification, and sensing. Photonics is closely related to quantum electronics, where quantum electronics deals with the theoretical part of it while photonics deal with its engineering applications. Though covering all light's technical applications over the whole spectrum, most photonic applications are in the range of visible and near-infrared light. The term photonics developed as an outgrowth of the first practical semiconductor light emitters invented in the early 1960s and optical fibers developed in the 1970s.

<span class="mw-page-title-main">Vertical-cavity surface-emitting laser</span> Type of semiconductor laser diode

The vertical-cavity surface-emitting laser is a type of semiconductor laser diode with laser beam emission perpendicular from the top surface, contrary to conventional edge-emitting semiconductor lasers which emit from surfaces formed by cleaving the individual chip out of a wafer. VCSELs are used in various laser products, including computer mice, fiber optic communications, laser printers, Face ID, and smartglasses.

<span class="mw-page-title-main">Mousepad</span> Mat on which a computer mouse is used

A mousepad or mousemat is a surface for placing and moving a computer mouse. A mousepad enhances the usability of the mouse compared to using a mouse directly on a table by providing a surface to allow it to measure movement accurately and without jitter. Some mousepads increase ergonomics by providing a padded wrist rest.

<span class="mw-page-title-main">Scroll wheel</span> Component of a computer mouse used for scrolling

A scroll wheel is a wheel used for scrolling. The term usually refers to such wheels found on computer mice. It is often made of hard plastic with a rubbery surface, centred around an internal rotary encoder. It is usually located between the left and right mouse buttons and is positioned perpendicular to the mouse surface. Sometimes the wheel can be pressed left and right, which is actually just two additional macros buttons.

<span class="mw-page-title-main">Interactive whiteboard</span> Large interactive display

An interactive whiteboard (IWB), also known as interactive board or smart board, is a large interactive display board in the form factor of a whiteboard. It can either be a standalone touchscreen computer used independently to perform tasks and operations, or a connectable apparatus used as a touchpad to control computers from a projector. They are used in a variety of settings, including classrooms at all levels of education, in corporate board rooms and work groups, in training rooms for professional sports coaching, in broadcasting studios, and others.

<span class="mw-page-title-main">Image sensor</span> Device that converts images into electronic signals

An image sensor or imager is a sensor that detects and conveys information used to form an image. It does so by converting the variable attenuation of light waves into signals, small bursts of current that convey the information. The waves can be light or other electromagnetic radiation. Image sensors are used in electronic imaging devices of both analog and digital types, which include digital cameras, camera modules, camera phones, optical mouse devices, medical imaging equipment, night vision equipment such as thermal imaging devices, radar, sonar, and others. As technology changes, electronic and digital imaging tends to replace chemical and analog imaging.

<span class="mw-page-title-main">Projection keyboard</span> Virtual device projected onto a surface

A projection keyboard is a form of computer input device whereby the image of a virtual keyboard is projected onto a surface: when a user touches the surface covered by an image of a key, the device records the corresponding keystroke. Some connect to Bluetooth devices, including many of the latest smartphone, tablet, and mini-PC devices with Android, iOS or Windows operating system.

Electro-optical MASINT is a subdiscipline of Measurement and Signature Intelligence, (MASINT) and refers to intelligence gathering activities which bring together disparate elements that do not fit within the definitions of Signals Intelligence (SIGINT), Imagery Intelligence (IMINT), or Human Intelligence (HUMINT).

<span class="mw-page-title-main">Apple pointing devices</span> Computer pointing devices made by Apple

Apple Inc. has designed and manufactured several models of mice, trackpads, and other pointing devices, primarily for use with Macintosh computers. Over the years, Apple has maintained a distinct form and function with its mice that reflect their design languages of that time. Apple's current external pointing devices are the Magic Mouse 2 and Magic Trackpad 2.

<span class="mw-page-title-main">Time-of-flight camera</span> Range imaging camera system

A time-of-flight camera, also known as time-of-flight sensor, is a range imaging camera system for measuring distances between the camera and the subject for each point of the image based on time-of-flight, the round trip time of an artificial light signal, as provided by a laser or an LED. Laser-based time-of-flight cameras are part of a broader class of scannerless LIDAR, in which the entire scene is captured with each laser pulse, as opposed to point-by-point with a laser beam such as in scanning LIDAR systems. Time-of-flight camera products for civil applications began to emerge around 2000, as the semiconductor processes allowed the production of components fast enough for such devices. The systems cover ranges of a few centimeters up to several kilometers.

A rotational mouse is a type of computer mouse which attempts to expand traditional mouse functionality. The objective of rotational mice is to facilitate three degrees of freedom (3DOF) for human-computer interaction by adding a third dimensional input, yaw, to the existing x and y dimensional inputs. There have been several attempts to develop rotating mice, using a variety of mechanisms to detect rotation.

<span class="mw-page-title-main">Input device</span> Device that provides data and signals to a computer

In computing, an input device is a piece of equipment used to provide data and control signals to an information processing system, such as a computer or information appliance. Examples of input devices include keyboards, computer mice, scanners, cameras, joysticks, and microphones.

<span class="mw-page-title-main">James R. Biard</span> American electrical engineer and inventor (1931–2022)

James Robert Biard was an American electrical engineer and inventor who held 73 U.S. patents. Some of his more significant patents include the first infrared light-emitting diode (LED), the optical isolator, Schottky clamped logic circuits, silicon Metal Oxide Semiconductor Read Only Memory, a low bulk leakage current avalanche photodetector, and fiber-optic data links. In 1980, Biard became a member of the staff of Texas A&M University as an Adjunct Professor of Electrical Engineering. In 1991, he was elected as a member into the National Academy of Engineering for contributions to semiconductor light-emitting diodes and lasers, Schotky-clamped logic, and read-only memories.

References

  1. 1 2 US 6433780,"Seeing eye mouse for a computer system"
  2. John Markoff (May 10, 1982). "Computer mice are scurrying out of R&D labs". InfoWorld. 4 (18): 10–11. ISSN   0199-6649.
  3. John Markoff (February 21, 1983). "In Focus: The Mouse that Rolled". InfoWorld. 5 (8). InfoWorld Media Group, Inc.: 28. ISSN   0199-6649.
  4. Sol Sherr (1988). Input Devices . Academic Press. ISBN   0126399700.
  5. Liz Karagianis (Fall 1997). "Steve Kirsch". MIT Spectrum. Archived from the original on March 22, 2008.
  6. "Portraits of MIT-Related Companies: Infoseek, Santa Clara, CA". MIT: The Impact of Innovation. MIT. Archived from the original on December 1, 1998. Retrieved 31 December 2006.
  7. Lyon, Richard F. (August 1981). "The Optical Mouse, and an Architectural Methodology for Smart Digital Sensors" (PDF). In H. T. Kung; Robert F. Sproull; Guy L. Steele (eds.). VLSI Systems and Computations. Computer Science Press. pp. 1–19. doi:10.1007/978-3-642-68402-9_1. ISBN   978-3-642-68404-3.
  8. Stan Augarten (1983). State of the Art: A Photographic History of the Integrated Circuit. Ticknor & Fields. pp. 60–61. ISBN   0-89919-195-9.
  9. "Xerox Mousepad". Digibarn.com. Retrieved 2010-05-29.
  10. USpatent 4751505
  11. "Optical mouse, the old way". Archived from the original on 2022-02-07. Retrieved 2023-04-29. [Box image] PC Mouse - State-of-the-art optical mouse [..] Mouse Systems Corp
  12. "Sun and Mouse Systems mice and optical pads". retrotechnology.com. Archived from the original on 2023-03-16. Retrieved 2023-04-29.
  13. "Old optical mouse". Wikimedia Commons. Archived from the original on 2021-02-18. Retrieved 2023-04-29.
  14. USpatent 4794384
  15. 1 2 Lyon, R. L. (2014). "The Optical Mouse: Early Biomimetic Embedded Vision". In Kisačanin, Branislav; Margrit Gelautz (eds.). Advances in Embedded Computer Vision. Springer. p. 20. ISBN   9783319093871 via Google Books.
  16. "Microsoft Press Release, April 19th 1999". Microsoft. 1999-04-19. Archived from the original on 2011-11-28. Retrieved 2011-05-11.
  17. Winn L. Rosch (2003). Winn L. Rosch hardware bible (6th ed.). Que Publishing. p. 756. ISBN   978-0-7897-2859-3.
  18. "Computer Engineering Tips – Mouse". Archived from the original on May 5, 2009. Retrieved 2006-12-31.
  19. "Logitech MX1000 Laser Cordless Mouse". CNET. November 4, 2004. Retrieved July 19, 2018.
  20. "Avago Technologies Announces Miniature Laser Navigation Sensors for Mouse Applications". January 28, 2008. Retrieved 2013-03-25.
  21. "Logitech Darkfield Innovation Brief" (PDF). Logitech. 2009.