Optical mouse

A Microsoft wireless optical mouse

An optical mouse is a computer mouse which uses a miniature camera and digital image processing to detect movement relative to a surface. Variations of the optical mouse have largely replaced the older mechanical mouse and its need for frequent cleaning.

The earliest optical mice detected movement on prepared surfaces, however they never gained wide acceptance. The modern optical mouse which uses digital image correlation and which works on almost any surface was invented in 2000 by Gary Gordon, Derek Knee, Rajeev Badyal and Jason Hartlove, and awarded US Patent 6,433,780. ^[1] Its technology is explained in an interview with one of its inventors made by the Computer History Museum.

Mechanical mice

Main article: Mechanical mouse

Though not commonly referred to as optical mice, nearly all mechanical mice tracked movement using LEDs and photodiodes to detect when beams of infrared light did and didn't pass through holes in a pair of incremental rotary encoder wheels (one for left/right, another for forward/back), driven by a rubberized ball. Thus, the primary distinction of “optical mice” is not their use of optics, but their complete lack of moving parts to track mouse movement, instead employing an entirely solid-state system.

Early optical mice

An early Xerox optical mouse chip, before the development of the inverted packaging design of Williams and Cherry

The first two optical mice, first demonstrated by two independent inventors in December 1980, had different basic designs: ^{[2] [3] [4]} One of these, invented by Steve Kirsch of MIT and Mouse Systems Corporation, ^{[5] [6]} used an infrared LED and a four-quadrant infrared sensor to detect grid lines printed with infrared absorbing ink on a special metallic surface. Predictive algorithms in the CPU of the mouse calculated the speed and direction over the grid. The other type, invented by Richard F. Lyon of Xerox, used a 16-pixel visible-light image sensor with integrated motion detection on the same n‑type (5 μm) MOS integrated circuit chip, ^{[7] [8]} and tracked the motion of light dots in a dark field of a printed paper or similar mouse pad. ^[9] The Kirsch and Lyon mouse types had very different behaviors, as the Kirsch mouse used an x-y coordinate system embedded in the pad, and would not work correctly when the pad was rotated, while the Lyon mouse used the x-y coordinate system of the mouse body, as mechanical mice do.

OEM-branded Mouse Systems (Kirsch) optical mouse showing underside and mousemat pattern

The optical mouse ultimately sold with the Xerox STAR office computer used an inverted sensor chip packaging approach patented by Lisa M. Williams and Robert S. Cherry of the Xerox Microelectronics Center. ^[10]

The Mouse Systems (Kirsch) design was commercialised and sold in PC compatible form by the company itself ^[11] alongside variants rebranded for OEM use with Sun Microsystems workstations ^[12] and by Data General. ^[13]

Modern optical mice

Optical sensor

Microscope photograph of the IntelliMouse Explorer sensor silicon die

The optical sensor from a Microsoft Wireless IntelliMouse Explorer (v. 1.0A)

Modern surface-independent optical mice work by using an optoelectronic sensor (essentially, a tiny low-resolution video camera) to take successive images of the surface on which the mouse operates. As computing power grew cheaper, it became possible to embed more powerful special-purpose image-processing chips in the mouse itself. This advance enabled the mouse to detect relative motion on a wide variety of surfaces, translating the movement of the mouse into the movement of the cursor and eliminating the need for a special mouse-pad. A surface-independent coherent light optical mouse design was patented by Stephen B. Jackson at Xerox in 1988, despite never being shown to work. ^[14]

Xerox's inventions were never massively commercially exploited, however, and optical mice would remain elusive in the personal computer market until Microsoft released the IntelliMouse with IntelliEye and IntelliMouse Explorer in 1999. ^{[15] [16]} These mice used technology developed by Hewlett-Packard under their Agilent Technologies subsidiary (see below). These mice worked on almost any surface, and represented a welcome improvement over mechanical mice, which would pick up dirt, track capriciously, invite rough handling, and need to be taken apart and cleaned frequently. Other manufacturers soon followed Microsoft's lead, including Apple for their Pro Mouse, ^[15] using components manufactured by Agilent (once they spun off from HP), and over the next several years mechanical mice became obsolete.

S5085 optical sensor IC die (CMOS sensor + driver)

The technology underlying the modern optical computer mouse is known as digital image correlation, a technology pioneered by the defense industry for tracking military targets. A simple binary-image version of digital image correlation was used in the 1980 Lyon optical mouse. Optical mice use image sensors to image naturally occurring texture in materials such as wood, cloth, mouse pads and Formica. These surfaces, when lit at a grazing angle by a light emitting diode, cast distinct shadows that resemble a hilly terrain lit at sunset. Images of these surfaces are captured in continuous succession and compared with each other to determine how far the mouse has moved.

Principle of operation

To understand how optical flow is used in optical mice, imagine two photographs of the same object except slightly offset from each other. Place both photographs on a light table to make them transparent, and slide one across the other until their images line up. The amount that the edges of one photograph overhang the other represents the offset between the images, and in the case of an optical computer mouse the distance it has moved.

Optical mice capture one thousand successive images or more per second. Depending on how fast the mouse is moving, each image will be offset from the previous one by a fraction of a pixel or as many as several pixels. Optical mice mathematically process these images using cross correlation to calculate how much each successive image is offset from the previous one.^{[ citation needed ]} The output of the optical sensor is usually delta X, Y coordinates. Some optical ICs allow to get image data as well. Mice usually embeds some kind of Image Acquisition System and DSP processors for fast data processing.

An optical mouse might use an image sensor having an 18 × 18 pixel array of monochromatic pixels. Its sensor would normally share the same ASIC as that used for storing and processing the images. One refinement would be accelerating the correlation process by using information from previous motions, and another refinement would be preventing deadbands when moving slowly by adding interpolation or frame-skipping.^{[ citation needed ]}

The development of the modern optical mouse at Hewlett-Packard Co. was supported by a succession of related projects during the 1990s at HP Laboratories. In 1992 William Holland was awarded US Patent 5,089,712 and John Ertel, William Holland, Kent Vincent, Rueiming Jamp, and Richard Baldwin were awarded US Patent 5,149,980 for measuring linear paper advance in a printer by correlating images of paper fibers. Ross R. Allen, David Beard, Mark T. Smith, and Barclay J. Tullis were awarded US Patents 5,578,813 (1996) and 5,644,139 (1997) for 2-dimensional optical navigational (i.e., position measurement) principles based on detecting and correlating microscopic, inherent features of the surface over which the navigation sensor travelled, and using position measurements of each end of a linear (document) image sensor to reconstruct an image of the document. This is the freehand scanning concept used in the HP CapShare 920 handheld scanner. By describing an optical means that explicitly overcame the limitations of wheels, balls, and rollers used in contemporary computer mice, the optical mouse was anticipated. These patents formed the basis for US Patent 5,729,008 (1998) awarded to Travis N. Blalock, Richard A. Baumgartner, Thomas Hornak, Mark T. Smith, and Barclay J. Tullis, where surface feature image sensing, image processing, and image correlation was realized by an integrated circuit to produce a position measurement. Improved precision of 2D optical navigation, needed for application of optical navigation to precise 2D measurement of media (paper) advance in HP DesignJet large format printers, was further refined in US Patent 6,195,475 awarded in 2001 to Raymond G. Beausoleil, Jr., and Ross R. Allen.

While the reconstruction of the image in the document scanning application (Allen et al.) required resolution by the optical navigators on the order of 1/600th of an inch, implementation of optical position measurement in computer mice not only benefit from the cost reductions inherent in navigating at lower resolution, but also enjoy the advantage of visual feedback to the user of the cursor position on the computer display. In 2002, Gary Gordon, Derek Knee, Rajeev Badyal and Jason Hartlove were awarded US Patent 6,433,780 ^[1] for an optical computer mouse that measured position using image correlation. Some small trackpads (such as those on Blackberry smartphones) work like an optical mouse.

Light source

LED mice

The blue-LED-based V-Mouse VM-101

Optical mice often used light-emitting diodes (LEDs) for illumination when first popularized. The color of the optical mouse's LEDs can vary, but red is most common, as red diodes are inexpensive and silicon photodetectors are very sensitive to red light. IR LEDs are also widely used. ^[17] Other colors are sometimes used, such as the blue LED of the V-Mouse VM-101 illustrated at right.

Laser mice

Although invisible to the naked eye, the light produced by this laser mouse is captured as the color purple because CCDs are sensitive to a broader light wavelength range than the human eye.

A laser mouse uses an infrared laser diode instead of an LED to illuminate the surface beneath their sensor. As early as 1998, Sun Microsystems provided a laser mouse with their Sun SPARCstation servers and workstations. ^[18] However, laser mice did not enter the mainstream consumer market until 2004, following the development by a team at Agilent Laboratories, Palo Alto, led by Doug Baney of a laser-based mouse based on a 850 nm VCSEL that offered a 20X improvement in tracking performance. Tong Xie, Marshall T. Depue, and Douglas M. Baney were awarded US patents 7,116,427 and 7,321,359 for their work on low power consumption broad navigability VCSEL-based consumer mice. Paul Machin at Logitech, in partnership with Agilent Technologies introduced the new technology as the MX 1000 laser mouse. This mouse uses a small infrared laser (VCSEL) instead of an LED and significantly increased the resolution of the image taken by the mouse. The laser illumination enabled superior surface tracking compared to LED-illuminated optical mice. ^[19]

In 2008, Avago Technologies introduced laser navigation sensors whose emitter was integrated into the IC using VCSEL technology. ^[20]

In August 2009, Logitech introduced mice with two lasers, to track better on glass and glossy surfaces; they dubbed them a "Darkfield" laser sensor. This however has little to do with dark field illumination as used by microscopes; according to Logitech's description ^[21] it uses a second laser from a different direction for better illumination if it detects low contrast.

Power

Manufacturers often engineer their optical mice—especially battery-powered wireless models—to save power when possible. To do this, the mouse dims or blinks the laser or LED when in standby mode (each mouse has a different standby time). A typical implementation (by Logitech) has four power states, where the sensor is pulsed at different rates per second:^{[ citation needed ]}

• 11500: full on, for accurate response while moving, illumination appears bright.
• 1100: fallback active condition while not moving, illumination appears dull.
• 110: standby
• 12: sleep state

Movement can be detected in any of these states; some mice turn the sensor fully off in the sleep state, requiring a button click to wake.

Optical mice utilizing infrared elements (LEDs or lasers) offer substantial increases in battery life over visible spectrum illumination. Some mice, such as the Logitech V450 848 nm laser mouse, are capable of functioning on two AA batteries for a full year, due to the low power requirements of the infrared laser.^{[ clarification needed ]}

Mice designed for use where low latency and high responsiveness are important, such as in playing video games, may omit power-saving features and require a wired connection to improve performance. Examples of mice which sacrifice power-saving in favor of performance are the Logitech G5 and the Razer Copperhead.

Optical versus mechanical mice

The Logitech iFeel optical mouse uses a red LED to project light onto the tracking surface.

Unlike mechanical mice, whose tracking mechanisms can become clogged with lint, optical mice have no moving parts (besides buttons and scroll wheels); therefore, they do not require maintenance other than removing debris that might collect under the light emitter. However, they generally cannot track on glossy and transparent surfaces, including some mouse-pads, causing the cursor to drift unpredictably during operation. Mice with less image-processing power also have problems tracking fast movement, whereas some high-quality mice can track faster than 2 m/s.

Some models of laser mouse can track on glossy and transparent surfaces, and have a much higher sensitivity.

As of 2006 ^[update] mechanical mice had lower average power requirements than their optical counterparts; the power used by mice is relatively small, and only an important consideration when the power is derived from batteries, with their limited capacity.

Optical models outperform mechanical mice on uneven, slick, soft, sticky, or loose surfaces, and generally in mobile situations lacking mouse pads. Because optical mice render movement based on an image which the LED (or infrared diode) illuminates, use with multicolored mouse pads may result in unreliable performance; however, laser mice do not suffer these problems and will track on such surfaces.

See also

• Trackball

References

1. US 6433780,"Seeing eye mouse for a computer system" 
2. John Markoff (May 10, 1982). "Computer mice are scurrying out of R&D labs". InfoWorld. 4 (18): 10–11. ISSN   0199-6649.
3. John Markoff (February 21, 1983). "In Focus: The Mouse that Rolled". InfoWorld. 5 (8). InfoWorld Media Group, Inc.: 28. ISSN   0199-6649.
4. Sol Sherr (1988). Input Devices . Academic Press. ISBN   0126399700.
5. Liz Karagianis (Fall 1997). "Steve Kirsch". MIT Spectrum. Archived from the original on March 22, 2008.
6. "Portraits of MIT-Related Companies: Infoseek, Santa Clara, CA". MIT: The Impact of Innovation. MIT. Archived from the original on December 1, 1998. Retrieved 31 December 2006.
7. Lyon, Richard F. (August 1981). "The Optical Mouse, and an Architectural Methodology for Smart Digital Sensors" (PDF). In H. T. Kung; Robert F. Sproull; Guy L. Steele (eds.). VLSI Systems and Computations. Computer Science Press. pp. 1–19. doi:10.1007/978-3-642-68402-9_1. ISBN   978-3-642-68404-3.
8. Stan Augarten (1983). State of the Art: A Photographic History of the Integrated Circuit. Ticknor & Fields. pp. 60–61. ISBN   0-89919-195-9.
9. "Xerox Mousepad". Digibarn.com. Retrieved 2010-05-29.
10. USpatent 4751505  
11. "Optical mouse, the old way". Archived from the original on 2022-02-07. Retrieved 2023-04-29. [Box image] PC Mouse - State-of-the-art optical mouse [..] Mouse Systems Corp
12. "Sun and Mouse Systems mice and optical pads". retrotechnology.com. Archived from the original on 2023-03-16. Retrieved 2023-04-29.
13. "Old optical mouse". Wikimedia Commons. Archived from the original on 2021-02-18. Retrieved 2023-04-29.
14. USpatent 4794384  
15. Lyon, R. L. (2014). "The Optical Mouse: Early Biomimetic Embedded Vision". In Kisačanin, Branislav; Margrit Gelautz (eds.). Advances in Embedded Computer Vision. Springer. p. 20. ISBN   9783319093871 – via Google Books.
16. "Microsoft Press Release, April 19th 1999". Microsoft. 1999-04-19. Archived from the original on 2011-11-28. Retrieved 2011-05-11.
17. Winn L. Rosch (2003). Winn L. Rosch hardware bible (6th ed.). Que Publishing. p. 756. ISBN   978-0-7897-2859-3.
18. "Computer Engineering Tips – Mouse". Archived from the original on May 5, 2009. Retrieved 2006-12-31.
19. "Logitech MX1000 Laser Cordless Mouse". CNET. November 4, 2004. Retrieved July 19, 2018.
20. "Avago Technologies Announces Miniature Laser Navigation Sensors for Mouse Applications". January 28, 2008. Retrieved 2013-03-25.
21. "Logitech Darkfield Innovation Brief" (PDF). Logitech. 2009.