Subpixel rendering

Last updated
A simulation of subpixel rendering Subpixel-rendering-RGB.png
A simulation of subpixel rendering
SubPixel.png
"Aa" rendered in subpixel
SubPixel.gif
The previous image, with the R, G and B channel separated and animated
Examples of pixel geometry, showing various arrangements of pixels and subpixels, which must be considered for subpixel rendering. LCD displays consisting of red, green, and blue subpixels (bottom right is the most typical example) are best suited to subpixel rendering. Pixel geometry 01 Pengo.jpg
Examples of pixel geometry, showing various arrangements of pixels and subpixels, which must be considered for subpixel rendering. LCD displays consisting of red, green, and blue subpixels (bottom right is the most typical example) are best suited to subpixel rendering.

Subpixel rendering is a method used to increase the effective resolution of a color display device. It takes advantage of each pixel's composition of individually addressable red, green, and blue components adjacent on the display matrix, called subpixels, and uses them as rendering units instead of pixels.

Contents

Subpixel rendering is primarily used for text rendering on standard DPI displays.

Despite the inherent color anomalies, it can also be used to render general graphics.

History

The origin of subpixel rendering as used today remains controversial. Apple Inc., IBM, and Microsoft patented various implementations with certain technical differences owing to the different purposes their technologies were intended for. [1]

Microsoft had several patents in the United States on subpixel rendering technology for text rendering on RGB Stripe layouts. The patents 6,219,025, 6,239,783, 6,307,566, 6,225,973, 6,243,070, 6,393,145, 6,421,054, 6,282,327, 6,624,828 were filed between October 7, 1998, and October 7, 1999, and expired on July 30, 2019. [2] Analysis of the patent by FreeType [3] indicates that the idea of subpixel rendering is not covered by the patent, but by the actual filter used as a last step to balance the color. Microsoft's patent describes the smallest filter possible that distributes each subpixel value to an equal amount of R,G, and B pixels. Any other filter will either be blurrier or will introduce color artifacts.

Apple was able to use it in Mac OS X due to a patent cross-licensing agreement. [4]

Characteristics

A single pixel on a color display is made of several subpixels, typically three arranged left-to-right as red, green, and blue (RGB). The components are easily visible when viewed with a small magnifying glass, such as a loupe. These pixel components appear as a single color to the human eye because of blurring by optics and spatial integration by nerve cells in the eye. However, the eye is much more sensitive to the location.[ clarification needed ] Therefore, turning on the G and B of one pixel and the R of the next pixel to the right will produce a white dot, but it will appear to be 1/3 of a pixel to the right of the white dot that would be seen from the RGB of only the first pixel. Subpixel rendering takes advantage of this to provide three times the horizontal resolution of the rendered image, though it has to blur this image to produce the correct color by ensuring the same amount of red, green, and blue are turned on as when no subpixel rendering is being done.

Subpixel rendering does not necessitate the use of antialiasing, and gives a smoother result regardless of whether antialiasing is used or not [5] since it artificially increases the resolution. However, it introduces color aliasing since subpixels are colored. Subsequent filtering applied to remove the color artifacts is a form of antialiasing, although its purpose is not smoothing jagged shapes as in conventional antialiasing.

Subpixel rendering requires the software to know the layout of the subpixels. The most common reason it is wrong is monitors that can be rotated 90 (or 180) degrees, though monitors are manufactured with other arrangements of the subpixels, such as BGR or in triangles, or with 4 colors like RGBW squares. On any such display the result of incorrect subpixel rendering will be worse than if no subpixel rendering was done at all (it will not produce color artifacts, but it will produce noisy edges).

Implementations

Apple II

Steve Gibson has claimed that the Apple II, introduced in 1977, supports an early form of subpixel rendering in its high-resolution (280×192) graphics mode. [6] The Wozniak patent only used 2 "sub-pixels". [7] The bytes that comprise the Apple II high-resolution screen buffer contain seven visible bits (each corresponding directly to a pixel) and a flag bit used to select between purple/green or blue/orange color sets. Each pixel, since it is represented by a single bit, is either on or off; there are no bits within the pixel itself for specifying color or brightness. Color is instead created as an artifact of the NTSC color encoding scheme, determined by horizontal position: pixels with even horizontal coordinates are always purple (or blue, if the flag bit is set), and odd pixels are always green (or orange). Two lit pixels next to each other are always white, regardless of whether the pair is even/odd or odd/even, and irrespective of the value of the flag bit. This is an approximation, but it is what most programmers of the time would have in mind while working with the Apple's high-resolution mode.

Gibson's example claims that because two adjacent bits make a white block, there are in fact two bits per pixel: one which activates the purple left half of the pixel, and the other which activates the green right half of the pixel. If the programmer instead activates the green right half of a pixel and the purple left half of the next pixel, then the result is a white block that is 1/2 pixel to the right, which is indeed an instance of subpixel rendering. However, it is not clear whether any programmers of the Apple II have considered the pairs of bits as pixels—instead calling each bit a pixel.

The flag bit in each byte affects color by shifting pixels half a pixel-width to the right. This half-pixel shift was exploited by some graphics software, such as HRCG (High-Resolution Character Generator), an Apple utility that displayed text using the high-resolution graphics mode, to smooth diagonals.

ClearType

Microsoft announced its subpixel rendering technology, called ClearType, at COMDEX in 1998. [8] Microsoft published a paper in May 2000, Displaced Filtering for Patterned Displays, describing the filtering behind ClearType. [9] It was then made available in Windows XP, but it was not activated by default until Windows Vista. (Windows XP OEMs, however, could and did change the default setting.) [10]

FreeType

FreeType, the library used by most current software on the X Window System, contains two open source implementations. The original implementation uses the ClearType antialiasing filters and carries the following notice: "The colour filtering algorithm of Microsoft's ClearType technology for subpixel rendering is covered by patents; for this reason the corresponding code in FreeType is disabled by default. Note that subpixel rendering per se is prior art; using a different colour filter thus easily circumvents Microsoft's patent claims." [3] [2]

FreeType offers a variety of color filters. Since version 2.6.2, the default filter is light, a filter that is both normalized (value sums up to 1) and color-balanced (eliminate color fringes at the cost of resolution). [11]

Since version 2.8.1, a second implementation exists, called Harmony, that "offers high quality LCD-optimized output without resorting to ClearType techniques of resolution tripling and filtering". This is the method enabled by default. When using this method, "each color channel is generated separately after shifting the glyph outline, capitalizing on the fact that the color grids on LCD panels are shifted by a third of a pixel. This output is indistinguishable from ClearType with a light 3-tap filter." [12] Since the Harmony method does not require additional filtering, it is not covered by the ClearType patents.

CoolType

Adobe created their own subpixel renderer called CoolType, allowing them to display documents the same way across various operating systems: Windows, MacOS, Linux etc. When it was launched around the year 2001, CoolType supported a wider range of fonts than Microsoft's ClearType, which at the time was limited to TrueType fonts, whereas Adobe's CoolType also supported PostScript fonts (and their OpenType equivalent as well). [13]

macOS

Mac OS X (later OS X, now macOS) used to use subpixel rendering as well, as part of Quartz 2D. However, it was removed after the introduction of Retina displays. Unlike Microsoft's implementation, which favors a tight fit to the grid (font hinting) to maximize legibility, Apple's implementation prioritizes the shape of the glyphs as set out by their designer. [14]

See also

Related Research Articles

ClearType is Microsoft's implementation of subpixel rendering technology in rendering text in a font system. ClearType attempts to improve the appearance of text on certain types of computer display screens by sacrificing color fidelity for additional intensity variation. This trade-off is asserted to work well on LCD flat panel monitors.

<span class="mw-page-title-main">Pixel</span> Physical point in a raster image

In digital imaging, a pixel, pel, or picture element is the smallest addressable element in a raster image, or the smallest addressable element in a dot matrix display device. In most digital display devices, pixels are the smallest element that can be manipulated through software.

<span class="mw-page-title-main">PNG</span> Family of lossless-compression image file formats

Portable Network Graphics is a raster-graphics file format that supports lossless data compression. PNG was developed as an improved, non-patented replacement for Graphics Interchange Format (GIF)—unofficially, the initials PNG stood for the recursive acronym "PNG's not GIF".

<span class="mw-page-title-main">RGB color model</span> Color model based on red, green, and blue

The RGB color model is an additive color model in which the red, green and blue primary colors of light are added together in various ways to reproduce a broad array of colors. The name of the model comes from the initials of the three additive primary colors, red, green, and blue.

TrueType is an outline font standard developed by Apple in the late 1980s as a competitor to Adobe's Type 1 fonts used in PostScript. It has become the most common format for fonts on the classic Mac OS, macOS, and Microsoft Windows operating systems.

<span class="mw-page-title-main">Video Graphics Array</span> Computer display standard and resolution

Video Graphics Array (VGA) is a video display controller and accompanying de facto graphics standard, first introduced with the IBM PS/2 line of computers in 1987, which became ubiquitous in the IBM PC compatible industry within three years. The term can now refer to the computer display standard, the 15-pin D-subminiature VGA connector, or the 640 × 480 resolution characteristic of the VGA hardware.

In digital signal processing, spatial anti-aliasing is a technique for minimizing the distortion artifacts (aliasing) when representing a high-resolution image at a lower resolution. Anti-aliasing is used in digital photography, computer graphics, digital audio, and many other applications.

OpenType is a format for scalable computer fonts. Derived from TrueType, it retains TrueType's basic structure but adds many intricate data structures for describing typographic behavior. OpenType is a registered trademark of Microsoft Corporation.

<span class="mw-page-title-main">Dots per inch</span> Measure of dot density

Dots per inch is a measure of spatial printing, video or image scanner dot density, in particular the number of individual dots that can be placed in a line within the span of 1 inch (2.54 cm). Similarly, dots per centimetre refers to the number of individual dots that can be placed within a line of 1 centimetre (0.394 in).

Color depth or colour depth, also known as bit depth, is either the number of bits used to indicate the color of a single pixel, or the number of bits used for each color component of a single pixel. When referring to a pixel, the concept can be defined as bits per pixel (bpp). When referring to a color component, the concept can be defined as bits per component, bits per channel, bits per color, and also bits per pixel component, bits per color channel or bits per sample (bps). Modern standards tend to use bits per component, but historical lower-depth systems used bits per pixel more often.

QuickDraw GX was a replacement for the QuickDraw (QD) 2D graphics engine and Printing Manager inside the classic Mac OS. Its underlying drawing platform was an object oriented, resolution-independent, retained mode system, making it much easier for programmers to perform common tasks. Additionally, GX added various curve-drawing commands that had been lacking from QD, as well as introducing TrueType as its basic font system.

<span class="mw-page-title-main">Font rasterization</span> Process of converting text from vector to raster

Font rasterization is the process of converting text from a vector description to a raster or bitmap description. This often involves some anti-aliasing on screen text to make it smoother and easier to read. It may also involve hinting—information embedded in the font data that optimizes rendering details for particular character sizes.

The Saffron Type System is a system for rendering high-quality scalable type on digital displays. It was developed by Mitsubishi Electric Research Laboratories, and is built on a core of adaptively-sampled distance field (ADF) technology. Saffron has been licensed to Adobe and Monotype and is shipping in numerous products such as the Adobe Flash Player and Amazon Kindle. Saffron has been implemented in both software and hardware.

Elan Graphics is a computer graphics architecture for Silicon Graphics computer workstations. Elan Graphics was developed in 1991 and was available as a high-end graphics option on workstations released during the mid-1990s as part of the Express Graphics architectures family. Elan Graphics gives the workstation real-time 2D and 3D graphics rendering capability similar to that of even high-end PCs made over ten years after Elan's introduction, with the exception of texture mapping, which had to be performed in software.

<span class="mw-page-title-main">Font hinting</span> Technique for improving font rendering

Font hinting, also known as instructing, is the use of mathematical instructions to adjust the display of an outline font so that it lines up with a rasterized grid. At low screen resolutions, hinting is critical for producing clear, legible text. It can be accompanied by antialiasing and subpixel rendering for further clarity.

Apple's Macintosh computer supports a wide variety of fonts. This support was one of the features that initially distinguished it from other systems.

CoolType is a software technology, introduced by Adobe Systems in 2000, to increase the legibility of text on color liquid-crystal displays (LCDs) like laptop or thin-film transistor (TFT) LCD monitors, especially to make reading long text, like E-Books, easier. Although it is primarily for LCDs, the legibility on cathode ray tube (CRT) monitors is also improved.

DirectWrite is a text layout and glyph rendering API by Microsoft. It was designed to replace GDI/GDI+ and Uniscribe for screen-oriented rendering and was first shipped with Windows 7 and Windows Server 2008 R2, as well as Windows Vista and Windows Server 2008. DirectWrite is hardware-accelerated when running on top of Direct2D, but can also use the CPU to render on any target, including a GDI bitmap.

PenTile matrix is a family of patented subpixel matrix schemes used in electronic device displays. PenTile is a trademark of Samsung. PenTile matrices are used in AMOLED and LCD displays.

References

  1. John Markoff, "Microsoft's Cleartype Sets Off Debate on Originality Archived 2017-04-21 at the Wayback Machine ", New York Times, December 7, 1998
  2. 1 2 David Turner (June 1, 2007). "ClearType Patents, FreeType and the Unix Desktop: an explanation". Archived from the original on 2009-03-31. Retrieved April 9, 2009.
  3. 1 2 "FreeType and Patents". FreeType.org. February 13, 2018. Archived from the original on 2018-11-10. Retrieved November 29, 2018.
  4. "LCD Rendering Patches". September 24, 2006. Archived from the original on 2011-06-03. Retrieved April 9, 2009.
  5. "A Treatise on Font Rasterisation With an Emphasis on Free Software". 2010-09-15. Retrieved 2023-11-11.
  6. "GRC - The Origins of Sub-Pixel Font Rendering". grc.com. Archived from the original on 2006-03-06. Retrieved 2006-03-02.
  7. David Turner (24 Sep 20:00 2006) LCD Rendering Patches Archived 2007-02-08 at the Wayback Machine
  8. ICT Bill Gates 1998 keynote comdex 1998, archived from the original on 2021-11-30, retrieved 2021-11-30
  9. Platt, John; Keely, Bert; Hill, Bill; Dresevic, Bodin; Betrisey, Claude; Mitchell, Don P.; Hitchcock, Greg; Blinn, Jim; Whitted, Turner (2000-05-01). "Displaced Filtering for Patterned Displays": 296–299. Archived from the original on 2021-11-30. Retrieved 2021-11-30.{{cite journal}}: Cite journal requires |journal= (help)
  10. Greg Hitchcock (with introduction by Steven Sinofsky) "Engineering Changes to ClearType in Windows 7 Archived 2012-12-18 at the Wayback Machine ", MSDN blogs, June 23, 2009
  11. "On slight hinting, proper text rendering, stem darkening and LCD filters". freetype.org. Archived from the original on 2020-11-12. Retrieved 2020-11-14.
  12. Lemberg, Werner (2017-09-16). "Announcing FreeType 2.8.1". Archived from the original on 2019-11-16. Retrieved 2019-04-30.
  13. Felici, James (April 2000) "ClearType, CoolType: The Eyes Have It", Seybold Report on Internet Publishing, Vol. 4 Issue 8
  14. "The Ails Of Typographic Anti-Aliasing". November 2, 2009. Archived from the original on 2014-08-09. Retrieved 2014-08-11.