Color filter array

Last updated
The Bayer color filter mosaic. Each two-by-two submosaic contains 2 green, 1 blue, and 1 red filter, each filter covering one pixel sensor. Bayer pattern on sensor.svg
The Bayer color filter mosaic. Each two-by-two submosaic contains 2 green, 1 blue, and 1 red filter, each filter covering one pixel sensor.

In digital imaging, a color filter array (CFA), or color filter mosaic (CFM), is a mosaic of tiny color filters placed over the pixel sensors of an image sensor to capture color information.

Contents

The term is also used in reference to e paper devices where it means a mosaic of tiny color filters placed over the grey scale display panel to reproduce color images.

Image sensor overview

Color filters are needed because the typical photosensors detect light intensity with little or no wavelength specificity and therefore cannot separate color information. [1] Since sensors are made of semiconductors, they obey solid-state physics.

The color filters filter the light by wavelength range, such that the separate filtered intensities include information about the color of light. For example, the Bayer filter (shown by the image) gives information about the intensity of light in red, green, and blue (RGB) wavelength regions. The raw image data captured by the image sensor is then converted to a full-color image (with intensities of all three primary colors represented at each pixel) by a demosaicing algorithm which is tailored for each type of color filter. The spectral transmittance of the CFA elements along with the demosaicing algorithm jointly determine the color rendition. [2] The sensor's passband quantum efficiency and span of the CFA's spectral responses are typically wider than the visible spectrum, thus all visible colors can be distinguished. The responses of the filters do not generally correspond to the CIE color matching functions, [3] so a color translation is required to convert the tristimulus values into a common, absolute color space. [4]

The Foveon X3 sensor uses a different structure such that a pixel utilizes properties of multi-junctions to stack blue, green, and red sensors on top of each other. This arrangement does not require a demosaicing algorithm because each pixel has information about each color. Dick Merrill of Foveon distinguishes the approaches as "vertical color filter" for the Foveon X3 versus "lateral color filter" for the CFA. [5] [6]

List of color filter arrays

ImageNameDescriptionPattern size (pixels)
Bayer pattern.svg Bayer filter Very common RGB filter. With one blue, one red, and two green.2×2
RGBE filter.svg RGBE filter Bayer-like with one of the green filters modified to "emerald"; used in a few Sony cameras.2×2
RYYB pattern.svg RYYB filterOne red, two yellow, and one blue;2×2
CYYM pattern.svg CYYM filter One cyan, two yellow, and one magenta; used in a few cameras of Kodak.2×2
CYGM pattern.svg CYGM filter One cyan, one yellow, one green, and one magenta; used in a few cameras.2×2
RGBW Bayer.svg RGBW BayerTraditional RGBW similar to Bayer and RGBE patterns.2×2
RGBW number 1.svg RGBW #1Three example RGBW filters from Kodak, with 50% white. (See Bayer filter#Modifications )4×4
RGBW number 2.svg RGBW #2
RGBW number 3.svg RGBW #32×4
XTrans matrix.png X-Trans Fujifilm-specific RGB matrix filter, with a large pattern, studied for diminishing Moiré effect.6×6
Quad Bayer Similar to Bayer filter, however with 4x blue, 4x red, and 8x green. [7]

Used by Sony, also known as Tetracell by Samsung and 4-cell by OmniVision. [8] [9]

4×4
RYYB Quad Bayer Similar to Quad Bayer filter, but with RYYB instead of RGGB. i.e. 4x blue, 4x red, and 8x yellow.

First used in the Leica camera sensor of the Huawei P30 series smartphones. [10]

Nonacell Similar to Bayer filter, however with 9x blue, 9x red, and 18x green. [11] 6×6
RCCC color filter array (R1C1).svg RCCCUsed in the automotive industry. [12] A monochrome sensor is desired for maximum sensitivity, with the red channel required for regions of interest such as traffic lights and rear lights. [13] [14] [15] 2x2
RCCB color filter array.svg RCCBUsed in the automotive industry. [12] Similar to the Bayer sensor except the green pixels are clear, providing more low-light sensitivity and less noise. [15] 2x2

RGBW sensor

An RGBW matrix (from red, green, blue, white) is a CFA that includes "white" or transparent filter elements that allow the photodiode to respond to all colors of light; that is, some cells are "panchromatic", and more of the light is detected, rather than absorbed, compared to the Bayer matrix. Sugiyama filed for a patent on such an arrangement in 2005. [16] Kodak announced several RGBW CFA patents in 2007, all of which have the property that when the panchromatic cells are ignored, the remaining color filtered cells are arranged such that their data can be processed with a standard Bayer demosaicing algorithm.

CYGM sensor

A CYGM matrix (cyan, yellow, green, magenta) is a CFA that uses mostly secondary colors, again to allow more of the incident light to be detected rather than absorbed. Other variants include CMY and CMYW matrices.

Manufacture of the image sensor CFA

Diazonaphthoquinone (DNQ)-novolac photoresist is one material used as the carrier for making color filters from color dyes or pigments. There is some interference between the dyes and the ultraviolet light needed to properly expose the polymer, though solutions have been found for this problem. [17] Color photoresists sometimes used include those with chemical monikers CMCR101R, CMCR101G, CMCR101B, CMCR106R, CMCR106G, and CMCR106B. [18]

A few sources [1] [19] discuss other specific chemical substances, attending optical properties, and optimal manufacturing processes of color filter arrays.

For instance, Nakamura said that materials for on-chip color filter arrays fall into two categories: pigment and dye. Pigment based CFAs have become the dominant option because they offer higher heat resistance and light resistance compared to dye based CFAs. In either case, thicknesses ranging up to 1 micrometre are readily available. [1]

Theuwissen says "Previously, the color filter was fabricated on a separate glass plate and glued to the CCD (Ishikawa 81), but nowadays, all single-chip color cameras are provided with an imager which has the color filter on-chip processed (Dillon 78) and not as a hybrid." [19] He provides a bibliography focusing on the number, types, aliasing effects, moire patterns, and spatial frequencies of the absorptive filters.

Some sources indicate that the CFA can be manufactured separately and affixed after the sensor has been manufactured, [20] [21] [22] while other sensors have the CFA manufactured directly on the surface of the imager. [22] [23] [24] Theuwissen makes no mention of the materials utilized in CFA manufacture.

At least one early example of an on-chip design utilized gelatin filters (Aoki et al., 1982). [25] The gelatin is sectionalized, via photolithography, and subsequently dyed. Aoki reveals that a CYWG arrangement was used, with the G filter being an overlap of the Y and C filters.

Filter materials are manufacturer specific. [26] Adams et al. state "Several factors influence the CFA's design. First, the individual CFA filters are usually layers of transmissive (absorptive) organic or pigment dyes. Ensuring that the dyes have the right mechanical properties—such as ease of application, durability, and resistance to humidity and other atmospheric stresses—is a challenging task. This makes it difficult, at best, to fine-tune the spectral responsivities.".

Given that the CFAs are deposited on the image sensor surface at the BEOL (back end of line, the later stages of the integrated circuit manufacturing line), where a low-temperature regime must be rigidly observed (due to the low melting temperature of the aluminum metalized "wires" and the substrate mobility of the dopants implanted within the bulk silicon), organics would be preferred over glass. On the other hand, some CVD silicon oxide processes are low temperature processes. [27]

Ocean Optics has indicated that their patented dichroic filter CFA process (alternating thin films of ZnS and Cryolite) can be applied to spectroscopic CCDs. [28] Gersteltec sells photoresists that possesses color filter properties. [29]

Some pigment and dye molecules used in CFAs

In U.S.P.# 4,808,501, Carl Chiulli cites the use of 5 chemicals, three of which are C.I. #12715, AKA Solvent Red 8; Solvent Yellow 88; and C.I. # 61551, Solvent Blue 36. In U.S.P. # 5,096,801 Koya et al., of Fuji Photo Film company, list some 150-200 chemical structures, mainly azo dyes and pyrazolone-diazenyl, but fail to provide chemical names, CAS Registry numbers, or Colour Index numbers.

Optically efficient CFA implementation

Nakamura [1] provides a schematic and bibliographic items illustrating the importance of microlenses, their f-number, and the interplay with the CFA and CCD array. [30] Further, a short discussion of anti-reflection films is offered, [31] though Janesick's [32] work appears is more concerned with photon–silicon interaction. Early work on microlenses [33] and on the three-CCD/prism cameras [34] stress the importance of a fully integrated design solution for CFAs. The camera system, as a whole, benefits from careful consideration of CFA technologies and their interplay with other sensor properties.

E-paper CFA

There are three primary methods for reproducing color on e paper displays. One uses micro spheres in various pigments, such as the limited color range three pigment Spectra displays or more faithful four pigment Advanced Color ePaper, both by E Ink. This method suffers from often slow refresh rates as with several pigments the display must perform refreshes for each pigment. As with grey scale units, after the display is updated the device does not require power to keep the image on screen.

The second common method uses a typical grey scale e paper display behind a transparent color layer. The color layer is a LCD based CFA. When displaying grey scale images the device runs at its native resolution, for instance, 300 pixels per inch (PPI). However, due to the CFA, the resolution of the device drops when displaying color images, say to 100 PPI. [35] When the image to be displayed consists of both a colored and a black and white section, for example when a book page comprises plain text as well as a color photo, some e book devices may display the photo at the reduced resolution while the text is at the normal resolution. As the CFA is LCD based, the CFA requires constant power to run and uses more energy.

The third method, as in ClearInk, uses a CFA consisting of a front layer of wells with hemispherical bottoms above a layer of fluid containing black charged spheres. When the spheres are away from the hemispheres, the hemispheres reflect brightly due to total internal reflection. When the black spheres are moved near to the hemispheres, the amount of reflection drops. The refresh rate on video versions of these devices is fast enough for video playback (33 Hz on the device, compared to 25 Hz for PAL television or 29.97 Hz for NTSC television). They require more energy to operate than a plain E Ink display but much less than a LCD based display. [36]

Related Research Articles

<span class="mw-page-title-main">Charge-coupled device</span> Device for the movement of electrical charge

A charge-coupled device (CCD) is an integrated circuit containing an array of linked, or coupled, capacitors. Under the control of an external circuit, each capacitor can transfer its electric charge to a neighboring capacitor. CCD sensors are a major technology used in digital imaging.

<span class="mw-page-title-main">RGB color model</span> Additive color model based on combining red, green, and blue

The RGB color model is an additive color model in which the red, green and blue primary colors of light are added together in various ways to reproduce a broad array of colors. The name of the model comes from the initials of the three additive primary colors, red, green, and blue.

<span class="mw-page-title-main">Photodiode</span> Converts light into current

A photodiode is a light-sensitive semiconductor diode. It produces current when it absorbs photons.

<span class="mw-page-title-main">Digital camera</span> Camera that captures photographs or video in digital format

A digital camera is a camera that captures photographs in digital memory. Most cameras produced today are digital, largely replacing those that capture images on photographic film. Digital cameras are now widely incorporated into mobile devices like smartphones with the same or more capabilities and features of dedicated cameras. High-end, high-definition dedicated cameras are still commonly used by professionals and those who desire to take higher-quality photographs.

Digital image processing is the use of a digital computer to process digital images through an algorithm. As a subcategory or field of digital signal processing, digital image processing has many advantages over analog image processing. It allows a much wider range of algorithms to be applied to the input data and can avoid problems such as the build-up of noise and distortion during processing. Since images are defined over two dimensions digital image processing may be modeled in the form of multidimensional systems. The generation and development of digital image processing are mainly affected by three factors: first, the development of computers; second, the development of mathematics ; third, the demand for a wide range of applications in environment, agriculture, military, industry and medical science has increased.

A thin-film transistor (TFT) is a special type of field-effect transistor (FET) where the transistor is thin relative to the plane of the device. TFTs are grown on a supporting substrate. A common substrate is glass, because the traditional application of TFTs is in liquid-crystal displays (LCDs). This differs from the conventional bulk metal oxide field effect transistor (MOSFET), where the semiconductor material typically is the substrate, such as a silicon wafer.

The Foveon X3 sensor is a digital camera image sensor designed by Foveon, Inc., and manufactured by Dongbu Electronics. It uses an array of photosites that consist of three vertically stacked photodiodes. Each of the three stacked photodiodes has a different spectral sensitivity, allowing it to respond differently to different wavelengths. The signals from the three photodiodes are then processed as additive color data that are transformed to a standard RGB color space.

A digital image is an image composed of picture elements, also known as pixels, each with finite, discrete quantities of numeric representation for its intensity or gray level that is an output from its two-dimensional functions fed as input by its spatial coordinates denoted with x, y on the x-axis and y-axis, respectively. Depending on whether the image resolution is fixed, it may be of vector or raster type. By itself, the term "digital image" usually refers to raster images or bitmapped images.

<span class="mw-page-title-main">Bayer filter</span> Color filter array

A Bayer filter mosaic is a color filter array (CFA) for arranging RGB color filters on a square grid of photosensors. Its particular arrangement of color filters is used in most single-chip digital image sensors used in digital cameras, camcorders, and scanners to create a color image. The filter pattern is half green, one quarter red and one quarter blue, hence is also called BGGR,RGBG, GRBG, or RGGB.

<span class="mw-page-title-main">Super CCD</span>

Super CCD is a proprietary charge-coupled device that has been developed by Fujifilm since 1999. The Super CCD uses octagonal, rather than rectangular, pixels. This allows a higher horizontal and vertical resolution to be achieved than a traditional sensor of an equivalent pixel count.

A demosaicing algorithm is a digital image process used to reconstruct a full color image from the incomplete color samples output from an image sensor overlaid with a color filter array (CFA). It is also known as CFA interpolation or color reconstruction.

<span class="mw-page-title-main">Three-CCD camera</span> Camera whose imaging system uses three separate charge-coupled devices

A three-CCD (3CCD) camera is a camera whose imaging system uses three separate charge-coupled devices (CCDs), each one receiving filtered red, green, or blue color ranges. Light coming in from the lens is split by a beam-splitter prism into three beams, which are then filtered to produce colored light in three color ranges or "bands". The system is employed by high quality still cameras, telecine systems, professional video cameras and some prosumer video cameras.

<span class="mw-page-title-main">Digital photography</span> Photography with a digital camera

Digital photography uses cameras containing arrays of electronic photodetectors interfaced to an analog-to-digital converter (ADC) to produce images focused by a lens, as opposed to an exposure on photographic film. The digitized image is stored as a computer file ready for further digital processing, viewing, electronic publishing, or digital printing.

<span class="mw-page-title-main">Image sensor</span> Device that converts an optical image into an electronic signal

An image sensor or imager is a sensor that detects and conveys information used to make an image. It does so by converting the variable attenuation of light waves into signals, small bursts of current that convey the information. The waves can be light or other electromagnetic radiation. Image sensors are used in electronic imaging devices of both analog and digital types, which include digital cameras, camera modules, camera phones, optical mouse devices, medical imaging equipment, night vision equipment such as thermal imaging devices, radar, sonar, and others. As technology changes, electronic and digital imaging tends to replace chemical and analog imaging.

<span class="mw-page-title-main">Active-pixel sensor</span> Image sensor consisting of an integrated circuit

An active-pixel sensor (APS) is an image sensor where each pixel sensor unit cell has a photodetector and one or more active transistors. In a metal–oxide–semiconductor (MOS) active-pixel sensor, MOS field-effect transistors (MOSFETs) are used as amplifiers. There are different types of APS, including the early NMOS APS and the now much more common complementary MOS (CMOS) APS, also known as the CMOS sensor. CMOS sensors are used in digital camera technologies such as cell phone cameras, web cameras, most modern digital pocket cameras, most digital single-lens reflex cameras (DSLRs), and mirrorless interchangeable-lens cameras (MILCs).

<span class="mw-page-title-main">CYGM filter</span>

In digital photography, the CYGM filter is an alternative color filter array to the Bayer filter (GRGB). It similarly uses a mosaic of pixel filters, of cyan, yellow, green and magenta, and so also requires demosaicing to produce a full-color image.

<span class="mw-page-title-main">RGBE filter</span>

In digital photography, the RGBE filter is an alternative color filter array to the Bayer filter (GRGB). It similarly uses a mosaic of pixel filters, of red, green, blue and "emerald", and so also requires demosaicing to produce a full-color image. It was developed by Sony and so far is used only in the ICX456 8-megapixel CCD and in the Sony Cyber-shot DSC-F828 camera.

Eric R. Fossum is an American physicist and engineer known for developing the CMOS image sensor. He is currently a professor at Thayer School of Engineering in Dartmouth College.

Nobukazu Teranishi is a Japanese engineer who researches image sensors, and is known for inventing the pinned photodiode, an important component of modern digital cameras. He was one of four recipients of the 2017 Queen Elizabeth Prize for Engineering. His industrial career wasa at NEC Corporation (1978–2000) and Panasonic Corporation (2000–13). As of 2018, he is a professor at the University of Hyogo and at Shizuoka University.

Orly Yadid-Pecht is a Professor of Electrical and Software Engineering and Alberta Innovates Technology Futures Strategic Chair of Integrated Intelligent Sensors at the University of Calgary. She develops CMOS based imaging devices for biomedical sensing. She is a Fellow of the Institute of Electrical and Electronics Engineers, SPIE and American Institute for Medical and Biological Engineering. She is an ASTech Award Winner for Technology. Yadid-Pecht holds several patents for new technologies, including sensors, health monitoring devices and drug delivery systems.

References

  1. 1 2 3 4 Nakamura, Junichi (2005). Image Sensors and Signal Processing for Digital Still Cameras. CRC Press. ISBN   978-0-8493-3545-7.
  2. ↑ "Color Correction for Image Sensors" (PDF). Image Sensor Solutions: Application Note. Revision 2.0. Kodak. 27 October 2003.[ permanent dead link ]
  3. ↑ Comparison of the spectral response of a Nikon D70 vs. Canon 10D, Christian Buil.
  4. ↑ Soo-Wook Jang; Eun-Su Kim; Sung-Hak Lee; Kyu-Ik Sohng (2005). Adaptive Colorimetric Characterization of Digital Camera with White Balance. Lecture Notes in Computer Science. Vol. 3656. Springer. pp. 712–719. doi:10.1007/11559573_87. ISBN   978-3-540-29069-8.
  5. ↑ "Digital Photography Essentials #003: "Color Separation"". Digital Outback Photo.[ permanent dead link ]
  6. ↑ Thomas Kreis (2006). Handbook of Holographic Interferometry: Optical and Digital Methods. Wiley-VCH. ISBN   3-527-60492-8.
  7. ↑ "Sony Releases Stacked CMOS Image Sensor for Smartphones with Industry's Highest 48 Effective Megapixels". Sony Global - Sony Global Headquarters. Retrieved 2019-08-16.
  8. ↑ "How Tetracell delivers crystal clear photos day and night | Samsung Semiconductor Global Website". www.samsung.com. Retrieved 2019-08-16.
  9. ↑ "Product Releases | News & Events | OmniVision". www.ovt.com. Retrieved 2019-08-16.
  10. ↑ "Part 4: Non-Bayer CFA, Phase Detection Autofocus (PDAF) | TechInsights". techinsights.com. Retrieved 2019-08-16.
  11. ↑ "Samsung's 108Mp ISOCELL Bright HM1 Delivers Brighter Ultra-High-Res Images with Industry-First Nonacell Technology". news.samsung.com. Retrieved 2020-02-14.
  12. 1 2 "RAW Files | imatest" . Retrieved 2021-01-03.
  13. ↑ "Interfacing Red/Clear Sensors to ADSP-BF609® Blackfin Processors (EE-358)" (PDF).
  14. ↑ "Sensor | imatest" . Retrieved 2021-01-03.
  15. 1 2 "IISS Image Pipe for Alternate CFA Formats" (PDF).
  16. ↑ US Patent Application 20050231618
  17. ↑ Miller Harris R. (1999). Conley, Will (ed.). "Color filter array for CCD and CMOS image sensors using a chemically amplified, thermally cured, pre-dyed, positive-tone photoresist for 365 nm lithography". Proceedings of SPIE. Advances in Resist Technology and Processing XVI. International Society for Optical Engineering. 3678 (2): 1083–1090. Bibcode:1999SPIE.3678.1083M. doi:10.1117/12.350159. ISSN   0277-786X. S2CID   109585024.
  18. ↑ "Microelectronics Fabrication Facility, Hong Kong University of Science and Technology". Archived from the original on 2011-07-21.{{cite journal}}: Cite journal requires |journal= (help)
  19. 1 2 Theuwissen, Albert (1995). Solid-State Imaging with charge coupled devices. Kluwer Academic Publishers. ISBN   978-0-7923-3456-9.
  20. ↑ Ishikawa; et al. (1981). "Color Reproduction of a Single Chip Color Camera with a Frame Transfer CCD". IEEE Journal of Solid-State Circuits. 16 (2): 101–103. Bibcode:1981IJSSC..16..101I. doi:10.1109/JSSC.1981.1051549.
  21. ↑ Takizawa; et al. (1983). "Field Integration Mode CCD Color Television Camera Using Frequency Interleaving Method". IEEE Transactions on Consumer Electronics (3): 358–364. doi:10.1109/TCE.1983.356322. S2CID   29448168.
  22. 1 2 Knop and Morf (August 1985). "A New Class of Mosaic Color Encoding Patterns for Single-Chip Cameras". IEEE Transactions on Electron Devices. 32 (8): 1390–1395. Bibcode:1985ITED...32.1390K. doi:10.1109/T-ED.1985.22134. S2CID   8503745.
  23. ↑ Dillon; et al. (February 1978). "Color Imaging System Using a Single CCD Area Array". IEEE Transactions on Electron Devices. 25 (2): 102–107. Bibcode:1978ITED...25..102D. doi:10.1109/T-ED.1978.19046.
  24. ↑ Tanaka; et al. (1990). "HDTV Single-Chip CCD Color Camera". IEEE Transactions on Consumer Electronics. 36 (3): 479–485. doi:10.1109/30.103163.
  25. ↑ Aoki; et al. (1982). "2/3-Inch Format MOS Single-Chip Color Imager". IEEE Transactions on Electron Devices. 29 (4): 745–750. Bibcode:1982ITED...29..745A. doi:10.1109/T-ED.1982.20772. S2CID   47298619.
  26. ↑ Adams; et al. (1998). "Color Processing in Digital Cameras" (PDF). IEEE Micro. 18 (6): 20–31. doi:10.1109/40.743681.
  27. ↑ Xiao (2001). Introduction to Semiconductor Manufacturing.
  28. ↑ "Dichroic Filter Array Patented Patterned Coatings Technology". Ocean Optics. Archived from the original on 2008-12-04. Retrieved 2008-11-17.
  29. ↑ "Swiss made SU-8 Photoepoxy Functional Products". Gersteltec Engineering Solutions. Archived from the original on 2010-10-10. Retrieved 2010-11-01.
  30. ↑ Agranov; et al. (January 2003). "Crosstalk and Microlens Study in a Color CMOS Image Sensor". IEEE Transactions on Electron Devices. 50 (1): 4–11. Bibcode:2003ITED...50....4A. doi:10.1109/ted.2002.806473. ISSN   0018-9383.
  31. ↑ Murakami; et al. (August 2000). "Technologies to Improve Photo-Sensitivity and Reduce VOD Shutter Voltage for CCD Image Sensors". IEEE Transactions on Electron Devices. 47 (8): 1566–1572. Bibcode:2000ITED...47.1566M. doi:10.1109/16.853032.
  32. ↑ Janesick, James (2001). Scientific Charge Coupled Devices. SPIE. ISBN   0-8194-3698-4.
  33. ↑ Ishihara, Y.; Tanigaki, K. (1983). "A high photosensitive IL-CCD image sensor with monolithic resin lens array". 1983 International Electron Devices Meeting. pp. 497–500. doi:10.1109/IEDM.1983.190552. S2CID   25401214.
  34. ↑ Murata; et al. (1983). "Development of a 3-MOS Color Camera". SMPTE Journal. 92 (12): 1270–1273. doi:10.5594/J04214.
  35. ↑ "Everything you need to know about e INK Kaleido 2 and Kaleido Plus". goodereader.com. 21 January 2021. Retrieved 2021-04-12.
  36. ↑ "Don't Buy an Ereader Yet: 6 Upcoming Color Epaper Technologies". MUO. 2020-04-06. Retrieved 2021-04-12.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.