This article has multiple issues. Please help improve it or discuss these issues on the talk page . (Learn how and when to remove these template messages)
|
In photography and videography, multi-exposure HDR capture is a technique that creates high dynamic range (HDR) images (or extended dynamic range images) by taking and combining multiple exposures of the same subject matter at different exposures. Combining multiple images in this way results in an image with a greater dynamic range than what would be possible by taking one single image. The technique can also be used to capture video by taking and combining multiple exposures for each frame of the video. The term "HDR" is used frequently to refer to the process of creating HDR images from multiple exposures. Many smartphones have an automated HDR feature that relies on computational imaging techniques to capture and combine multiple exposures.
A single image captured by a camera provides a finite range of luminosity inherent to the medium, whether it is a digital sensor or film. Outside this range, tonal information is lost and no features are visible; tones that exceed the range are "burned out" and appear pure white in the brighter areas, while tones that fall below the range are "crushed" and appear pure black in the darker areas. The ratio between the maximum and the minimum tonal values that can be captured in a single image is known as the dynamic range. In photography, dynamic range is measured in exposure value (EV) differences, also known as stops.
The human eye's response to light is non-linear: halving the light level does not halve the perceived brightness of a space, it makes it look only slightly dimmer. For most illumination levels, the response is approximately logarithmic. [1] [2] Human eyes adapt fairly rapidly to changes in light levels. HDR can thus produce images that look more like what a human sees when looking at the subject.
This technique can be applied to produce images that preserve local contrast for a natural rendering, or exaggerate local contrast for artistic effect. HDR is useful for recording many real-world scenes containing a wider range of brightness than can be captured directly, typically both bright, direct sunlight and deep shadows. [3] [4] [5] [6] Due to the limitations of printing and display contrast, the extended dynamic range of HDR images must be compressed to the range that can be displayed. The method of rendering a high dynamic range image to a standard monitor or printing device is called tone mapping; it reduces the overall contrast of an HDR image to permit display on devices or prints with lower dynamic range.
One aim of HDR is to present a similar range of luminance to that experienced through the human visual system. The human eye, through non-linear response, adaptation of the iris, and other methods, adjusts constantly to a broad range of luminance present in the environment. The brain continuously interprets this information so that a viewer can see in a wide range of light conditions.
Device | Stops | Contrast ratio | |
---|---|---|---|
Single exposure | |||
Human eye: close objects | 7.5 | 150...200 | |
Human eye: 4° angular separation | 13 | 8000...10000 | |
Human eye (static) | 10...14 [7] | 1000...15000 | |
Negative film (Kodak VISION3) | 13 [8] | 8000 | |
1/1.7" camera (Nikon Coolpix P340) | 11.9 [9] | 3800 | |
1" camera (Canon PowerShot G7 X) | 12.7 [9] | 6600 | |
Four-thirds DSLR camera (Panasonic Lumix DC-GH5) | 13.0 [9] | 8200 | |
APS DSLR camera (Nikon D7200) | 14.6 [9] | 24800 | |
Full-frame DSLR camera (Nikon D810) | 14.8 [9] | 28500 | |
Most cameras are limited to a much narrower range of exposure values within a single image, due to the dynamic range of the capturing medium. With a limited dynamic range, tonal differences can be captured only within a certain range of brightness. Outside of this range, no details can be distinguished: when the tone being captured exceeds the range in bright areas, these tones appear as pure white, and when the tone being captured does not meet the minimum threshold, these tones appear as pure black. Images captured with non-HDR cameras that have a limited exposure range (low dynamic range, LDR), may lose detail in highlights or shadows.
Modern CMOS image sensors have improved dynamic range and can often capture a wider range of tones in a single exposure [10] reducing the need to perform multi-exposure HDR. Color film negatives and slides consist of multiple film layers that respond to light differently. Original film (especially negatives versus transparencies or slides) feature a very high dynamic range (in the order of 8 for negatives and 4 to 4.5 for positive transparencies).
Multi-exposure HDR is used in photography and also in extreme dynamic range applications such as welding or automotive work. In security cameras the term "wide dynamic range" is used instead of HDR.
A fast-moving subject, or camera movement between the multiple exposures, will generate a "ghost" effect or a staggered-blur strobe effect due to the merged images not being identical. Unless the subject is static and the camera mounted on a tripod there may be a tradeoff between extended dynamic range and sharpness. Sudden changes in the lighting conditions (strobed LED light) can also interfere with the desired results, by producing one or more HDR layers that do have the luminosity expected by an automated HDR system, though one might still be able to produce a reasonable HDR image manually in software by rearranging the image layers to merge in order of their actual luminosity.
Because of the nonlinearity of some sensors image artifacts can be common.
Camera characteristics such as gamma curves, sensor resolution, noise, photometric calibration and color calibration affect resulting high-dynamic-range images. [11]
High-dynamic-range photographs are generally composites of multiple standard dynamic range images, often captured using exposure bracketing. Afterwards, photo manipulation software merges the input files into a single HDR image, which is then also tone mapped in accordance with the limitations of the planned output or display.
Any camera that allows manual exposure control can perform multi-exposure HDR image capture, although one equipped with automatic exposure bracketing (AEB) facilitates the process. Some cameras have an AEB feature that spans a far greater dynamic range than others, from ±0.6 in simpler cameras to ±18 EV in top professional cameras, as of 2020. [update] [12]
The exposure value (EV) refers to the amount of light applied to the light-sensitive detector, whether film or digital sensor such as a CCD. An increase or decrease of one stop is defined as a doubling or halving of the amount of light captured. Revealing detail in the darkest of shadows requires an increased EV, while preserving detail in very bright situations requires very low EVs.
EV is controlled using one of two photographic controls: varying either the size of the aperture or the exposure time. A set of images with multiple EVs intended for HDR processing should be captured only by altering the exposure time; altering the aperture size also would affect the depth of field and so the resultant multiple images would be quite different, preventing their final combination into a single HDR image.
Multi-exposure HDR photography generally is limited to still scenes because any movement between successive images will impede or prevent success in combining them afterward. Also, because the photographer must capture three or more images to obtain the desired luminance range, taking such a full set of images takes extra time. Photographers have developed calculation methods and techniques to partially overcome these problems, but the use of a sturdy tripod is advised to minimize framing differences between exposures.
Tonal information and details from shadow areas can be recovered from images that are deliberately overexposed (i.e., with positive EV compared to the correct scene exposure), while similar tonal information from highlight areas can be recovered from images that are deliberately underexposed (negative EV). The process of selecting and extracting shadow and highlight information from these over/underexposed images and then combining them with image(s) that are exposed correctly for the overall scene is known as exposure fusion. Exposure fusion can be performed manually, relying on the HDR operator's judgment, experience, and training, but usually, fusion is performed automatically by software.
Information stored in high-dynamic-range images typically corresponds to the physical values of luminance or radiance that can be observed in the real world. This is different from traditional digital images, which represent colors as they should appear on a monitor or a paper print. Therefore, HDR image formats are often called scene-referred, in contrast to traditional digital images, which are device-referred or output-referred. Furthermore, traditional images are usually encoded for the human visual system (maximizing the visual information stored in the fixed number of bits), which is usually called gamma encoding or gamma correction . The values stored for HDR images are often gamma compressed using mathematical functions such as power laws logarithms, or floating point linear values, since fixed-point linear encodings are increasingly inefficient over higher dynamic ranges. [13] [14] [15]
HDR images often do not use fixed ranges per color channel, other than traditional images, to represent many more colors over a much wider dynamic range (multiple channels). For that purpose, they do not use integer values to represent the single color channels (e.g., 0–255 in an 8 bit per pixel interval for red, green and blue) but instead use a floating point representation. Common values are 16-bit (half precision) or 32-bit floating-point numbers to represent HDR pixels. However, when the appropriate transfer function is used, HDR pixels for some applications can be represented with a color depth that has as few as 10 to 12 bits (1024 to 4096 values) for luminance and 8 bits (256 values) for chrominance without introducing any visible quantization artifacts. [13] [16]
Tone mapping reduces the dynamic range, or contrast ratio, of an entire image while retaining localized contrast. Although it is a distinct operation, tone mapping is often applied to HDR files by the same software package.
Tone mapping is often needed because the dynamic range that can be displayed is often lower than the dynamic range of the captured or processed image. [10] HDR displays can receive a higher dynamic range signal than SDR displays, reducing the need for tone mapping.
HDR can be done via several methods:
This is an example of four standard dynamic range images that are combined to produce three resulting tone mapped images:
This is an example of a scene with a very wide dynamic range:
Several software applications are available on the PC, Mac, and Linux platforms for producing HDR files and tone mapped images. [19] Notable titles include:
Several camera manufacturers offer built-in multi-exposure HDR features. For example, the Pentax K-7 DSLR has an HDR mode that makes 3 or 5 exposures and outputs (only) a tone mapped HDR image in a JPEG file. [20] The Canon PowerShot G12, Canon PowerShot S95, and Canon PowerShot S100 offer similar features in a smaller format. [21] Nikon's approach is called 'Active D-Lighting' which applies exposure compensation and tone mapping to the image as it comes from the sensor, with the emphasis being on creating a realistic effect. [22]
Some smartphones provide HDR modes for their cameras, and most mobile platforms have apps that provide multi-exposure HDR picture taking. [23] Google released a HDR+ mode for the Nexus 5 and Nexus 6 smartphones in 2014, which automatically captures a series of images and combines them into a single still image, as detailed by Marc Levoy. Unlike traditional HDR, Levoy's implementation of HDR+ uses multiple images underexposed by using a short shutter speed, which are then aligned and averaged by pixel, improving dynamic range and reducing noise. By selecting the sharpest image as the baseline for alignment, the effect of camera shake is reduced. [24]
Some of the sensors on modern phones and cameras may combine two images on-chip so that a wider dynamic range without in-pixel compression is directly available to the user for display or processing.[ citation needed ]
Although not as established as for still photography capture, it is also possible to capture and combine multiple images for each frame of a video in order to increase the dynamic range captured by the camera. [25] This can be done via multiple methods:
Some cameras designed for use in security applications can automatically provide two or more images for each frame, with changing exposure.[ citation needed ] For example, a sensor for 30fps video will give out 60fps with the odd frames at a short exposure time and the even frames at a longer exposure time.
In 2020, Qualcomm announced Snapdragon 888, a mobile SoC able to do computational multi-exposure HDR video capture in 4K and also to record it in a format compatible with HDR displays. [29]
In 2021, the Xiaomi Mi 11 Ultra smartphone is able to do computational multi-exposure HDR for video capture. [30]
HDR capture can be implemented on surveillance cameras, even inexpensive models. This is usually termed a wide dynamic range (WDR) function [31] Examples include CarCam Tiny, Prestige DVR-390, and DVR-478. [32]
The idea of using several exposures to adequately reproduce a too-extreme range of luminance was pioneered as early as the 1850s by Gustave Le Gray to render seascapes showing both the sky and the sea. Such rendering was impossible at the time using standard methods, as the luminosity range was too extreme. Le Gray used one negative for the sky, and another one with a longer exposure for the sea, and combined the two into one picture in positive. [33]
External image | |
---|---|
Schweitzer at the Lamp, by W. Eugene Smith [34] [35] |
Manual tone mapping was accomplished by dodging and burning – selectively increasing or decreasing the exposure of regions of the photograph to yield better tonality reproduction. This was effective because the dynamic range of the negative is significantly higher than would be available on the finished positive paper print when that is exposed via the negative in a uniform manner. An excellent example is the photograph Schweitzer at the Lamp by W. Eugene Smith, from his 1954 photo essay A Man of Mercy on Albert Schweitzer and his humanitarian work in French Equatorial Africa. The image took five days to reproduce the tonal range of the scene, which ranges from a bright lamp (relative to the scene) to a dark shadow. [35]
Ansel Adams elevated dodging and burning to an art form. Many of his famous prints were manipulated in the darkroom with these two methods. Adams wrote a comprehensive book on producing prints called The Print, which prominently features dodging and burning, in the context of his Zone System. [36]
With the advent of color photography, tone mapping in the darkroom was no longer possible due to the specific timing needed during the developing process of color film. Photographers looked to film manufacturers to design new film stocks with improved response, or continued to shoot in black and white to use tone mapping methods.[ citation needed ]
Color film capable of directly recording high-dynamic-range images was developed by Charles Wyckoff and EG&G "in the course of a contract with the Department of the Air Force". [37] This XR film had three emulsion layers, an upper layer having an ASA speed rating of 400, a middle layer with an intermediate rating, and a lower layer with an ASA rating of 0.004. The film was processed in a manner similar to color films, and each layer produced a different color. [38] The dynamic range of this extended range film has been estimated as 1:108. [39] It has been used to photograph nuclear explosions, [40] for astronomical photography, [41] for spectrographic research, [42] and for medical imaging. [43] Wyckoff's detailed pictures of nuclear explosions appeared on the cover of Life magazine in the mid-1950s.
Georges Cornuéjols and licensees of his patents (Brdi, Hymatom) introduced the principle of HDR video image, in 1986, by interposing a matricial LCD screen in front of the camera's image sensor, [44] increasing the sensors dynamic by five stops.
The concept of neighborhood tone mapping was applied to video cameras in 1988 by a group from the Technion in Israel, led by Oliver Hilsenrath and Yehoshua Y. Zeevi. Technion researchers filed for a patent on this concept in 1991, [45] and several related patents in 1992 and 1993. [46]
In February and April 1990, Georges Cornuéjols introduced the first real-time HDR camera that combined two images captured successively by a sensor [47] or simultaneously [48] by two sensors of the camera. This process is known as bracketing used for a video stream.
In 1991, the first commercial video camera was introduced that performed real-time capturing of multiple images with different exposures, and producing an HDR video image, by Hymatom, licensee of Georges Cornuéjols.
Also in 1991, Georges Cornuéjols introduced the HDR+ image principle by non-linear accumulation of images to increase the sensitivity of the camera: [47] for low-light environments, several successive images are accumulated, thus increasing the signal-to-noise ratio.
In 1993, another commercial medical camera producing an HDR video image, by the Technion. [46]
Modern HDR imaging uses a completely different approach, based on making a high-dynamic-range luminance or light map using only global image operations (across the entire image), and then tone mapping the result. Global HDR was first introduced in 1993 [3] resulting in a mathematical theory of differently exposed pictures of the same subject matter that was published in 1995 by Steve Mann and Rosalind Picard. [4]
On October 28, 1998, Ben Sarao created one of the first nighttime HDR+G (high dynamic range + graphic) image of STS-95 on the launch pad at NASA's Kennedy Space Center. It consisted of four film images of the space shuttle at night that were digitally composited with additional digital graphic elements. The image was first exhibited at NASA Headquarters Great Hall, Washington DC, in 1999 and then published in Hasselblad Forum. [49]
The advent of consumer digital cameras produced a new demand for HDR imaging to improve the light response of digital camera sensors, which had a much smaller dynamic range than film. Steve Mann developed and patented the global-HDR method for producing digital images having extended dynamic range at the MIT Media Lab. [50] Mann's method involved a two-step procedure: First, generate one floating point image array by global-only image operations (operations that affect all pixels identically, without regard to their local neighborhoods). Second, convert this image array, using local neighborhood processing (tone-remapping, etc.), into an HDR image. The image array generated by the first step of Mann's process is called a lightspace image, lightspace picture, or radiance map. Another benefit of global-HDR imaging is that it provides access to the intermediate light or radiance map, which has been used for computer vision, and other image processing operations. [50]
In February 2001, the Dynamic Ranger technique was demonstrated, using multiple photos with different exposure levels to accomplish high dynamic range similar to the naked eye. [51]
In the early 2000s, several scholarly research efforts used consumer-grade sensors and cameras. [52] A few companies such as RED and Arri have been developing digital sensors capable of a higher dynamic range. [53] [54] RED EPIC-X can capture time-sequential HDRx images [17] with a user-selectable 1–3 stops of additional highlight latitude in the "x" channel. The "x" channel can be merged with the normal channel in post production software. The Arri Alexa camera uses a dual-gain architecture to generate an HDR image from two exposures captured at the same time. [27]
With the advent of low-cost consumer digital cameras, many amateurs began posting tone-mapped HDR time-lapse videos on the Internet, essentially a sequence of still photographs in quick succession. In 2010, the independent studio Soviet Montage produced an example of HDR video from disparately exposed video streams using a beam splitter and consumer grade HD video cameras. [55] Similar methods have been described in the academic literature in 2001 and 2007. [56] [57]
In 2005, Adobe Systems introduced several new features in Photoshop CS2 including Merge to HDR, 32 bit floating point image support, and HDR tone mapping. [58]
On June 30, 2016, Microsoft added support for the digital compositing of HDR images to Windows 10 using the Universal Windows Platform. [59]
A charge-coupled device (CCD) is an integrated circuit containing an array of linked, or coupled, capacitors. Under the control of an external circuit, each capacitor can transfer its electric charge to a neighboring capacitor. CCD sensors are a major technology used in digital imaging.
Dynamic range is the ratio between the largest and smallest values that a certain quantity can assume. It is often used in the context of signals, like sound and light. It is measured either as a ratio or as a base-10 (decibel) or base-2 logarithmic value of the ratio between the largest and smallest signal values.
In photography, exposure is the amount of light per unit area reaching a frame of photographic film or the surface of an electronic image sensor. It is determined by shutter speed, lens F-number, and scene luminance. Exposure is measured in units of lux-seconds, and can be computed from exposure value (EV) and scene luminance in a specified region.
A Bayer filter mosaic is a color filter array (CFA) for arranging RGB color filters on a square grid of photosensors. Its particular arrangement of color filters is used in most single-chip digital image sensors used in digital cameras, and camcorders to create a color image. The filter pattern is half green, one quarter red and one quarter blue, hence is also called BGGR, RGBG, GRBG, or RGGB.
Computational photography refers to digital image capture and processing techniques that use digital computation instead of optical processes. Computational photography can improve the capabilities of a camera, or introduce features that were not possible at all with film-based photography, or reduce the cost or size of camera elements. Examples of computational photography include in-camera computation of digital panoramas, high-dynamic-range images, and light field cameras. Light field cameras use novel optical elements to capture three dimensional scene information which can then be used to produce 3D images, enhanced depth-of-field, and selective de-focusing. Enhanced depth-of-field reduces the need for mechanical focusing systems. All of these features use computational imaging techniques.
Tone mapping is a technique used in image processing and computer graphics to map one set of colors to another to approximate the appearance of high-dynamic-range (HDR) images in a medium that has a more limited dynamic range. Print-outs, CRT or LCD monitors, and projectors all have a limited dynamic range that is inadequate to reproduce the full range of light intensities present in natural scenes. Tone mapping addresses the problem of strong contrast reduction from the scene radiance to the displayable range while preserving the image details and color appearance important to appreciate the original scene content.
Image noise is random variation of brightness or color information in images, and is usually an aspect of electronic noise. It can be produced by the image sensor and circuitry of a scanner or digital camera. Image noise can also originate in film grain and in the unavoidable shot noise of an ideal photon detector. Image noise is an undesirable by-product of image capture that obscures the desired information. Typically the term “image noise” is used to refer to noise in 2D images, not 3D images.
A camera raw image file contains unprocessed or minimally processed data from the image sensor of either a digital camera, a motion picture film scanner, or other image scanner. Raw files are so named because they are not yet processed, and contain large amounts of potentially redundant data. Normally, the image is processed by a raw converter, in a wide-gamut internal color space where precise adjustments can be made before conversion to a viewable file format such as JPEG or PNG for storage, printing, or further manipulation. There are dozens of raw formats in use by different manufacturers of digital image capture equipment.
A motion picture film scanner is a device used in digital filmmaking to scan original film for storage as high-resolution digital intermediate files.
Digital photography uses cameras containing arrays of electronic photodetectors interfaced to an analog-to-digital converter (ADC) to produce images focused by a lens, as opposed to an exposure on photographic film. The digitized image is stored as a computer file ready for further digital processing, viewing, electronic publishing, or digital printing. It is a form of digital imaging based on gathering visible light.
High dynamic range (HDR), also known as wide dynamic range, extended dynamic range, or expanded dynamic range, is a signal with a higher dynamic range than usual.
In digital photography, exposing to the right (ETTR) is the technique of adjusting the exposure of an image as high as possible at base ISO to collect the maximum amount of light and thus get the optimum performance out of the digital image sensor.
The merits of digital versus film photography were considered by photographers and filmmakers in the early 21st century after consumer digital cameras became widely available. Digital photography and digital cinematography have both advantages and disadvantages relative to still film and motion picture film photography. In the 21st century, photography came to be predominantly digital, but traditional photochemical methods continue to serve many users and applications.
Image quality can refer to the level of accuracy with which different imaging systems capture, process, store, compress, transmit and display the signals that form an image. Another definition refers to image quality as "the weighted combination of all of the visually significant attributes of an image". The difference between the two definitions is that one focuses on the characteristics of signal processing in different imaging systems and the latter on the perceptual assessments that make an image pleasant for human viewers.
Monochrome photography is photography where each position on an image can record and show a different amount of light, but not a different hue. It includes all forms of black-and-white photography, which produce images containing shades of neutral grey ranging from black to white. Other hues besides grey, such as sepia, cyan, blue, or brown can also be used in monochrome photography. In the contemporary world, monochrome photography is mostly used for artistic purposes and certain technical imaging applications, rather than for visually accurate reproduction of scenes.
In cinematography and photography, flashing is the exposure of the film or digital sensors to uniform light prior to exposing them to the scene. It is used as a method of contrast control to bring out detail in darker areas. This adds a bias to the overall light input recorded by the sensor. When used for artistic effects, it can be used to add a colour cast to shadows without significantly affecting highlights. Flashing is usually described as a percentage of exposure increase to the film's base fog level. While the flash itself is often a neutral color temperature, the flash exposure could be any color: the color of the flash will be imbued disproportionately into the shadows of the image.
The Arri Alexa is a digital motion picture camera system developed by Arri. The Arri Alexa was introduced in April 2010 and was Arri's first major transition into digital cinematography, after previous efforts including the Arriflex D-20 and D-21.
The Nikon D3200 is a 24.2-megapixel DX format DSLR Nikon F-mount camera officially launched by Nikon on April 19, 2012. It is marketed as an entry-level DSLR camera for beginners and experienced DSLR hobbyists who are ready for more advanced specs and performance.
Luminance HDR, formerly Qtpfsgui, is graphics software used for the creation and manipulation of high-dynamic-range images. Released under the terms of the GPL, it is available for Linux, Windows and Mac OS X. Luminance HDR supports several High Dynamic Range (HDR) as well as Low Dynamic Range (LDR) file formats.
Pixel Camera is a camera phone application developed by Google for the Android operating system. Development with zoom lenses for the application began in 2011 at the Google X research incubator led by Marc Levoy, which was developing image fusion technology for Google Glass. It was publicly released for Android 4.4+ on the Google Play on April 16, 2014. The app was initially released as Google Camera and supported on all devices running Android 4.4 KitKat and higher. However, in October 2023, coinciding with the release of the Pixel 8 series, it was renamed to Pixel Camera and became officially supported only on Google Pixel devices.
Images that store a depiction of the scene in a range of intensities commensurate with the scene are what we call HDR, or 'radiance maps'. On the other hand, we call images suitable for display with current display technology LDR.
{{cite book}}
: |work=
ignored (help)