Image quality

Last updated October 31, 2023

Image quality can refer to the level of accuracy with which different imaging systems capture, process, store, compress, transmit and display the signals that form an image. Another definition refers to image quality as "the weighted combination of all of the visually significant attributes of an image".^[1]^: 598 The difference between the two definitions is that one focuses on the characteristics of signal processing in different imaging systems and the latter on the perceptual assessments that make an image pleasant for human viewers.

Image quality should not be mistaken with image fidelity. Image fidelity refers to the ability of a process to render a given copy in a perceptually similar way to the original (without distortion or information loss), i.e., through a digitization or conversion process from analog media to digital image.

The process of determining the level of accuracy is called Image Quality Assessment (IQA). Image quality assessment is part of the quality of experience measures. Image quality can be assessed using two methods: subjective and objective. Subjective methods are based on the perceptual assessment of a human viewer about the attributes of an image or set of images, while objective methods are based on computational models that can predict perceptual image quality.^[2]^: vii Objective and subjective methods aren't necessarily consistent or accurate between each other: a human viewer might perceive stark differences in quality in a set of images where a computer algorithm might not.

Subjective methods are costly, require a large number of people, and are impossible to automate in real-time. Therefore, the goal of image quality assessment research is to design algorithms for objective assessment that are also consistent with subjective assessments.^[3] The development of such algorithms has a lot of potential applications. They can be used to monitor image quality in control quality systems, to benchmark image processing systems and algorithms and to optimize imaging systems.^[2]^: 2^[3]^: 430

Image quality factors

The image formation process is affected by several distortions between the moment in which the signals travel through to and reach the capture surface, and the device or mean in which signals are displayed. Although optical aberrations can cause great distortions in image quality, they are not part of the field of Image Quality Assessment. Optical aberrations caused by lenses belong to the optics area and not to the signal processing areas.

In an ideal model, there's no quality loss between the emission of the signal and the surface in which the signal is being captured on. For example, a digital image is formed by electromagnetic radiation or other waves as they pass through or reflect off objects. That information is then captured and converted into digital signals by an image sensor. The sensor, however, has nonidealities that limit its performance.

Image quality assessment methods

Image quality can be assessed using objective or subjective methods. In the objective method, image quality assessments are performed by different algorithms that analyze the distortions and degradations introduced in an image. Subjective image quality assessments are a method based on the way in which humans experience or perceive image quality. Objective and subjective methods of quality assessment don't necessarily correlate with each other. An algorithm might have a similar value for an image and its altered or degraded versions, while a subjective method might perceive a stark contrast in quality for the same image and its versions.

Subjective methods

Subjective methods for image quality assessment belong to the larger area of psychophysics research, a field that studies the relationship between physical stimulus and human perceptions. A subjective IQA method will typically consist on applying mean opinion score techniques, where a number of viewers rate their opinions based on their perceptions of image quality. These opinions are afterwards mapped onto numerical values.

These methods can be classified depending on the availability of the source and test images:

Single-stimulus: the viewer only has the test image and is not aware of the source image.
Double-stimulus: the viewer has both the source and test image.

Since visual perception can be affected by environmental and viewing conditions, the International Telecommunication Union produced a set of recommendations for standardized testing methods for subjective image quality assessment.^[4]

Objective methods

Wang & Bovik (2006) classify the objective methods with the following criteria: (a) the availability of an original image; (b) on the basis of their application scopes and (c) on the model of a Human Visual System simulation to assess quality.^[5] Keelan (2002) classifies the methods based on (a) direct experimental measurements; (b) system modeling and (c) visual assessment against calibrated standards.^[6]^: 173

Full-reference (FR) methods – FR metrics try to assess the quality of a test image by comparing it with a reference image that is assumed to have perfect quality, e.g. the original of an image versus a JPEG-compressed version of the image.
Reduced-reference (RR) methods – RR metrics assess the quality of a test and reference image based on a comparison of features extracted from both images.
No-reference (NR) methods – NR metrics try to assess the quality of a test image without any reference to the original one.

Image quality metrics can also be classified in terms of measuring only one specific type of degradation (e.g., blurring, blocking, or ringing), or taking into account all possible signal distortions, that is, multiple kinds of artifacts.^[7]

Image quality attributes

Sharpness determines the amount of detail an image can convey. System sharpness is affected by the lens (design and manufacturing quality, focal length, aperture, and distance from the image center) and sensor (pixel count and anti-aliasing filter). In the field, sharpness is affected by camera shake (a good tripod can be helpful), focus accuracy, and atmospheric disturbances (thermal effects and aerosols). Lost sharpness can be restored by sharpening, but sharpening has limits. Oversharpening, can degrade image quality by causing "halos" to appear near contrast boundaries. Images from many compact digital cameras are sometimes oversharpened to compensate for lower image quality.
Noise is a random variation of image density, visible as grain in film and pixel level variations in digital images. It arises from the effects of basic physics— the photon nature of light and the thermal energy of heat— inside image sensors. Typical noise reduction (NR) software reduces the visibility of noise by smoothing the image, excluding areas near contrast boundaries. This technique works well, but it can obscure fine, low contrast detail.
Dynamic range (or exposure range) is the range of light levels a camera can capture, usually measured in f-stops, EV (exposure value), or zones (all factors of two in exposure). It is closely related to noise: high noise implies low dynamic range.
Tone reproduction is the relationship between scene luminance and the reproduced image brightness.
Contrast , also known as gamma, is the slope of the tone reproduction curve in a log-log space. High contrast usually involves loss of dynamic range — loss of detail, or clipping, in highlights or shadows.
Color accuracy is an important but ambiguous image quality factor. Many viewers prefer enhanced color saturation; the most accurate color isn't necessarily the most pleasing. Nevertheless, it is important to measure a camera's color response: its color shifts, saturation, and the effectiveness of its white balance algorithms.
Distortion is an aberration that causes straight lines to curve. It can be troublesome for architectural photography and metrology (photographic applications involving measurement). Distortion tends to be noticeable in low cost cameras, including cell phones, and low cost DSLR lenses. It is usually very easy to see in wide angle photos. It can be now be corrected in software.
Vignetting , or light falloff, darkens images near the corners. It can be significant with wide angle lenses.
Exposure accuracy can be an issue with fully automatic cameras and with video cameras where there is little or no opportunity for post-exposure tonal adjustment. Some even have exposure memory: exposure may change after very bright or dark objects appear in a scene.
Lateral chromatic aberration (LCA), also called "color fringing", including purple fringing, is a lens aberration that causes colors to focus at different distances from the image center. It is most visible near corners of images. LCA is worst with asymmetrical lenses, including ultrawides, true telephotos and zooms. It is strongly affected by demosaicing.
Lens flare , including "veiling glare" is stray light in lenses and optical systems caused by reflections between lens elements and the inside barrel of the lens. It can cause image fogging (loss of shadow detail and color) as well as "ghost" images that can occur in the presence of bright light sources in or near the field of view.
Color moiré is artificial color banding that can appear in images with repetitive patterns of high spatial frequencies, like fabrics or picket fences. It is affected by lens sharpness, the anti-aliasing (lowpass) filter (which softens the image), and demosaicing software. It tends to be worst with the sharpest lenses.
Artifacts – software (especially operations performed during RAW conversion) can cause significant visual artifacts, including data compression and transmission losses (e.g. Low quality JPEG), oversharpening "halos" and loss of fine, low-contrast detail.

Related Research Articles

Peak signal-to-noise ratio (PSNR) is an engineering term for the ratio between the maximum possible power of a signal and the power of corrupting noise that affects the fidelity of its representation. Because many signals have a very wide dynamic range, PSNR is usually expressed as a logarithmic quantity using the decibel scale.

The science of photography is the use of chemistry and physics in all aspects of photography. This applies to the camera, its lenses, physical operation of the camera, electronic camera internals, and the process of developing film in order to take and develop pictures properly.

Tone mapping is a technique used in image processing and computer graphics to map one set of colors to another to approximate the appearance of high-dynamic-range (HDR) images in a medium that has a more limited dynamic range. Print-outs, CRT or LCD monitors, and projectors all have a limited dynamic range that is inadequate to reproduce the full range of light intensities present in natural scenes. Tone mapping addresses the problem of strong contrast reduction from the scene radiance to the displayable range while preserving the image details and color appearance important to appreciate the original scene content.

Video quality is a characteristic of a video passed through a video transmission or processing system that describes perceived video degradation. Video processing systems may introduce some amount of distortion or artifacts in the video signal that negatively impacts the user's perception of a system. For many stakeholders in video production and distribution, assurance of video quality is an important task.

<span class="mw-page-title-main">Image noise</span> Visible interference in an image

Image noise is random variation of brightness or color information in images, and is usually an aspect of electronic noise. It can be produced by the image sensor and circuitry of a scanner or digital camera. Image noise can also originate in film grain and in the unavoidable shot noise of an ideal photon detector. Image noise is an undesirable by-product of image capture that obscures the desired information. Typically the term “image noise” is used to refer to noise in 2D images, not 3D images.

The structural similarityindex measure (SSIM) is a method for predicting the perceived quality of digital television and cinematic pictures, as well as other kinds of digital images and videos. SSIM is used for measuring the similarity between two images. The SSIM index is a full reference metric; in other words, the measurement or prediction of image quality is based on an initial uncompressed or distortion-free image as reference.

In optics, defocus is the aberration in which an image is simply out of focus. This aberration is familiar to anyone who has used a camera, videocamera, microscope, telescope, or binoculars. Optically, defocus refers to a translation of the focus along the optical axis away from the detection surface. In general, defocus reduces the sharpness and contrast of the image. What should be sharp, high-contrast edges in a scene become gradual transitions. Fine detail in the scene is blurred or even becomes invisible. Nearly all image-forming optical devices incorporate some form of focus adjustment to minimize defocus and maximize image quality.

The following outline is provided as an overview of and topical guide to photography:

Perceptual Evaluation of Audio Quality (PEAQ) is a standardized algorithm for objectively measuring perceived audio quality, developed in 1994-1998 by a joint venture of experts within Task Group 6Q of the International Telecommunication Union's Radiocommunication Sector (ITU-R). It was originally released as ITU-R Recommendation BS.1387 in 1998 and last updated in 2023. It utilizes software to simulate perceptual properties of the human ear and then integrates multiple model output variables into a single metric.

Perceptual Evaluation of Speech Quality (PESQ) is a family of standards comprising a test methodology for automated assessment of the speech quality as experienced by a user of a telephony system. It was standardized as Recommendation ITU-T P.862 in 2001. PESQ is used for objective voice quality testing by phone manufacturers, network equipment vendors and telecom operators. Its usage requires a license. The first edition of PESQ's successor POLQA entered into force in 2011.

Perceptual Evaluation of Video Quality(PEVQ) is an end-to-end (E2E) measurement algorithm to score the picture quality of a video presentation by means of a 5-point mean opinion score (MOS). It is, therefore, a video quality model. PEVQ was benchmarked by the Video Quality Experts Group (VQEG) in the course of the Multimedia Test Phase 2007–2008. Based on the performance results, in which the accuracy of PEVQ was tested against ratings obtained by human viewers, PEVQ became part of the new International Standard.

Foveated imaging is a digital image processing technique in which the image resolution, or amount of detail, varies across the image according to one or more "fixation points". A fixation point indicates the highest resolution region of the image and corresponds to the center of the eye's retina, the fovea.

Alan Conrad Bovik is an American engineer, vision scientist, and educator. He is a professor at the University of Texas at Austin (UT-Austin), where he holds the Cockrell Family Regents Endowed Chair in the Cockrell School of Engineering and is Director of the Laboratory for Image and Video Engineering (LIVE). He is a faculty member in the UT-Austin Department of Electrical and Computer Engineering, the Machine Learning Laboratory, the Institute for Neuroscience, and the Wireless Networking and Communications Group.

Scene statistics is a discipline within the field of perception. It is concerned with the statistical regularities related to scenes. It is based on the premise that a perceptual system is designed to interpret scenes.

Image restoration is the operation of taking a corrupt/noisy image and estimating the clean, original image. Corruption may come in many forms such as motion blur, noise and camera mis-focus. Image restoration is performed by reversing the process that blurred the image and such is performed by imaging a point source and use the point source image, which is called the Point Spread Function (PSF) to restore the image information lost to the blurring process.

Perceptual Objective Listening Quality Analysis (POLQA) was the working title of an ITU-T standard that covers a model to predict speech quality by means of analyzing digital speech signals. The model was standardized as Recommendation ITU-T P.863 in 2011. The second edition of the standard appeared in 2014, and the third, currently in-force edition was adopted in 2018 under the title Perceptual objective listening quality prediction.

The MOtion-tuned Video Integrity Evaluation (MOVIE) index is a model and set of algorithms for predicting the perceived quality of digital television and cinematic pictures, as well as other kinds of digital images and videos.

Video Multimethod Assessment Fusion (VMAF) is an objective full-reference video quality metric developed by Netflix in cooperation with the University of Southern California, The IPI/LS2N lab Nantes Université, and the Laboratory for Image and Video Engineering (LIVE) at The University of Texas at Austin. It predicts subjective video quality based on a reference and distorted video sequence. The metric can be used to evaluate the quality of different video codecs, encoders, encoding settings, or transmission variants.

Computational imaging is the process of indirectly forming images from measurements using algorithms that rely on a significant amount of computing. In contrast to traditional imaging, computational imaging systems involve a tight integration of the sensing system and the computation in order to form the images of interest. The ubiquitous availability of fast computing platforms, the advances in algorithms and modern sensing hardware is resulting in imaging systems with significantly enhanced capabilities. Computational Imaging systems cover a broad range of applications include computational microscopy, tomographic imaging, MRI, ultrasound imaging, computational photography, Synthetic Aperture Radar (SAR), seismic imaging etc. The integration of the sensing and the computation in computational imaging systems allows for accessing information which was otherwise not possible. For example:

Image fidelity, often referred to as the ability to discriminate between two images or how closely the image represents the real source distribution. Different from image quality, which is often referred to as the subject preference for one image over another, image fidelity represents to the ability of a process to render an image accurately, without any visible distortion or information loss. The two terms are often used interchangeably, but they are not the same.

References

↑ Burningham, Norman; Pizlo, Zygmunt; Allebach, Jan P. (2002). "Image Quality Metrics". In Hornak, Joseph P. (ed.). Encyclopedia of imaging science and technology. New York: Wiley. doi:10.1002/0471443395.img038. ISBN 978-0-471-33276-3.
1 2 Wang, Zhou; Bovik, Alan C. (2006). "Preface". Modern image quality assessment. San Rafael: Morgan & Claypool Publishers. ISBN 978-1598290226.
1 2 Sheikh, Hamid Rahim; Bovik, Alan C. (February 2006). "Image Information and Visual Quality". IEEE Transactions on Image Processing. 15 (2): 430–444. Bibcode:2006ITIP...15..430S. CiteSeerX 10.1.1.477.2659 . doi:10.1109/TIP.2005.859378. PMID 16479813.
↑ P.910 : Subjective video quality assessment methods for multimedia applications. International Telecommunication Union. 6 April 2008. ^{[ dead link ]}
↑ Zhou Wang; Alan C. Bovik (2006). Modern Image Quality Assessment. pp. 11–15. ISBN 1-59829-022-3. OL 9866061M. Wikidata Q55757889.
↑ Keelan, Brian W. (2002). Handbook of image quality : characterization and prediction. New York, NY: Marcel Dekker, Inc. ISBN 978-0-8247-0770-5.
↑ Shahid, Muhammad; Rossholm, Andreas; Lövström, Benny; Zepernick, Hans-Jürgen (2014-08-14). "No-reference image and video quality assessment: a classification and review of recent approaches". EURASIP Journal on Image and Video Processing. 2014: 40. doi: 10.1186/1687-5281-2014-40 . ISSN 1687-5281.