Compression artifact

Last updated

Original image, with good text edges and color grade Sego lily cm-150.png
Original image, with good text edges and color grade
Loss of edge clarity and tone "fuzziness" in heavy JPEG compression Sego lily cm-150.jpg
Loss of edge clarity and tone "fuzziness" in heavy JPEG compression

A compression artifact (or artefact) is a noticeable distortion of media (including images, audio, and video) caused by the application of lossy compression. Lossy data compression involves discarding some of the media's data so that it becomes small enough to be stored within the desired disk space or transmitted (streamed) within the available bandwidth (known as the data rate or bit rate). If the compressor cannot store enough data in the compressed version, the result is a loss of quality, or introduction of artifacts. The compression algorithm may not be intelligent enough to discriminate between distortions of little subjective importance and those objectionable to the user.

Contents

The most common digital compression artifacts are DCT blocks, caused by the discrete cosine transform (DCT) compression algorithm used in many digital media standards, such as JPEG, MP3, and MPEG video file formats. [1] [2] [3] These compression artifacts appear when heavy compression is applied, [1] and occur often in common digital media, such as DVDs, common computer file formats such as JPEG, MP3 and MPEG files, and some alternatives to the compact disc, such as Sony's MiniDisc format. Uncompressed media (such as on Laserdiscs, Audio CDs, and WAV files) or losslessly compressed media (such as FLAC or PNG) do not suffer from compression artifacts.

The minimization of perceivable artifacts is a key goal in implementing a lossy compression algorithm. However, artifacts are occasionally intentionally produced for artistic purposes, a style known as glitch art [4] or datamoshing. [5]

Technically speaking, a compression artifact is a particular class of data error that is usually the consequence of quantization in lossy data compression. Where transform coding is used, it typically assumes the form of one of the basis functions of the coder's transform space.

Images

Illustration of the effect of JPEG compression on a slightly noisy image with a mixture of text and whitespace. Text is a screen capture from a Wikipedia conversation with noise added (intensity 10 in Paint.NET). One frame of the animation was saved as a JPEG (quality 90) and reloaded. Both frames were then zoomed by a factor of 4 (nearest neighbor interpolation). Jpeg-text-artifacts.gif
Illustration of the effect of JPEG compression on a slightly noisy image with a mixture of text and whitespace. Text is a screen capture from a Wikipedia conversation with noise added (intensity 10 in Paint.NET). One frame of the animation was saved as a JPEG (quality 90) and reloaded. Both frames were then zoomed by a factor of 4 (nearest neighbor interpolation).

When performing block-based discrete cosine transform (DCT) [1] coding for quantization, as in JPEG-compressed images, several types of artifacts can appear.

Other lossy algorithms, which use pattern matching to deduplicate similar symbols, are prone to introducing hard to detect errors in printed text. For example, the numbers "6" and "8" may get replaced. This has been observed to happen with JBIG2 in certain photocopier machines. [6] [7]

Block boundary artifacts

Block coding artifacts in a JPEG image. Flat blocks are caused by coarse quantization. Discontinuities at transform block boundaries are visible. The macroblocking effect (JPEG).png
Block coding artifacts in a JPEG image. Flat blocks are caused by coarse quantization. Discontinuities at transform block boundaries are visible.

At low bit rates, any lossy block-based coding scheme introduces visible artifacts in pixel blocks and at block boundaries. These boundaries can be transform block boundaries, prediction block boundaries, or both, and may coincide with macroblock boundaries. The term macroblocking is commonly used regardless of the artifact's cause. Other names include blocking, [8] tiling, [9] mosaicing, pixelating, quilting, and checkerboarding.

Block-artifacts are a result of the very principle of block transform coding. The transform (for example the discrete cosine transform) is applied to a block of pixels, and to achieve lossy compression, the transform coefficients of each block are quantized. The lower the bit rate, the more coarsely the coefficients are represented and the more coefficients are quantized to zero. Statistically, images have more low-frequency than high-frequency content, so it is the low-frequency content that remains after quantization, which results in blurry, low-resolution blocks. In the most extreme case only the DC-coefficient, that is the coefficient which represents the average color of a block, is retained, and the transform block is only a single color after reconstruction.

Because this quantization process is applied individually in each block, neighboring blocks quantize coefficients differently. This leads to discontinuities at the block boundaries. These are most visible in flat areas, where there is little detail to mask the effect.

Image artifact reduction

Various approaches have been proposed to reduce image compression effects, but to use standardized compression/decompression techniques and retain the benefits of compression (for instance, lower transmission and storage costs), many of these methods focus on "post-processing"—that is, processing images when received or viewed. No post-processing technique has been shown to improve image quality in all cases; consequently, none has garnered widespread acceptance, though some have been implemented and are in use in proprietary systems. Many photo editing programs, for instance, have proprietary JPEG artifact reduction algorithms built-in. Consumer equipment often calls this post-processing "MPEG Noise Reduction". [10]

Boundary artifact in JPEG can be turned into more pleasing "grains" not unlike those in high ISO photographic films. Instead of just multiplying the quantized coefficients with the quantisation step Q pertaining to the 2D-frequency, intelligent noise in the form of a random number in the interval [-Q/2; Q/2] can be added to the dequantized coefficient. This method can be added as an integral part to JPEG decompressors working on the trillions of existing and future JPEG images. As such it is not a "post-processing" technique. [11]

The ringing issue can be reduced at encode time by overshooting the DCT values, clamping the rings away. [12]

Posterization generally only happens at low quality, when the DC values are given too little importance. Tuning the quantization table helps. [13]

Video

Example of image with artifacts due to a transmission error Macroblocking error.png
Example of image with artifacts due to a transmission error

When motion prediction is used, as in MPEG-1, MPEG-2 or MPEG-4, compression artifacts tend to remain on several generations of decompressed frames, and move with the optic flow of the image, leading to a peculiar effect, part way between a painting effect and "grime" that moves with objects in the scene.

Data errors in the compressed bit-stream, possibly due to transmission errors, can lead to errors similar to large quantization errors, or can disrupt the parsing of the data stream entirely for a short time, leading to "break-up" of the picture. Where gross errors have occurred in the bit-stream, decoders continue to apply updates to the damaged picture for a short interval, creating a "ghost image" effect, until receiving the next independently compressed frame. In MPEG picture coding, these are known as "I-frames", with the 'I' standing for "intra". Until the next I-frame arrives, the decoder can perform error concealment.

Motion compensation block boundary artifacts

Block boundary discontinuities can occur at edges of motion compensation prediction blocks. In motion compensated video compression, the current picture is predicted by shifting blocks (macroblocks, partitions, or prediction units) of pixels from previously decoded frames. If two neighboring blocks use different motion vectors, there will be a discontinuity at the edge between the blocks.

Mosquito noise

Video compression artifacts include cumulative results of compression of the comprising still images, for instance ringing or other edge busyness in successive still images appear in sequence as a shimmering blur of dots around edges, called mosquito noise, as they resemble mosquitoes swarming around the object. [14] [15] The so-called "mosquito noise" is caused by the block-based discrete cosine transform (DCT) compression algorithm used in most video coding standards, such as the MPEG formats. [3]

Video artifact reduction

The artifacts at block boundaries can be reduced by applying a deblocking filter. As in still image coding, it is possible to apply a deblocking filter to the decoder output as post-processing.

In motion-predicted video coding with a closed prediction loop, the encoder uses the decoder output as the prediction reference from which future frames are predicted. To that end, the encoder conceptually integrates a decoder. If this "decoder" performs a deblocking, the deblocked picture is then used as a reference picture for motion compensation, which improves coding efficiency by preventing a propagation of block artifacts across frames. This is referred to as an in-loop deblocking filter. Standards which specify an in-loop deblocking filter include VC-1, H.263 Annex J, H.264/AVC, and H.265/HEVC.

Audio

Lossy audio compression typically works with a psychoacoustic model—a model of human hearing perception. Lossy audio formats typically involve the use of a time/frequency domain transform, such as a modified discrete cosine transform. With the psychoacoustic model, masking effects such as frequency masking and temporal masking are exploited, so that sounds that should be imperceptible are not recorded. For example, in general, human beings are unable to perceive a quiet tone played simultaneously with a similar but louder tone. A lossy compression technique might identify this quiet tone and attempt to remove it. Also, quantization noise can be "hidden" where they would be masked by more prominent sounds. With low compression, a conservative psy-model is used with small block sizes.

When the psychoacoustic model is inaccurate, when the transform block size is restrained, or when aggressive compression is used, this may result in compression artifacts. Compression artifacts in compressed audio typically show up as ringing, pre-echo, "birdie artifacts", drop-outs, rattling, warbling, metallic ringing, an underwater feeling, hissing, or "graininess".

An example of compression artifacts in audio is applause in a relatively highly compressed audio file (e.g. 96 kbit/sec MP3). In general, musical tones have repeating waveforms and more predictable variations in volume, whereas applause is essentially random, therefore hard to compress. A highly compressed track of applause may have "metallic ringing" and other compression artifacts.

Artistic use

Video glitch art (epilepsy warning)
Example of datamoshing
An example of a "deep fried" image made from a photo of a can of Diet Pepsi DeepfriedPepsiCan.jpg
An example of a "deep fried" image made from a photo of a can of Diet Pepsi

Compression artifacts may intentionally be used as a visual style, sometimes known as glitch art . Rosa Menkman's glitch art makes use of compression artifacts, [16] particularly the discrete cosine transform blocks (DCT blocks) found in most digital media data compression formats such as JPEG digital images and MP3 digital audio. [2] In still images, an example is Jpegs by German photographer Thomas Ruff, which uses intentional JPEG artifacts as the basis of the picture's style. [17] [18]

In video art, one technique used is datamoshing, where two videos are interleaved so intermediate frames are interpolated from two separate sources. Another technique involves simply transcoding from one lossy video format to another, which exploits the difference in how the separate video codecs process motion and color information. [19] The technique was pioneered by artists Bertrand Planes in collaboration with Christian Jacquemin in 2006 with DivXPrime, [20] Sven König, Takeshi Murata, Jacques Perconte and Paul B. Davis in collaboration with Paperrad, and more recently used by David OReilly and within music videos for Chairlift and by Nabil Elderkin in the "Welcome to Heartbreak" music video for Kanye West. [21] [22]

There is also a genre of internet memes where often nonsensical images are purposefully heavily compressed sometimes multiple times for comedic effect. Images created using this technique are often referred to as "deep fried." [23]

See also

Related Research Articles

In information theory, data compression, source coding, or bit-rate reduction is the process of encoding information using fewer bits than the original representation. Any particular compression is either lossy or lossless. Lossless compression reduces bits by identifying and eliminating statistical redundancy. No information is lost in lossless compression. Lossy compression reduces bits by removing unnecessary or less important information. Typically, a device that performs data compression is referred to as an encoder, and one that performs the reversal of the process (decompression) as a decoder.

<span class="mw-page-title-main">JPEG</span> Lossy compression method for reducing the size of digital images

JPEG is a commonly used method of lossy compression for digital images, particularly for those images produced by digital photography. The degree of compression can be adjusted, allowing a selectable tradeoff between storage size and image quality. JPEG typically achieves 10:1 compression with little perceptible loss in image quality. Since its introduction in 1992, JPEG has been the most widely used image compression standard in the world, and the most widely used digital image format, with several billion JPEG images produced every day as of 2015.

<span class="mw-page-title-main">Lossy compression</span> Data compression approach that reduces data size while discarding or changing some of it

In information technology, lossy compression or irreversible compression is the class of data compression methods that uses inexact approximations and partial data discarding to represent the content. These techniques are used to reduce data size for storing, handling, and transmitting content. Higher degrees of approximation create coarser images as more details are removed. This is opposed to lossless data compression which does not degrade the data. The amount of data reduction possible using lossy compression is much higher than using lossless techniques.

MPEG-1 is a standard for lossy compression of video and audio. It is designed to compress VHS-quality raw digital video and CD audio down to about 1.5 Mbit/s without excessive quality loss, making video CDs, digital cable/satellite TV and digital audio broadcasting (DAB) practical.

<span class="mw-page-title-main">Image compression</span> Reduction of image size to save storage and transmission costs

Image compression is a type of data compression applied to digital images, to reduce their cost for storage or transmission. Algorithms may take advantage of visual perception and the statistical properties of image data to provide superior results compared with generic data compression methods which are used for other digital data.

Transform coding is a type of data compression for "natural" data like audio signals or photographic images. The transformation is typically lossless on its own but is used to enable better quantization, which then results in a lower quality copy of the original input.

<span class="mw-page-title-main">Motion compensation</span> Video compression technique, used to efficiently predict and generate video frames

Motion compensation in computing is an algorithmic technique used to predict a frame in a video given the previous and/or future frames by accounting for motion of the camera and/or objects in the video. It is employed in the encoding of video data for video compression, for example in the generation of MPEG-2 files. Motion compensation describes a picture in terms of the transformation of a reference picture to the current picture. The reference picture may be previous in time or even from the future. When images can be accurately synthesized from previously transmitted/stored images, the compression efficiency can be improved.

<span class="mw-page-title-main">Video codec</span> Digital video processing

A video codec is software or hardware that compresses and decompresses digital video. In the context of video compression, codec is a portmanteau of encoder and decoder, while a device that only compresses is typically called an encoder, and one that only decompresses is a decoder.

A discrete cosine transform (DCT) expresses a finite sequence of data points in terms of a sum of cosine functions oscillating at different frequencies. The DCT, first proposed by Nasir Ahmed in 1972, is a widely used transformation technique in signal processing and data compression. It is used in most digital media, including digital images, digital video, digital audio, digital television, digital radio, and speech coding. DCTs are also important to numerous other applications in science and engineering, such as digital signal processing, telecommunication devices, reducing network bandwidth usage, and spectral methods for the numerical solution of partial differential equations.

<span class="mw-page-title-main">JPEG 2000</span> Image compression standard and coding system

JPEG 2000 (JP2) is an image compression standard and coding system. It was developed from 1997 to 2000 by a Joint Photographic Experts Group committee chaired by Touradj Ebrahimi, with the intention of superseding their original JPEG standard, which is based on a discrete cosine transform (DCT), with a newly designed, wavelet-based method. The standardized filename extension is .jp2 for ISO/IEC 15444-1 conforming files and .jpx for the extended part-2 specifications, published as ISO/IEC 15444-2. The registered MIME types are defined in RFC 3745. For ISO/IEC 15444-1 it is image/jp2.

H.261 is an ITU-T video compression standard, first ratified in November 1988. It is the first member of the H.26x family of video coding standards in the domain of the ITU-T Study Group 16 Video Coding Experts Group. It was the first video coding standard that was useful in practical terms.

Cinepak is a lossy video codec developed by Peter Barrett at SuperMac Technologies, and released in 1991 with the Video Spigot, and then in 1992 as part of Apple Computer's QuickTime video suite. One of the first video compression tools to achieve full motion video on CD-ROM, it was designed to encode 320×240 resolution video at 1× CD-ROM transfer rates. The original name of this codec was Compact Video, which is why its FourCC identifier is CVID. The codec was ported to Microsoft Windows in 1993. It was also used on fourth- and fifth-generation game consoles, such as the Atari Jaguar CD, Sega CD, Sega Saturn, and 3DO. libavcodec includes a Cinepak decoder and an encoder, both licensed under the terms of the LGPL.

Quantization, involved in image processing, is a lossy compression technique achieved by compressing a range of values to a single quantum (discrete) value. When the number of discrete symbols in a given stream is reduced, the stream becomes more compressible. For example, reducing the number of colors required to represent a digital image makes it possible to reduce its file size. Specific applications include DCT data quantization in JPEG and DWT data quantization in JPEG 2000.

<span class="mw-page-title-main">Digital artifact</span> Computer graphic effect

Digital artifact in information science, is any undesired or unintended alteration in data introduced in a digital process by an involved technique and/or technology.

JPEG XR is an image compression standard for continuous tone photographic images, based on the HD Photo specifications that Microsoft originally developed and patented. It supports both lossy and lossless compression, and is the preferred image format for Ecma-388 Open XML Paper Specification documents.

The macroblock is a processing unit in image and video compression formats based on linear block transforms, typically the discrete cosine transform (DCT). A macroblock typically consists of 16×16 samples, and is further subdivided into transform blocks, and may be further subdivided into prediction blocks. Formats which are based on macroblocks include JPEG, where they are called MCU blocks, H.261, MPEG-1 Part 2, H.262/MPEG-2 Part 2, H.263, MPEG-4 Part 2, and H.264/MPEG-4 AVC. In H.265/HEVC, the macroblock as a basic processing unit has been replaced by the coding tree unit.

<span class="mw-page-title-main">Progressive Graphics File</span> File format

PGF is a wavelet-based bitmapped image format that employs lossless and lossy data compression. PGF was created to improve upon and replace the JPEG format. It was developed at the same time as JPEG 2000 but with a focus on speed over compression ratio.

A video coding format is a content representation format of digital video content, such as in a data file or bitstream. It typically uses a standardized video compression algorithm, most commonly based on discrete cosine transform (DCT) coding and motion compensation. A specific software, firmware, or hardware implementation capable of compression or decompression in a specific video coding format is called a video codec.

<span class="mw-page-title-main">Nasir Ahmed (engineer)</span> Indian-American electrical engineer and computer scientist (born 1940)

Nasir Ahmed is an Indian-American electrical engineer and computer scientist. He is Professor Emeritus of Electrical and Computer Engineering at University of New Mexico (UNM). He is best known for inventing the discrete cosine transform (DCT) in the early 1970s. The DCT is the most widely used data compression transformation, the basis for most digital media standards and commonly used in digital signal processing. He also described the discrete sine transform (DST), which is related to the DCT.

ZPEG is a motion video technology that applies a human visual acuity model to a decorrelated transform-domain space, thereby optimally reducing the redundancies in motion video by removing the subjectively imperceptible. This technology is applicable to a wide range of video processing problems such as video optimization, real-time motion video compression, subjective quality monitoring, and format conversion.

References

  1. 1 2 3 Katsaggelos, Aggelos K.; Babacan, S. Derin; Chun-Jen, Tsai (2009). "Chapter 15 - Iterative Image Restoration". The Essential Guide to Image Processing. Academic Press. pp. 349–383. ISBN   9780123744579.
  2. 1 2 Alikhani, Darya (1 April 2015). "Beyond resolution: Rosa Menkman's glitch art". POSTmatter. Archived from the original on 19 October 2019. Retrieved 19 October 2019.
  3. 1 2 "Mosquito noise". PC Magazine . Retrieved 19 October 2019.
  4. ↑ Geere, Duncan (13 December 2011). "Glitch art created by 'databending'". Wired. Retrieved 23 December 2011.
  5. ↑ Baker-Smith, Ben (28 April 2009). "Datamoshing – The Beauty of Glitch". Bitsynthesis.com. Archived from the original on 16 November 2010. Retrieved 28 April 2009.
  6. ↑ "Xerox scanners/photocopiers randomly alter numbers in scanned documents". 2 August 2013. Retrieved 4 August 2013.
  7. ↑ "Confused Xerox copiers rewrite documents, expert finds". BBC News. 6 August 2013. Retrieved 6 August 2013.
  8. ↑ Amiri, Sekine Asadi; Hassanpour, Hamid (1 April 2018). "Image compression using JPEG with reduced blocking effects via adaptive down-sampling and self-learning image sparse representation". Multimedia Tools and Applications. 77 (7): 8677–8693. doi:10.1007/s11042-017-4763-1. ISSN   1573-7721 . Retrieved 8 March 2024.
  9. ↑ Watkinson, John (2004). The MPEG handbook by John Watkinson. Taylor & Francis. ISBN   9780240805788.
  10. ↑ "PC Magazine, Definition of blocking artifacts". Archived from the original on 7 October 2012. Retrieved 23 September 2020.
  11. ↑ Hudson, Graham; Léger, Alain; Niss, Birger; Sebestyén, István; Vaaben, Jørgen (31 August 2018). "JPEG.1 standard 25 years: past, present and future reasons for success". Journal of Electronic Imaging. 27 (4): 1. doi: 10.1117/1.JEI.27.4.040901 . S2CID   52164892.
  12. ↑ Richter, Thomas (September 2016). "JPEG on STEROIDS: Common optimization techniques for JPEG image compression". 2016 IEEE International Conference on Image Processing (ICIP). pp. 61–65. doi:10.1109/ICIP.2016.7532319. ISBN   978-1-4673-9961-6. S2CID   14922251.
  13. ↑ "kornelski/jpeg-compressor". GitHub . 16 November 2020.
  14. ↑ Le Dinh, Phuc-Tue; Patry, Jacques. "Video compression artifacts and MPEG noise reduction". Embedded. Retrieved 19 February 2016.
  15. ↑ "3.9 mosquito noise: Form of edge busyness distortion sometimes associated with movement, characterized by moving artifacts and/or blotchy noise patterns superimposed over the objects (resembling a mosquito flying around a person's head and shoulders)." ITU-T Rec. P.930 (08/96) Principles of a reference impairment system for video
  16. ↑ Menkman, Rosa (October 2011). The Glitch Moment(um) (PDF). Institute of Network Cultures. ISBN   978-90-816021-6-7 . Retrieved 19 October 2019.
  17. ↑ jpegs, Thomas Ruff, Aperture, May 31, 2009, 132 pp., ISBN   978-1-59711-093-8
  18. ↑ Review: jpegs by Thomas Ruff, by Jörg Colberg, April 17, 2009
  19. ↑ Anoniem zei (19 February 2009). "From compression artifact to filter". Rosa-menkman.blogspot.com. Retrieved 23 December 2011.
  20. ↑ Jacquemin, Christian (2008). "Le bug dans l'oeuvre DivXPrime de Bertrand Planes: Invention et mutation. In, Ivan Toulouse and Daniel Danétis, editors, Eurêka: Le moment de l'invention, un dialogue entre art et science, L'Harmattan, Paris" (PDF). pp. 245–256. Retrieved 5 November 2012.
  21. ↑ Pixel Bleed, by John Michael Boling. Rhizome. February 25, 2009.
  22. ↑ Rodriguez, Jayson (18 February 2009). "Kanye West Rushes New Video Onto His Web Site – MTV News". Mtv.com. Archived from the original on 19 February 2009. Retrieved 23 December 2011.
  23. ↑ Matsakis, Louise (30 August 2017). "How to Deep-Fry a Meme". Vice . Retrieved 27 July 2021.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.