Bilateral filter

Last updated
Left: original image. Right: image processed with bilateral filter Bilateral Filter.jpg
Left: original image. Right: image processed with bilateral filter

A bilateral filter is a non-linear, edge-preserving, and noise-reducing smoothing filter for images. It replaces the intensity of each pixel with a weighted average of intensity values from nearby pixels. This weight can be based on a Gaussian distribution. Crucially, the weights depend not only on Euclidean distance of pixels, but also on the radiometric differences (e.g., range differences, such as color intensity, depth distance, etc.). This preserves sharp edges.

Contents

Definition

The bilateral filter is defined as [1] [2]

and normalization term, , is defined as

where

is the filtered image;
is the original input image to be filtered;
are the coordinates of the current pixel to be filtered;
is the window centered in , so is another pixel;
is the range kernel for smoothing differences in intensities (this function can be a Gaussian function);
is the spatial (or domain) kernel for smoothing differences in coordinates (this function can be a Gaussian function).

The weight is assigned using the spatial closeness (using the spatial kernel ) and the intensity difference (using the range kernel ). [2] Consider a pixel located at that needs to be denoised in image using its neighbouring pixels and one of its neighbouring pixels is located at . Then, assuming the range and spatial kernels to be Gaussian kernels, the weight assigned for pixel to denoise the pixel is given by

where σd and σr are smoothing parameters, and I(i, j) and I(k, l) are the intensity of pixels and respectively.

After calculating the weights, normalize them:

where is the denoised intensity of pixel .

Parameters

Limitations

The bilateral filter in its direct form can introduce several types of image artifacts:

There exist several extensions to the filter that deal with these artifacts, like the scaled bilateral filter that uses downscaled image for computing the weights. [5] Alternative filters, like the guided filter, [6] have also been proposed as an efficient alternative without these limitations.

Implementations

Adobe Photoshop implements a bilateral filter in its surface blur tool. GIMP implements a bilateral filter in its Filters → Blur tools; and it is called Selective Gaussian Blur. The free G'MIC plugin Repair → Smooth [bilateral] for GIMP adds more control. [7] A simple trick to efficiently implement a bilateral filter is to exploit Poisson-disk subsampling. [1]

The bilateral filter has been shown to be an application of the short time kernel of the Beltrami flow [8] [9] [10] that was introduced as an edge preserving selective smoothing mechanism before the bilateral filter.

Other edge-preserving smoothing filters include: anisotropic diffusion, [11] weighted least squares, [12] edge-avoiding wavelets, [13] geodesic editing, [14] guided filtering, [15] iterative guided filtering [16] and domain transforms. [17]

See also

Related Research Articles

Digital image processing is the use of a digital computer to process digital images through an algorithm. As a subcategory or field of digital signal processing, digital image processing has many advantages over analog image processing. It allows a much wider range of algorithms to be applied to the input data and can avoid problems such as the build-up of noise and distortion during processing. Since images are defined over two dimensions digital image processing may be modeled in the form of multidimensional systems. The generation and development of digital image processing are mainly affected by three factors: first, the development of computers; second, the development of mathematics ; third, the demand for a wide range of applications in environment, agriculture, military, industry and medical science has increased.

In digital signal processing, spatial anti-aliasing is a technique for minimizing the distortion artifacts (aliasing) when representing a high-resolution image at a lower resolution. Anti-aliasing is used in digital photography, computer graphics, digital audio, and many other applications.

Edge detection includes a variety of mathematical methods that aim at identifying edges, defined as curves in a digital image at which the image brightness changes sharply or, more formally, has discontinuities. The same problem of finding discontinuities in one-dimensional signals is known as step detection and the problem of finding signal discontinuities over time is known as change detection. Edge detection is a fundamental tool in image processing, machine vision and computer vision, particularly in the areas of feature detection and feature extraction.

<span class="mw-page-title-main">Canny edge detector</span> Image edge detection algorithm

The Canny edge detector is an edge detection operator that uses a multi-stage algorithm to detect a wide range of edges in images. It was developed by John F. Canny in 1986. Canny also produced a computational theory of edge detection explaining why the technique works.

Noise reduction is the process of removing noise from a signal. Noise reduction techniques exist for audio and images. Noise reduction algorithms may distort the signal to some degree. Noise rejection is the ability of a circuit to isolate an undesired signal component from the desired signal component, as with common-mode rejection ratio.

In mathematics, the discrete Laplace operator is an analog of the continuous Laplace operator, defined so that it has meaning on a graph or a discrete grid. For the case of a finite-dimensional graph, the discrete Laplace operator is more commonly called the Laplacian matrix.

Scale-space theory is a framework for multi-scale signal representation developed by the computer vision, image processing and signal processing communities with complementary motivations from physics and biological vision. It is a formal theory for handling image structures at different scales, by representing an image as a one-parameter family of smoothed images, the scale-space representation, parametrized by the size of the smoothing kernel used for suppressing fine-scale structures. The parameter in this family is referred to as the scale parameter, with the interpretation that image structures of spatial size smaller than about have largely been smoothed away in the scale-space level at scale .

<span class="mw-page-title-main">Gaussian blur</span> Type of image blur produced by a Gaussian function

In image processing, a Gaussian blur is the result of blurring an image by a Gaussian function.

In imaging science, difference of Gaussians (DoG) is a feature enhancement algorithm that involves the subtraction of one Gaussian blurred version of an original image from another, less blurred version of the original. In the simple case of grayscale images, the blurred images are obtained by convolving the original grayscale images with Gaussian kernels having differing width. Blurring an image using a Gaussian kernel suppresses only high-frequency spatial information. Subtracting one image from the other preserves spatial information that lies between the range of frequencies that are preserved in the two blurred images. Thus, the DoG is a spatial band-pass filter that attenuates frequencies in the original grayscale image that are far from the band center.

<span class="mw-page-title-main">Corner detection</span> Approach used in computer vision systems

Corner detection is an approach used within computer vision systems to extract certain kinds of features and infer the contents of an image. Corner detection is frequently used in motion detection, image registration, video tracking, image mosaicing, panorama stitching, 3D reconstruction and object recognition. Corner detection overlaps with the topic of interest point detection.

As applied in the field of computer vision, graph cut optimization can be employed to efficiently solve a wide variety of low-level computer vision problems, such as image smoothing, the stereo correspondence problem, image segmentation, object co-segmentation, and many other computer vision problems that can be formulated in terms of energy minimization. Many of these energy minimization problems can be approximated by solving a maximum flow problem in a graph. Under most formulations of such problems in computer vision, the minimum energy solution corresponds to the maximum a posteriori estimate of a solution. Although many computer vision algorithms involve cutting a graph, the term "graph cuts" is applied specifically to those models which employ a max-flow/min-cut optimization.

<span class="mw-page-title-main">Mean shift</span> Mathematical technique

Mean shift is a non-parametric feature-space mathematical analysis technique for locating the maxima of a density function, a so-called mode-seeking algorithm. Application domains include cluster analysis in computer vision and image processing.

In the fields of computer vision and image analysis, the Harris affine region detector belongs to the category of feature detection. Feature detection is a preprocessing step of several algorithms that rely on identifying characteristic points or interest points so to make correspondences between images, recognize textures, categorize objects or build panoramas.

<span class="mw-page-title-main">Pyramid (image processing)</span> Type of multi-scale signal representation

Pyramid, or pyramid representation, is a type of multi-scale signal representation developed by the computer vision, image processing and signal processing communities, in which a signal or an image is subject to repeated smoothing and subsampling. Pyramid representation is a predecessor to scale-space representation and multiresolution analysis.

In image processing and computer vision, anisotropic diffusion, also called Perona–Malik diffusion, is a technique aiming at reducing image noise without removing significant parts of the image content, typically edges, lines or other details that are important for the interpretation of the image. Anisotropic diffusion resembles the process that creates a scale space, where an image generates a parameterized family of successively more and more blurred images based on a diffusion process. Each of the resulting images in this family are given as a convolution between the image and a 2D isotropic Gaussian filter, where the width of the filter increases with the parameter. This diffusion process is a linear and space-invariant transformation of the original image. Anisotropic diffusion is a generalization of this diffusion process: it produces a family of parameterized images, but each resulting image is a combination between the original image and a filter that depends on the local content of the original image. As a consequence, anisotropic diffusion is a non-linear and space-variant transformation of the original image.

The random walker algorithm is an algorithm for image segmentation. In the first description of the algorithm, a user interactively labels a small number of pixels with known labels, e.g., "object" and "background". The unlabeled pixels are each imagined to release a random walker, and the probability is computed that each pixel's random walker first arrives at a seed bearing each label, i.e., if a user places K seeds, each with a different label, then it is necessary to compute, for each pixel, the probability that a random walker leaving the pixel will first arrive at each seed. These probabilities may be determined analytically by solving a system of linear equations. After computing these probabilities for each pixel, the pixel is assigned to the label for which it is most likely to send a random walker. The image is modeled as a graph, in which each pixel corresponds to a node which is connected to neighboring pixels by edges, and the edges are weighted to reflect the similarity between the pixels. Therefore, the random walk occurs on the weighted graph.

<span class="mw-page-title-main">Step detection</span>

In statistics and signal processing, step detection is the process of finding abrupt changes in the mean level of a time series or signal. It is usually considered as a special case of the statistical method known as change detection or change point detection. Often, the step is small and the time series is corrupted by some kind of noise, and this makes the problem challenging because the step may be hidden by the noise. Therefore, statistical and/or signal processing algorithms are often required.

<span class="mw-page-title-main">Foreground detection</span>

Foreground detection is one of the major tasks in the field of computer vision and image processing whose aim is to detect changes in image sequences. Background subtraction is any technique which allows an image's foreground to be extracted for further processing.

In image analysis, the average with limited data validity is an image filter for feature-preserving noise removal, consisting in a smoothing filter that only involves pixels satisfying some validity criterion. If some feature of noise elements is known, it is possible to use it to define a criterion to detect invalid pixels, and selectively smooth only invalid pixels using data coming only from valid pixels, thus avoiding to affect other features of the image.

Shrinkage fields is a random field-based machine learning technique that aims to perform high quality image restoration using low computational overhead.

References

  1. 1 2 Banterle, F.; Corsini, M.; Cignoni, P.; Scopigno, R. (2011). "A Low-Memory, Straightforward and Fast Bilateral Filter Through Subsampling in Spatial Domain". Computer Graphics Forum. 31 (1): 19–32. doi:10.1111/j.1467-8659.2011.02078.x. S2CID   18288647.
  2. 1 2 Tomasi, C; Manduchi, R (1998). Bilateral filtering for gray and color images (PDF). Sixth International Conference on Computer Vision. Bombay. pp. 839–846. doi:10.1109/ICCV.1998.710815. Archived from the original (PDF) on 2008-08-29. Retrieved 2009-11-18.
  3. Kornprobst, Pierre (2007). "Limitations? - A Gentle Introductionto Bilateral Filteringand its Applications" (PDF). Retrieved 7 May 2017.
  4. He, Kaiming; Sun, Jian; Tang, Xiaoou. "Guided Image Filtering" (PDF). Archived from the original (PDF) on 9 December 2017. Retrieved 7 May 2017.
  5. Aswatha, Shashaank M.; Mukhopadhyay, Jayanta; Bhowmick, Partha (December 2011). "Image Denoising by Scaled Bilateral Filtering". 2011 Third National Conference on Computer Vision, Pattern Recognition, Image Processing and Graphics. pp. 122–125. doi:10.1109/NCVPRIPG.2011.33. ISBN   978-1-4577-2102-1. S2CID   25738863.
  6. He, Kaiming. "Guided Image Filtering". Archived from the original on 19 December 2017. Retrieved 7 May 2017.
  7. "G'MIC - GREYC's Magic for Image Computing: An Open and Full-Featured Framework for Image Processing". gmic.eu. Archived from the original on 2014-12-27.
  8. R. Kimmel, R. Malladi, and N. Sochen. Images as embedding maps and minimal surfaces: Movies, color, and volumetric medical images. IEEE CVPR'97, pp. 350-355, Puerto Rico, June 17–19, 1997. https://www.cs.technion.ac.il/~ron/PAPERS/cvpr97.pdf
  9. R. Kimmel, R. Malladi, and N. Sochen. Images as Embedded Maps and Minimal Surfaces: Movies, Color, Texture, and Volumetric Medical Images. International Journal of Computer Vision, 39(2):111-129, Sept. 2000. https://www.cs.technion.ac.il/~ron/PAPERS/KimMalSoc_IJCV2000.pdf
  10. N. Sochen, R. Kimmel, and A.M. Bruckstein. Diffusions and confusions in signal and image processing, Journal of Mathematical Imaging and Vision, 14(3):195-209, 2001.https://www.cs.technion.ac.il/~ron/PAPERS/SocKimBru_JMIV2001.pdf
  11. Durand, Frédo and Dorsey, Julie. "Fast Bilateral Filtering for the Display of High-Dynamic-Range Images" In ACM Transactions on Graphics, Volume 21, Issue 3, pp 257–266. https://people.csail.mit.edu/fredo/PUBLI/Siggraph2002/DurandBilateral.pdf
  12. Farbman, Zeev, Raanan Fattal, Dani Lischinski, and Richard Szeliski. "Edge-preserving decompositions for multi-scale tone and detail manipulation." In ACM Transactions on Graphics, vol. 27, no. 3 (2008): 67. http://www.cs.huji.ac.il/~danix/epd/
  13. Fattal, Raanan. "Edge-avoiding wavelets and their applications." In ACM Transactions on Graphics vol. 28, no. 3 (2009): 22. http://www.cs.huji.ac.il/~raananf/projects/eaw/
  14. Criminisi, Antonio, Toby Sharp, Carsten Rother, and Patrick Pérez. "Geodesic image and video editing." In ACM Transactions on Graphphics (TOG), vol. 29, no. 5 (2010): 134. http://research.microsoft.com/apps/pubs/default.aspx?id=81528
  15. He, Kaiming, Jian Sun, and Xiaoou Tang. "Guided image filtering." In Computer Vision–ECCV 2010, pp. 1-14. Springer Berlin Heidelberg, 2010. http://kaiminghe.com/eccv10/index.html Archived 2017-12-19 at the Wayback Machine
  16. Tatar, Nurollah, et al. "High-Resolution Satellite Stereo Matching by Object-Based Semiglobal Matching and Iterative Guided Edge-Preserving Filter." IEEE Geoscience and Remote Sensing Letters (2020): 1-5.
  17. Gastal, Eduardo S. L., and Manuel M. Oliveira. "Domain transform for edge-aware image and video processing." In ACM Transactions on Graphics, vol. 30, no. 4 (2011): 69. http://inf.ufrgs.br/~eslgastal/DomainTransform/