Bilateral filter

Last updated
Left: original image. Right: image processed with bilateral filter Bilateral Filter.jpg
Left: original image. Right: image processed with bilateral filter

A bilateral filter is a non-linear, edge-preserving, and noise-reducing smoothing filter for images. It replaces the intensity of each pixel with a weighted average of intensity values from nearby pixels. This weight can be based on a Gaussian distribution. Crucially, the weights depend not only on Euclidean distance of pixels, but also on the radiometric differences (e.g., range differences, such as color intensity, depth distance, etc.). This preserves sharp edges.

Contents

Definition

The bilateral filter is defined as [1] [2]

and normalization term, , is defined as

where

is the filtered image;
is the original input image to be filtered;
are the coordinates of the current pixel to be filtered;
is the window centered in , so is another pixel;
is the range kernel for smoothing differences in intensities (this function can be a Gaussian function);
is the spatial (or domain) kernel for smoothing differences in coordinates (this function can be a Gaussian function).

The weight is assigned using the spatial closeness (using the spatial kernel ) and the intensity difference (using the range kernel ). [2] Consider a pixel located at that needs to be denoised in image using its neighbouring pixels and one of its neighbouring pixels is located at . Then, assuming the range and spatial kernels to be Gaussian kernels, the weight assigned for pixel to denoise the pixel is given by

where σd and σr are smoothing parameters, and I(i, j) and I(k, l) are the intensity of pixels and respectively.

After calculating the weights, normalize them:

where is the denoised intensity of pixel .

Parameters

Limitations

The bilateral filter in its direct form can introduce several types of image artifacts:

There exist several extensions to the filter that deal with these artifacts, like the scaled bilateral filter that uses downscaled image for computing the weights. [5] Alternative filters, like the guided filter, [6] have also been proposed as an efficient alternative without these limitations.

Implementations

Adobe Photoshop implements a bilateral filter in its surface blur tool. GIMP implements a bilateral filter in its Filters-->Blur tools; and it is called Selective Gaussian Blur. The free G'MIC plugin Repair → Smooth [bilateral] for GIMP adds more control. [7] A simple trick to efficiently implement a bilateral filter is to exploit Poisson-disk subsampling. [1]

The bilateral filter has been shown to be an application of the short time kernel of the Beltrami flow [8] [9] [10] that was introduced as an edge preserving selective smoothing mechanism before the bilateral filter.

Other edge-preserving smoothing filters include: anisotropic diffusion, weighted least squares, [11] edge-avoiding wavelets, [12] geodesic editing, [13] guided filtering, [14] iterative guided filtering [15] and domain transforms. [16]

See also

Related Research Articles

In digital signal processing, spatial anti-aliasing is a technique for minimizing the distortion artifacts known as aliasing when representing a high-resolution image at a lower resolution. Anti-aliasing is used in digital photography, computer graphics, digital audio, and many other applications.

Edge detection includes a variety of mathematical methods that aim at identifying points in a digital image at which the image brightness changes sharply or, more formally, has discontinuities. The points at which image brightness changes sharply are typically organized into a set of curved line segments termed edges. The same problem of finding discontinuities in one-dimensional signals is known as step detection and the problem of finding signal discontinuities over time is known as change detection. Edge detection is a fundamental tool in image processing, machine vision and computer vision, particularly in the areas of feature detection and feature extraction.

Sobel operator

The Sobel operator, sometimes called the Sobel–Feldman operator or Sobel filter, is used in image processing and computer vision, particularly within edge detection algorithms where it creates an image emphasising edges. It is named after Irwin Sobel and Gary Feldman, colleagues at the Stanford Artificial Intelligence Laboratory (SAIL). Sobel and Feldman presented the idea of an "Isotropic 3x3 Image Gradient Operator" at a talk at SAIL in 1968. Technically, it is a discrete differentiation operator, computing an approximation of the gradient of the image intensity function. At each point in the image, the result of the Sobel–Feldman operator is either the corresponding gradient vector or the norm of this vector. The Sobel–Feldman operator is based on convolving the image with a small, separable, and integer-valued filter in the horizontal and vertical directions and is therefore relatively inexpensive in terms of computations. On the other hand, the gradient approximation that it produces is relatively crude, in particular for high-frequency variations in the image.

Canny edge detector

The Canny edge detector is an edge detection operator that uses a multi-stage algorithm to detect a wide range of edges in images. It was developed by John F. Canny in 1986. Canny also produced a computational theory of edge detection explaining why the technique works.

Noise reduction is the process of removing noise from a signal. Noise reduction techniques exist for audio and images. Noise reduction algorithms tend to alter signals to a greater or lesser degree.

In mathematics, the discrete Laplace operator is an analog of the continuous Laplace operator, defined so that it has meaning on a graph or a discrete grid. For the case of a finite-dimensional graph, the discrete Laplace operator is more commonly called the Laplacian matrix.

Gaussian blur

In image processing, a Gaussian blur is the result of blurring an image by a Gaussian function.

In imaging science, difference of Gaussians (DoG) is a feature enhancement algorithm that involves the subtraction of one Gaussian blurred version of an original image from another, less blurred version of the original. In the simple case of grayscale images, the blurred images are obtained by convolving the original grayscale images with Gaussian kernels having differing width. Blurring an image using a Gaussian kernel suppresses only high-frequency spatial information. Subtracting one image from the other preserves spatial information that lies between the range of frequencies that are preserved in the two blurred images. Thus, the DoG is a spatial band-pass filter that attenuates frequencies in the original grayscale image that are far from the band center.

Corner detection

Corner detection is an approach used within computer vision systems to extract certain kinds of features and infer the contents of an image. Corner detection is frequently used in motion detection, image registration, video tracking, image mosaicing, panorama stitching, 3D reconstruction and object recognition. Corner detection overlaps with the topic of interest point detection.

In image processing and computer vision, a scale space framework can be used to represent an image as a family of gradually smoothed images. This framework is very general and a variety of scale space representations exist. A typical approach for choosing a particular type of scale space representation is to establish a set of scale-space axioms, describing basic properties of the desired scale-space representation and often chosen so as to make the representation useful in practical applications. Once established, the axioms narrow the possible scale-space representations to a smaller class, typically with only a few free parameters.

Mean shift is a non-parametric feature-space analysis technique for locating the maxima of a density function, a so-called mode-seeking algorithm. Application domains include cluster analysis in computer vision and image processing.

In the fields of computer vision and image analysis, the Harris affine region detector belongs to the category of feature detection. Feature detection is a preprocessing step of several algorithms that rely on identifying characteristic points or interest points so to make correspondences between images, recognize textures, categorize objects or build panoramas.

Pyramid (image processing) type of multi-scale signal representation

Pyramid, or pyramid representation, is a type of multi-scale signal representation developed by the computer vision, image processing and signal processing communities, in which a signal or an image is subject to repeated smoothing and subsampling. Pyramid representation is a predecessor to scale-space representation and multiresolution analysis.

In image processing and computer vision, anisotropic diffusion, also called Perona–Malik diffusion, is a technique aiming at reducing image noise without removing significant parts of the image content, typically edges, lines or other details that are important for the interpretation of the image. Anisotropic diffusion resembles the process that creates a scale space, where an image generates a parameterized family of successively more and more blurred images based on a diffusion process. Each of the resulting images in this family are given as a convolution between the image and a 2D isotropic Gaussian filter, where the width of the filter increases with the parameter. This diffusion process is a linear and space-invariant transformation of the original image. Anisotropic diffusion is a generalization of this diffusion process: it produces a family of parameterized images, but each resulting image is a combination between the original image and a filter that depends on the local content of the original image. As a consequence, anisotropic diffusion is a non-linear and space-variant transformation of the original image.

Step detection eezz

In statistics and signal processing, step detection is the process of finding abrupt changes in the mean level of a time series or signal. It is usually considered as a special case of the statistical method known as change detection or change point detection. Often, the step is small and the time series is corrupted by some kind of noise, and this makes the problem challenging because the step may be hidden by the noise. Therefore, statistical and/or signal processing algorithms are often required.

Non-local means is an algorithm in image processing for image denoising. Unlike "local mean" filters, which take the mean value of a group of pixels surrounding a target pixel to smooth the image, non-local means filtering takes a mean of all pixels in the image, weighted by how similar these pixels are to the target pixel. This results in much greater post-filtering clarity, and less loss of detail in the image compared with local mean algorithms.

In image Noise reduction, local pixel grouping is the algorithm to remove noise from images using principal component analysis (PCA).

Receptive field profiles registered by cell recordings have shown that mammalian vision has developed receptive fields tuned to different sizes and orientations in the image domain as well as to different image velocities in space-time. Corresponding cell recordings in the auditory system has shown that mammals have developed receptive fields tuned to different frequencies as well as temporal transients. This article describes normative theories that have been developed to explain these properties of sensory receptive fields based on structural properties of the environment. Beyond theoretical explanation of biological phenomena, these theories can also be used for computational modelling of biological receptive fields and for building algorithms for artificial perception based on sensory data.

In image analysis, the average with limited data validity is an image filter for feature-preserving noise removal, consisting in a smoothing filter that only involves pixels satisfying some validity criterion. If some feature of noise elements is known, it is possible to use it to define a criterion to detect invalid pixels, and selectively smooth only invalid pixels using data coming only from valid pixels, thus avoiding to affect other features of the image.

Shrinkage fields is a random field-based machine learning technique that aims to perform high quality image restoration using low computational overhead.

References

  1. 1 2 Banterle, F.; Corsini, M.; Cignoni, P.; Scopigno, R. (2011). "A Low-Memory, Straightforward and Fast Bilateral Filter Through Subsampling in Spatial Domain". Computer Graphics Forum. 31 (1): 19–32. doi:10.1111/j.1467-8659.2011.02078.x. S2CID   18288647.
  2. 1 2 Tomasi, C; Manduchi, R (1998). Bilateral filtering for gray and color images (PDF). Sixth International Conference on Computer Vision. Bombay. pp. 839–846. doi:10.1109/ICCV.1998.710815.
  3. Kornprobst, Pierre (2007). "Limitations? - A Gentle Introductionto Bilateral Filteringand its Applications" (PDF). Retrieved 7 May 2017.
  4. He, Kaiming; Sun, Jian; Tang, Xiaoou. "Guided Image Filtering" (PDF). Retrieved 7 May 2017.
  5. Aswatha, Shashaank M.; Mukhopadhyay, Jayanta; Bhowmick, Partha (December 2011). "Image Denoising by Scaled Bilateral Filtering". 2011 Third National Conference on Computer Vision, Pattern Recognition, Image Processing and Graphics: 122–125. doi:10.1109/NCVPRIPG.2011.33. ISBN   978-1-4577-2102-1. S2CID   25738863.
  6. He, Kaiming. "Guided Image Filtering" . Retrieved 7 May 2017.
  7. http://gmic.eu/gimp.shtml
  8. R. Kimmel, R. Malladi, and N. Sochen. Images as embedding maps and minimal surfaces: Movies, color, and volumetric medical images. IEEE CVPR'97, pp. 350-355, Puerto Rico, June 17–19, 1997. https://www.cs.technion.ac.il/~ron/PAPERS/cvpr97.pdf
  9. R. Kimmel, R. Malladi, and N. Sochen. Images as Embedded Maps and Minimal Surfaces: Movies, Color, Texture, and Volumetric Medical Images. International Journal of Computer Vision, 39(2):111-129, Sept. 2000. https://www.cs.technion.ac.il/~ron/PAPERS/KimMalSoc_IJCV2000.pdf
  10. N. Sochen, R. Kimmel, and A.M. Bruckstein. Diffusions and confusions in signal and image processing, Journal of Mathematical Imaging and Vision, 14(3):195-209, 2001.https://www.cs.technion.ac.il/~ron/PAPERS/SocKimBru_JMIV2001.pdf
  11. Farbman, Zeev, Raanan Fattal, Dani Lischinski, and Richard Szeliski. "Edge-preserving decompositions for multi-scale tone and detail manipulation." In ACM Transactions on Graphics, vol. 27, no. 3 (2008): 67. http://www.cs.huji.ac.il/~danix/epd/
  12. Fattal, Raanan. "Edge-avoiding wavelets and their applications." In ACM Transactions on Graphics vol. 28, no. 3 (2009): 22. http://www.cs.huji.ac.il/~raananf/projects/eaw/
  13. Criminisi, Antonio, Toby Sharp, Carsten Rother, and Patrick Pérez. "Geodesic image and video editing." In ACM Transactions on Graphphics (TOG), vol. 29, no. 5 (2010): 134. http://research.microsoft.com/apps/pubs/default.aspx?id=81528
  14. He, Kaiming, Jian Sun, and Xiaoou Tang. "Guided image filtering." In Computer Vision–ECCV 2010, pp. 1-14. Springer Berlin Heidelberg, 2010. http://kaiminghe.com/eccv10/index.html
  15. Tatar, Nurollah, et al. "High-Resolution Satellite Stereo Matching by Object-Based Semiglobal Matching and Iterative Guided Edge-Preserving Filter." IEEE Geoscience and Remote Sensing Letters (2020): 1-5.
  16. Gastal, Eduardo S. L., and Manuel M. Oliveira. "Domain transform for edge-aware image and video processing." In ACM Transactions on Graphics, vol. 30, no. 4 (2011): 69. http://inf.ufrgs.br/~eslgastal/DomainTransform/