Edge-preserving smoothing

Last updated

Edge-preserving smoothing is an image processing technique that smooths away noise or textures while retaining sharp edges. Examples are the median, bilateral, guided, and anisotropic diffusion filters.

Contents

Introduction

In many applications, e.g., medical or satellite imaging, the edges are key features and thus must be preserved sharp and undistorted in smoothing/denoising. [1] Edge-preserving filters are designed to automatically limit the smoothing at “edges” in images measured, e.g., by high gradient magnitudes.

For example, the motivation for anisotropic diffusion (also called nonuniform or variable conductance diffusion) is that a Gaussian smoothed image is a single time slice of the solution to the heat equation, that has the original image as its initial conditions. Anisotropic diffusion includes a variable conductance term that is determined using the differential structure of the image, such that the heat does not propagate over the edges of the image.

The edge-preserving filters can conveniently be formulated in a general context of graph-based signal processing, where the graph adjacency matrix is first determined using the differential structure of the image, then the graph Laplacian is formulated (analogous to the anisotropic diffusion operator), and finally the approximate low-pass filter is constructed to amplify the eigenvectors of the graph Laplacian corresponding to its smallest eigenvalues.

Since the edges only implicitly appear in constructing the edge-preserving filters, a typical filter uses some parameters, that can be tuned, to balance between aggressive averaging and edge preservation. A common default choice for the parameters of the filter is aimed for natural images and results in strong denoising at the cost of some smoothing of the edges.

Iterative filters

Requirements of the strict edge preservation commonly limit the smoothing power of the filter, such that a single application of the filter still results in unacceptably large noise away from the edges. A repetitive application of the filter may be useful to reduce the noise, leading to the idea of combining the filter with an iterative method, e.g., the Chebyshev iteration and the conjugate gradient method are proposed in [2] for graph-based image denoising.

Due to the interpretation of the edge-preserving filters as low-pass graph-based filters, iterative eigenvalue solvers, such as LOBPCG, can be used for denoising; see, [3] e.g., to accelerate the repeated application of the total variation denoising.

Edge-enhancing smoothing

Anisotropic diffusion generates small conductance at the location of the edge of the image to prevent the heat flow over the edge, thus making the anisotropic diffusion filter edge-preserving. In the graph-based interpretation, the small conductance corresponds to a small weight of an edge of the graph describing a probability of a random walk over the edge in the Markov chain on the graph. If the graph weight was negative, that would correspond to a negative conductivity in the heat equation, stimulating the heat concentration at the graph vertices connected by the graph edge, rather than the normal heat dissipation. While not-physical for the heat equation, this effect results in sharpening corners of one-dimensional signals, when used in graph-based smoothing filters, as shown in reference [4] that also provides an alternative physical interpretation using the wave equation describing mechanical vibrations of a mass-spring system with some repulsive springs.

Edge-preserving upsampling

Signal upsampling via the traditional interpolation followed by smoothing for denoising evidently distorts the edges in the original ideal or downsampled signal. The edge-preserving interpolation followed by the edge-preserving filters is proposed in [5] e.g., to upsample a no-flash RGB photo guided using a high resolution flash RGB photo, and a depth image guided using a high resolution RGB photo.

Related Research Articles

Nonlinear dimensionality reduction Summary of algorithms for nonlinear dimensionality reduction

High-dimensional data, meaning data that requires more than two or three dimensions to represent, can be difficult to interpret. One approach to simplification is to assume that the data of interest lie on an embedded non-linear manifold within the higher-dimensional space. If the manifold is of low enough dimension, the data can be visualised in the low-dimensional space.

Canny edge detector

The Canny edge detector is an edge detection operator that uses a multi-stage algorithm to detect a wide range of edges in images. It was developed by John F. Canny in 1986. Canny also produced a computational theory of edge detection explaining why the technique works.

Noise reduction

Noise reduction is the process of removing noise from a signal. Noise reduction techniques exist for audio and images. Noise reduction algorithms tend to alter signals to a greater or lesser degree.

Image segmentation

In digital image processing and computer vision, image segmentation is the process of partitioning a digital image into multiple segments. The goal of segmentation is to simplify and/or change the representation of an image into something that is more meaningful and easier to analyze. Image segmentation is typically used to locate objects and boundaries in images. More precisely, image segmentation is the process of assigning a label to every pixel in an image such that pixels with the same label share certain characteristics.

Anisotropic filtering Method of enhancing the image quality of textures on surfaces of computer graphics

In 3D computer graphics, anisotropic filtering is a method of enhancing the image quality of textures on surfaces of computer graphics that are at oblique viewing angles with respect to the camera where the projection of the texture appears to be non-orthogonal.

Parks–McClellan filter design algorithm

The Parks–McClellan algorithm, published by James McClellan and Thomas Parks in 1972, is an iterative algorithm for finding the optimal Chebyshev finite impulse response (FIR) filter. The Parks–McClellan algorithm is utilized to design and implement efficient and optimal FIR filters. It uses an indirect method for finding the optimal filter coefficients.

Scale-space segmentation

Scale-space segmentation or multi-scale segmentation is a general framework for signal and image segmentation, based on the computation of image descriptors at multiple scales of smoothing.

The topological derivative is, conceptually, a derivative of a shape functional with respect to infinitesimal changes in its topology, such as adding an infinitesimal hole or crack. When used in higher dimensions than one, the term topological gradient is also used to name the first-order term of the topological asymptotic expansion, dealing only with infinitesimal singular domain perturbations. It has applications in shape optimization, topology optimization, image processing and mechanical modeling.

Compressed sensing is a signal processing technique for efficiently acquiring and reconstructing a signal, by finding solutions to underdetermined linear systems. This is based on the principle that, through optimization, the sparsity of a signal can be exploited to recover it from far fewer samples than required by the Nyquist–Shannon sampling theorem. There are two conditions under which recovery is possible. The first one is sparsity, which requires the signal to be sparse in some domain. The second one is incoherence, which is applied through the isometric property, which is sufficient for sparse signals.

Spectral clustering

In multivariate statistics and the clustering of data, spectral clustering techniques make use of the spectrum (eigenvalues) of the similarity matrix of the data to perform dimensionality reduction before clustering in fewer dimensions. The similarity matrix is provided as an input and consists of a quantitative assessment of the relative similarity of each pair of points in the dataset.

Bilateral filter

A bilateral filter is a non-linear, edge-preserving, and noise-reducing smoothing filter for images. It replaces the intensity of each pixel with a weighted average of intensity values from nearby pixels. This weight can be based on a Gaussian distribution. Crucially, the weights depend not only on Euclidean distance of pixels, but also on the radiometric differences. This preserves sharp edges.

Contourlets form a multiresolution directional tight frame designed to efficiently approximate images made of smooth regions separated by smooth boundaries. The contourlet transform has a fast implementation based on a Laplacian pyramid decomposition followed by directional filterbanks applied on each bandpass subband.

Pyramid (image processing)

Pyramid, or pyramid representation, is a type of multi-scale signal representation developed by the computer vision, image processing and signal processing communities, in which a signal or an image is subject to repeated smoothing and subsampling. Pyramid representation is a predecessor to scale-space representation and multiresolution analysis.

In image processing and computer vision, anisotropic diffusion, also called Perona–Malik diffusion, is a technique aiming at reducing image noise without removing significant parts of the image content, typically edges, lines or other details that are important for the interpretation of the image. Anisotropic diffusion resembles the process that creates a scale space, where an image generates a parameterized family of successively more and more blurred images based on a diffusion process. Each of the resulting images in this family are given as a convolution between the image and a 2D isotropic Gaussian filter, where the width of the filter increases with the parameter. This diffusion process is a linear and space-invariant transformation of the original image. Anisotropic diffusion is a generalization of this diffusion process: it produces a family of parameterized images, but each resulting image is a combination between the original image and a filter that depends on the local content of the original image. As a consequence, anisotropic diffusion is a non-linear and space-variant transformation of the original image.

Total variation denoising

In signal processing, total variation denoising, also known as total variation regularization, is a process, most often used in digital image processing, that has applications in noise removal. It is based on the principle that signals with excessive and possibly spurious detail have high total variation, that is, the integral of the absolute gradient of the signal is high. According to this principle, reducing the total variation of the signal subject to it being a close match to the original signal, removes unwanted detail whilst preserving important details such as edges. The concept was pioneered by Rudin, Osher, and Fatemi in 1992 and so is today known as the ROF model.

Locally Optimal Block Preconditioned Conjugate Gradient (LOBPCG) is a matrix-free method for finding the largest eigenvalues and the corresponding eigenvectors of a symmetric positive definite generalized eigenvalue problem

Non-local means is an algorithm in image processing for image denoising. Unlike "local mean" filters, which take the mean value of a group of pixels surrounding a target pixel to smooth the image, non-local means filtering takes a mean of all pixels in the image, weighted by how similar these pixels are to the target pixel. This results in much greater post-filtering clarity, and less loss of detail in the image compared with local mean algorithms.

Diffusion map

Diffusion maps is a dimensionality reduction or feature extraction algorithm introduced by Coifman and Lafon which computes a family of embeddings of a data set into Euclidean space whose coordinates can be computed from the eigenvectors and eigenvalues of a diffusion operator on the data. The Euclidean distance between points in the embedded space is equal to the "diffusion distance" between probability distributions centered at those points. Different from linear dimensionality reduction methods such as principal component analysis (PCA) and multi-dimensional scaling (MDS), diffusion maps is part of the family of nonlinear dimensionality reduction methods which focus on discovering the underlying manifold that the data has been sampled from. By integrating local similarities at different scales, diffusion maps give a global description of the data-set. Compared with other methods, the diffusion map algorithm is robust to noise perturbation and computationally inexpensive.

Block-matching and 3D filtering Algorithm for noise reduction in images

Block-matching and 3D filtering (BM3D) is a 3-D block-matching algorithm used primarily for noise reduction in images.

Gradient vector flow

Gradient vector flow (GVF), a computer vision framework introduced by Chenyang Xu and Jerry L. Prince , is the vector field that is produced by a process that smooths and diffuses an input vector field. It is usually used to create a vector field from images that points to object edges from a distance. It's widely used in image analysis and computer vision applications for object tracking, shape recognition, segmentation, and edge detection. In particular, it's commonly used in conjunction with active contour model.

References

  1. Tatar, Nurollah, et al. "High-Resolution Satellite Stereo Matching by Object-Based Semiglobal Matching and Iterative Guided Edge-Preserving Filter." IEEE Geoscience and Remote Sensing Letters (2020): 1-5.
  2. Tian, D.; Mansour, H.; Knyazev, A.; Vetro, A. (2014). Chebyshev and Conjugate Gradient Filters for Graph Image Denoising. IEEE International Conference on Multimedia and Expo Workshops (ICMEW). pp. 1–6. arXiv: 1509.01624 . doi:10.1109/ICMEW.2014.6890711.
  3. Knyazev, A.; Malyshev, A. (2015). Accelerated graph-based spectral polynomial filters. 2015 IEEE 25th International Workshop on Machine Learning for Signal Processing (MLSP), Boston, MA. pp. 1–6. arXiv: 1509.02468 . doi:10.1109/MLSP.2015.7324315.
  4. Knyazev, A. (2015). Edge-enhancing Filters with Negative Weights. IEEE Global Conference on Signal and Information Processing (GlobalSIP), Orlando, FL, 14-16 Dec.2015. pp. 260–264. arXiv: 1509.02491 . doi:10.1109/GlobalSIP.2015.7418197.
  5. Knyazev, A.; Malyshev, A. (2017). Signal reconstruction via operator guiding. SampTA 2017: Sampling Theory and Applications, 12th International Conference, July 3–7, 2017, Tallinn, Estonia. pp. 630–634. arXiv: 1705.03493 . doi:10.1109/SAMPTA.2017.8024424.