Phase-stretch Adaptive Gradient-field Extractor

Last updated June 11, 2022

Phase-Stretch Adaptive Gradient-Field Extractor (PAGE) is an edge detection algorithm based on physics of electromagnetic diffraction and dispersion. A computational imaging algorithm, it identifies edges, their orientations and sharpness in a digital image where the image brightness changes abruptly. Edge detection is a basic operation performed by the eye and is crucial to visual perception.

PAGE embeds an original image into a set of feature maps that selects semantic information at different scales, orientations, and spatial frequencies that can be used for object representation and classification. The algorithm performs exceptionally well as an edge and texture extractor, in particular in low light level and low contrast impaired images. As a visualization aid, the edge angle is typically encoded into color in the output image. The code was first released in February, 2022. The code was then significantly refactored and improved to support GPU acceleration. In May 2022, it became one algorithm in PhyCV: the first physics-inspired computer vision library.

Operation principle

Phase-stretch Adaptive Gradient-field Extractor (PAGE) is a physics inspired feature engineering algorithm^[1] that computes a feature set composed of edges at different spatial frequencies (and hence spatial scales) and orientations.^[2]^[3] Metaphorically speaking, PAGE emulates the physics of birefringent (orientation-dependent) diffractive propagation through a physical medium with a specific diffractive structure. The propagation converts a real-valued image into a complex function. Related information is contained in the real and imaginary components of the output. The output represents the phase of the complex function. PAGE builds on the Phase stretch transform (PST),^[4] another physics-inspired edge detection algorithm. The Phase stretch transform algorithm evolved from the research on a class of real time measurement and sensing methods known as the photonic times stretch including time stretch analog-to-digital converter,^[5] Time stretch dispersive Fourier transform ^[6] and serial time-encoded amplified microscopy.^[7]

In a birefringent optical medium, the dielectric constant of the medium and hence, its refractive index is a function of spatial frequency and the polarization in the transverse plane. To understand the analogy between PAGE and electromagnetic propagation equations, let's consider an optical field with two linearly orthogonal polarizations propagating through a medium. The Fourier content of the incoming signal,

${\tilde {E}}_{i}\left(u,v;z\right)=FFT^{2}\left\{E_{i}(x,y;z)\right\}$

can be decomposed into the two orthogonal polarizations as

${\tilde {E}}_{i}\left(z\right)={\tilde {E}}_{x}(z)+{\tilde {E}}_{y}(z)$

where $FFT\left\{\right\}$ is the fast Fourier transform over the transversal coordinates and $(u,v)$ are spatial frequency variables. As the propagation constant $\beta =2\pi n/\lambda$ is a function of refractive index, the two orthogonal polarizations ${\tilde {E}}_{x}$ and ${\tilde {E}}_{y}$ will have different propagation constants and hence, a phase difference at the output given by the following equation:

$\Delta \phi =\phi _{x}-\phi _{y}=\Delta \beta ={\frac {\omega _{m}}{c}}\mid {n_{x}-n_{y}}\mid$

By controlling the value of $n_{x}$ and $n_{x}$ , as well the dependence of refractive index on frequency $n_{x}(\omega )$ and $n_{y}(\omega )$ , coherent detection at the output detects a hyper-dimensional feature set from a 2D image that corresponds to edges at user-defined specific orientations and spatial frequencies. We note that in the above definition of the phase, we have set the propagation length to 1.

Physical and mathematical foundations of Phase-Stretch Adaptive Gradient-Field Extractor

The first step is to apply an optional smoothening kernel in the frequency domain to reduce noise. This is typically performed in the frequency domain (after a Fourier transform), but it can also be done in the spatial domain using convolution. The image is then multiplied by a phase kernel that emulates the birefringence and frequency channelized diffractive propagation. Next, the image is transformed back into the spatial domain followed by a calculation of the spatial phase representing the desired feature vectors. The final step of PAGE is to apply thresholding and morphological operations on the generated feature vectors to produce the final output. For a color image, these operations are performed separately on all color channels and the results are then combined in a single image, although each channel can also be viewed separately.

Mathematically, this sequence of operations can be represented by the following equations. The birefringent Stretch operator $\mathbb {S} \left\{\right\}$ is defined as follows:

$E_{o}\left[x,y\right]=\mathbb {S} \left\{E_{i}\left[x,y\right]\right\}=IFFT^{2}{\bigg \{}{\tilde {K}}\left[u,v,\theta \right]\cdot {}{\tilde {L}}\left[u,v\right]\cdot {}FFT^{2}{\Big \{}E_{i}\left[x,y\right]{\Big \}}{\bigg \}}$

where $E_{o}\left[x,y\right]$ is a complex quantity defined as,

$E_{o}\left[x,y\right]=\left\vert {}E_{o}\left[x,y\right]\right\vert {}e^{j\theta {}\left[x,y\right]}$

In the above equations, $E_{i}\left[x,y\right]$ is the input image, $x$ and $y$ are the spatial variables, $FFT^{2}$ is the two-dimensional Fast Fourier Transform, $IFFT^{2}$ is the two-dimensional Inverse Fast Fourier Transform, and $u$ and $v$ are frequency variables. The function ${\tilde {K}}[u,v,\theta ]$ is called the PAGE kernel and the function ${\tilde {L}}[u,v]$ is a denoising kernel, both implemented in frequency domain. For the results shown on this Wiki page, ${\tilde {L}}\left[u,v\right]$ is a gaussian filter whose cut off frequency is determined by the sigma of the gaussian filter.

The PAGE operator $\mathbb {P} \left\{\right\}$ is then be defined as the phase of the output of the stretch operation $\mathbb {S} \left\{\right\}$ applied on the input image $E_{i}\left[x,y\right]$ :

$\mathbb {P} \left\{E_{i}\left[x,y\right]\right\}=\measuredangle {\Big \{}\mathbb {S} {\big \{}E_{i}\left[x,y\right]{\big \}}{\Big \}}$

where $\measuredangle \langle \cdot \rangle$ is the angle operator.

PAGE filter banks

PAGE filter banks are defined by the PAGE kernel ${\tilde {K}}\left[u,v,\theta \right]$ and are designed to compute semantic information from an image at different orientations and frequencies. The PAGE kernel ${\tilde {K}}\left[u,v,\theta \right]$ , consists of a phase filter which is a function of frequency variable $u$ and $v$ , and the angle variable $\theta$ which controls the directionality of the edge. The spectral phase operator is expressed as a product of two phase functions, $\phi _{1}$ and $\phi _{2}$ . The first component $\phi _{1}$ is a symmetric gaussian filter that selects the spatial frequency range of the edges that are detected. Default center frequency is 0, which indicates a baseband filter, the center frequency and bandwidth of which can be changed to probe edges with different sharpness. In other words, it enables the filtering of edges occurring over different spatial scales. The second component, $\phi _{2}$ , performs the edge-detection. Since the output is based on the phase, it needs to be a complex-valued function. The PAGE operation transforms a real-value input to a complex-value quantity from which the phase is extracted.

A change of basis leads to the transformed frequency variables $u^{\prime }$ and $v^{\prime }$

$u^{\prime }=u\cdot cos(\theta )+v\cdot sin(\theta )$

$v^{\prime }=u\cdot sin(\theta )+v\cdot cos(\theta )$

such that the frequency vector rotates about the origin with $\theta$

$u^{\prime }+jv^{\prime }$

The PAGE kernel ${\tilde {K}}\left[u,v,\theta \right]$ is defined as a function of frequency variable $u$ and $v$ and angle $\theta$ as follows:

${\tilde {K}}\left[u,v,\theta \right]={\tilde {K}}\left[u^{\prime },v^{\prime }\right]=\exp {\Big \{}j\cdot \phi _{1}({u^{\prime }})\cdot \phi _{2}({v^{\prime }}){\Big \}}$

where

$\phi _{1}({u^{\prime }})=S_{u^{\prime }}\cdot {\frac {1}{\sigma _{u^{\prime }}{\sqrt {2\pi }}}}\cdot exp^{{-({|u^{\prime }|}-\mu _{u^{\prime }})^{2}}/{2\sigma _{u^{\prime }}^{2}}}$

$\phi _{2}({v^{\prime }})=S_{v^{\prime }}\cdot {\frac {1}{{|v^{\prime }|}\sigma _{v^{\prime }}{\sqrt {2\pi }}}}\cdot exp^{{-(ln({|v^{\prime }|})-\mu _{v^{\prime }})^{2}}/{2\sigma _{v^{\prime }}^{2}}}$

For all simulation examples here, the phase functions $\phi _{1}({u^{\prime }})$ and $\phi _{2}({v^{\prime }})$ are normalized in the range (0,1) for all values of $\theta$ and then multiplied by $S_{u^{\prime }}$ and $S_{v^{\prime }}$ respectively, such that the strength of each kernel is mutable for different applications and image conditions.

Feature extraction

PAGE has the potential to be used as a preprocessing step for machine learning tasks such as image classification. An important step in any classification task is feature extraction. Feature extraction algorithms of note include histogram of oriented gradients, scale-invariant feature transform, and shape context descriptors. In each case, images are reduced to certain key features that aid in the tasks of object detection and classification. PAGE serves as a physics-inspired feature extractor and descriptor. It is able to return a hyper-dimensional feature mapping in which regions of great change in intensity are highlighted and grouped based on directionality.

Applications

PAGE has the potential to be used as a preprocessing step for machine learning tasks such as image classification. An important step in any classification task is feature extraction. In such applications, images are reduced to certain key features that aid in the tasks of object detection and classification. PAGE serves as a physics-inspired feature extractor and descriptor. It returns a hyper-dimensional feature mapping in which regions of great change in intensity are highlighted and grouped based on directionality. Given that it is selective over edge width and orientation, it is able to return a rich feature matrix with high representational power.

PAGE has a diverse set of applications that span several fields. Diagnosis and classification of retinopathy, for example, are medically important tasks highly dependent upon segmentation of blood vessels of varied width and orientation. This segmentation, and further image analysis, can be accomplished through a directional edge filter such as PAGE. Similarly, digital subtraction angiography creates an image of blood vessels using a contrast medium that can be used in pathology for soft tissue. Such imagery can be processed using the PAGE filter for diagnosis and visualization purposes. Further applications of note requiring directional edge information such as that computed by PAGE include fingerprint, written character, and flora and fauna recognition.

Originally introduced in 2020, PAGE builds on the Phase Stretch Transform (PST).^[8] Local Flow PST (LF-PST) is another algorithm introduced in 2020 that is based on PST and also performs orientation and scale dependent edge detection.^[9] Local Flow PST has shown exceptional results in retina vessel detection for application to retinopathy.

Related Research Articles

In fluid dynamics, potential flow describes the velocity field as the gradient of a scalar function: the velocity potential. As a result, a potential flow is characterized by an irrotational velocity field, which is a valid approximation for several applications. The irrotationality of a potential flow is due to the curl of the gradient of a scalar always being equal to zero.

In mechanics and geometry, the 3D rotation group, often denoted SO(3), is the group of all rotations about the origin of three-dimensional Euclidean space $under the operation of composition.$

Fourier optics is the study of classical optics using Fourier transforms (FTs), in which the waveform being considered is regarded as made up of a combination, or superposition, of plane waves. It has some parallels to the Huygens–Fresnel principle, in which the wavefront is regarded as being made up of a combination of spherical wavefronts whose sum is the wavefront being studied. A key difference is that Fourier optics considers the plane waves to be natural modes of the propagation medium, as opposed to Huygens–Fresnel, where the spherical waves originate in the physical medium.

In control theory and signal processing, a linear, time-invariant system is said to be minimum-phase if the system and its inverse are causal and stable.

In quantum mechanics and quantum field theory, the propagator is a function that specifies the probability amplitude for a particle to travel from one place to another in a given period of time, or to travel with a certain energy and momentum. In Feynman diagrams, which serve to calculate the rate of collisions in quantum field theory, virtual particles contribute their propagator to the rate of the scattering event described by the respective diagram. These may also be viewed as the inverse of the wave operator appropriate to the particle, and are, therefore, often called (causal) Green's functions.

This is a list of some vector calculus formulae for working with common curvilinear coordinate systems.

Projectile motion Motion of launched objects due to gravity

Projectile motion is a form of motion experienced by an object or particle that is projected near Earth's surface and moves along a curved path under the action of gravity only. This curved path was shown by Galileo to be a parabola, but may also be a straight line in the special case when it is thrown directly upwards. The study of such motions is called ballistics, and such a trajectory is a ballistic trajectory. The only force of mathematical significance that is actively exerted on the object is gravity, which acts downward, thus imparting to the object a downward acceleration towards the Earth’s center of mass. Because of the object's inertia, no external force is needed to maintain the horizontal velocity component of the object's motion. Taking other forces into account, such as aerodynamic drag or internal propulsion, requires additional analysis. A ballistic missile is a missile only guided during the relatively brief initial powered phase of flight, and whose remaining course is governed by the laws of classical mechanics.

In physics and engineering, a phasor is a complex number representing a sinusoidal function whose amplitude (A), angular frequency (ω), and initial phase (θ) are time-invariant. It is related to a more general concept called analytic representation, which decomposes a sinusoid into the product of a complex constant and a factor depending on time and frequency. The complex constant, which depends on amplitude and phase, is known as a phasor, or complex amplitude, and sinor or even complexor.

In quantum field theory, a quartic interaction is a type of self-interaction in a scalar field. Other types of quartic interactions may be found under the topic of four-fermion interactions. A classical free scalar field $satisfies the Klein-Gordon equation. If a scalar field is denoted, a quartic interaction is represented by adding a potential energy term to the Lagrangian density. The coupling constant is dimensionless in 4-dimensional spacetime.$

In mathematics, a volume element provides a means for integrating a function with respect to volume in various coordinate systems such as spherical coordinates and cylindrical coordinates. Thus a volume element is an expression of the form

In special functions, a topic in mathematics, spin-weighted spherical harmonics are generalizations of the standard spherical harmonics and—like the usual spherical harmonics—are functions on the sphere. Unlike ordinary spherical harmonics, the spin-weighted harmonics are $U(1)$ gauge fields rather than scalar fields: mathematically, they take values in a complex line bundle. The spin-weighted harmonics are organized by degree $l$ , just like ordinary spherical harmonics, but have an additional spin weight $s$ that reflects the additional $U(1)$ symmetry. A special basis of harmonics can be derived from the Laplace spherical harmonics $Y lm$ , and are typically denoted by $s Y lm$ , where $l$ and $m$ are the usual parameters familiar from the standard Laplace spherical harmonics. In this special basis, the spin-weighted spherical harmonics appear as actual functions, because the choice of a polar axis fixes the $U(1)$ gauge ambiguity. The spin-weighted spherical harmonics can be obtained from the standard spherical harmonics by application of spin raising and lowering operators. In particular, the spin-weighted spherical harmonics of spin weight $s = 0$ are simply the standard spherical harmonics:

In geometry, various formalisms exist to express a rotation in three dimensions as a mathematical transformation. In physics, this concept is applied to classical mechanics where rotational kinematics is the science of quantitative description of a purely rotational motion. The orientation of an object at a given instant is described with the same tools, as it is defined as an imaginary rotation from a reference placement in space, rather than an actually observed rotation from a previous placement in space.

In general relativity, a point mass deflects a light ray with impact parameter $by an angle approximately equal to$

In fluid dynamics, the mild-slope equation describes the combined effects of diffraction and refraction for water waves propagating over bathymetry and due to lateral boundaries—like breakwaters and coastlines. It is an approximate model, deriving its name from being originally developed for wave propagation over mild slopes of the sea floor. The mild-slope equation is often used in coastal engineering to compute the wave-field changes near harbours and coasts.

The contrast transfer function (CTF) mathematically describes how aberrations in a transmission electron microscope (TEM) modify the image of a sample. This contrast transfer function (CTF) sets the resolution of high-resolution transmission electron microscopy (HRTEM), also known as phase contrast TEM.

An anamorphic stretch transform (AST) also referred to as warped stretch transform is a physics-inspired signal transform that emerged from time stretch dispersive Fourier transform. The transform can be applied to analog temporal signals such as communication signals, or to digital spatial data such as images. The transform reshapes the data in such a way that its output has properties conducive for data compression and analytics. The reshaping consists of warped stretching in the Fourier domain. The name "Anamorphic" is used because of the metaphoric analogy between the warped stretch operation and warping of images in anamorphosis and surrealist artworks.

The spectrum of a chirp pulse describes its characteristics in terms of its frequency components. This frequency-domain representation is an alternative to the more familiar time-domain waveform, and the two versions are mathematically related by the Fourier transform.
The spectrum is of particular interest when pulses are subject to signal processing. For example, when a chirp pulse is compressed by its matched filter, the resulting waveform contains not only a main narrow pulse but, also, a variety of unwanted artifacts many of which are directly attributable to features in the chirp's spectral characteristics.
The simplest way to derive the spectrum of a chirp, now that computers are widely available, is to sample the time-domain waveform at a frequency well above the Nyquist limit and call up an FFT algorithm to obtain the desired result. As this approach was not an option for the early designers, they resorted to analytic analysis, where possible, or to graphical or approximation methods, otherwise. These early methods still remain helpful, however, as they give additional insight into the behavior and properties of chirps.

Phase stretch transform (PST) is a computational approach to signal and image processing. One of its utilities is for feature detection and classification. PST is related to time stretch dispersive Fourier transform. It transforms the image by emulating propagation through a diffractive medium with engineered 3D dispersive property. The operation relies on symmetry of the dispersion profile and can be understood in terms of dispersive eigenfunctions or stretch modes. PST performs similar functionality as phase-contrast microscopy, but on digital images. PST can be applied to digital images and temporal data. It is a physics-based feature engineering algorithm.

The Peierls substitution method, named after the original work by Rudolf Peierls is a widely employed approximation for describing tightly-bound electrons in the presence of a slowly varying magnetic vector potential.

Steered-Response Power Phase Transform (SRP-PHAT) is a popular algorithm for acoustic source localization, well known for its robust performance in adverse acoustic environments. The algorithm can be interpreted as a beamforming-based approach that searches for the candidate position that maximizes the output of a steered delay-and-sum beamformer.

References

↑ Physics-based Feature Engineering. Jalali et al. Optics, Photonics and Laser Technology, 2019
↑ Suthar, Madhuri, and Bahram Jalali. "Phase-stretch adaptive gradient-field extractor (page)." Coding Theory. IntechOpen, 2020. 143.
↑ MacPhee, Callen, Madhuri Suthar, and Bahram Jalali. "Phase-Stretch Adaptive Gradient-Field Extractor (PAGE)." arXiv preprint arXiv:2202.03570 (2022).
↑ M. H. Asghari, and B. Jalali, "Physics-inspired image edge detection," IEEE Global Signal and Information Processing Symposium (GlobalSIP 2014), paper: WdBD-L.1, Atlanta, December 2014.
↑ Bhushan, A. S. et al. “Time-stretched analogue-to-digital conversion.” Electronics Letters 34 (1998): 839-841.
↑ Mahjoubfar, A., Churkin, D., Barland, S. et al. Time stretch and its applications. Nature Photon 11, 341–351 (2017). https://doi.org/10.1038/nphoton.2017.76
↑ {K. Goda, K. K. Tsia, and B. Jalali, "Serial Time Encoded Amplified Microscopy," in Conference on Lasers and Electro-Optics/International Quantum Electronics Conference, OSA Technical Digest (CD) (Optica Publishing Group, 2009), paper CTuAA3.
↑ M. H. Asghari, and B. Jalali, "Physics-inspired image edge detection," IEEE Global Signal and Information Processing Symposium (GlobalSIP 2014), paper: WdBD-L.1, Atlanta, December 2014.
↑ Challoob M., Gao Y. (2020) A Local Flow Phase Stretch Transform for Robust Retinal Vessel Detection. In: Blanc-Talon J., Delmas P., Philips W., Popescu D., Scheunders P. (eds) Advanced Concepts for Intelligent Vision Systems. ACIVS 2020. Lecture Notes in Computer Science, vol 12002. Springer, Cham. https://doi.org/10.1007/978-3-030-40605-9_22

Open Source Code on Github

https://github.com/JalaliLabUCLA/Phase-Stretch-Adaptive-Gradient-field-Extractor/

Category:Image processing Category:Computational physics

This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.

[1] Physics-based Feature Engineering. Jalali et al. Optics, Photonics and Laser Technology, 2019

[2] Suthar, Madhuri, and Bahram Jalali. "Phase-stretch adaptive gradient-field extractor (page)." Coding Theory. IntechOpen, 2020. 143.

[3] MacPhee, Callen, Madhuri Suthar, and Bahram Jalali. "Phase-Stretch Adaptive Gradient-Field Extractor (PAGE)." arXiv preprint arXiv:2202.03570 (2022).

[4] M. H. Asghari, and B. Jalali, "Physics-inspired image edge detection," IEEE Global Signal and Information Processing Symposium (GlobalSIP 2014), paper: WdBD-L.1, Atlanta, December 2014.

[5] Bhushan, A. S. et al. “Time-stretched analogue-to-digital conversion.” Electronics Letters 34 (1998): 839-841.

[6] Mahjoubfar, A., Churkin, D., Barland, S. et al. Time stretch and its applications. Nature Photon 11, 341–351 (2017). https://doi.org/10.1038/nphoton.2017.76

[7] {K. Goda, K. K. Tsia, and B. Jalali, "Serial Time Encoded Amplified Microscopy," in Conference on Lasers and Electro-Optics/International Quantum Electronics Conference, OSA Technical Digest (CD) (Optica Publishing Group, 2009), paper CTuAA3.

[8] M. H. Asghari, and B. Jalali, "Physics-inspired image edge detection," IEEE Global Signal and Information Processing Symposium (GlobalSIP 2014), paper: WdBD-L.1, Atlanta, December 2014.

[9] Challoob M., Gao Y. (2020) A Local Flow Phase Stretch Transform for Robust Retinal Vessel Detection. In: Blanc-Talon J., Delmas P., Philips W., Popescu D., Scheunders P. (eds) Advanced Concepts for Intelligent Vision Systems. ACIVS 2020. Lecture Notes in Computer Science, vol 12002. Springer, Cham. https://doi.org/10.1007/978-3-030-40605-9_22

[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]