Focus recovery based on the linear canonical transform

Last updated May 04, 2023

For digital image processing, the Focus recovery from a defocused image is an ill-posed problem since it loses the component of high frequency. Most of the methods for focus recovery are based on depth estimation theory.^[1] The Linear canonical transform (LCT) gives a scalable kernel to fit many well-known optical effects. Using LCTs to approximate an optical system for imaging and inverting this system, theoretically permits recovery of a defocused image.

Depth of field and perceptual focus

The object is put at the different positions whereas causes to effective focus. Larger aperture.PNG — The object is put at the different positions whereas causes to effective focus.

In photography, depth of field (DOF) means an effective focal length. It is usually used for stressing an object and deemphasizing the background (and/or the foreground). The important measure related to DOF is the lens aperture. Decreasing the diameter of aperture increases focus and lowers resolution and vice versa.

The Huygens–Fresnel principle and DOF

The observation points at two different fields Huygens-Fresnel field.PNG — The observation points at two different fields

The Huygens–Fresnel principle describes diffraction of wave propagation between two fields. It belongs to Fourier optics rather than geometric optics. The disturbance of diffraction depends on two circumstance parameters, the size of aperture and the interfiled distance.

Consider a source field and a destination field, field 1 and field 0, respectively. P₁(x₁,y₁) is the position in the source field, P₀(x₀,y₀) is the position in the destination field. The Huygens–Fresnel principle gives the diffraction formula for two fields U(x₀,y₀), U(x₁,y₁) as following:

\mathbf {U} (x_{0},y_{0})={\frac {1}{j\lambda }}\int \!\int \mathbf {U} (x_{1},y_{1}){\frac {e^{jkr_{01}}}{r_{01}}}\cos \theta dx_{1}dy_{1}

where θ denotes the angle between $r_{01}$ and $z$ . Replace cos θ by ${\frac {r_{01}}{z}}$ and $r_{01}$ by
$[(x_{0}-x_{1})^{2}+(y_{0}-y_{1})^{2}+z^{2}]^{1/2}$

we get

\mathbf {U} (x_{0},y_{0})={\frac {1}{j\lambda z}}\int \!\int \mathbf {U} (x_{1},y_{1}){\frac {\exp(jkz[1+({\frac {x_{0}-x_{1}}{z}})^{2}+({\frac {y_{0}-y_{1}}{z}})^{2}]^{1/2})}{1+({\frac {x_{0}-x_{1}}{z}})^{2}+({\frac {y_{0}-y_{1}}{z}})^{2}}}dx_{1}dy_{1}

The further distance z or the smaller aperture (x₁,y₁) causes a greater diffraction. A larger DOF can lead to a more effective focused wave distribution. This seems to be a conflict. Here are the notations:

Diffraction
In a real imaging environment, the depths of objects comparing to the aperture are usually not enough to lead to serious diffraction.
However, a long enough depth of the object can truly blurs the image.
Effective Focus
Small aperture, small blurring radius, few wave information.
Loses details in comparing to a large aperture.

In conclusion, diffraction explains a micro behavior whereas DOF shows a macro behavior. Both of them are related to aperture size.

Linear canonical transform

As the meaning of "canonical", the linear canonical transform (LCT) is a scalable transform that connects to many important kernels such as the Fresnel transform, Fraunhofer transform and the fractional Fourier transform. It can be easily controlled by its four parameters, a, b, c, d (3 degrees of freedom). The definition:

A general imaging system with two free space propagation and one thin lens passing Imaging system.PNG — A general imaging system with two free space propagation and one thin lens passing

L_{M}(f(u))=\int L_{M}(u,u')f(u')du'

where

L_{M}(u,u')={\begin{cases}{\sqrt {\frac {1}{b}}}e^{-j\pi /4}e^{[j\pi ({\frac {d}{b}}u^{2})-2{\frac {1}{b}}uu'+{\frac {a}{b}}u'^{2}]},&{\mbox{if }}b\neq 0\\{\sqrt {d}}e^{{\frac {j}{2}}cdu^{2}}\delta (u'-du),&{\mbox{if }}b=0\end{cases}}

Consider a general imaging system with object distance z₀, focal length of the thin lens f and an imaging distance z₁. The effect of the propagation in freespace acts as nearly a chirp convolution, that is, the formula of diffraction. Besides, the effect of the propagation in thin lens acts as a chirp multiplication. The parameters are all simplified as paraxial approximations while meeting the freespace propagation. It does not consider aperture size.

From the properties of the LCT, it is possible to obtain those 4 parameters for this optical system as:

{\begin{bmatrix}1-{\frac {z_{1}}{f}}\quad &\lambda z_{0}-{\frac {\lambda z_{0}z_{1}}{f}}+\lambda z_{1}\\-{\frac {1}{\lambda f}}\quad &1-{\frac {z_{0}}{f}}\end{bmatrix}}

Once the values of z₁, z₀ and f are known, the LCT can simulate any optical system.

Notes

↑ Most depth recovery methods are simply based on camera focus and defocus. Among those approaches, they usually fall in a depth discontinuity problem.

Related Research Articles

$Diffraction Phenomenon of the motion of waves$

Diffraction is the interference or bending of waves around the corners of an obstacle or through an aperture into the region of geometrical shadow of the obstacle/aperture. The diffracting object or aperture effectively becomes a secondary source of the propagating wave. Italian scientist Francesco Maria Grimaldi coined the word diffraction and was the first to record accurate observations of the phenomenon in 1660.

In physics, the Lorentz transformations are a six-parameter family of linear transformations from a coordinate frame in spacetime to another frame that moves at a constant velocity relative to the former. The respective inverse transformation is then parameterized by the negative of this velocity. The transformations are named after the Dutch physicist Hendrik Lorentz.

In vector calculus, Green's theorem relates a line integral around a simple closed curve $C$ to a double integral over the plane region $D$ bounded by $C$ . It is the two-dimensional special case of Stokes' theorem.

Fourier optics is the study of classical optics using Fourier transforms (FTs), in which the waveform being considered is regarded as made up of a combination, or superposition, of plane waves. It has some parallels to the Huygens–Fresnel principle, in which the wavefront is regarded as being made up of a combination of spherical wavefronts whose sum is the wavefront being studied. A key difference is that Fourier optics considers the plane waves to be natural modes of the propagation medium, as opposed to Huygens–Fresnel, where the spherical waves originate in the physical medium.

$Airy disk Diffraction pattern in optics$

In optics, the Airy disk and Airy pattern are descriptions of the best-focused spot of light that a perfect lens with a circular aperture can make, limited by the diffraction of light. The Airy disk is of importance in physics, optics, and astronomy.

In mathematics, the matrix exponential is a matrix function on square matrices analogous to the ordinary exponential function. It is used to solve systems of linear differential equations. In the theory of Lie groups, the matrix exponential gives the exponential map between a matrix Lie algebra and the corresponding Lie group.

In the theory of stochastic processes, the Karhunen–Loève theorem, also known as the Kosambi–Karhunen–Loève theorem states that a stochastic process can be represented as an infinite linear combination of orthogonal functions, analogous to a Fourier series representation of a function on a bounded interval. The transformation is also known as Hotelling transform and eigenvector transform, and is closely related to principal component analysis (PCA) technique widely used in image processing and in data analysis in many fields.

In optics, the Fraunhofer diffraction equation is used to model the diffraction of waves when plane waves are incident on a diffracting object, and the diffraction pattern is viewed at a sufficiently long distance from the object, and also when it is viewed at the focal plane of an imaging lens. In contrast, the diffraction pattern created near the diffracting object and is given by the Fresnel diffraction equation.

A first class constraint is a dynamical quantity in a constrained Hamiltonian system whose Poisson bracket with all the other constraints vanishes on the constraint surface in phase space. To calculate the first class constraint, one assumes that there are no second class constraints, or that they have been calculated previously, and their Dirac brackets generated.

Optical resolution describes the ability of an imaging system to resolve detail, in the object that is being imaged.

$Fresnel diffraction Diffraction$

In optics, the Fresnel diffraction equation for near-field diffraction is an approximation of the Kirchhoff–Fresnel diffraction that can be applied to the propagation of waves in the near field. It is used to calculate the diffraction pattern created by waves passing through an aperture or around an object, when viewed from relatively close to the object. In contrast the diffraction pattern in the far field region is given by the Fraunhofer diffraction equation.

In Hamiltonian mechanics, the linear canonical transformation (LCT) is a family of integral transforms that generalizes many classical transforms. It has 4 parameters and 1 constraint, so it is a 3-dimensional family, and can be visualized as the action of the special linear group SL₂(R) on the time–frequency plane (domain). As this defines the original function up to a sign, this translates into an action of its double cover on the original function space.

A ratio distribution is a probability distribution constructed as the distribution of the ratio of random variables having two other known distributions. Given two random variables X and Y, the distribution of the random variable Z that is formed as the ratio Z = X/Y is a ratio distribution.

The derivatives of scalars, vectors, and second-order tensors with respect to second-order tensors are of considerable use in continuum mechanics. These derivatives are used in the theories of nonlinear elasticity and plasticity, particularly in the design of algorithms for numerical simulations.

Common integrals in quantum field theory are all variations and generalizations of Gaussian integrals to the complex plane and to multiple dimensions. Other integrals can be approximated by versions of the Gaussian integral. Fourier integrals are also considered.

The multislice algorithm is a method for the simulation of the elastic interaction of an electron beam with matter, including all multiple scattering effects. The method is reviewed in the book by Cowley. The algorithm is used in the simulation of high resolution Transmission electron microscopy micrographs, and serves as a useful tool for analyzing experimental images. Here we describe relevant background information, the theoretical basis of the technique, approximations used, and several software packages that implement this technique. Moreover, we delineate some of the advantages and limitations of the technique and important considerations that need to be taken into account for real-world use.

Kirchhoff's diffraction formula can be used to model the propagation of light in a wide range of configurations, either analytically or using numerical modelling. It gives an expression for the wave disturbance when a monochromatic spherical wave is the incoming wave of a situation under consideration. This formula is derived by applying the Kirchhoff integral theorem, which uses the Green's second identity to derive the solution to the homogeneous scalar wave equation, to a spherical wave with some approximations.

In optics, the Fraunhofer diffraction equation is used to model the diffraction of waves when the diffraction pattern is viewed at a long distance from the diffracting object, and also when it is viewed at the focal plane of an imaging lens.

In the science of fluid flow, Stokes' paradox is the phenomenon that there can be no creeping flow of a fluid around a disk in two dimensions; or, equivalently, the fact there is no non-trivial steady-state solution for the Stokes equations around an infinitely long cylinder. This is opposed to the 3-dimensional case, where Stokes' method provides a solution to the problem of flow around a sphere.

Lightfieldmicroscopy (LFM) is a scanning-free 3-dimensional (3D) microscopic imaging method based on the theory of light field. This technique allows sub-second (~10 Hz) large volumetric imaging with ~1 μm spatial resolution in the condition of weak scattering and semi-transparence, which has never been achieved by other methods. Just as in traditional light field rendering, there are two steps for LFM imaging: light field capture and processing. In most setups, a microlens array is used to capture the light field. As for processing, it can be based on two kinds of representations of light propagation: the ray optics picture and the wave optics picture. The Stanford University Computer Graphics Laboratory published their first prototype LFM in 2006 and has been working on the cutting edge since then.

References

Haldun M. Ozaktas; Zeev Zalevsky; M. Alper Kutay (2001). The fractional Fourier transform with applications in optics and signal processing. New York: John Wiley & Sons. ISBN 978-0-471-96346-2.
M. Sorel and J. Flusser, "Space-variant restoration of images degraded by camera motion blur", IEEE Transactions on Image Processing, vol. 17, pp. 105–116, Feb. 2008.
"The way a zoom lens works". Jos. Schneider Optische Werke GmbH. February 2008. Archived from the original on 2012-05-08.
B. Barshan, M. Alper Kutay and H. M. Ozaktas, "Optimal filtering with linear ca-nonical transformations", Optics Communications, vol. 135, pp. 32–36, Feb. 1997.

This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.

[1] Most depth recovery methods are simply based on camera focus and defocus. Among those approaches, they usually fall in a depth discontinuity problem.

[1]