# Time–frequency analysis

Last updated

In signal processing, time–frequency analysis comprises those techniques that study a signal in both the time and frequency domains simultaneously, using various time–frequency representations. Rather than viewing a 1-dimensional signal (a function, real or complex-valued, whose domain is the real line) and some transform (another function whose domain is the real line, obtained from the original via some transform), time–frequency analysis studies a two-dimensional signal – a function whose domain is the two-dimensional real plane, obtained from the signal via a time–frequency transform.  

## Contents

The mathematical motivation for this study is that functions and their transform representation are tightly connected, and they can be understood better by studying them jointly, as a two-dimensional object, rather than separately. A simple example is that the 4-fold periodicity of the Fourier transform – and the fact that two-fold Fourier transform reverses direction – can be interpreted by considering the Fourier transform as a 90° rotation in the associated time–frequency plane: 4 such rotations yield the identity, and 2 such rotations simply reverse direction (reflection through the origin).

The practical motivation for time–frequency analysis is that classical Fourier analysis assumes that signals are infinite in time or periodic, while many signals in practice are of short duration, and change substantially over their duration. For example, traditional musical instruments do not produce infinite duration sinusoids, but instead begin with an attack, then gradually decay. This is poorly represented by traditional methods, which motivates time–frequency analysis.

One of the most basic forms of time–frequency analysis is the short-time Fourier transform (STFT), but more sophisticated techniques have been developed, notably wavelets and least-squares spectral analysis methods for unevenly spaced data.

## Motivation

In signal processing, time–frequency analysis  is a body of techniques and methods used for characterizing and manipulating signals whose statistics vary in time, such as transient signals.

It is a generalization and refinement of Fourier analysis, for the case when the signal frequency characteristics are varying with time. Since many signals of interest – such as speech, music, images, and medical signals – have changing frequency characteristics, time–frequency analysis has broad scope of applications.

Whereas the technique of the Fourier transform can be extended to obtain the frequency spectrum of any slowly growing locally integrable signal, this approach requires a complete description of the signal's behavior over all time. Indeed, one can think of points in the (spectral) frequency domain as smearing together information from across the entire time domain. While mathematically elegant, such a technique is not appropriate for analyzing a signal with indeterminate future behavior. For instance, one must presuppose some degree of indeterminate future behavior in any telecommunications systems to achieve non-zero entropy (if one already knows what the other person will say one cannot learn anything).

To harness the power of a frequency representation without the need of a complete characterization in the time domain, one first obtains a time–frequency distribution of the signal, which represents the signal in both the time and frequency domains simultaneously. In such a representation the frequency domain will only reflect the behavior of a temporally localized version of the signal. This enables one to talk sensibly about signals whose component frequencies vary in time.

For instance rather than using tempered distributions to globally transform the following function into the frequency domain one could instead use these methods to describe it as a signal with a time varying frequency.

$x(t)={\begin{cases}\cos(\pi t);&t<10\\\cos(3\pi t);&10\leq t<20\\\cos(2\pi t);&t>20\end{cases}}$ Once such a representation has been generated other techniques in time–frequency analysis may then be applied to the signal in order to extract information from the signal, to separate the signal from noise or interfering signals, etc.

## Time–frequency distribution functions

### Formulations

There are several different ways to formulate a valid time–frequency distribution function, resulting in several well-known time–frequency distributions, such as:

More information about the history and the motivation of development of time–frequency distribution can be found in the entry Time–frequency representation.

### Ideal TF distribution function

A time–frequency distribution function ideally has the following properties:[ citation needed ]

1. High resolution in both time and frequency, to make it easier to be analyzed and interpreted.
2. No cross-term to avoid confusing real components from artifacts or noise.
3. A list of desirable mathematical properties to ensure such methods benefit real-life application.
4. Lower computational complexity to ensure the time needed to represent and process a signal on a time–frequency plane allows real-time implementations.

Below is a brief comparison of some selected time–frequency distribution functions. 

 Clarity Cross-term Good mathematical properties[ clarification needed ] Computational complexity Gabor transform Worst No Worst Low Wigner distribution function Best Yes Best High Gabor–Wigner distribution function Good Almost eliminated Good High Cone-shape distribution function Good No (eliminated, in time) Good Medium (if recursively defined)

To analyze the signals well, choosing an appropriate time–frequency distribution function is important. Which time–frequency distribution function should be used depends on the application being considered, as shown by reviewing a list of applications.  The high clarity of the Wigner distribution function (WDF) obtained for some signals is due to the auto-correlation function inherent in its formulation; however, the latter also causes the cross-term problem. Therefore, if we want to analyze a single-term signal, using the WDF may be the best approach; if the signal is composed of multiple components, some other methods like the Gabor transform, Gabor-Wigner distribution or Modified B-Distribution functions may be better choices.

As an illustration, magnitudes from non-localized Fourier analysis cannot distinguish the signals:

$x_{1}(t)={\begin{cases}\cos(\pi t);&t<10\\\cos(3\pi t);&10\leq t<20\\\cos(2\pi t);&t>20\end{cases}}$ $x_{2}(t)={\begin{cases}\cos(\pi t);&t<10\\\cos(2\pi t);&10\leq t<20\\\cos(3\pi t);&t>20\end{cases}}$ But time–frequency analysis can.

## TF analysis and Random Process 

For a random process x(t), we cannot find the explicit value of x(t).

The value of x(t) is expressed as a probability function.

• Auto-covariance function $R_{x}(t,\tau )$ $R_{x}(t,\tau )=E[x(t+\tau /2)x^{*}(t-\tau /2)]$ In usual, we suppose that $E[x(t)]=0$ for any t,

$E[x(t+\tau /2)x^{*}(t-\tau /2)]$ $=\iint x(t+\tau /2,\xi _{1})x^{*}(t-\tau /2,\xi _{2})P(\xi _{1},\xi _{2})d\xi _{1}d\xi _{2}$ (alternative definition of the auto-covariance function) ${\overset {\land }{R_{x}}}(t,\tau )=E[x(t)x(t+\tau )]$ • Power spectral density (PSD) $S_{x}(t,f)$ $S_{x}(t,f)=\int _{-\infty }^{\infty }R_{x}(t,\tau )e^{-j2\pi f\tau }d\tau$ $E[W_{x}(t,f)]=\int _{-\infty }^{\infty }E[x(t+\tau /2)x^{*}(t-\tau /2)]\cdot e^{-j2\pi f\tau }\cdot d\tau$ $=\int _{-\infty }^{\infty }R_{x}(t,\tau )\cdot e^{-j2\pi f\tau }\cdot d\tau$ $=S_{x}(t,f)$ $E[A_{X}(\eta ,\tau )]=\int _{-\infty }^{\infty }E[x(t+\tau /2)x^{*}(t-\tau /2)]e^{-j2\pi t\eta }dt$ $=\int _{-\infty }^{\infty }R_{x}(t,\tau )e^{-j2\pi t\eta }dt$ $R_{x}(t_{1},\tau )=R_{x}(t_{2},\tau )=R_{x}(\tau )$ for any $t$ , Therefore, $R_{x}(\tau )=E[x(\tau /2)x^{*}(-\tau /2)]$ $=\iint x(\tau /2,\xi _{1})x^{*}(-\tau /2,\xi _{2})P(\xi _{1},\xi _{2})d\xi _{1}d\xi _{2}$ PSD, $S_{x}(f)=\int _{-\infty }^{\infty }R_{x}(\tau )e^{-j2\pi f\tau }d\tau$ White noise:

$S_{x}(f)=\sigma$ , where $\sigma$ is some constant.

• When x(t) is stationary,

$E[W_{x}(t,f)]=S_{x}(f)$ , (invariant with $t$ )

$E[A_{x}(\eta ,\tau )]=\int _{-\infty }^{\infty }R_{x}(\tau )\cdot e^{-j2\pi t\eta }\cdot dt$ $=R_{x}(\tau )\int _{-\infty }^{\infty }e^{-j2\pi t\eta }\cdot dt$ $=R_{x}(\tau )\delta (\eta )$ , (nonzero only when $\eta =0$ )

• For white noise,

$E[W_{g}(t,f)]=\sigma$ $E[A_{x}(\eta ,\tau )]=\sigma \delta (\tau )\delta (\eta )$ Filter Design for White noise

$E_{x}$ : energy of the signal

$A$ : area of the time frequency distribution of the signal

The PSD of the white noise is $S_{n}(f)=\sigma$ $SNR\approx 10\log _{10}{\frac {E_{x}}{\iint \limits _{(t,f)\in {\text{signal part}}}S_{x}(t,f)dtdf}}$ $SNR\approx 10\log _{10}{\frac {E_{x}}{\sigma \mathrm {A} }}$ • If $E[W_{x}(t,f)]$ varies with $t$ and $E[A_{x}(\eta ,\tau )]$ is nonzero when $\eta =0$ , then $x(t)$ is a non-stationary random process.
• If
1. $h(t)=x_{1}(t)+x_{2}(t)+x_{3}(t)+......+x_{k}(t)$ 2. $x_{n}(t)$ 's have zero mean for all $t$ 's
3. $x_{n}(t)$ 's are mutually independent for all $t$ 's and $\tau$ 's

$E[x_{m}(t+\tau /2)x_{n}^{*}(t-\tau /2)]=E[x_{m}(t+\tau /2)]E[x_{n}^{*}(t-\tau /2)]=0$ if $m\neq n$ , then

$E[W_{h}(t,f)]=\sum _{n=1}^{k}E[W_{x_{n}}(t,f)]$ $E[A_{h}(\eta ,\tau )]=\sum _{n=1}^{k}E[A_{x_{n}}(\eta ,\tau )]$ 1. Random process for STFT (Short Time Fourier Transform)

$E[x(t)]\neq 0$ should be satisfied. Otherwise, $E[X(t,f)]=E[\int _{t-B}^{t+B}x(\tau )w(t-\tau )e^{-j2\pi f\tau }d\tau ]$ $=\int _{t-B}^{t+B}E[x(\tau )]w(t-\tau )e^{-j2\pi f\tau }d\tau$ for zero-mean random process, $E[X(t,f)]=0$ 1. Decompose by the AF and the FRFT Any non-stationary random process can be expressed as a summation of the fractional Fourier transform (or chirp multiplication) of stationary random process.

## Applications

The following applications need not only the time–frequency distribution functions but also some operations to the signal. The Linear canonical transform (LCT) is really helpful. By LCTs, the shape and location on the time–frequency plane of a signal can be in the arbitrary form that we want it to be. For example, the LCTs can shift the time–frequency distribution to any location, dilate it in the horizontal and vertical direction without changing its area on the plane, shear (or twist) it, and rotate it (Fractional Fourier transform). This powerful operation, LCT, make it more flexible to analyze and apply the time–frequency distributions.

### Instantaneous frequency estimation

The definition of instantaneous frequency is the time rate of change of phase, or

${\frac {1}{2\pi }}{\frac {d}{dt}}\phi (t),$ where $\phi (t)$ is the instantaneous phase of a signal. We can know the instantaneous frequency from the time–frequency plane directly if the image is clear enough. Because the high clarity is critical, we often use WDF to analyze it.

### TF filtering and signal decomposition

The goal of filter design is to remove the undesired component of a signal. Conventionally, we can just filter in the time domain or in the frequency domain individually as shown below.

The filtering methods mentioned above can’t work well for every signal which may overlap in the time domain or in the frequency domain. By using the time–frequency distribution function, we can filter in the Euclidean time–frequency domain or in the fractional domain by employing the fractional Fourier transform. An example is shown below.

Filter design in time–frequency analysis always deals with signals composed of multiple components, so one cannot use WDF due to cross-term. The Gabor transform, Gabor–Wigner distribution function, or Cohen's class distribution function may be better choices.

The concept of signal decomposition relates to the need to separate one component from the others in a signal; this can be achieved through a filtering operation which require a filter design stage. Such filtering is traditionally done in the time domain or in the frequency domain; however, this may not be possible in the case of non-stationary signals that are multicomponent as such components could overlap in both the time domain and also in the frequency domain; as a consequence, the only possible way to achieve component separation and therefore a signal decomposition is to implement a time–frequency filter.

### Sampling theory

By the Nyquist–Shannon sampling theorem, we can conclude that the minimum number of sampling points without aliasing is equivalent to the area of the time–frequency distribution of a signal. (This is actually just an approximation, because the TF area of any signal is infinite.) Below is an example before and after we combine the sampling theory with the time–frequency distribution:

It is noticeable that the number of sampling points decreases after we apply the time–frequency distribution.

When we use the WDF, there might be the cross-term problem (also called interference). On the other hand, using Gabor transform causes an improvement in the clarity and readability of the representation, therefore improving its interpretation and application to practical problems.

Consequently, when the signal we tend to sample is composed of single component, we use the WDF; however, if the signal consists of more than one component, using the Gabor transform, Gabor-Wigner distribution function, or other reduced interference TFDs may achieve better results.

The Balian–Low theorem formalizes this, and provides a bound on the minimum number of time–frequency samples needed.

### Modulation and multiplexing

Conventionally, the operation of modulation and multiplexing concentrates in time or in frequency, separately. By taking advantage of the time–frequency distribution, we can make it more efficient to modulate and multiplex. All we have to do is to fill up the time–frequency plane. We present an example as below.

As illustrated in the upper example, using the WDF is not smart since the serious cross-term problem make it difficult to multiplex and modulate.

### Electromagnetic wave propagation

We can represent an electromagnetic wave in the form of a 2 by 1 matrix

${\begin{bmatrix}x\\y\end{bmatrix}},$ which is similar to the time–frequency plane. When electromagnetic wave propagates through free-space, the Fresnel diffraction occurs. We can operate with the 2 by 1 matrix

${\begin{bmatrix}x\\y\end{bmatrix}}$ by LCT with parameter matrix

${\begin{bmatrix}a&b\\c&d\end{bmatrix}}={\begin{bmatrix}1&\lambda z\\0&1\end{bmatrix}},$ where z is the propagation distance and $\lambda$ is the wavelength. When electromagnetic wave pass through a spherical lens or be reflected by a disk, the parameter matrix should be

${\begin{bmatrix}a&b\\c&d\end{bmatrix}}={\begin{bmatrix}1&0\\-{\frac {1}{\lambda f}}&1\end{bmatrix}}$ and

${\begin{bmatrix}a&b\\c&d\end{bmatrix}}={\begin{bmatrix}1&0\\{\frac {1}{\lambda R}}&1\end{bmatrix}}$ respectively, where ƒ is the focal length of the lens and R is the radius of the disk. These corresponding results can be obtained from

${\begin{bmatrix}a&b\\c&d\end{bmatrix}}{\begin{bmatrix}x\\y\end{bmatrix}}.$ ### Optics, acoustics, and biomedicine

Light is an electromagnetic wave, so time–frequency analysis applies to optics in the same way as for general electromagnetic wave propagation.

Similarly, it is a characteristic of acoustic signals, that their frequency components undergo abrupt variations in time and would hence be not well represented by a single frequency component analysis covering their entire durations.

As acoustic signals are used as speech in communication between the human-sender and -receiver, their undelayedly transmission in technical communication systems is crucial, which makes the use of simpler TFDs, such as the Gabor transform, suitable to analyze these signals in real-time by reducing computational complexity.

If frequency analysis speed is not a limitation, a detailed feature comparison with well defined criteria should be made before selecting a particular TFD. Another approach is to define a signal dependent TFD that is adapted to the data. In biomedicine, one can use time–frequency distribution to analyze the electromyography (EMG), electroencephalography (EEG), electrocardiogram (ECG) or otoacoustic emissions (OAEs).

## History

Early work in time–frequency analysis can be seen in the Haar wavelets (1909) of Alfréd Haar, though these were not significantly applied to signal processing. More substantial work was undertaken by Dennis Gabor, such as Gabor atoms (1947), an early form of wavelets, and the Gabor transform, a modified short-time Fourier transform. The Wigner–Ville distribution (Ville 1948, in a signal processing context) was another foundational step.

Particularly in the 1930s and 1940s, early time–frequency analysis developed in concert with quantum mechanics (Wigner developed the Wigner–Ville distribution in 1932 in quantum mechanics, and Gabor was influenced by quantum mechanics – see Gabor atom); this is reflected in the shared mathematics of the position-momentum plane and the time–frequency plane – as in the Heisenberg uncertainty principle (quantum mechanics) and the Gabor limit (time–frequency analysis), ultimately both reflecting a symplectic structure.

An early practical motivation for time–frequency analysis was the development of radar – see ambiguity function.

## Related Research Articles In mathematical physics, the Dirac delta distribution, also known as the unit impulse, is a generalized function or distribution over the real numbers, whose value is zero everywhere except at zero, and whose integral over the entire real line is equal to one. In physics and mathematics, the Fourier transform (FT) is a transform that converts a function into a form that describes the frequencies present in the original function. The output of the transform is a complex-valued function of frequency. The term Fourier transform refers to both this complex-valued function and the mathematical operation. When a distinction needs to be made the Fourier transform is sometimes called the frequency domain representation of the original function. The Fourier transform is analogous to decomposing the sound of a musical chord into terms of the intensity of its constituent pitches. The power spectrum of a time series describes the distribution of power into frequency components composing that signal. According to Fourier analysis, any physical signal can be decomposed into a number of discrete frequencies, or a spectrum of frequencies over a continuous range. The statistical average of a certain signal or sort of signal as analyzed in terms of its frequency content, is called its spectrum. The short-time Fourier transform (STFT), is a Fourier-related transform used to determine the sinusoidal frequency and phase content of local sections of a signal as it changes over time. In practice, the procedure for computing STFTs is to divide a longer time signal into shorter segments of equal length and then compute the Fourier transform separately on each shorter segment. This reveals the Fourier spectrum on each shorter segment. One then usually plots the changing spectra as a function of time, known as a spectrogram or waterfall plot, such as commonly used in software defined radio (SDR) based spectrum displays. Full bandwidth displays covering the whole range of an SDR commonly use fast Fourier transforms (FFTs) with 2^24 points on desktop computers.

In mathematics and signal processing, the Hilbert transform is a specific singular integral that takes a function, u(t) of a real variable and produces another function of a real variable H(u)(t). The Hilbert transform is given by the Cauchy principal value of the convolution with the function (see § Definition). The Hilbert transform has a particularly simple representation in the frequency domain: It imparts a phase shift of ±90° (π2 radians) to every frequency component of a function, the sign of the shift depending on the sign of the frequency (see § Relationship with the Fourier transform). The Hilbert transform is important in signal processing, where it is a component of the analytic representation of a real-valued signal u(t). The Hilbert transform was first introduced by David Hilbert in this setting, to solve a special case of the Riemann–Hilbert problem for analytic functions.

Stransform as a time–frequency distribution was developed in 1994 for analyzing geophysics data. In this way, the S transform is a generalization of the short-time Fourier transform (STFT), extending the continuous wavelet transform and overcoming some of its disadvantages. For one, modulation sinusoids are fixed with respect to the time axis; this localizes the scalable Gaussian window dilations and translations in S transform. Moreover, the S transform doesn't have a cross-term problem and yields a better signal clarity than Gabor transform. However, the S transform has its own disadvantages: the clarity is worse than Wigner distribution function and Cohen's class distribution function.

In mathematics, in the area of harmonic analysis, the fractional Fourier transform (FRFT) is a family of linear transformations generalizing the Fourier transform. It can be thought of as the Fourier transform to the n-th power, where n need not be an integer — thus, it can transform a function to any intermediate domain between time and frequency. Its applications range from filter design and signal analysis to phase retrieval and pattern recognition.

In pulsed radar and sonar signal processing, an ambiguity function is a two-dimensional function of propagation delay and Doppler frequency , . It represents the distortion of a returned pulse due to the receiver matched filter of the return from a moving target. The ambiguity function is defined by the properties of the pulse and of the filter, and not any particular target scenario. In mathematics, a Dirac comb is a periodic function with the formula

In mathematics, the Fourier sine and cosine transforms are forms of the Fourier transform that do not use complex numbers or require negative frequency. They are the forms originally used by Joseph Fourier and are still preferred in some applications, such as signal processing or statistics.

In applied mathematics, the Wiener–Khinchin theorem or Wiener–Khintchine theorem, also known as the Wiener–Khinchin–Einstein theorem or the Khinchin–Kolmogorov theorem, states that the autocorrelation function of a wide-sense-stationary random process has a spectral decomposition given by the power spectrum of that process. The Wigner distribution function (WDF) is used in signal processing as a transform in time-frequency analysis.

In Hamiltonian mechanics, the linear canonical transformation (LCT) is a family of integral transforms that generalizes many classical transforms. It has 4 parameters and 1 constraint, so it is a 3-dimensional family, and can be visualized as the action of the special linear group SL2(R) on the time–frequency plane (domain). As this defines the original function up to a sign, this translates into an action of its double cover on the original function space. The Gabor transform, named after Dennis Gabor, is a special case of the short-time Fourier transform. It is used to determine the sinusoidal frequency and phase content of local sections of a signal as it changes over time. The function to be transformed is first multiplied by a Gaussian function, which can be regarded as a window function, and the resulting function is then transformed with a Fourier transform to derive the time-frequency analysis. The window function means that the signal near the time being analyzed will have higher weight. The Gabor transform of a signal x(t) is defined by this formula:

A Modified Wigner distribution function is a variation of the Wigner distribution function (WD) with reduced or removed cross-terms.

The Gabor transform, named after Dennis Gabor, and the Wigner distribution function, named after Eugene Wigner, are both tools for time-frequency analysis. Since the Gabor transform does not have high clarity, and the Wigner distribution function has a "cross term problem", a 2007 study by S. C. Pei and J. J. Ding proposed a new combination of the two transforms that has high clarity and no cross term problem. Since the cross term does not appear in the Gabor transform, the time frequency distribution of the Gabor transform can be used as a filter to filter out the cross term in the output of the Wigner distribution function.

Bilinear time–frequency distributions, or quadratic time–frequency distributions, arise in a sub-field of signal analysis and signal processing called time–frequency signal processing, and, in the statistical analysis of time series data. Such methods are used where one needs to deal with a situation where the frequency composition of a signal may be changing over time; this sub-field used to be called time–frequency signal analysis, and is now more often called time–frequency signal processing due to the progress in using these methods to a wide range of signal-processing problems.

Choi–Williams distribution function is one of the members of Cohen's class distribution function. It was first proposed by Hyung-Ill Choi and William J. Williams in 1989. This distribution function adopts exponential kernel to suppress the cross-term. However, the kernel gain does not decrease along the axes in the ambiguity domain. Consequently, the kernel function of Choi–Williams distribution function can only filter out the cross-terms that result from the components that differ in both time and frequency center.

The cone-shape distribution function, also known as the Zhao–Atlas–Marks time-frequency distribution,, is one of the members of Cohen's class distribution function. It was first proposed by Yunxin Zhao, Les E. Atlas, and Robert J. Marks II in 1990. The distribution's name stems from the twin cone shape of the distribution's kernel function on the plane. The advantage of the cone kernel function is that it can completely remove the cross-term between two components having the same center frequency. Cross-term results from components with the same time center, however, cannot be completely removed by the cone-shaped kernel.

Time–frequency analysis for music signals is one of the applications of time–frequency analysis. Musical sound can be more complicated than human vocal sound, occupying a wider band of frequency. Music signals are time-varying signals; while the classic Fourier transform is not sufficient to analyze them, time–frequency analysis is an efficient tool for such use. Time–frequency analysis is extended from the classic Fourier approach. Short-time Fourier transform (STFT), Gabor transform (GT) and Wigner distribution function (WDF) are famous time–frequency methods, useful for analyzing music signals such as notes played on a piano, a flute or a guitar.

1. L. Cohen, "Time–Frequency Analysis," Prentice-Hall, New York, 1995. ISBN   978-0135945322
2. E. Sejdić, I. Djurović, J. Jiang, “Time-frequency feature representation using energy concentration: An overview of recent advances,” Digital Signal Processing, vol. 19, no. 1, pp. 153-183, January 2009.
3. P. Flandrin, "Time–frequency/Time–Scale Analysis," Wavelet Analysis and its Applications, Vol. 10 Academic Press, San Diego, 1999.
4. Shafi, Imran; Ahmad, Jamil; Shah, Syed Ismail; Kashif, F. M. (2009-06-09). "Techniques to Obtain Good Resolution and Concentrated Time-Frequency Distributions: A Review". EURASIP Journal on Advances in Signal Processing. 2009 (1): 673539. Bibcode:2009EJASP2009..109S. doi:. ISSN   1687-6180.
5. A. Papandreou-Suppappola, Applications in Time–Frequency Signal Processing (CRC Press, Boca Raton, Fla., 2002)
6. Ding, Jian-Jiun (2022). Time frequency analysis and wavelet transform class notes. Taipei, Taiwan: Graduate Institute of Communication Engineering, National Taiwan University (NTU).