Head-related transfer function

HRTF filtering effect

A head-related transfer function (HRTF) is a response that characterizes how an ear receives a sound from a point in space. As sound strikes the listener, the size and shape of the head, ears, ear canal, density of the head, size and shape of nasal and oral cavities, all transform the sound and affect how it is perceived, boosting some frequencies and attenuating others. Generally speaking, the HRTF boosts frequencies from 2–5 kHz with a primary resonance of +17 dB at 2,700 Hz. But the response curve is more complex than a single bump, affects a broad frequency spectrum, and varies significantly from person to person.

A pair of HRTFs for two ears can be used to synthesize a binaural sound that seems to come from a particular point in space. It is a transfer function, describing how a sound from a specific point will arrive at the ear (generally at the outer end of the auditory canal). Some consumer home entertainment products designed to reproduce surround sound from stereo (two-speaker) headphones use HRTFs. Some forms of HRTF processing have also been included in computer software to simulate surround sound playback from loudspeakers.

Sound localization

Humans have just two ears, but can locate sounds in three dimensions – in range (distance), in direction above and below (elevation), in front and to the rear, as well as to either side (azimuth). This is possible because the brain, inner ear, and the external ears (pinna) work together to make inferences about location. This ability to localize sound sources may have developed in humans and ancestors as an evolutionary necessity since the eyes can only see a fraction of the world around a viewer, and vision is hampered in darkness, while the ability to localize a sound source works in all directions, to varying accuracy, ^[1] regardless of the surrounding light.

Humans estimate the location of a source by taking cues derived from one ear (monaural cues), and by comparing cues received at both ears (difference cues or binaural cues). Among the difference cues are time differences of arrival and intensity differences. The monaural cues come from the interaction between the sound source and the human anatomy, in which the original source sound is modified before it enters the ear canal for processing by the auditory system. These modifications encode the source location and may be captured via an impulse response which relates the source location and the ear location. This impulse response is termed the head-related impulse response (HRIR). Convolution of an arbitrary source sound with the HRIR converts the sound to that which would have been heard by the listener if it had been played at the source location, with the listener's ear at the receiver location. HRIRs have been used to produce virtual surround sound. ^{[2] [3] [ example needed ]}

The HRTF is the Fourier transform of HRIR.

HRTFs for left and right ear (expressed above as HRIRs) describe the filtering of a sound source (x(t)) before it is perceived at the left and right ears as x_L(t) and x_R(t), respectively.

The HRTF can also be described as the modifications to a sound from a direction in free air to the sound as it arrives at the eardrum. These modifications include the shape of the listener's outer ear, the shape of the listener's head and body, the acoustic characteristics of the space in which the sound is played, and so on. All these characteristics will influence how (or whether) a listener can accurately tell what direction a sound is coming from.

In the AES69-2015 standard, ^[4] the Audio Engineering Society (AES) has defined the SOFA file format for storing spatially oriented acoustic data like head-related transfer functions (HRTFs). SOFA software libraries and files are collected at the Sofa Conventions website. ^[5]

How HRTF works

The associated mechanism varies between individuals, as their head and ear shapes differ.

HRTF describes how a given sound wave input (parameterized as frequency and source location) is filtered by the diffraction and reflection properties of the head, pinna, and torso, before the sound reaches the transduction machinery of the eardrum and inner ear (see auditory system). Biologically, the source-location-specific prefiltering effects of these external structures aid in the neural determination of source location, particularly the determination of the source's elevation. ^[6]

Technical derivation

A sample of frequency response of ears:

• green curve: left ear  X_L(f)
• blue curve:   right ear X_R(f)

for a sound source from upward front.

An example of how the HRTF tilt with azimuth taken from a point of reference is derived

Linear systems analysis defines the transfer function as the complex ratio between the output signal spectrum and the input signal spectrum as a function of frequency. Blauert (1974; cited in Blauert, 1981) initially defined the transfer function as the free-field transfer function (FFTF). Other terms include free-field to eardrum transfer function and the pressure transformation from the free-field to the eardrum. Less specific descriptions include the pinna transfer function, the outer ear transfer function, the pinna response, or directional transfer function (DTF).

The transfer function H(f) of any linear time-invariant system at frequency f is:

H(f) = Output(f) / Input(f)

One method used to obtain the HRTF from a given source location is therefore to measure the head-related impulse response (HRIR), h(t), at the ear drum for the impulse Δ(t) placed at the source. The HRTF H(f) is the Fourier transform of the HRIR h(t).

Even when measured for a "dummy head" of idealized geometry, HRTF are complicated functions of frequency and the three spatial variables. For distances greater than 1 m from the head, however, the HRTF can be said to attenuate inversely with range. It is this far field HRTF, H(f, θ, φ), that has most often been measured. At closer range, the difference in level observed between the ears can grow quite large, even in the low-frequency region within which negligible level differences are observed in the far field.

HRTFs are typically measured in an anechoic chamber to minimize the influence of early reflections and reverberation on the measured response. HRTFs are measured at small increments of θ such as 15° or 30° in the horizontal plane, with interpolation used to synthesize HRTFs for arbitrary positions of θ. Even with small increments, however, interpolation can lead to front-back confusion, and optimizing the interpolation procedure is an active area of research.

In order to maximize the signal-to-noise ratio (SNR) in a measured HRTF, it is important that the impulse being generated be of high volume. In practice, however, it can be difficult to generate impulses at high volumes and, if generated, they can be damaging to human ears, so it is more common for HRTFs to be directly calculated in the frequency domain using a frequency-swept sine wave or by using maximum length sequences. User fatigue is still a problem, however, highlighting the need for the ability to interpolate based on fewer measurements.

The head-related transfer function is involved in resolving the cone of confusion, a series of points where ITD and ILD are identical for sound sources from many locations around the 0 part of the cone. When a sound is received by the ear it can either go straight down the ear into the ear canal or it can be reflected off the pinnae of the ear, into the ear canal a fraction of a second later. The sound will contain many frequencies, so therefore many copies of this signal will go down the ear all at different times depending on their frequency (according to reflection, diffraction, and their interaction with high and low frequencies and the size of the structures of the ear.) These copies overlap each other, and during this, certain signals are enhanced (where the phases of the signals match) while other copies are canceled out (where the phases of the signal do not match). Essentially, the brain is looking for frequency notches in the signal that correspond to particular known directions of sound.^{[ citation needed ]}

If another person's ears were substituted, the individual would not immediately be able to localize sound, as the patterns of enhancement and cancellation would be different from those patterns the person's auditory system is used to. However, after some weeks, the auditory system would adapt to the new head-related transfer function. ^[7] The inter-subject variability in the spectra of HRTFs has been studied through cluster analyses. ^[8]

Assessing the variation through changes between the person's ear, we can limit our perspective with the degrees of freedom of the head and its relation with the spatial domain. Through this, we eliminate the tilt and other co-ordinate parameters that add complexity. For the purpose of calibration we are only concerned with the direction level to our ears, ergo a specific degree of freedom. Some of the ways in which we can deduce an expression to calibrate the HRTF are:

1. Localization of sound in Virtual Auditory space ^[9]
2. HRTF Phase synthesis ^[10]
3. HRTF Magnitude synthesis ^[11]

Localization of sound in virtual auditory space

A basic assumption in the creation of a virtual auditory space is that if the acoustical waveforms present at a listener's eardrums are the same under headphones as in free field, then the listener's experience should also be the same.

Typically, sounds generated from headphones are perceived as originating from within the head. In the virtual auditory space, the headphones should be able to "externalize" the sound. Using the HRTF, sounds can be spatially positioned using the technique described below. ^[9]

Let x₁(t) represent an electrical signal driving a loudspeaker and y₁(t) represent the signal received by a microphone inside the listener's eardrum. Similarly, let x₂(t) represent the electrical signal driving a headphone and y₂(t) represent the microphone response to the signal. The goal of the virtual auditory space is to choose x₂(t) such that y₂(t) = y₁(t). Applying the Fourier transform to these signals, we come up with the following two equations:

Y₁ = X₁LFM, and

Y₂ = X₂HM,

where L is the transfer function of the loudspeaker in the free field, F is the HRTF, M is the microphone transfer function, and H is the headphone-to-eardrum transfer function. Setting Y₁ = Y₂, and solving for X₂ yields

X₂ = X₁LF/H.

By observation, the desired transfer function is

T= LF/H.

Therefore, theoretically, if x₁(t) is passed through this filter and the resulting x₂(t) is played on the headphones, it should produce the same signal at the eardrum. Since the filter applies only to a single ear, another one must be derived for the other ear. This process is repeated for many places in the virtual environment to create an array of head-related transfer functions for each position to be recreated while ensuring that the sampling conditions are set by the Nyquist criteria.

HRTF phase synthesis

There is less reliable phase estimation in the very low part of the frequency band, and in the upper frequencies the phase response is affected by the features of the pinna. Earlier studies also show that the HRTF phase response is mostly linear and that listeners are insensitive to the details of the interaural phase spectrum as long as the interaural time delay (ITD) of the combined low-frequency part of the waveform is maintained. This is the modeled phase response of the subject HRTF as a time delay, dependent on the direction and elevation. ^[10]

A scaling factor is a function of the anthropometric features. For example, a training set of N subjects would consider each HRTF phase and describe a single ITD scaling factor as the average delay of the group. This computed scaling factor can estimate the time delay as function of the direction and elevation for any given individual. Converting the time delay to phase response for the left and the right ears is trivial.

The HRTF phase can be described by the ITD scaling factor. This is in turn quantified by the anthropometric data of a given individual taken as the source of reference. For a generic case we consider β as a sparse vector

${\displaystyle \beta =[\beta _{1},\beta _{2},\ldots ,\beta _{N}]^{T}}$

that represents the subject's anthropometric features as a linear superposition of the anthropometric features from the training data (y^' = β^T X), and then apply the same sparse vector directly on the scaling vector H. We can write this task as a minimization problem, for a non-negative shrinking parameter λ:

${\displaystyle \beta =\operatorname {argmin} \limits _{\beta }\left(\sum _{a=1}^{A}\left(y_{a}-\sum _{n=1}^{N}\beta _{n}X_{n}^{2}\right)+\lambda \sum _{n=1}^{N}\beta _{n}\right)}$

From this, ITD scaling factor value H^' is estimated as:

${\displaystyle H'=\sum _{n=1}^{N}\beta _{n}H_{n}.}$

where The ITD scaling factors for all persons in the dataset are stacked in a vector H ∈ R^N, so the value Hⁿ corresponds to the scaling factor of the n-th person.

HRTF magnitude synthesis

We solve the above minimization problem using Least Absolute Shrinkage and Selection Operator (LASSO). We assume that the HRTFs are represented by the same relation as the anthropometric features. ^[11] Therefore, once we learn the sparse vector β from the anthropometric features, we directly apply it to the HRTF tensor data and the subject's HRTF values H^' given by:

${\displaystyle H'_{d,k}=\sum _{n=1}^{N}\beta _{n}H_{n,d,k}}$

where The HRTFs for each subject are described by a tensor of size D × K, where D is the number of HRTF directions and K is the number of frequency bins. All H_n,d,k corresponds to all the HRTFs of the training set are stacked in a new tensor H ∈ R^N×D×K, so the value H_n,d,k corresponds to the k-th frequency bin for d-th HRTF direction of the n-th person. Also H^'_d,k corresponds to k-th frequency for every d-th HRTF direction of the synthesized HRTF.

HRTF from geometry

Accumulation of HRTF data has made it possible for a computer program to infer an approximate HRTF from head geometry. Two programs are known to do so, both open-source: Mesh2HRTF, ^[12] which runs physical simulation on a full 3D-mesh of the head, and EAC, which uses a neural network trained from existing HRTFs and works from photo and other rough measurements. ^[13]

Recording and playback technology

Recordings processed via an HRTF, such as in a computer gaming environment (see A3D, EAX, and OpenAL), which approximates the HRTF of the listener, can be heard through stereo headphones or speakers and interpreted as if they comprise sounds coming from all directions, rather than just two points on either side of the head. The perceived accuracy of the result depends on how closely the HRTF data set matches the characteristics of one's own ears, though a generic HRTF may be preferred to an accurate one measured from one's one ear. ^[14] Some vendors like Apple and Sony offer a variety of HRTFs to be selected by the user's ear shape. ^[15]

Windows 10 and above come with Microsoft Spatial Sound included, the same spatial audio framework used on Xbox One and Hololens 2. On a Windows PC or an Xbox One, the framework can use several different downstream audio processors, including Windows Sonic for Headphones, Dolby Atmos, and DTS Headphone:X, to apply an HRTF. The framework can render both fixed-position surround sound sources and dynamic "object" sources that can move in space. ^[16]

Apple similarly has Spatial Sound for its devices used with headphones produced by Apple or Beats. For music playback to headphones, Dolby Atmos can be enabled and the HRTF applied. ^[17] The HRTF (or rather, the object positions) can vary with head tracking to maintain the illusion of direction. ^[18] Qualcomm Snapdragon has a similar head-tracked spatial audio system, used by some brands of Android phones. ^[19] YouTube uses head-tracked HRTF with 360-degree and VR videos. ^[20]

Linux is currently unable to directly process any of the proprietary spatial audio (surround plus dynamic objects) formats. SoundScape Renderer offers directional synthesis. ^[21] PulseAudio and PipeWire each can provide virtual surround (fixed-location channels) using an HRTF. Recent PipeWire versions are also able to provide dynamic spatial rendering using HRTFs, ^[22] however integration with applications is still in progress. Users can configure their own positional and dynamic sound sources, as well as simulate a surround speaker setup using existing configurations.

The cross-platform OpenAL Soft, an implementation of OpenAL, uses HRTFs for improved localization. ^[23]

Windows and Linux spatial audio systems support any model of stereo headphones, while Apple only allows spatial audio to be used with Apple or Beats-branded Bluetooth headsets.^{[ citation needed ]}

See also

• 3D sound reconstruction
• A3D
• Binaural recording
• Dolby Atmos
• Dummy head recording
• Environmental audio extensions
• Head shadow
• OpenAL
• QSound
• Sensaura
• Sound localization
• Sound Retrieval System
• Soundbar
• Transfer function

References

1. Daniel Starch (1908). Perimetry of the localization of sound. State University of Iowa. p. 35 ff.
2. Begault, D.R. (1994) 3D sound for virtual reality and multimedia. AP Professional.
3. So, R.H.Y., Leung, N.M., Braasch, J. and Leung, K.L. (2006) A low cost, Non-individualized surround sound system based upon head-related transfer functions. An Ergonomics study and prototype development. Applied Ergonomics, 37, pp. 695–707.
4. "AES Standard AES69-2015: AES standard for file exchange - Spatial acoustic data file format". www.aes.org. Retrieved 2016-12-30.
5. "Sofa Conventions Website". Acoustics Research Institute, a research institute of the Austrian Academy of Sciences.
6. Blauert, J. (1997) Spatial hearing: the psychophysics of human sound localization. MIT Press.
7. Hofman, Paul M.; Van Riswick, JG; Van Opstal, AJ (September 1998). "Relearning sound localization with new ears" (PDF). Nature Neuroscience. 1 (5): 417–421. doi:10.1038/1633. PMID   10196533. S2CID   10088534.
8. So, R.H.Y., Ngan, B., Horner, A., Leung, K.L., Braasch, J. and Blauert, J. (2010) Toward orthogonal non-individualized head-related transfer functions for forward and backward directional sound: cluster analysis and an experimental study. Ergonomics, 53(6), pp.767-781.
9. Carlile, S. (1996). Virtual Auditory Space: Generation and Applications (1 ed.). Berlin, Heidelberg: Springer. ISBN   9783662225967.
10. Tashev, Ivan (2014). "HRTF phase synthesis via sparse representation of anthropometric features". 2014 Information Theory and Applications Workshop (ITA). pp. 1–5. doi:10.1109/ITA.2014.6804239. ISBN   978-1-4799-3589-5. S2CID   13232557.
11. Bilinski, Piotr; Ahrens, Jens; Thomas, Mark RP; Tashev, Ivan; Platt, John C (2014). "HRTF magnitude synthesis via sparse representation of anthropometric features" (PDF). 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE ICASSP, Florence, Italy. pp. 4468–4472. doi:10.1109/ICASSP.2014.6854447. ISBN   978-1-4799-2893-4. S2CID   5619011.
12. Ziegelwanger, H., and Kreuzer, W., Majdak, P. (2015). "Mesh2HRTF: An open-source software package for the numerical calculation of head-related transfer functions," in Proceedings of the 22nd International Congress on Sound and Vibration, Florence, Italy.
13. Carvalho, Davi (17 April 2023). "EAC - Individualized HRTF Synthesis". GitHub .
14. Armstrong, Cal; Thresh, Lewis; Murphy, Damian; Kearney, Gavin (23 October 2018). "A Perceptual Evaluation of Individual and Non-Individual HRTFs: A Case Study of the SADIE II Database". Applied Sciences. 8 (11): 2029. doi: 10.3390/app8112029 .
15. "Spatial Audio: Part 1 - Current Formats & The Rise Of HRTF - The Broadcast Bridge - Connecting IT to Broadcast". The Broadcast Bridge. 7 December 2022.
16. "Spatial Sound for app developers for Windows, Xbox, and Hololens 2 - Win32 apps". learn.microsoft.com. 27 April 2023.
17. "About Spatial Audio with Dolby Atmos in Apple Music". Apple Support. 27 March 2023.
18. "Listen with spatial audio for AirPods and Beats". Apple Support. 19 July 2023.
19. "Spatial Audio". www.qualcomm.com.
20. "Use spatial audio in 360-degree and VR videos - YouTube Help". support.google.com.
21. "SoundScape Renderer". spatialaudio.net. 9 January 2013.
22. "Filter Chain". gitlab.freedesktop.org/pipewire/pipewire. 14 April 2023.
23. "OpenAL Soft - Software 3D Audio". openal-soft.org.

External links

• Spatial Sound Tutorial
• CIPIC HRTF Database
• Listen HRTF Database
• High-resolution HRTF and 3D ear model database (48 subjects)
• AIR Database (HRTF database in reverberant environments)
• Full Sphere HRIR/HRTF Database of the Neumann KU100
• MIT Database (one dataset)
• ARI (Acoustics Research Institute) Database (90+ datasets)