The Adventure of Spatial Sound Reproduction

The Adventure of
Spatial Sound Reproduction
Sascha Spors
Universität Rostock, Institute of Communications Engineering
27.August 2014

View Slide

Loudspeakers
Headphones
Mathematics
Acoustics
Perception
Stereophony
Binaural Synthesis
Wave Field Synthesis
Ambisonics
Directional Audio Coding
Transaural
Surround
authentic
plausible
immersive
3D
natural sound
with height
surrounding
analytic
Sascha Spors | Spatial Sound Reproduction | Introduction 1

View Slide

1861 – Philipp Reis: The Telephone
Sascha Spors | Spatial Sound Reproduction | History 2

View Slide

1881 – Clément Ader: Théâtrophone

View Slide

1940 – Walt Disney: Fantasia/Fantasound

View Slide

1940 – Walt Disney: Fantasia/Fantasound
[Garity, WM.E. and Hawkins, J.N.A., Fantasound, Journal of the Society of Motion Picture Engineers, pp.127-146, 1941]

View Slide

1995 – Vector-base Amplitude Panning

View Slide

Physical versus Listener Centric Viewpoint
Sascha Spors | Spatial Sound Reproduction | Today 6

View Slide

Music Production
sound event
sound event
sound scene
recording room
audio signals
DSP
recording/
playback
virtual scene
scene
representation
auditory event
auditory event
auditory scene
sound event
listening room
from [Sascha Spors et al., Spatial Sound with Loudspeakers and its Perception: A review of the current state, IEEE Proceedings, 2013]

View Slide

Widespread Sound Reproduction Approaches
1. Binaural Synthesis
2. Stereophony
3. Sound Field Synthesis
Wave Field Synthesis
Near-Field Compensated Higher-Order Ambisonics
4. Sparse Reproduction

View Slide

Dynamic Binaural Synthesis
Introduction Binaural Synthesis Stereophony Sound Field Synthesis Comparison Evaluation Production Outlook and Conclusions
Dynamic Binaural Synthesis using HRIRs
Synthesis of the signals at the ear-drum by ﬁltering of the virtual source signal with
head-related impulse responses (HRIRs)
hL
(t) hR
(t)
hL
(t)
hR
(t)
s(t)
HRIR
database
position
virt. source
position
head
head-tracking and dynamic ﬁltering required for good results
characteristics of source and room captured in HRIR/BRIRs
Sascha Spors
Spatial Sound Reproduction
ICSA 2011
10 / 32
Sascha Spors | Spatial Sound Reproduction | Binaural 9

View Slide

Authenticity of Individualized Dynamic Binaural Synthesis
[Brinkmann et al. 2014]
from [Brinkmann et al., Assessing the Authenticity of Individual Dynamic Binaural Synthesis, EAA Joint Symposium on Auralization and Ambisonics, Berlin, 2014]
Sascha Spors | Spatial Sound Reproduction | Binaural 10

View Slide

Stereophony
L R
Sascha Spors | Spatial Sound Reproduction | Stereophony 11

View Slide

Localization of Phantom Source
[Wierstorf et al. 2014]
Estimated Localization Size of Sweet-Spot
from [Wierstorf and Spors, Predicting Localization Accuracy for Stereophonic Downmixes in Wave Field Synthesis, Forum Acusticum, Krakow, 2014]

View Slide

Coloration of Phantom Source
[Wierstorf et al. 2014, Wittek 2007]
ref.
center off-center
anchor
−1
−0.5
0
0.5
1
−0.5
0
0.5
inter-system coloration (mean, CI)
data from [Wierstorf et al., Coloration in Wave Field Synthesis, 2014]
no
difference
extremely
different real
stereo
perceived
intra-system coloration
system
noise
data from [Helmut Wittek, Perceptual Differences between
Waveﬁeld Synthesis and Stereophony, 2007]

View Slide

Attributes
Spatial Audio Quality Inventory (SAQI) [Lindau et al. 2014]
from [Alexander Lindau, Spatial Audio Quality Inventory – Test Manual, 2014]

View Slide

Sound Field Synthesis
Sascha Spors | Spatial Sound Reproduction | Sound Field Synthesis 15

View Slide

Synthesized Sound Fields
Virtual Point Source
Wave Field Sythesis
x / m
y / m
−2 −1 0 1 2
−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
Higher-Order Ambisonics
x / m
y / m
−2 −1 0 1 2
−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
56 loudspeakers, 3 m diameter, 2.5-dimensional synthesis, virtual point source at (0 2,5) m

View Slide

Localization of a Virtual Point Source
[Wierstorf 2014]
0.9◦
0.9◦
0.6◦
0.17m
WFS
5.0◦
1.4◦
0.7◦
M = 28
NFC-HOA
from [Hagen Wierstorf, Perceptual Assessment of Sound Field Synthesis, Phd thesis, 2014]

View Slide

Inter-System Coloration in Wave Field Synthesis
[Wierstorf et al. 2014]
no
difference
very
different
ref
stereo
0.3
cm
0.5
cm
1
cm
2
cm
4
cm
8
cm
17
cm
34
cm
67
cm
low
perceived coloration
system
WFS
noise
speech
listener at (0, 0, 0) m
from [Wierstorf et al., Coloration in Wave Field Synthesis, 2014]

View Slide

Sparsity of Natural Sound Fields
Circular Microphone Array – Amsterdam Concertgebouw [Hulsebos 2004]
from [Edo Hulsebos, Auralization using Wave Field Synthesis, Phd thesis, 2004]
Sascha Spors | Spatial Sound Reproduction | Sparse Reproduction 19

View Slide

Sparse Representation of Sound Fields
Circular Microphone Array – Room Simulation [Hulsebos 2004]

View Slide

Reproduction of Direct and Diffuse Contributions
[Hulsebos 2004]

View Slide

The Human Auditory System
pinna
ear canal
tympanic
membrane
(eardrum)
auditory
nerve
malleus
cochlea
incus
stapes
}
ossicles
oval
window
from [N. R. Carlson, Physiologische Psychologie, 2004]

View Slide

Directional Audio Coding (DirAC)
[Pulkki 2007]
from [Ville Pulkki, Spatial Sound Reproduction with Directional Audio Coding, JAES, 2007]

View Slide

Reproduction of Diffuse Sound Fields
[Hiyama et al. 2002]
from [Koichiro Hiyama et al., The minimum number of loudspeakers and its arrangement for reproducing the spatial impression of diffuse sound ﬁeld, AES, 2002]

View Slide

An Integrative View on Spatial Sound Reproduction
Signal Processing
n-D Signals & Systems
Implementation
Human Perception
Psychoacoustics
Evaluation
Acoustics
Engineering Acoustics
Mathematical Foundations
Sascha Spors | Spatial Sound Reproduction | Outlook 25

View Slide

http://spatialaudio.net
Thanks to Hagen Wierstorf, Frank Schultz, Till Rettberg
Sascha Spors | Spatial Sound Reproduction | Outlook 26

View Slide

The Adventure of Spatial Sound Reproduction

The Adventure of Spatial Sound Reproduction

Sascha Spors

More Decks by Sascha Spors

Other Decks in Science

Featured

Transcript