The Adventure of Spatial Sound Reproduction

Keynote held at the 55th International Converence of the Audio Engineering Society (AES) in Helsinki.

Transcript

1. The Adventure of Spatial Sound Reproduction Sascha Spors Universität Rostock, Institute of Communications Engineering 27.August 2014
2. Loudspeakers Headphones Mathematics Acoustics Perception Stereophony Binaural Synthesis Wave Field Synthesis Ambisonics Directional Audio Coding Transaural Surround authentic plausible immersive 3D natural sound with height surrounding analytic Sascha Spors | Spatial Sound Reproduction | Introduction 1
3. Loudspeakers Headphones Mathematics Acoustics Perception Stereophony Binaural Synthesis Wave Field Synthesis Ambisonics Directional Audio Coding Transaural Surround authentic plausible immersive 3D natural sound with height surrounding analytic Sascha Spors | Spatial Sound Reproduction | Introduction 1
4. Loudspeakers Headphones Mathematics Acoustics Perception Stereophony Binaural Synthesis Wave Field Synthesis Ambisonics Directional Audio Coding Transaural Surround authentic plausible immersive 3D natural sound with height surrounding analytic Sascha Spors | Spatial Sound Reproduction | Introduction 1
5. Loudspeakers Headphones Mathematics Acoustics Perception Stereophony Binaural Synthesis Wave Field Synthesis Ambisonics Directional Audio Coding Transaural Surround authentic plausible immersive 3D natural sound with height surrounding analytic Sascha Spors | Spatial Sound Reproduction | Introduction 1
6. Loudspeakers Headphones Mathematics Acoustics Perception Stereophony Binaural Synthesis Wave Field Synthesis Ambisonics Directional Audio Coding Transaural Surround authentic plausible immersive 3D natural sound with height surrounding analytic Sascha Spors | Spatial Sound Reproduction | Introduction 1
7. 1861 – Philipp Reis: The Telephone Sascha Spors | Spatial Sound Reproduction | History 2
8. 1881 – Clément Ader: Théâtrophone Sascha Spors | Spatial Sound Reproduction | History 3
9. 1881 – Clément Ader: Théâtrophone Sascha Spors | Spatial Sound Reproduction | History 3
10. 1881 – Clément Ader: Théâtrophone Sascha Spors | Spatial Sound Reproduction | History 3
11. 1940 – Walt Disney: Fantasia/Fantasound Sascha Spors | Spatial Sound Reproduction | History 4
12. 1940 – Walt Disney: Fantasia/Fantasound [Garity, WM.E. and Hawkins, J.N.A., Fantasound, Journal of the Society of Motion Picture Engineers, pp.127-146, 1941] Sascha Spors | Spatial Sound Reproduction | History 4
13. 1940 – Walt Disney: Fantasia/Fantasound [Garity, WM.E. and Hawkins, J.N.A., Fantasound, Journal of the Society of Motion Picture Engineers, pp.127-146, 1941] Sascha Spors | Spatial Sound Reproduction | History 4
14. 1995 – Vector-base Amplitude Panning Sascha Spors | Spatial Sound Reproduction | History 5
15. Physical versus Listener Centric Viewpoint Sascha Spors | Spatial Sound Reproduction | Today 6
16. Music Production sound event sound event sound scene recording room audio signals DSP recording/ playback virtual scene scene representation auditory event auditory event auditory scene sound event listening room from [Sascha Spors et al., Spatial Sound with Loudspeakers and its Perception: A review of the current state, IEEE Proceedings, 2013] Sascha Spors | Spatial Sound Reproduction | Today 7
17. Widespread Sound Reproduction Approaches 1. Binaural Synthesis 2. Stereophony 3. Sound Field Synthesis Wave Field Synthesis Near-Field Compensated Higher-Order Ambisonics 4. Sparse Reproduction Sascha Spors | Spatial Sound Reproduction | Today 8
18. Dynamic Binaural Synthesis Introduction Binaural Synthesis Stereophony Sound Field Synthesis Comparison Evaluation Production Outlook and Conclusions Dynamic Binaural Synthesis using HRIRs Synthesis of the signals at the ear-drum by filtering of the virtual source signal with head-related impulse responses (HRIRs) hL (t) hR (t) hL (t) hR (t) s(t) HRIR database position virt. source position head head-tracking and dynamic filtering required for good results characteristics of source and room captured in HRIR/BRIRs Sascha Spors Spatial Sound Reproduction ICSA 2011 10 / 32 Sascha Spors | Spatial Sound Reproduction | Binaural 9
19. Authenticity of Individualized Dynamic Binaural Synthesis [Brinkmann et al. 2014] from [Brinkmann et al., Assessing the Authenticity of Individual Dynamic Binaural Synthesis, EAA Joint Symposium on Auralization and Ambisonics, Berlin, 2014] Sascha Spors | Spatial Sound Reproduction | Binaural 10
20. Stereophony L R Sascha Spors | Spatial Sound Reproduction | Stereophony 11
21. Localization of Phantom Source [Wierstorf et al. 2014] Estimated Localization Size of Sweet-Spot from [Wierstorf and Spors, Predicting Localization Accuracy for Stereophonic Downmixes in Wave Field Synthesis, Forum Acusticum, Krakow, 2014] Sascha Spors | Spatial Sound Reproduction | Stereophony 12
22. Coloration of Phantom Source [Wierstorf et al. 2014, Wittek 2007] ref. center off-center anchor −1 −0.5 0 0.5 1 −0.5 0 0.5 inter-system coloration (mean, CI) data from [Wierstorf et al., Coloration in Wave Field Synthesis, 2014] no difference extremely different real stereo perceived intra-system coloration system noise data from [Helmut Wittek, Perceptual Differences between Wavefield Synthesis and Stereophony, 2007] Sascha Spors | Spatial Sound Reproduction | Stereophony 13
23. Attributes Spatial Audio Quality Inventory (SAQI) [Lindau et al. 2014] from [Alexander Lindau, Spatial Audio Quality Inventory – Test Manual, 2014] Sascha Spors | Spatial Sound Reproduction | Stereophony 14
24. Sound Field Synthesis Sascha Spors | Spatial Sound Reproduction | Sound Field Synthesis 15
25. Synthesized Sound Fields Virtual Point Source Wave Field Sythesis x / m y / m −2 −1 0 1 2 −2 −1.5 −1 −0.5 0 0.5 1 1.5 2 Higher-Order Ambisonics x / m y / m −2 −1 0 1 2 −2 −1.5 −1 −0.5 0 0.5 1 1.5 2 56 loudspeakers, 3 m diameter, 2.5-dimensional synthesis, virtual point source at (0 2,5) m Sascha Spors | Spatial Sound Reproduction | Sound Field Synthesis 16
26. Synthesized Sound Fields Virtual Point Source Wave Field Sythesis x / m y / m −2 −1 0 1 2 −2 −1.5 −1 −0.5 0 0.5 1 1.5 2 Higher-Order Ambisonics x / m y / m −2 −1 0 1 2 −2 −1.5 −1 −0.5 0 0.5 1 1.5 2 56 loudspeakers, 3 m diameter, 2.5-dimensional synthesis, virtual point source at (0 2,5) m Sascha Spors | Spatial Sound Reproduction | Sound Field Synthesis 16
27. Synthesized Sound Fields Virtual Point Source Wave Field Sythesis x / m y / m −2 −1 0 1 2 −2 −1.5 −1 −0.5 0 0.5 1 1.5 2 Higher-Order Ambisonics x / m y / m −2 −1 0 1 2 −2 −1.5 −1 −0.5 0 0.5 1 1.5 2 56 loudspeakers, 3 m diameter, 2.5-dimensional synthesis, virtual point source at (0 2,5) m Sascha Spors | Spatial Sound Reproduction | Sound Field Synthesis 16
28. Localization of a Virtual Point Source [Wierstorf 2014] 0.9◦ 0.9◦ 0.6◦ 0.17m WFS 5.0◦ 1.4◦ 0.7◦ M = 28 NFC-HOA from [Hagen Wierstorf, Perceptual Assessment of Sound Field Synthesis, Phd thesis, 2014] Sascha Spors | Spatial Sound Reproduction | Sound Field Synthesis 17
29. Inter-System Coloration in Wave Field Synthesis [Wierstorf et al. 2014] no difference very different ref stereo 0.3 cm 0.5 cm 1 cm 2 cm 4 cm 8 cm 17 cm 34 cm 67 cm low perceived coloration system WFS noise speech listener at (0, 0, 0) m from [Wierstorf et al., Coloration in Wave Field Synthesis, 2014] Sascha Spors | Spatial Sound Reproduction | Sound Field Synthesis 18
30. Sparsity of Natural Sound Fields Circular Microphone Array – Amsterdam Concertgebouw [Hulsebos 2004] from [Edo Hulsebos, Auralization using Wave Field Synthesis, Phd thesis, 2004] Sascha Spors | Spatial Sound Reproduction | Sparse Reproduction 19
31. Sparse Representation of Sound Fields Circular Microphone Array – Room Simulation [Hulsebos 2004] from [Edo Hulsebos, Auralization using Wave Field Synthesis, Phd thesis, 2004] Sascha Spors | Spatial Sound Reproduction | Sparse Reproduction 20
32. Reproduction of Direct and Diffuse Contributions [Hulsebos 2004] from [Edo Hulsebos, Auralization using Wave Field Synthesis, Phd thesis, 2004] Sascha Spors | Spatial Sound Reproduction | Sparse Reproduction 21
33. The Human Auditory System pinna ear canal tympanic membrane (eardrum) auditory nerve malleus cochlea incus stapes } ossicles oval window from [N. R. Carlson, Physiologische Psychologie, 2004] Sascha Spors | Spatial Sound Reproduction | Sparse Reproduction 22
34. Directional Audio Coding (DirAC) [Pulkki 2007] from [Ville Pulkki, Spatial Sound Reproduction with Directional Audio Coding, JAES, 2007] Sascha Spors | Spatial Sound Reproduction | Sparse Reproduction 23
35. Reproduction of Diffuse Sound Fields [Hiyama et al. 2002] from [Koichiro Hiyama et al., The minimum number of loudspeakers and its arrangement for reproducing the spatial impression of diffuse sound field, AES, 2002] Sascha Spors | Spatial Sound Reproduction | Sparse Reproduction 24
36. An Integrative View on Spatial Sound Reproduction Signal Processing n-D Signals & Systems Implementation Human Perception Psychoacoustics Evaluation Acoustics Engineering Acoustics Mathematical Foundations Sascha Spors | Spatial Sound Reproduction | Outlook 25
37. http://spatialaudio.net Thanks to Hagen Wierstorf, Frank Schultz, Till Rettberg Sascha Spors | Spatial Sound Reproduction | Outlook 26