Simulating Room Acoustics

Talk given at the summer school of the FET-Open project Two!Ears on the generation of ears signals using the binaural simulator.

Transcript

1. Simulating Room Acoustics Sascha Spors et al. Universität Rostock Institut für Nachrichtentechnik Two!Ears Summer School 21st September 2015, Toulouse
2. Model Architecture Spors et al. | 21.9.2015 | Two!Ears – Simulating Room Acoustics | Introduction 1 / 21
3. Scenario-Based Development and Evaluation Process real environment observer source model ear signals vision feedback Real World virtual environment virtual observer virtual source model ear signals vision feedback Virtual World Spors et al. | 21.9.2015 | Two!Ears – Simulating Room Acoustics | Introduction 2 / 21
4. Scenario-Based Development and Evaluation Process real environment HATS source model ear signals vision feedback Real World simulation framework model ear signals vision feedback scenario database Virtual World Spors et al. | 21.9.2015 | Two!Ears – Simulating Room Acoustics | Introduction 2 / 21
5. Scenario-Based Development and Evaluation Process real-world input less controllable but potentially richer interactive and exploratory nature of model signals depend on feedback from upper stages block/frame-based computation of signals simulation framework model ear signals vision feedback scenario database Virtual World Spors et al. | 21.9.2015 | Two!Ears – Simulating Room Acoustics | Introduction 2 / 21
6. Techniques for Binaural Synthesis Degrees of realism capture of real-environments model/data-based synthesis of virtual environments numeric simulation of virtual environments Ear signals captured with individuals head and torso simulator (HATS) Underlying signal processing techniques recording/playback of ear-signals capture/convolution of source signals with impulse responses Spors et al. | 21.9.2015 | Two!Ears – Simulating Room Acoustics | Introduction 3 / 21
7. Pre-Recorded Binaural Signals Recording defined position in the (blocked) ear canal Playback headphone compensation Pros/Cons + complex scenarios – fixed head orientation – fixed scenarios Applications diffuse background noise Spors et al. | 21.9.2015 | Two!Ears – Simulating Room Acoustics | Binaural Synthesis Techniques 4 / 21
8. Head-Related Impulse Responses (HRIRs) Recording identify impulse responses from a source to the left/right ears → HRIRs database of HRIRs for different source positions head/torso rotation equivalent to source at fixed distance Playback convolution of dry source signal with HRIRs Pros/Cons + copes for head orientation + variable position of source/receiver – no diffuse sources Applications (anechoic) scenarios with limited number of (static) sources Spors et al. | 21.9.2015 | Two!Ears – Simulating Room Acoustics | Binaural Synthesis Techniques 5 / 21
9. Example – QU KEMAR with 3m Source Distance Spors et al. | 21.9.2015 | Two!Ears – Simulating Room Acoustics | Binaural Synthesis Techniques 6 / 21
10. Example – QU KEMAR with 3m Source Distance Left/right HRIR for source 60o to the left side 0 1 2 3 4 5 time (ms) −1.0 −0.5 0.0 0.5 1.0 left ear right ear Spors et al. | 21.9.2015 | Two!Ears – Simulating Room Acoustics | Binaural Synthesis Techniques 6 / 21
11. Example – QU KEMAR with 3m Source Distance Left HRIRs for varying source angle −180 −135 −90 −45 0 45 90 135 angle (deg) 0 1 2 3 4 5 time (ms) 0 −10 −20 −30 −40 −50 −60 −70 relative level (dB) Spors et al. | 21.9.2015 | Two!Ears – Simulating Room Acoustics | Binaural Synthesis Techniques 6 / 21
12. Example – QU KEMAR with 3m Source Distance Left HRTFs for varying source angle −180 −135 −90 −45 0 45 90 135 angle (deg) 0 5 10 15 20 frequency (kHz) 20 10 0 −10 −20 −30 −40 −50 relative level (dB) Spors et al. | 21.9.2015 | Two!Ears – Simulating Room Acoustics | Binaural Synthesis Techniques 6 / 21
13. The SoundScape Renderer (SSR) A versatile framework for spatial sound reproduction techniques: WFS, HOA, VBAP, binaural synthesis, ... Linux and OS X open source software (GPLv2) http://spatialaudio.net/ssr Spors et al. | 21.9.2015 | Two!Ears – Simulating Room Acoustics | Binaural Synthesis Techniques 7 / 21
14. Binaural Room Impulse Responses (BRIRs) Recording identify room impulse responses from a source to the left/right ears → BRIRs database of BRIRs for different head-orientations Playback convolution of dry source signal with BRIRs Pros/Cons + copes for head rotation + includes room acoustics – fixed position of source/receiver Applications reverberant scenarios with low number of static sources Spors et al. | 21.9.2015 | Two!Ears – Simulating Room Acoustics | Binaural Synthesis Techniques 8 / 21
15. Example – QU KEMAR in Auditorium Spors et al. | 21.9.2015 | Two!Ears – Simulating Room Acoustics | Binaural Synthesis Techniques 9 / 21
16. Example – QU KEMAR in Auditorium Left/right ear HRIR for head rotated 60o to the right side 0 50 100 150 200 time (ms) −1.0 −0.5 0.0 0.5 1.0 left ear right ear Spors et al. | 21.9.2015 | Two!Ears – Simulating Room Acoustics | Binaural Synthesis Techniques 9 / 21
17. Example – QU KEMAR in Auditorium Left HRIRs for varying head rotation −90 −45 0 45 angle (deg) 10 20 30 40 50 60 70 80 90 100 time (ms) −60 −54 −48 −42 −36 −30 −24 −18 −12 −6 0 relative level (dB) Spors et al. | 21.9.2015 | Two!Ears – Simulating Room Acoustics | Binaural Synthesis Techniques 9 / 21
18. Example – QU KEMAR in Auditorium Left HRTFs for varying head rotation −90 −45 0 45 angle (deg) 0 5 10 15 20 frequency (kHz) 25 20 15 10 5 0 −5 −10 −15 −20 −25 −30 relative level (dB) Spors et al. | 21.9.2015 | Two!Ears – Simulating Room Acoustics | Binaural Synthesis Techniques 9 / 21
19. Dynamic Binaural Room Impulse Responses Recording time-variant room impulse responses from a moving source to the left/right ears Playback time-variant convolution of dry source signal Pros/Cons + includes acoustic properties of room – fixed trajectory of source/receiver – fixed head orientation Applications reverberant scenario with low number of dynamic sources Spors et al. | 21.9.2015 | Two!Ears – Simulating Room Acoustics | Binaural Synthesis Techniques 10 / 21
20. Data-based Binaural Synthesis Recording identify impulse responses from a source to a microphone array decomposition of captured sound field into plane waves Playback filtering of plane waves with HRIRs → BRIRs, convolution of source signal Pros/Cons + copes for small translatory movements + head rotation possible + individual HRIRs – nearly fixed position of source/receiver Spors et al. | 21.9.2015 | Two!Ears – Simulating Room Acoustics | Binaural Synthesis Techniques 11 / 21
21. Room Acoustic Models Numerical/Wave-based (numerical) solution of the wave equation includes all effects due to the wave nature of sound complex for large rooms/small wavelegths Geometrical decomposes sound propagation into rays of sound effects due to wave nature are not fully covered applicable for large rooms/small wavelengths Statistical statistical properties of perceived room impression underlying assumption: diffuse sound field buildup and decay of sound, stationary sound field in rooms Spors et al. | 21.9.2015 | Two!Ears – Simulating Room Acoustics | Numeric Simulation 12 / 21
22. Numerical Simulation of Room Acoustics Various simulation environments compute the sound field in a room numerically State of the art hybrid model: mirror image model, raytracing closed (and coupled) rooms with arbitrary geometry frequency depend boundary conditions, directive sources/receivers estimation of various room acoustic measures, auralization Selected frameworks ODEON – Room Acoustics Software, http://www.odeon.dk/ EASE – Enhanced Acoustic Simulator for Engineers, http://ease.afmg.eu/ CATT – Acoustic, http://www.catt.se/ RAVEN – Room Acoustics for Virtual ENvironments, RWTH Aachen Spors et al. | 21.9.2015 | Two!Ears – Simulating Room Acoustics | Numeric Simulation 13 / 21
23. Measurement vs Simulation of Acoustic Environments Measurement/Recording + fine structure of responses + capture of diffuse sound fields – effort for multiple source/listener positions and environments Numeric Simulation + arbitrary position of source/receiver + individual HRIRs – limited natural fine structure – high computational complexity for complex/large environments Caution: perceptual equivalence does not guarantee physical equivalence! Spors et al. | 21.9.2015 | Two!Ears – Simulating Room Acoustics | Numeric Simulation 14 / 21
24. Summary – Techniques for Binaural Synthesis technique diffuse sound field moving sources head- orientation head- translation realism application pre-recorded binaural signals yes yes no no high background noise static HRIR/BRIR limited limited yes limited high static sound sources dynamic BRIR no yes no no high moving sound sources data-based synthesis yes no yes small-scale high static sound sources numerical simulation limited yes yes yes limited acoustic navigation Spors et al. | 21.9.2015 | Two!Ears – Simulating Room Acoustics | Numeric Simulation 15 / 21
25. Database of Audio-Visual Data synthesis of ear/eye signals requires data characterizing the environment evaluation requires physical and perceptual labels central database of audio-visual data and perceptual labels Infrastructure public and project-internal database software interface for seamless access common data formats https://github.com/TWOEARS/data Spors et al. | 21.9.2015 | Two!Ears – Simulating Room Acoustics | Database 16 / 21
26. Data Formats Binary Data available datasets are stored in various formats sounds converted to Waveform Audio File Format (WAV) IRs converted to Spatially Oriented Format for Acoustics (SOFA) Audio-Visual Scene Description XML-based description of scene references to binary data supported by binaural simulator Spors et al. | 21.9.2015 | Two!Ears – Simulating Room Acoustics | Database 17 / 21
27. Current Contents of Database Head-related transfer functions (SCUT, MIT, KEMAR, KU100, ...) Binaural room impulse responses (various rooms at TUB, URO, Surrey, ...) Sounds (monaural, binaural) (speech, environmental noise, ...) Perceptual labels (on-/offsets, head movements, localisation/coloration) Spors et al. | 21.9.2015 | Two!Ears – Simulating Room Acoustics | Database 18 / 21
28. The Binaural Simulator Acoustic Scene Listener Sources & Environment XML Parser Scene Description File MEX SoundScape Renderer Robot Interface Knowledge Sources scene objects frame size, sample rate, ... ear signals listener & sources head rotation & translatory movement current head orientation control ear signals https://github.com/TWOEARS/binaural-simulator Spors et al. | 21.9.2015 | Two!Ears – Simulating Room Acoustics | Implementation 19 / 21
29. Features Virtual scenes composed from pre-recorded binaural signals virtual sources synthesized by HRIRs/BRIRs moving virtual sources by concatenation of static positions spatial room response/ambience via plane wave decompositions virtual rooms by two-dimensional mirror image source model Outlook integration of numerical room acoustic simulation (RAVEN) three-dimensional scenes and HRIRs/BRIRs Spors et al. | 21.9.2015 | Two!Ears – Simulating Room Acoustics | Implementation 20 / 21
30. Conclusions/Outlook framework for scenario-based research and development open source, systematic evaluation ⇒ reproducible research best practice for other research projects Spors et al. | 21.9.2015 | Two!Ears – Simulating Room Acoustics | Conclusions 21 / 21