Scenario-Based Development and Evaluation Process real environment observer source model ear signals vision feedback Real World virtual environment virtual observer virtual source model ear signals vision feedback Virtual World Spors et al. | 21.9.2015 | Two!Ears – Simulating Room Acoustics | Introduction 2 / 21
Scenario-Based Development and Evaluation Process real environment HATS source model ear signals vision feedback Real World simulation framework model ear signals vision feedback scenario database Virtual World Spors et al. | 21.9.2015 | Two!Ears – Simulating Room Acoustics | Introduction 2 / 21
Scenario-Based Development and Evaluation Process real-world input less controllable but potentially richer interactive and exploratory nature of model signals depend on feedback from upper stages block/frame-based computation of signals simulation framework model ear signals vision feedback scenario database Virtual World Spors et al. | 21.9.2015 | Two!Ears – Simulating Room Acoustics | Introduction 2 / 21
Techniques for Binaural Synthesis Degrees of realism capture of real-environments model/data-based synthesis of virtual environments numeric simulation of virtual environments Ear signals captured with individuals head and torso simulator (HATS) Underlying signal processing techniques recording/playback of ear-signals capture/convolution of source signals with impulse responses Spors et al. | 21.9.2015 | Two!Ears – Simulating Room Acoustics | Introduction 3 / 21
Head-Related Impulse Responses (HRIRs) Recording identify impulse responses from a source to the left/right ears → HRIRs database of HRIRs for different source positions head/torso rotation equivalent to source at fixed distance Playback convolution of dry source signal with HRIRs Pros/Cons + copes for head orientation + variable position of source/receiver – no diffuse sources Applications (anechoic) scenarios with limited number of (static) sources Spors et al. | 21.9.2015 | Two!Ears – Simulating Room Acoustics | Binaural Synthesis Techniques 5 / 21
Example – QU KEMAR with 3m Source Distance Left/right HRIR for source 60o to the left side 0 1 2 3 4 5 time (ms) −1.0 −0.5 0.0 0.5 1.0 left ear right ear Spors et al. | 21.9.2015 | Two!Ears – Simulating Room Acoustics | Binaural Synthesis Techniques 6 / 21
The SoundScape Renderer (SSR) A versatile framework for spatial sound reproduction techniques: WFS, HOA, VBAP, binaural synthesis, ... Linux and OS X open source software (GPLv2) http://spatialaudio.net/ssr Spors et al. | 21.9.2015 | Two!Ears – Simulating Room Acoustics | Binaural Synthesis Techniques 7 / 21
Binaural Room Impulse Responses (BRIRs) Recording identify room impulse responses from a source to the left/right ears → BRIRs database of BRIRs for different head-orientations Playback convolution of dry source signal with BRIRs Pros/Cons + copes for head rotation + includes room acoustics – fixed position of source/receiver Applications reverberant scenarios with low number of static sources Spors et al. | 21.9.2015 | Two!Ears – Simulating Room Acoustics | Binaural Synthesis Techniques 8 / 21
Example – QU KEMAR in Auditorium Left/right ear HRIR for head rotated 60o to the right side 0 50 100 150 200 time (ms) −1.0 −0.5 0.0 0.5 1.0 left ear right ear Spors et al. | 21.9.2015 | Two!Ears – Simulating Room Acoustics | Binaural Synthesis Techniques 9 / 21
Dynamic Binaural Room Impulse Responses Recording time-variant room impulse responses from a moving source to the left/right ears Playback time-variant convolution of dry source signal Pros/Cons + includes acoustic properties of room – fixed trajectory of source/receiver – fixed head orientation Applications reverberant scenario with low number of dynamic sources Spors et al. | 21.9.2015 | Two!Ears – Simulating Room Acoustics | Binaural Synthesis Techniques 10 / 21
Data-based Binaural Synthesis Recording identify impulse responses from a source to a microphone array decomposition of captured sound field into plane waves Playback filtering of plane waves with HRIRs → BRIRs, convolution of source signal Pros/Cons + copes for small translatory movements + head rotation possible + individual HRIRs – nearly fixed position of source/receiver Spors et al. | 21.9.2015 | Two!Ears – Simulating Room Acoustics | Binaural Synthesis Techniques 11 / 21
Room Acoustic Models Numerical/Wave-based (numerical) solution of the wave equation includes all effects due to the wave nature of sound complex for large rooms/small wavelegths Geometrical decomposes sound propagation into rays of sound effects due to wave nature are not fully covered applicable for large rooms/small wavelengths Statistical statistical properties of perceived room impression underlying assumption: diffuse sound field buildup and decay of sound, stationary sound field in rooms Spors et al. | 21.9.2015 | Two!Ears – Simulating Room Acoustics | Numeric Simulation 12 / 21
Database of Audio-Visual Data synthesis of ear/eye signals requires data characterizing the environment evaluation requires physical and perceptual labels central database of audio-visual data and perceptual labels Infrastructure public and project-internal database software interface for seamless access common data formats https://github.com/TWOEARS/data Spors et al. | 21.9.2015 | Two!Ears – Simulating Room Acoustics | Database 16 / 21
Data Formats Binary Data available datasets are stored in various formats sounds converted to Waveform Audio File Format (WAV) IRs converted to Spatially Oriented Format for Acoustics (SOFA) Audio-Visual Scene Description XML-based description of scene references to binary data supported by binaural simulator Spors et al. | 21.9.2015 | Two!Ears – Simulating Room Acoustics | Database 17 / 21
Conclusions/Outlook framework for scenario-based research and development open source, systematic evaluation ⇒ reproducible research best practice for other research projects Spors et al. | 21.9.2015 | Two!Ears – Simulating Room Acoustics | Conclusions 21 / 21
spors
ynakano
koro485
ailaboocu
33yutaroyutaroyutaro
nyoho
fr0gger
keyakkie
wd15
s1ok69oo
rabernat
ykamit
reverentgeek
lemiorhan
hursman
morganepeng
chriscoyier
shpigford
jacobian
samlambert
andyhume
denniskardys
holman
yeseniaperezcruz