A Spatial Audio Quality Inventory for Virtual Acoustic Environments (SAQI)

Lindau, A.; Erbes, V.; Maempel, H.-J.; Lepa, S., Brinkmann, F.; Weinzierl, S.: "A Spatial Audio Quality Inventory for Virtual Acoustic Environments (SAQI)", Talk given at EAA Joint Symposium on Auralization and Ambisonics, Berlin, 2014

Transcript

1. 1 Fachgebiet Audiokommunikation Audio communication group A Spatial Audio Quality Inventory for Virtual Acoustic Environments (SAQI) Alexander Lindau1, Vera Erbes1, Steffen Lepa1, Hans-Joachim Maempel2, Fabian Brinkmann1, Stefan Weinzierl1 1Audio Communication Group, Technische Universität Berlin 2Federal Institute for Music Research, Berlin EAA Joint Symposium on Auralization and Ambisonics Berlin 2014
2. 2 How to perceptually evaluate Virtual Acoustic Environment?  Assessment of selected singular auditive qualities[1,2]  Assessment of overall perceptual accuracy[3,4]  Assessment according to differentiate sensory catalogues[5,6]  Ad-hoc vocabularies vs. empirically substantiated elicitation approaches Fachgebiet Audiokommunikation Audio communication group [1] Wierstorf, H. et al. (2013): "Localization in Wave Field Synthesis and HOA …", in: Proc. of the AIA-DAGA 2013 Conference on Acoustics. Meran [2] Wittek, H. et al. (2004): "Spatial perception in Wave Field Synthesis rendered sound fields: Distance of real and virtual nearby sources", AES Convention. Berlin, preprint no. 6000 [3] Lindau, A.; Weinzierl, S. (2012): "Assessing the Plausibility of Virtual Acoustic Environments", in: Acta Acustica united with Acustica, Vol. 98, No. 5, pp. 804-810 [4] Brinkmann, F. et al. (2014): "Assessing the Authenticity of Individual Dynamic Binaural Synthesis", EAA Joint Symposium on Auralization and Ambisonics. Berlin, pp. 62-68 [5] Silzle, A. (2007): Generation of Quality Taxonomies for Auditory Virtual Environments by Means of Systematic Expert Survey. doct diss., Ruhr-Universität Bochum, Aachen: Shaker [6] Lindau, A. et al. (2014): "A Spatial Audio Quality Inventory for Virtual Acoustic Environments (SAQI)", in: EAA Joint Symposium on Auralization and Ambisonics. Berlin
3. 3 Fachgebiet Audiokommunikation Audio communication group Prior Research Auditive Quality Lokki (2001) Pellegrini (2001) Zacharov (2001) Lorho (2005) Silzle (2007) Tone color √ √ √ √ √ Spaciousness √ √ √ √ √ Localizability √ √ √ √ √ Steadiness √ √ √ √ Size √ √ √ √ Distance √ √ Ensemble balance √ √ Externalization √ √ Artefacts √ √ Loudness √ √ Responsiveness √ √ Lokki, T; Järveläinen, H (2001): “Subjective evaluation of auralization of physics-based room acoustics modeling”, in: Proc. of ICAD. Espoo Pellegrini, R. S. (2001): “Quality Assessment of Auditory Virtual Environments”, in: Proc. of ICAD 2001, Espoo Zacharov, N.; Koivuniemi, K. (2001): "Unravelling the perception of spatial sound reproduction: Techniques and experimental design." In: Proc. of the 19th Int. AES Conf. Elmau Lorho, G. (2005): “Individual Vocabulary Profiling of Spatial Enhancement Systems for Stereo Headphone Reproduction”, in: Proc. of the 119th AES Conv. New York, Silzle, A. (2007): “Quality Taxonomies for Auditory Virtual Environments”, in: Proc. of the 122nd AES Conv., Vienna
4. 4 Fachgebiet Audiokommunikation Audio communication group Methods for creating consensual sensory vocabularies  Verbal vs. non-verbal  Stimulus-based vs. non-stimulus-based  Individual vs. group-based  Laymen vs. experts Nominal Group method, Delphi method, Focus Group method
5. 5 Fachgebiet Audiokommunikation Audio communication group “Create a consensus vocabulary … Discussion Guideline
6. 6 Fachgebiet Audiokommunikation Audio communication group “Create a consensus vocabulary covering all auditive qualities that may arise when comparing virtual acoustic environments … Discussion Guideline
7. 7 Fachgebiet Audiokommunikation Audio communication group “Create a consensus vocabulary covering all auditive qualities that may arise when comparing virtual acoustic environments of all types and in all stages of development… Discussion Guideline
8. 8 Fachgebiet Audiokommunikation Audio communication group “Create a consensus vocabulary covering all auditive qualities that may arise when comparing virtual acoustic environments of all types and in all stages of development with each other, or with respect to a given or imagined reference (e.g., the corresponding reality).” Discussion Guideline
9. 9 Fachgebiet Audiokommunikation Audio communication group “Create a consensus vocabulary covering all auditive qualities that may arise when comparing virtual acoustic environments of all types and in all stages of development with each other, or with respect to a given or imagined reference (e.g. the corresponding reality).” Quality names:  Describe perceptual process  Complete but still practically relevant  Formulated clearly and unambiguously  Non-overlapping Discussion Guideline
10. 10 Fachgebiet Audiokommunikation Audio communication group Dual-moderator, two-way Focus Group
11. 11 Fachgebiet Audiokommunikation Audio communication group Dual-moderator, two-way Focus Group
12. 12 Fachgebiet Audiokommunikation Audio communication group Dual-moderator, two-way Focus Group
13. 13 Fachgebiet Audiokommunikation Audio communication group Dual-moderator, two-way Focus Group
14. 14 Fachgebiet Audiokommunikation Audio communication group Dual-moderator, two-way Focus Group
15. 15 Moderated group discussion observers Fachgebiet Audiokommunikation Audio communication group Dual-moderator, two-way Focus Group
16. 16 Moderated group discussion Fachgebiet Audiokommunikation Audio communication group Dual-moderator, two-way Focus Group
17. 17 Moderated roundtable discussion Fachgebiet Audiokommunikation Audio communication group Dual-moderator, two-way Focus Group
18. 18 Fachgebiet Audiokommunikation Audio communication group Dual-moderator, two-way Focus Group
19. 19 Fachgebiet Audiokommunikation Audio communication group Validation 1. Internal post-check of semantic consensus 2. External post-check of understandability 3. Semantic analysis 4. Resolved by 5 experts Final German vocabulary
20. 20 Fachgebiet Audiokommunikation Audio communication group Translation GER  EN  Translation guidelines by Hambleton (2001)  Translation by 5 bilingual experts (two natives, 3 non-natives)  Individual translations of German quality names  Finalized in online group meeting  Back-translation by 3 German experts working abroad Hambleton, R. K. (2001): “The Next Generation of the ITC Test Translation and Adaptation Guidelines”, in: Europ. J. Psych. Ass., 17(3), pp. 164-172
21. 21 Category Auditory Quality Timbre (8) Tone color bright-dark, High-/Mid-/Low-frequency tone color, Sharpness, Roughness*, Comb filter coloration*, Metallic tone color Tonalness (3) Tonalness, Pitch, Doppler effect Geometry (10) Horizontal/Vertical direction, Front-back position, Distance, Depth, Width, Height, Externalization, Localizability, Spatial disintegration Room (3) Reverberation level, Reverberation time, Envelopment by reverberation Time behavior (7) Pre-/Post-echoes, Temporal disintegration, Crispness, Speed, Sequence of events, Responsiveness Dynamics (3) Loudness, Dynamic range, Dynamic compression effects* Artifacts (7) Pitched/Impulsive/Noise-like artifact, Alien source, Ghost source, Distortion, Tactile vibration General (7) (Overall) Difference, Clarity, Speech intelligibility, Naturalness, Presence, Degree-of-Liking, Other Fachgebiet Audiokommunikation Audio communication group Spatial Audio Quality Inventory (SAQI-EN) * Illustrative audio examples have been created.
22. Test Stimulus Exemplary SAQI Test Procedure
23. Imagined Reference Test Stimulus Exemplary SAQI Test Procedure
24. Imagined Reference Test Stimulus Reference Stimulus or Exemplary SAQI Test Procedure
25. Imagined Reference Test Stimulus Reference Stimulus or Overall Difference? whisPER 1.8.0 Matlab® toolbox Exemplary SAQI Test Procedure
26. Imagined Reference Test Stimulus Reference Stimulus or Overall Difference? End of Test No whisPER 1.8.0 Matlab® toolbox Exemplary SAQI Test Procedure
27. Imagined Reference Test Stimulus Reference Stimulus or Overall Difference? End of Test No whisPER 1.8.0 Matlab® toolbox Yes Unipolar Rating Scale Upper Scale Label Lower Scale Label Exemplary SAQI Test Procedure
28. Imagined Reference Test Stimulus Reference Stimulus or Overall Difference? End of Test No whisPER 1.8.0 Matlab® toolbox Yes Unipolar Rating Scale Upper Scale Label Lower Scale Label Exemplary SAQI Test Procedure
29. Imagined Reference Test Stimulus Reference Stimulus or Overall Difference? End of Test No (optional) Unipolar Rating Scale Upper Scale Label Lower Scale Label whisPER 1.8.0 Matlab® toolbox Yes Modifications II Modifications II Check Boxes Modifications I A B1 B2 C1 C2 F2 Exemplary SAQI Test Procedure
30. Imagined Reference Test Stimulus Reference Stimulus or Overall Difference? End of Test No (optional) Unipolar Rating Scale Upper Scale Label Lower Scale Label whisPER 1.8.0 Matlab® toolbox Yes Modifications II Modifications II Check Boxes Modifications I A B1 B2 C1 C2 F2 (optional) Check Boxes Assessment entities A D C B E F Other: Exemplary SAQI Test Procedure
31. Imagined Reference Test Stimulus Reference Stimulus or Overall Difference? End of Test No (optional) Unipolar Rating Scale Upper Scale Label Lower Scale Label whisPER 1.8.0 Matlab® toolbox Yes Modifications II Modifications II Check Boxes Modifications I A B1 B2 C1 C2 F2 (optional) Check Boxes Assessment entities A D C B E F Other: Difference Quality 1? “Circumscription …” Exemplary SAQI Test Procedure
32. Imagined Reference Test Stimulus Reference Stimulus or Overall Difference? End of Test No (optional) Unipolar Rating Scale Upper Scale Label Lower Scale Label whisPER 1.8.0 Matlab® toolbox Yes Modifications II Modifications II Check Boxes Modifications I A B1 B2 C1 C2 F2 (optional) Check Boxes Assessment entities A D C B E F Other: Difference Quality 1? “Circumscription …” Yes No Exemplary SAQI Test Procedure
33. Imagined Reference Test Stimulus Reference Stimulus or Overall Difference? End of Test No (optional) Unipolar Rating Scale Upper Scale Label Lower Scale Label whisPER 1.8.0 Matlab® toolbox Yes Modifications II Modifications II Check Boxes Modifications I A B1 B2 C1 C2 F2 (optional) Check Boxes Assessment entities A D C B E F Other: Difference Quality 1? “Circumscription …” (optional) (optional) Bipolar Rating Scale Left Hand Label Upper Scale Label Lower Scale Label Check Boxes Assessment entities A D C B E F Other: Yes No Modifications II Modifications II Check Boxes Modifications I A B1 B2 C1 C2 F2 Exemplary SAQI Test Procedure
34. Imagined Reference Test Stimulus Reference Stimulus or Overall Difference? End of Test No (optional) Unipolar Rating Scale Upper Scale Label Lower Scale Label whisPER 1.8.0 Matlab® toolbox (optional) Difference Quality 1? “Circumscription …” (optional) (optional) Bipolar Rating Scale Left Hand Label Upper Scale Label Lower Scale Label Check Boxes Assessment entities A D C B E F Other: Check Boxes Assessment entities A D C B E F Other: Yes No Yes Modifications II Modifications II Check Boxes Modifications I A B1 B2 C1 C2 F2 Modifications II Modifications II Check Boxes Modifications I A B1 B2 C1 C2 F2 Exemplary SAQI Test Procedure
35. Imagined Reference Test Stimulus Reference Stimulus or Overall Difference? End of Test No (optional) Unipolar Rating Scale Upper Scale Label Lower Scale Label whisPER 1.8.0 Matlab® toolbox (optional) Difference Quality 1? “Circumscription …” (optional) (optional) Difference Quality N? “Circumscription …” (optional) (optional) Bipolar Rating Scale Left Hand Label Upper Scale Label Lower Scale Label Bipolar Rating Scale Left Hand Label Upper Scale Label Lower Scale Label Check Boxes Assessment entities A D C B E F Other: Check Boxes Assessment entities A D C B E F Other: Check Boxes Assessment entities A D C B E F Other: “XYZ…” Yes No Yes Yes No Modifications II Modifications II Check Boxes Modifications I A B1 B2 C1 C2 F2 Modifications II Modifications II Check Boxes Modifications I A B1 B2 C1 C2 F2 Modifications II Modifications II Check Boxes Modifications I A B1 B2 C1 C2 F2 Exemplary SAQI Test Procedure
36. Imagined Reference Test Stimulus Reference Stimulus or Overall Difference? End of Test No (optional) Unipolar Rating Scale Upper Scale Label Lower Scale Label whisPER 1.8.0 Matlab® toolbox (optional) Difference Quality 1? “Circumscription …” (optional) (optional) Difference Quality N? “Circumscription …” (optional) (optional) Other: Bipolar Rating Scale Left Hand Label Upper Scale Label Lower Scale Label Bipolar Rating Scale Left Hand Label Upper Scale Label Lower Scale Label Check Boxes Assessment entities A D C B E F Other: Check Boxes Assessment entities A D C B E F Other: Check Boxes Assessment entities A D C B E F Other: “XYZ…” Yes No Yes Yes No Modifications II Modifications II Check Boxes Modifications I A B1 B2 C1 C2 F2 Modifications II Modifications II Check Boxes Modifications I A B1 B2 C1 C2 F2 Modifications II Modifications II Check Boxes Modifications I A B1 B2 C1 C2 F2 Other Differences? Exemplary SAQI Test Procedure
37. Imagined Reference Test Stimulus Reference Stimulus or Overall Difference? End of Test No (optional) Unipolar Rating Scale Upper Scale Label Lower Scale Label whisPER 1.8.0 Matlab® toolbox (optional) Difference Quality 1? “Circumscription …” (optional) (optional) Difference Quality N? “Circumscription …” (optional) (optional) Other: Bipolar Rating Scale Left Hand Label Upper Scale Label Lower Scale Label Bipolar Rating Scale Left Hand Label Upper Scale Label Lower Scale Label Check Boxes Assessment entities A D C B E F Other: Check Boxes Assessment entities A D C B E F Other: Check Boxes Assessment entities A D C B E F Other: “XYZ…” Yes No Yes Yes No Yes No Modifications II Modifications II Check Boxes Modifications I A B1 B2 C1 C2 F2 Modifications II Modifications II Check Boxes Modifications I A B1 B2 C1 C2 F2 Modifications II Modifications II Check Boxes Modifications I A B1 B2 C1 C2 F2 Other Differences? Exemplary SAQI Test Procedure
38. Imagined Reference Test Stimulus Reference Stimulus or Overall Difference? End of Test No (optional) Unipolar Rating Scale Upper Scale Label Lower Scale Label whisPER 1.8.0 Matlab® toolbox (optional) Difference Quality 1? “Circumscription …” (optional) (optional) Difference Quality N? “Circumscription …” (optional) (optional) Other Differences? Other: Bipolar Rating Scale Left Hand Label Upper Scale Label Lower Scale Label Bipolar Rating Scale Left Hand Label Upper Scale Label Lower Scale Label Bipolar Rating Scale Left Hand Label Upper Scale Label Lower Scale Label Check Boxes Assessment entities A D C B E F Other: Check Boxes Assessment entities A D C B E F Other: Check Boxes Assessment entities A D C B E F Other: “XYZ…” Yes No Yes Yes No Yes No (optional) (optional) Check Boxes Assessment entities A D C B E F Other: Modifications II Modifications II Check Boxes Modifications I A B1 B2 C1 C2 F2 Modifications II Modifications II Check Boxes Modifications I A B1 B2 C1 C2 F2 Modifications II Modifications II Check Boxes Modifications I A B1 B2 C1 C2 F2 Modifications II Modifications II Check Boxes Modifications I A B1 B2 C1 C2 F2 Exemplary SAQI Test Procedure
39. 39 Resources  SAQI-EN/GER in free Matlab® listening test toolbox WhisPER 1.8.0  DOI 10.14279/depositonce-31  SAQI Test Manual v.1.0  DOI 10.14279/depositonce-1  Explanations, practical guidelines  Audio examples  Test data from real SAQI tests  Functions for plotting and convenient data import to SPSS Fachgebiet Audiokommunikation Audio communication group
40. 40 Fachgebiet Audiokommunikation Audio communication group First Applications – Non-individual dynamic binaural synthesis
41. 41 Fachgebiet Audiokommunikation Audio communication group First Applications – Non-individual dynamic binaural synthesis
42. 42 Focus Group Members Fabian Brinkmann, Benjamin Bernschütz, Clemens Büttner, Diemer de Vries, Vera Erbes, Alexander Fuß, Matthias Geier, Martin Guski, Michael Horn, Stefan Klockgether, Steffen Lepa, Alexander Lindau, Hans-Joachim Maempel, Johannes Nowak, Rob Opdam, Zora Schärer, Frank Schultz, Sascha Spors, Michael Vorländer, Stefan Weinzierl and Hagen Wierstorf. Validation of German SAQI Matthias Frank, Markus Noisternig, Sönke Pelzer, Andreas Silze, and Franz Zotter. English SAQI Jude Brereton, Kees de Visser, Brian Gygi, Charalampos Saitis, and Steven van de Par English-German back-translation Frank Melchior, Nils Peters and Ulrich Reiter. Fachgebiet Audiokommunikation Audio communication group
43. 43 Fachgebiet Audiokommunikation Audio communication group Thank you! alexander.lindau@tu-berlin.de