A Spatial Audio Quality Inventory for Virtual Acoustic Environments (SAQI)

A Spatial Audio Quality Inventory for Virtual Acoustic Environments (SAQI)

Lindau, A.; Erbes, V.; Maempel, H.-J.; Lepa, S., Brinkmann, F.; Weinzierl, S.: "A Spatial Audio Quality Inventory for Virtual Acoustic Environments (SAQI)", Talk given at EAA Joint Symposium on Auralization and Ambisonics, Berlin, 2014

724ec4ba746d1411fe8600b4d533a8d9?s=128

Dr. Alexander Lindau

April 04, 2014
Tweet

Transcript

  1. 1 Fachgebiet Audiokommunikation Audio communication group A Spatial Audio Quality

    Inventory for Virtual Acoustic Environments (SAQI) Alexander Lindau1, Vera Erbes1, Steffen Lepa1, Hans-Joachim Maempel2, Fabian Brinkmann1, Stefan Weinzierl1 1Audio Communication Group, Technische Universität Berlin 2Federal Institute for Music Research, Berlin EAA Joint Symposium on Auralization and Ambisonics Berlin 2014
  2. 2 How to perceptually evaluate Virtual Acoustic Environment?  Assessment

    of selected singular auditive qualities[1,2]  Assessment of overall perceptual accuracy[3,4]  Assessment according to differentiate sensory catalogues[5,6]  Ad-hoc vocabularies vs. empirically substantiated elicitation approaches Fachgebiet Audiokommunikation Audio communication group [1] Wierstorf, H. et al. (2013): "Localization in Wave Field Synthesis and HOA …", in: Proc. of the AIA-DAGA 2013 Conference on Acoustics. Meran [2] Wittek, H. et al. (2004): "Spatial perception in Wave Field Synthesis rendered sound fields: Distance of real and virtual nearby sources", AES Convention. Berlin, preprint no. 6000 [3] Lindau, A.; Weinzierl, S. (2012): "Assessing the Plausibility of Virtual Acoustic Environments", in: Acta Acustica united with Acustica, Vol. 98, No. 5, pp. 804-810 [4] Brinkmann, F. et al. (2014): "Assessing the Authenticity of Individual Dynamic Binaural Synthesis", EAA Joint Symposium on Auralization and Ambisonics. Berlin, pp. 62-68 [5] Silzle, A. (2007): Generation of Quality Taxonomies for Auditory Virtual Environments by Means of Systematic Expert Survey. doct diss., Ruhr-Universität Bochum, Aachen: Shaker [6] Lindau, A. et al. (2014): "A Spatial Audio Quality Inventory for Virtual Acoustic Environments (SAQI)", in: EAA Joint Symposium on Auralization and Ambisonics. Berlin
  3. 3 Fachgebiet Audiokommunikation Audio communication group Prior Research Auditive Quality

    Lokki (2001) Pellegrini (2001) Zacharov (2001) Lorho (2005) Silzle (2007) Tone color √ √ √ √ √ Spaciousness √ √ √ √ √ Localizability √ √ √ √ √ Steadiness √ √ √ √ Size √ √ √ √ Distance √ √ Ensemble balance √ √ Externalization √ √ Artefacts √ √ Loudness √ √ Responsiveness √ √ Lokki, T; Järveläinen, H (2001): “Subjective evaluation of auralization of physics-based room acoustics modeling”, in: Proc. of ICAD. Espoo Pellegrini, R. S. (2001): “Quality Assessment of Auditory Virtual Environments”, in: Proc. of ICAD 2001, Espoo Zacharov, N.; Koivuniemi, K. (2001): "Unravelling the perception of spatial sound reproduction: Techniques and experimental design." In: Proc. of the 19th Int. AES Conf. Elmau Lorho, G. (2005): “Individual Vocabulary Profiling of Spatial Enhancement Systems for Stereo Headphone Reproduction”, in: Proc. of the 119th AES Conv. New York, Silzle, A. (2007): “Quality Taxonomies for Auditory Virtual Environments”, in: Proc. of the 122nd AES Conv., Vienna
  4. 4 Fachgebiet Audiokommunikation Audio communication group Methods for creating consensual

    sensory vocabularies  Verbal vs. non-verbal  Stimulus-based vs. non-stimulus-based  Individual vs. group-based  Laymen vs. experts Nominal Group method, Delphi method, Focus Group method
  5. 5 Fachgebiet Audiokommunikation Audio communication group “Create a consensus vocabulary

    … Discussion Guideline
  6. 6 Fachgebiet Audiokommunikation Audio communication group “Create a consensus vocabulary

    covering all auditive qualities that may arise when comparing virtual acoustic environments … Discussion Guideline
  7. 7 Fachgebiet Audiokommunikation Audio communication group “Create a consensus vocabulary

    covering all auditive qualities that may arise when comparing virtual acoustic environments of all types and in all stages of development… Discussion Guideline
  8. 8 Fachgebiet Audiokommunikation Audio communication group “Create a consensus vocabulary

    covering all auditive qualities that may arise when comparing virtual acoustic environments of all types and in all stages of development with each other, or with respect to a given or imagined reference (e.g., the corresponding reality).” Discussion Guideline
  9. 9 Fachgebiet Audiokommunikation Audio communication group “Create a consensus vocabulary

    covering all auditive qualities that may arise when comparing virtual acoustic environments of all types and in all stages of development with each other, or with respect to a given or imagined reference (e.g. the corresponding reality).” Quality names:  Describe perceptual process  Complete but still practically relevant  Formulated clearly and unambiguously  Non-overlapping Discussion Guideline
  10. 10 Fachgebiet Audiokommunikation Audio communication group Dual-moderator, two-way Focus Group

  11. 11 Fachgebiet Audiokommunikation Audio communication group Dual-moderator, two-way Focus Group

  12. 12 Fachgebiet Audiokommunikation Audio communication group Dual-moderator, two-way Focus Group

  13. 13 Fachgebiet Audiokommunikation Audio communication group Dual-moderator, two-way Focus Group

  14. 14 Fachgebiet Audiokommunikation Audio communication group Dual-moderator, two-way Focus Group

  15. 15 Moderated group discussion observers Fachgebiet Audiokommunikation Audio communication group

    Dual-moderator, two-way Focus Group
  16. 16 Moderated group discussion Fachgebiet Audiokommunikation Audio communication group Dual-moderator,

    two-way Focus Group
  17. 17 Moderated roundtable discussion Fachgebiet Audiokommunikation Audio communication group Dual-moderator,

    two-way Focus Group
  18. 18 Fachgebiet Audiokommunikation Audio communication group Dual-moderator, two-way Focus Group

  19. 19 Fachgebiet Audiokommunikation Audio communication group Validation 1. Internal post-check

    of semantic consensus 2. External post-check of understandability 3. Semantic analysis 4. Resolved by 5 experts Final German vocabulary
  20. 20 Fachgebiet Audiokommunikation Audio communication group Translation GER  EN

     Translation guidelines by Hambleton (2001)  Translation by 5 bilingual experts (two natives, 3 non-natives)  Individual translations of German quality names  Finalized in online group meeting  Back-translation by 3 German experts working abroad Hambleton, R. K. (2001): “The Next Generation of the ITC Test Translation and Adaptation Guidelines”, in: Europ. J. Psych. Ass., 17(3), pp. 164-172
  21. 21 Category Auditory Quality Timbre (8) Tone color bright-dark, High-/Mid-/Low-frequency

    tone color, Sharpness, Roughness*, Comb filter coloration*, Metallic tone color Tonalness (3) Tonalness, Pitch, Doppler effect Geometry (10) Horizontal/Vertical direction, Front-back position, Distance, Depth, Width, Height, Externalization, Localizability, Spatial disintegration Room (3) Reverberation level, Reverberation time, Envelopment by reverberation Time behavior (7) Pre-/Post-echoes, Temporal disintegration, Crispness, Speed, Sequence of events, Responsiveness Dynamics (3) Loudness, Dynamic range, Dynamic compression effects* Artifacts (7) Pitched/Impulsive/Noise-like artifact, Alien source, Ghost source, Distortion, Tactile vibration General (7) (Overall) Difference, Clarity, Speech intelligibility, Naturalness, Presence, Degree-of-Liking, Other Fachgebiet Audiokommunikation Audio communication group Spatial Audio Quality Inventory (SAQI-EN) * Illustrative audio examples have been created.
  22. Test Stimulus Exemplary SAQI Test Procedure

  23. Imagined Reference Test Stimulus Exemplary SAQI Test Procedure

  24. Imagined Reference Test Stimulus Reference Stimulus or Exemplary SAQI Test

    Procedure
  25. Imagined Reference Test Stimulus Reference Stimulus or Overall Difference? whisPER

    1.8.0 Matlab® toolbox Exemplary SAQI Test Procedure
  26. Imagined Reference Test Stimulus Reference Stimulus or Overall Difference? End

    of Test No whisPER 1.8.0 Matlab® toolbox Exemplary SAQI Test Procedure
  27. Imagined Reference Test Stimulus Reference Stimulus or Overall Difference? End

    of Test No whisPER 1.8.0 Matlab® toolbox Yes Unipolar Rating Scale Upper Scale Label Lower Scale Label Exemplary SAQI Test Procedure
  28. Imagined Reference Test Stimulus Reference Stimulus or Overall Difference? End

    of Test No whisPER 1.8.0 Matlab® toolbox Yes Unipolar Rating Scale Upper Scale Label Lower Scale Label Exemplary SAQI Test Procedure
  29. Imagined Reference Test Stimulus Reference Stimulus or Overall Difference? End

    of Test No (optional) Unipolar Rating Scale Upper Scale Label Lower Scale Label whisPER 1.8.0 Matlab® toolbox Yes Modifications II Modifications II Check Boxes Modifications I A B1 B2 C1 C2 F2 Exemplary SAQI Test Procedure
  30. Imagined Reference Test Stimulus Reference Stimulus or Overall Difference? End

    of Test No (optional) Unipolar Rating Scale Upper Scale Label Lower Scale Label whisPER 1.8.0 Matlab® toolbox Yes Modifications II Modifications II Check Boxes Modifications I A B1 B2 C1 C2 F2 (optional) Check Boxes Assessment entities A D C B E F Other: Exemplary SAQI Test Procedure
  31. Imagined Reference Test Stimulus Reference Stimulus or Overall Difference? End

    of Test No (optional) Unipolar Rating Scale Upper Scale Label Lower Scale Label whisPER 1.8.0 Matlab® toolbox Yes Modifications II Modifications II Check Boxes Modifications I A B1 B2 C1 C2 F2 (optional) Check Boxes Assessment entities A D C B E F Other: Difference Quality 1? “Circumscription …” Exemplary SAQI Test Procedure
  32. Imagined Reference Test Stimulus Reference Stimulus or Overall Difference? End

    of Test No (optional) Unipolar Rating Scale Upper Scale Label Lower Scale Label whisPER 1.8.0 Matlab® toolbox Yes Modifications II Modifications II Check Boxes Modifications I A B1 B2 C1 C2 F2 (optional) Check Boxes Assessment entities A D C B E F Other: Difference Quality 1? “Circumscription …” Yes No Exemplary SAQI Test Procedure
  33. Imagined Reference Test Stimulus Reference Stimulus or Overall Difference? End

    of Test No (optional) Unipolar Rating Scale Upper Scale Label Lower Scale Label whisPER 1.8.0 Matlab® toolbox Yes Modifications II Modifications II Check Boxes Modifications I A B1 B2 C1 C2 F2 (optional) Check Boxes Assessment entities A D C B E F Other: Difference Quality 1? “Circumscription …” (optional) (optional) Bipolar Rating Scale Left Hand Label Upper Scale Label Lower Scale Label Check Boxes Assessment entities A D C B E F Other: Yes No Modifications II Modifications II Check Boxes Modifications I A B1 B2 C1 C2 F2 Exemplary SAQI Test Procedure
  34. Imagined Reference Test Stimulus Reference Stimulus or Overall Difference? End

    of Test No (optional) Unipolar Rating Scale Upper Scale Label Lower Scale Label whisPER 1.8.0 Matlab® toolbox (optional) Difference Quality 1? “Circumscription …” (optional) (optional) Bipolar Rating Scale Left Hand Label Upper Scale Label Lower Scale Label Check Boxes Assessment entities A D C B E F Other: Check Boxes Assessment entities A D C B E F Other: Yes No Yes Modifications II Modifications II Check Boxes Modifications I A B1 B2 C1 C2 F2 Modifications II Modifications II Check Boxes Modifications I A B1 B2 C1 C2 F2 Exemplary SAQI Test Procedure
  35. Imagined Reference Test Stimulus Reference Stimulus or Overall Difference? End

    of Test No (optional) Unipolar Rating Scale Upper Scale Label Lower Scale Label whisPER 1.8.0 Matlab® toolbox (optional) Difference Quality 1? “Circumscription …” (optional) (optional) Difference Quality N? “Circumscription …” (optional) (optional) Bipolar Rating Scale Left Hand Label Upper Scale Label Lower Scale Label Bipolar Rating Scale Left Hand Label Upper Scale Label Lower Scale Label Check Boxes Assessment entities A D C B E F Other: Check Boxes Assessment entities A D C B E F Other: Check Boxes Assessment entities A D C B E F Other: “XYZ…” Yes No Yes Yes No Modifications II Modifications II Check Boxes Modifications I A B1 B2 C1 C2 F2 Modifications II Modifications II Check Boxes Modifications I A B1 B2 C1 C2 F2 Modifications II Modifications II Check Boxes Modifications I A B1 B2 C1 C2 F2 Exemplary SAQI Test Procedure
  36. Imagined Reference Test Stimulus Reference Stimulus or Overall Difference? End

    of Test No (optional) Unipolar Rating Scale Upper Scale Label Lower Scale Label whisPER 1.8.0 Matlab® toolbox (optional) Difference Quality 1? “Circumscription …” (optional) (optional) Difference Quality N? “Circumscription …” (optional) (optional) Other: Bipolar Rating Scale Left Hand Label Upper Scale Label Lower Scale Label Bipolar Rating Scale Left Hand Label Upper Scale Label Lower Scale Label Check Boxes Assessment entities A D C B E F Other: Check Boxes Assessment entities A D C B E F Other: Check Boxes Assessment entities A D C B E F Other: “XYZ…” Yes No Yes Yes No Modifications II Modifications II Check Boxes Modifications I A B1 B2 C1 C2 F2 Modifications II Modifications II Check Boxes Modifications I A B1 B2 C1 C2 F2 Modifications II Modifications II Check Boxes Modifications I A B1 B2 C1 C2 F2 Other Differences? Exemplary SAQI Test Procedure
  37. Imagined Reference Test Stimulus Reference Stimulus or Overall Difference? End

    of Test No (optional) Unipolar Rating Scale Upper Scale Label Lower Scale Label whisPER 1.8.0 Matlab® toolbox (optional) Difference Quality 1? “Circumscription …” (optional) (optional) Difference Quality N? “Circumscription …” (optional) (optional) Other: Bipolar Rating Scale Left Hand Label Upper Scale Label Lower Scale Label Bipolar Rating Scale Left Hand Label Upper Scale Label Lower Scale Label Check Boxes Assessment entities A D C B E F Other: Check Boxes Assessment entities A D C B E F Other: Check Boxes Assessment entities A D C B E F Other: “XYZ…” Yes No Yes Yes No Yes No Modifications II Modifications II Check Boxes Modifications I A B1 B2 C1 C2 F2 Modifications II Modifications II Check Boxes Modifications I A B1 B2 C1 C2 F2 Modifications II Modifications II Check Boxes Modifications I A B1 B2 C1 C2 F2 Other Differences? Exemplary SAQI Test Procedure
  38. Imagined Reference Test Stimulus Reference Stimulus or Overall Difference? End

    of Test No (optional) Unipolar Rating Scale Upper Scale Label Lower Scale Label whisPER 1.8.0 Matlab® toolbox (optional) Difference Quality 1? “Circumscription …” (optional) (optional) Difference Quality N? “Circumscription …” (optional) (optional) Other Differences? Other: Bipolar Rating Scale Left Hand Label Upper Scale Label Lower Scale Label Bipolar Rating Scale Left Hand Label Upper Scale Label Lower Scale Label Bipolar Rating Scale Left Hand Label Upper Scale Label Lower Scale Label Check Boxes Assessment entities A D C B E F Other: Check Boxes Assessment entities A D C B E F Other: Check Boxes Assessment entities A D C B E F Other: “XYZ…” Yes No Yes Yes No Yes No (optional) (optional) Check Boxes Assessment entities A D C B E F Other: Modifications II Modifications II Check Boxes Modifications I A B1 B2 C1 C2 F2 Modifications II Modifications II Check Boxes Modifications I A B1 B2 C1 C2 F2 Modifications II Modifications II Check Boxes Modifications I A B1 B2 C1 C2 F2 Modifications II Modifications II Check Boxes Modifications I A B1 B2 C1 C2 F2 Exemplary SAQI Test Procedure
  39. 39 Resources  SAQI-EN/GER in free Matlab® listening test toolbox

    WhisPER 1.8.0  DOI 10.14279/depositonce-31  SAQI Test Manual v.1.0  DOI 10.14279/depositonce-1  Explanations, practical guidelines  Audio examples  Test data from real SAQI tests  Functions for plotting and convenient data import to SPSS Fachgebiet Audiokommunikation Audio communication group
  40. 40 Fachgebiet Audiokommunikation Audio communication group First Applications – Non-individual

    dynamic binaural synthesis
  41. 41 Fachgebiet Audiokommunikation Audio communication group First Applications – Non-individual

    dynamic binaural synthesis
  42. 42 Focus Group Members Fabian Brinkmann, Benjamin Bernschütz, Clemens Büttner,

    Diemer de Vries, Vera Erbes, Alexander Fuß, Matthias Geier, Martin Guski, Michael Horn, Stefan Klockgether, Steffen Lepa, Alexander Lindau, Hans-Joachim Maempel, Johannes Nowak, Rob Opdam, Zora Schärer, Frank Schultz, Sascha Spors, Michael Vorländer, Stefan Weinzierl and Hagen Wierstorf. Validation of German SAQI Matthias Frank, Markus Noisternig, Sönke Pelzer, Andreas Silze, and Franz Zotter. English SAQI Jude Brereton, Kees de Visser, Brian Gygi, Charalampos Saitis, and Steven van de Par English-German back-translation Frank Melchior, Nils Peters and Ulrich Reiter. Fachgebiet Audiokommunikation Audio communication group
  43. 43 Fachgebiet Audiokommunikation Audio communication group Thank you! alexander.lindau@tu-berlin.de