Upgrade to PRO for Only $50/Year—Limited-Time Offer! 🔥

Auralization of road vehicles using spectral mo...

Avatar for Chinmay Pendharkar Chinmay Pendharkar
April 08, 2013
Research
0
200

Auralization of road vehicles using spectral modeling synthesis

Masters Thesis presentation on auralization of road vehicles using spectral modeling synthesis

Avatar for Chinmay Pendharkar

Chinmay Pendharkar

April 08, 2013
Tweet

More Decks by Chinmay Pendharkar

See All by Chinmay Pendharkar
Audio Fundamentals - Pd
Avatar for Chinmay Pendharkar notthetup
0
97
DNSSEC and Bind
Avatar for Chinmay Pendharkar notthetup
1
150
Garageband And Podcasting
Avatar for Chinmay Pendharkar notthetup
0
59
Audio Fundamentals - HTML5 Audio
Avatar for Chinmay Pendharkar notthetup
0
120
Audio Fundamentals - Oscillators
Avatar for Chinmay Pendharkar notthetup
0
60
Audio Fundamentals - Basics
Avatar for Chinmay Pendharkar notthetup
0
120
Robots and Pi
Avatar for Chinmay Pendharkar notthetup
2
120
What I’ve learnt about Environmental Sound Design
Avatar for Chinmay Pendharkar notthetup
0
240
Audio Editing with Audacity
Avatar for Chinmay Pendharkar notthetup
0
70

Other Decks in Research

See All in Research
Satellites Reveal Mobility: A Commuting Origin-destination Flow Generator for Global Cities
Avatar for SatAI.challenge satai
3
260
Earth AI: Unlocking Geospatial Insights with Foundation Models and Cross-Modal Reasoning
Avatar for SatAI.challenge satai
2
200
[Devfest Incheon 2025] 모두를 위한 친절한 언어모델(LLM) 학습 가이드
Avatar for Beomi beomi
2
1.3k
「リアル×スキマ時間」​を活用したUXリサーチ ​〜新規事業を前に進めるためのUXリサーチプロセスの設計〜
Avatar for PERSOL CAREER Dev | techtekt techtekt
PRO
0
210
生成AI による論文執筆サポート・ワークショップ ─ サーベイ／リサーチクエスチョン編 / Workshop on AI-Assisted Paper Writing Support: Survey/Research Question Edition
Avatar for Kenji Saito ks91
PRO
0
130
Learning to (Learn at Test Time): RNNs with Expressive Hidden States
Avatar for Hiroto Kurita kurita
1
300
大規模言語モデルにおけるData-Centric AIと合成データの活用 / Data-Centric AI and Synthetic Data in Large Language Models
Avatar for tsurubee tsurubee
1
460
空間音響処理における物理法則に基づく機械学習
Avatar for NII S. Koyama's Lab skoyamalab
0
140
長期・短期メモリを活用したエージェントの個別最適化
Avatar for AITC - DENTSU SOKEN isidaitc
0
360
Community Driveプロジェクト（CDPJ）の中間報告
Avatar for SMARTふくしラボ smartfukushilab1
0
110
SREのためのテレメトリー技術の探究 / Telemetry for SRE
Avatar for Yuuki Tsubouchi (yuuk1) yuukit
13
2.6k
POI: Proof of Identity
Avatar for MATSUMOTO Katsuyoshi katsyoshi
0
120

Featured

See All Featured
Mobile First: as difficult as doing things right
Avatar for Swwweet swwweet
225
10k
Agile Leadership in an Agile Organization
Avatar for Dr. Kim W Petersen kimpetersen
PRO
0
58
How to Grow Your eCommerce with AI & Automation
Avatar for Katarina Dahlin katarinadahlin
PRO
0
78
The Pragmatic Product Professional
Avatar for Laura Van Doore lauravandoore
37
7.1k
The Mindset for Success: Future Career Progression
Avatar for Greg Gifford greggifford
PRO
0
200
How to build a perfect <img>
Avatar for Jono Alderson jonoalderson
0
4.7k
The Cult of Friendly URLs
Avatar for Andy Hume andyhume
79
6.7k
Music & Morning Musume
Avatar for Bryan Veloso bryan
46
7k
The Art of Programming - Codeland 2020
Avatar for Erika Heidi erikaheidi
56
14k
Redefining SEO in the New Era of Traffic Generation
Avatar for Szymon szymonslowik
1
170
DevOps and Value Stream Thinking: Enabling flow, efficiency and business value
Avatar for Helen Beal helenjbeal
1
70
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
Avatar for panda_program panda_program
122
21k

Transcript

  1. Auralization of road vehicles using spectral modeling synthesis Chinmay Pendharkar,

    Applied Acoustics, Chalmers, 2011 Monday, April 8, 13

  2. • LISTEN Project • Auralization • Sound Synthesis • Spectral

    Modeling Synthesis (SMS) • SMS Implementation for Vehicle Sounds • Listening Test • Results and Analysis • Q & A Organization Monday, April 8, 13

  3. LISTEN Project Monday, April 8, 13

  4. LISTEN Project • Collaboration - KTH, Konstfack, SU, II. Monday,

    April 8, 13

  5. LISTEN Project • Collaboration - KTH, Konstfack, SU, II. •

    Software Demonstrator/Simulator Monday, April 8, 13

  6. LISTEN Project • Collaboration - KTH, Konstfack, SU, II. •

    Software Demonstrator/Simulator • Qualitative listening of urban soundscapes Monday, April 8, 13

  7. LISTEN Demonstrator Monday, April 8, 13

  8. LISTEN Demonstrator • Auralization of Urban Soundscapes Monday, April 8,

    13

  9. LISTEN Demonstrator • Auralization of Urban Soundscapes • Qualitative vs

    Quantitative Analysis Monday, April 8, 13

  10. LISTEN Demonstrator • Auralization of Urban Soundscapes • Qualitative vs

    Quantitative Analysis Monday, April 8, 13

  11. Auralization • Source + Propagation Monday, April 8, 13

  12. Auralization Monday, April 8, 13

  13. Auralization • Source Modeling Monday, April 8, 13

  14. Auralization • Source Modeling • Vehicles as sound sources. •

    Amplitude / Level • Frequency • Directivity • Non-Stationary sound. Monday, April 8, 13

  15. Auralization • Recordings - Large and Inflexible • 6 seconds

    - 853KB • For each velocity, vehicle type, ... Monday, April 8, 13

  16. Auralization Monday, April 8, 13

  17. Auralization • What if..? Monday, April 8, 13

  18. Auralization • What if..? • Extract the important ‘features’ Monday,

    April 8, 13

  19. Auralization • What if..? • Extract the important ‘features’ •

    Calculate waveforms ‘on the fly’ Monday, April 8, 13

  20. Auralization • What if..? • Extract the important ‘features’ •

    Calculate waveforms ‘on the fly’ • Parameters to change the sound Vehicle Model Speed Monday, April 8, 13

  21. Sound Synthesis Monday, April 8, 13

  22. Sound Synthesis • Electronic Generation of Audio Waveforms Monday, April

    8, 13

  23. Sound Synthesis • Electronic Generation of Audio Waveforms • Model

    to Waveform Monday, April 8, 13

  24. Sound Synthesis • Electronic Generation of Audio Waveforms • Model

    to Waveform • Computer Music Origin Monday, April 8, 13

  25. Sound Synthesis • Electronic Generation of Audio Waveforms • Model

    to Waveform • Computer Music Origin • Current Uses • Electronic Music Production, • Games • Interactive Media Monday, April 8, 13

  26. Sound Synthesis Monday, April 8, 13

  27. Sound Synthesis • Flexible Model Monday, April 8, 13

  28. Sound Synthesis • Flexible • Real Time Model Monday, April

    8, 13

  29. Sound Synthesis • Flexible • Real Time • Low Memory

    Requirement Model ~10KB vs 1MB Monday, April 8, 13

  30. Sound Synthesis Monday, April 8, 13

  31. Sound Synthesis • Types Monday, April 8, 13

  32. Sound Synthesis • Types • Additive Synthesis Monday, April 8,

    13

  33. Sound Synthesis • Types • Additive Synthesis • Frequency Modulation

    Synthesis Monday, April 8, 13

  34. Sound Synthesis • Types • Additive Synthesis • Frequency Modulation

    Synthesis • Physical Synthesis Monday, April 8, 13

  35. Sound Synthesis • Types • Additive Synthesis • Frequency Modulation

    Synthesis • Physical Synthesis • Spectral Modeling Synthesis Monday, April 8, 13

  36. Sound Synthesis Monday, April 8, 13

  37. Sound Synthesis • Vehicle Sounds • Tonal Component - Engine

    Noise • Noise Component - Tyre Noise Monday, April 8, 13

  38. Sound Synthesis • Vehicle Sounds • Tonal Component - Engine

    Noise • Noise Component - Tyre Noise • Non - Stationary Monday, April 8, 13

  39. Spectral Modeling Synthesis (SMS) Monday, April 8, 13

  40. Spectral Modeling Synthesis (SMS) • Xavier Serra and Julius Smith,

    1990 Monday, April 8, 13

  41. Spectral Modeling Synthesis (SMS) • Xavier Serra and Julius Smith,

    1990 • Modeling musical instruments Monday, April 8, 13

  42. Spectral Modeling Synthesis (SMS) • Xavier Serra and Julius Smith,

    1990 • Modeling musical instruments • Deterministic + Stochastic Decomposition Monday, April 8, 13

  43. Spectral Modeling Synthesis (SMS) • Xavier Serra and Julius Smith,

    1990 • Modeling musical instruments • Deterministic + Stochastic Decomposition • Tonal Sounds + Noise Monday, April 8, 13

  44. SMS Demo Monday, April 8, 13

  45. SMS Demo • Original • Tonal (Deterministic) • Noise (Stochastic)

    • ReSynthesis Monday, April 8, 13

  46. SMS Demo • Original • Tonal (Deterministic) • Noise (Stochastic)

    • ReSynthesis Monday, April 8, 13

  47. SMS Demo • Original • Tonal (Deterministic) • Noise (Stochastic)

    • ReSynthesis Monday, April 8, 13

  48. SMS Demo • Original • Tonal (Deterministic) • Noise (Stochastic)

    • ReSynthesis Monday, April 8, 13

  49. SMS Demo • Original • Tonal (Deterministic) • Noise (Stochastic)

    • ReSynthesis Monday, April 8, 13

  50. • LISTEN Project • Auralization • Sound Synthesis • Spectral

    Modeling Synthesis (SMS) • SMS Implementation for Vehicle Sounds • Listening Test • Results and Analysis • Q & A Organization Monday, April 8, 13

  51. SMS Algorithm STFT Analysis Original Audio Peak Detection Peak Tracking

    Peak Trajectories Tonal ReSynth Residual Audio Residual Parameterization Residual Parameters Noise ReSynth ReSynth Audio Monday, April 8, 13

  52. SMS Algorithm STFT Analysis Original Audio Peak Detection Peak Tracking

    Peak Trajectories Tonal ReSynth Residual Audio Residual Parameterization Residual Parameters Noise ReSynth ReSynth Audio Tonal Analysis Monday, April 8, 13

  53. SMS Algorithm STFT Analysis Original Audio Peak Detection Peak Tracking

    Peak Trajectories Tonal ReSynth Residual Audio Residual Parameterization Residual Parameters Noise ReSynth ReSynth Audio Tonal Analysis Noise Analysis Monday, April 8, 13

  54. SMS Algorithm STFT Analysis Original Audio Peak Detection Peak Tracking

    Peak Trajectories Tonal ReSynth Residual Audio Residual Parameterization Residual Parameters Noise ReSynth ReSynth Audio Tonal Analysis Noise Analysis Synthesis Monday, April 8, 13

  55. SMS Algorithm Tonal Analysis • Original Audio - Peak Trajectories

    STFT Analysis Original Audio Peak Detection Peak Tracking Peak Trajectories Monday, April 8, 13

  56. SMS Algorithm STFT Monday, April 8, 13

  57. SMS Algorithm STFT • Short Term Fourier Transform (STFT) Monday,

    April 8, 13

  58. SMS Algorithm STFT • Short Term Fourier Transform (STFT) •

    Fourier Analysis of Short Time Instances Monday, April 8, 13

  59. SMS Algorithm STFT • Short Term Fourier Transform (STFT) •

    Fourier Analysis of Short Time Instances • Sliding Window Analysis Monday, April 8, 13

  60. SMS Algorithm Peak Detection Monday, April 8, 13

  61. SMS Algorithm Peak Detection • Gradient Based Detection Monday, April

    8, 13

  62. SMS Algorithm Peak Detection • Gradient Based Detection Monday, April

    8, 13

  63. SMS Algorithm Peak Detection • Gradient Based Detection Monday, April

    8, 13

  64. SMS Algorithm Peak Detection • Gradient Based Detection • Interpolation

    due to DFT Sampling. Monday, April 8, 13

  65. SMS Algorithm Peak Detection • Gradient Based Detection • Interpolation

    due to DFT Sampling. Monday, April 8, 13

  66. SMS Algorithm Peak Tracking Monday, April 8, 13

  67. SMS Algorithm Peak Tracking • Frequency Guides - Peak Trajectories

    Monday, April 8, 13

  68. SMS Algorithm Peak Tracking • Frequency Guides - Peak Trajectories

    • Gradually Varying Sinusoids Monday, April 8, 13

  69. SMS Algorithm Peak Tracking • Frequency Guides - Peak Trajectories

    • Gradually Varying Sinusoids • Doppler Shift Monday, April 8, 13

  70. SMS Algorithm Peak Tracking • Frequency Guides - Peak Trajectories

    • Gradually Varying Sinusoids • Doppler Shift Monday, April 8, 13

  71. SMS Algorithm Peak Trajectories Monday, April 8, 13

  72. SMS Algorithm Noise Analysis • Original Audio & Tonal Synthesis

    - Residual Parameters Original Audio Peak Trajectories Tonal ReSynth Residual Audio Residual Parameterization Residual Parameters Monday, April 8, 13

  73. SMS Algorithm Residual Calculation Monday, April 8, 13

  74. SMS Algorithm Residual Calculation • Original - Tonal = Residual

    Monday, April 8, 13

  75. SMS Algorithm Residual Calculation • Original - Tonal = Residual

    • Spectrum Subtraction Monday, April 8, 13

  76. SMS Algorithm Residual Calculation • Original - Tonal = Residual

    • Spectrum Subtraction Monday, April 8, 13

  77. SMS Algorithm Residual Parameterization Monday, April 8, 13

  78. SMS Algorithm Residual Parameterization • Perceptual Model Monday, April 8,

    13

  79. SMS Algorithm Residual Parameterization • Perceptual Model • Noise Energy

    per Critical Band Monday, April 8, 13

  80. SMS Algorithm Residual Parameterization • Perceptual Model • Noise Energy

    per Critical Band Monday, April 8, 13

  81. SMS Algorithm - Residual Parameters Monday, April 8, 13

  82. SMS Algorithm Complete Model Monday, April 8, 13

  83. SMS Algorithm Complete Model • Peak Trajectories Monday, April 8,

    13

  84. SMS Algorithm Complete Model • Peak Trajectories Monday, April 8,

    13

  85. SMS Algorithm Complete Model • Peak Trajectories • Residual Parameters

    Monday, April 8, 13

  86. SMS Algorithm Complete Model • Peak Trajectories • Residual Parameters

    Monday, April 8, 13

  87. SMS Algorithm Complete Model • Peak Trajectories • Residual Parameters

    6 seconds - 80KB Monday, April 8, 13

  88. SMS Algorithm Synthesis • Peak Trajectories - Tonal Sound •

    Residual Parameters - Noise Peak Trajectories Tonal ReSynth Residual Parameters Noise ReSynth ReSynth Audio Monday, April 8, 13

  89. SMS Algorithm Tonal Synthesis Monday, April 8, 13

  90. SMS Algorithm Tonal Synthesis • Peak Trajectories - Sinusoid Waveform

    Monday, April 8, 13

  91. SMS Algorithm Tonal Synthesis • Peak Trajectories - Sinusoid Waveform

    • x[n] = A[n]*cos(w[n]) Monday, April 8, 13

  92. SMS Algorithm Tonal Synthesis • Peak Trajectories - Sinusoid Waveform

    • x[n] = A[n]*cos(w[n]) + + = = Monday, April 8, 13

  93. SMS Algorithm Residual Synthesis Monday, April 8, 13

  94. SMS Algorithm Residual Synthesis • Residual Parameters - Noise Monday,

    April 8, 13

  95. SMS Algorithm Residual Synthesis • Residual Parameters - Noise •

    Interpolation Monday, April 8, 13

  96. SMS Algorithm Residual Synthesis • Residual Parameters - Noise •

    Interpolation • Random Phase Assignment Monday, April 8, 13

  97. SMS Algorithm Residual Synthesis • Residual Parameters - Noise •

    Interpolation • Random Phase Assignment • Inverse Fourier Transform Monday, April 8, 13

  98. SMS Algorithm Residual Synthesis • Residual Parameters - Noise •

    Interpolation • Random Phase Assignment • Inverse Fourier Transform • Overlap and Add Synthesis Monday, April 8, 13

  99. SMS Algorithm Residual Synthesis • Overlap and Add Synthesis Monday,

    April 8, 13

  100. SMS Algorithm STFT Analysis Original Audio Peak Detection Peak Tracking

    Peak Trajectories Tonal ReSynth Residual Audio Residual Parameterization Residual Parameters Noise ReSynth ReSynth Audio Tonal Analysis Noise Analysis Synthesis Monday, April 8, 13

  101. • LISTEN Project • Auralization • Sound Synthesis • Spectral

    Modeling Synthesis (SMS) • SMS Implementation for Vehicle Sounds • Listening Test • Results and Analysis • Q & A Organization Monday, April 8, 13

  102. SMS for Vehicle Sounds • Multi Pass Analysis • Trajectory

    Energy Filtering • Residual parameter smoothing Monday, April 8, 13

  103. SMS for Vehicle Sounds • Multi Pass Analysis • Analysis

    Window - Frequency Dependence Monday, April 8, 13

  104. SMS for Vehicle Sounds • Multi Pass Analysis - Pyramid

    Analysis • Vary window length for each ‘Pass’ Monday, April 8, 13

  105. SMS for Vehicle Sounds Monday, April 8, 13

  106. SMS for Vehicle Sounds • Multi Pass Analysis Monday, April

    8, 13

  107. SMS for Vehicle Sounds • Multi Pass Analysis • Frequency

    band - Pass Monday, April 8, 13

  108. SMS for Vehicle Sounds • Multi Pass Analysis • Frequency

    band - Pass Pass 1 Pass 2 Pass 3 Monday, April 8, 13

  109. SMS for Vehicle Sounds • Trajectory Energy Filtering • Too

    many trajectories - Filter by ‘Importance’ Monday, April 8, 13

  110. SMS for Vehicle Sounds Monday, April 8, 13

  111. SMS for Vehicle Sounds • Fluctuating Residual Parameter Monday, April

    8, 13

  112. SMS for Vehicle Sounds • Fluctuating Residual Parameter • Smoothen

    Monday, April 8, 13

  113. SMS for Vehicle Sounds • Fluctuating Residual Parameter • Smoothen

    Monday, April 8, 13

  114. SMS on Vehicle Sounds • Bus (39km/h) • Original •

    Tonal • Noise • ReSynth Monday, April 8, 13

  115. SMS on Vehicle Sounds • Bus (39km/h) • Original •

    Tonal • Noise • ReSynth Monday, April 8, 13

  116. SMS on Vehicle Sounds • Bus (39km/h) • Original •

    Tonal • Noise • ReSynth Monday, April 8, 13

  117. SMS on Vehicle Sounds • Bus (39km/h) • Original •

    Tonal • Noise • ReSynth Monday, April 8, 13

  118. SMS on Vehicle Sounds • Bus (39km/h) • Original •

    Tonal • Noise • ReSynth Monday, April 8, 13

  119. SMS on Vehicle Sounds • Open Astra (45 km/h) •

    Volvo V70 (50 km/h) • IV Medium-Heavy (40km/h) • Bus (86km/h) Monday, April 8, 13

  120. SMS on Vehicle Sounds • Open Astra (45 km/h) •

    Volvo V70 (50 km/h) • IV Medium-Heavy (40km/h) • Bus (86km/h) Monday, April 8, 13

  121. SMS on Vehicle Sounds • Open Astra (45 km/h) •

    Volvo V70 (50 km/h) • IV Medium-Heavy (40km/h) • Bus (86km/h) Monday, April 8, 13

  122. SMS on Vehicle Sounds • Open Astra (45 km/h) •

    Volvo V70 (50 km/h) • IV Medium-Heavy (40km/h) • Bus (86km/h) Monday, April 8, 13

  123. SMS on Vehicle Sounds • Open Astra (45 km/h) •

    Volvo V70 (50 km/h) • IV Medium-Heavy (40km/h) • Bus (86km/h) Monday, April 8, 13

  124. • LISTEN Project • Auralization • Sound Synthesis • Spectral

    Modeling Synthesis (SMS) • SMS Implementation for Vehicle Sounds • Listening Test • Results and Analysis • Q & A Organization Monday, April 8, 13

  125. Listening Test Monday, April 8, 13

  126. Listening Test • Perceptual Similarity of Sounds Monday, April 8,

    13

  127. Listening Test • Perceptual Similarity of Sounds • 18 Sounds

    - 72 Combination Monday, April 8, 13

  128. Listening Test • Perceptual Similarity of Sounds • 18 Sounds

    - 72 Combination • Paired Sound Test Monday, April 8, 13

  129. Listening Test • Perceptual Similarity of Sounds • 18 Sounds

    - 72 Combination • Paired Sound Test • 3 Questions • Realism • Annoyance • Speed Monday, April 8, 13

  130. Listening Test • Perceptual Similarity of Sounds • 18 Sounds

    - 72 Combination • Paired Sound Test • 3 Questions • Realism • Annoyance • Speed • 27 Participants Monday, April 8, 13

  131. Listening Test Results Monday, April 8, 13

  132. Listening Test Results • Significant Difference Match Pair t-test -

    90% Monday, April 8, 13

  133. Spectral Analysis Monday, April 8, 13

  134. • LISTEN Project • Auralization • Sound Synthesis • Spectral

    Modeling Synthesis (SMS) • SMS Implementation for Vehicle Sounds • Listening Test • Results and Analysis • Q & A Organization Monday, April 8, 13

  135. Analysis - Applications Monday, April 8, 13

  136. Analysis - Applications • Applying propagation to Sound Model Monday,

    April 8, 13

  137. Analysis - Applications • Applying propagation to Sound Model •

    Direct Frequency-Domain Changes Monday, April 8, 13

  138. Analysis - Applications • Applying propagation to Sound Model •

    Direct Frequency-Domain Changes • Less Memory Requirement Monday, April 8, 13

  139. Analysis - Applications • Applying propagation to Sound Model •

    Direct Frequency-Domain Changes • Less Memory Requirement • Interaction Real-Time Synthesis Monday, April 8, 13

  140. Analysis - Applications Monday, April 8, 13

  141. Analysis - Applications • Binaural calculation of propagation Monday, April

    8, 13

  142. Analysis - Applications • Binaural calculation of propagation • Multiple

    source height - issue Monday, April 8, 13

  143. Analysis - Applications • Binaural calculation of propagation • Multiple

    source height - issue • SMS can ‘separate’ engine and tyre Monday, April 8, 13

  144. Conclusion Monday, April 8, 13

  145. Conclusion • SMS works great for vehicle sounds. Monday, April

    8, 13

  146. Conclusion • SMS works great for vehicle sounds. • Less

    memory and more flexible Monday, April 8, 13

  147. Conclusion • SMS works great for vehicle sounds. • Less

    memory and more flexible • Fine tuning for effectively and efficiency. Monday, April 8, 13

  148. Conclusion • SMS works great for vehicle sounds. • Less

    memory and more flexible • Fine tuning for effectively and efficiency. • Can produce perceptually similar sounds. Monday, April 8, 13

  149. Future Work Monday, April 8, 13

  150. Future Work • Automated Parameter Estimation Monday, April 8, 13

  151. Future Work • Automated Parameter Estimation • Improved Noise Model

    Monday, April 8, 13

  152. Future Work • Automated Parameter Estimation • Improved Noise Model

    • Transients Modeling Monday, April 8, 13

  153. Future Work • Automated Parameter Estimation • Improved Noise Model

    • Transients Modeling • Support for Binaural Propagation calculation Monday, April 8, 13

  154. Thanks! Monday, April 8, 13