The Future of Audio Processing in the Web

WebAssembly opens a door for professional-grade audio software for the Web, allowing to compile plugins written in C, C++, or Faust. Faust is not only the tragic play by Goethe but a programming language for digital signal processing (DSP). During the talk, I compile Faust plugins right in online IDE into WebAssembly and load them as Web Audio Modules into a web-based guitar processing app.

Transcript

1. IN THE WEB T H E F U T U R E O F A U D I O P R O C E S S I N G
2. VITALII BOBROV • Lead SW Engineer @ EPAM Poland • Web Audio experimentator • @AngularWroclaw org and tech speaker @bobrov1989 https://bobrov.dev
3. I H AV E A P R O B L E M — I L O V E P E D A L S http://js-rocks.web.app
4. I H AV E A P R O B L E M — I L O V E P E D A L S
5. I H AV E A P R O B L E M — I L O V E P E D A L S
6. BUT YOU HEAR IT Y O U M I G H T N O T K N O W I T
7. ARE USED IN RECORDS B O T H A N A L O G A N D D I G I TA L
8. PROFESSIONALS W E B A U D I O F O R
9. W E B A U D I O H I S T O RY • First draft in 2010 • Initial implementation in 2012 • Includes limited number of of nodes • Doesn’t work with raw audio data https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_API
10. I T ’ S S O Y O U N G
11. T E C H N O L O G I E S U S E D F O R D S P • C/C++ • JUCE • iPlug2 • LV2 • Assembly
12. ARE LATENCY AND LOW LEVEL T H E M A I N S T O P P E R S
13. AUDIO WORKLET L AT E N C Y S O LV E D B Y
14. MAIN THREAD https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_API/Using_AudioWorklet
15. AUDIO WORKLET THREAD
16. H O W I T S O LV E S T H E P R O B L E M • Works with raw audio buffers • Executes in a separate thread • Acts as any other Web Audio node
17. N O L AT E N C Y P R O C E S S I N G
18. BROWSERS SUPPORT https://caniuse.com/mdn-api_audioworklet
19. GIVES A FULL FREEDOM A U D I O W O R K L E T
20. WEB ASSEMBLY P E R F O R M A N C E S O LV E D B Y
21. T E C H N O L O G I E S F O R W E B D S P • C/C++ with Emscripten • Rust • DSL, ex. Faust
22. WEB AUDIO FOR A PRO USAGE W E B A S S E M B LY E N A B L E S
23. PORTABILITY T H E M I S S I N G P I E C E
24. D A W P L U G I N S TA N D A R D S • VST • Apple Audio Units • AVID AAX
25. WEB AUDIO PLUGINS P O RTA B L E
26. WAP INTEGRATION https://github.com/micbuffa/WebAudioPlugins
27. WA P F E AT U R E S • Accessible by URL • Use WA and Audio Worklets for integration • Web Components for UI
28. FAUST FA S T WAY T O C R E AT E A WA P
29. FA U S T ( F U N C T I O N A L A U D I O S T R E A M ) • DSL to be compiled to C++, LLVM bit code, WA • Huge helpers library • Produces DSP and UI https://faust.grame.fr
30. FAUST IDE Q U I C K A N D E A S Y D S P C R E AT I O N W I T H https://faustide.grame.fr
31. D E M O T I M E
32. THANK YOU W E B I S A G R E AT P L A C E F O R T H E M U S I C @bobrov1989 https://bobrov.dev https://speakerdeck.com/bobrov1989/the-future-of-audio-processing-in-the-web