Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The Future of Audio Processing in the Web

The Future of Audio Processing in the Web

WebAssembly opens a door for professional-grade audio software for the Web, allowing to compile plugins written in C, C++, or Faust. Faust is not only the tragic play by Goethe but a programming language for digital signal processing (DSP). During the talk, I compile Faust plugins right in online IDE into WebAssembly and load them as Web Audio Modules into a web-based guitar processing app.

4f0880beebecf17d29eb709246055e14?s=128

Vitalii Bobrov

September 18, 2020
Tweet

Transcript

  1. IN THE WEB T H E F U T U

    R E O F A U D I O P R O C E S S I N G
  2. VITALII BOBROV • Lead SW Engineer @ EPAM Poland •

    Web Audio experimentator • @AngularWroclaw org and tech speaker @bobrov1989 https://bobrov.dev
  3. I H AV E A P R O B L

    E M — I L O V E P E D A L S http://js-rocks.web.app
  4. I H AV E A P R O B L

    E M — I L O V E P E D A L S
  5. I H AV E A P R O B L

    E M — I L O V E P E D A L S
  6. BUT YOU HEAR IT Y O U M I G

    H T N O T K N O W I T
  7. ARE USED IN RECORDS B O T H A N

    A L O G A N D D I G I TA L
  8. PROFESSIONALS W E B A U D I O F

    O R
  9. W E B A U D I O H I

    S T O RY • First draft in 2010 • Initial implementation in 2012 • Includes limited number of of nodes • Doesn’t work with raw audio data https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_API
  10. I T ’ S S O Y O U N

    G
  11. T E C H N O L O G I

    E S U S E D F O R D S P • C/C++ • JUCE • iPlug2 • LV2 • Assembly
  12. ARE LATENCY AND LOW LEVEL T H E M A

    I N S T O P P E R S
  13. AUDIO WORKLET L AT E N C Y S O

    LV E D B Y
  14. MAIN THREAD https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_API/Using_AudioWorklet

  15. AUDIO WORKLET THREAD

  16. H O W I T S O LV E S

    T H E P R O B L E M • Works with raw audio buffers • Executes in a separate thread • Acts as any other Web Audio node
  17. N O L AT E N C Y P R

    O C E S S I N G
  18. BROWSERS SUPPORT https://caniuse.com/mdn-api_audioworklet

  19. GIVES A FULL FREEDOM A U D I O W

    O R K L E T
  20. WEB ASSEMBLY P E R F O R M A

    N C E S O LV E D B Y
  21. T E C H N O L O G I

    E S F O R W E B D S P • C/C++ with Emscripten • Rust • DSL, ex. Faust
  22. WEB AUDIO FOR A PRO USAGE W E B A

    S S E M B LY E N A B L E S
  23. PORTABILITY T H E M I S S I N

    G P I E C E
  24. D A W P L U G I N S

    TA N D A R D S • VST • Apple Audio Units • AVID AAX
  25. WEB AUDIO PLUGINS P O RTA B L E

  26. WAP INTEGRATION https://github.com/micbuffa/WebAudioPlugins

  27. WA P F E AT U R E S •

    Accessible by URL • Use WA and Audio Worklets for integration • Web Components for UI
  28. FAUST FA S T WAY T O C R E

    AT E A WA P
  29. FA U S T ( F U N C T

    I O N A L A U D I O S T R E A M ) • DSL to be compiled to C++, LLVM bit code, WA • Huge helpers library • Produces DSP and UI https://faust.grame.fr
  30. FAUST IDE Q U I C K A N D

    E A S Y D S P C R E AT I O N W I T H https://faustide.grame.fr
  31. D E M O T I M E

  32. THANK YOU W E B I S A G R

    E AT P L A C E F O R T H E M U S I C @bobrov1989 https://bobrov.dev https://speakerdeck.com/bobrov1989/the-future-of-audio-processing-in-the-web