$30 off During Our Annual Pro Sale. View Details »

rMIXr: how we learned to stop worrying and love the graph

rMIXr: how we learned to stop worrying and love the graph

In this talk we present a case study in the use of the web audio APIs. Specifically, our use of them for the creation of a rapidly developed
prototype application. The app, called rMIXr (http://rmixr.com), is a simple digital audio workstation (DAW) for fan remix contests. We created rMIXr in 48 hours at the Midem Hack Day in June 2015. We’ll give a brief demo of the app and show multi-channel sync. We'll also show various effects as well as cutting/time-slicing.
During the talk we will go through our solutions to these issues, showing by example a number of practical ways to get things done with the Web Audio APIs. We will walk through our abstractions and object models for building a fully functional DAW in the browser. These abstractions were necessary because we included a number of effects from third party libraries. Some of these effects libraries do not comply with the “source and sink” model that the web audio API predicates. Our abstractions gave us a more flexible graph structure that was critical to making this application work. While not perfect, it was certainly excellent for the rapid prototype scenario of the hackday. We’ll also talk about some potential improvements to our abstractions. We will then show how to use localstorage to communicate across browser windows and how this approach can be harnessed to quickly create flexible UI elements with multiple windows.
Slides as given at the 2016 Web Audio Conference in Atlanta, GA. Full abstract at https://smartech.gatech.edu/handle/1853/54668

Ben Fields

April 05, 2016
Tweet

More Decks by Ben Fields

Other Decks in Technology

Transcript

  1. rMIXr: how we learned to stop worrying and love the

    graph
  2. Ben Fields @alsothings Sam Phippen @samphippen

  3. this talk represents our experience using the web audio APIs

    to go from nothing to a reasonably complete prototype in 48 hours about a year ago. actual API usage may vary.
  4. ⚠⚠⚠⚠⚠⚠⚠⚠⚠⚠ ACTUAL API USAGE MAY VARY ⚠⚠⚠⚠⚠⚠⚠⚠⚠⚠

  5. Midem Hack Day

  6. [[[[midem branding]]]]

  7. None
  8. DJ Spotify festivalBag

  9. rMIXr: white-labelled DAW in the browser

  10. None
  11. None
  12. None
  13. Effects in rMIXr

  14. `

  15. None
  16. Source Sink

  17. Source Sink .connect

  18. Source Sink .connect

  19. Source Sink Fx/ Analysis

  20. Source Sink Fx/ Analysis .connect

  21. Source Sink Fx/ Analysis .connect .connect

  22. None
  23. None
  24. None
  25. None
  26. OMG NO

  27. Source Sink Fx/ Analysis .connect .connect

  28. Converb Converb .input Source Sink .connect .connect

  29. None
  30. None
  31. None
  32. None
  33. None
  34. None
  35. None
  36. How to do UI for effects?

  37. None
  38. None
  39. None
  40. None
  41. None
  42. None
  43. Icon font didn’t load lol

  44. None
  45. None
  46. None
  47. None
  48. None
  49. Ben will explain how this works

  50. The local storage API is totally a message bus

  51. Last write wins is totally a good consistency model

  52. None
  53. server side audio rendering is for suckers

  54. thus: need to serialise state and associate it with page

    location
  55. None
  56. every time you change a setting,

  57. every time you change a setting, the new state is

    serialised to the url
  58. generalise to *all* UI elements

  59. None
  60. None
  61. None
  62. None
  63. a channel’s state channel strip’s state

  64. None
  65. hash decoding all the way down

  66. http://rmixr.com/queen/#{"channels":[{"effects":[],"loopPoints": [],"notmuted":true},{"effects":[{"name":"threebandEQ","params": {"bassgain":"21","midgain":"-24","treblegain":"16"}}, {"name":"chorus","params":{"rate":2.41,"feedback":0.2,"delay": 0.05}}],"loopPoints":[]},{"effects":[],"loopPoints": [],"notmuted":true},{"effects":[{"name":"converb","params": {"dry":0.5,"wet":0.59}}],"loopPoints":[],"notmuted":false}, {"effects":[{"name":"stereoPan","params": {"pan":-0.1}}],"loopPoints":[],"notmuted":true},{"effects": [{"name":"stereoPan","params":{"pan":0.11}}],"loopPoints":

    [],"notmuted":false}]}
  67. the fragment id *is* the application state engine

  68. at least tweeting and entry submission is easy

  69. None
  70. thanks twitter.

  71. None
  72. at least tweeting and entry submission is easy

  73. at least the back button works

  74. {this is where we attempt a demo}

  75. so:

  76. unifying abstractions

  77. local storage can be a messaging queue

  78. even single page web apps can be RESTful

  79. Let’s have some questions a!/samphippen !/alsothings