Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Beats on the web - A journey into performance

55c3671e959e903a7880eaf0cf61f3e4?s=47 Ritesh Kumar
September 27, 2019

Beats on the web - A journey into performance

React is a brilliant JavaScript library for building simple and complex user interfaces. It is quite performant when used correctly but there are times when the library alone is not enough to achieve what we want. This talk focuses on using the power of React and the platform together to build a fast and performant music application. It will involve using the different threads, web features and React in sync. I will be talking about the journey of converting an idea into a product and different decisions I had to take both in regards to design and implementation.

55c3671e959e903a7880eaf0cf61f3e4?s=128

Ritesh Kumar

September 27, 2019
Tweet

Transcript

  1. Beats on the web Ritesh Kumar Front-end, pspdfkit @ritz078

  2. @ritz078

  3. Music can’t be delayed

  4. None
  5. MIDI

  6. Percussions / Beats Instruments

  7. None
  8. • Loading and Parsing MIDI files • Scheduling notes •

    Visualization sync
  9. JavaScript is single threaded

  10. None
  11. Use worker in main thread

  12. Load File & parse

  13. Instruments Percussion

  14. Scheduling Notes

  15. The JS timers (setTimeout, setInterval) call can be delayed if

    the thread is blocked.
  16. JS Clock has a maximum precision of 1ms

  17. 44.1 KHz Analog audio is recorded by sampling it 44.1

    times per ms Sampling Frequency
  18. In case of inactive tabs, JS Clocks are throttled in

    background
  19. performance.now()

  20. performance.now() Different value for different threads

  21. AudioContext

  22. Scheduler Events Timeline 1 2 3 4 5 6 7

    8 9 1 2 3 4 5 6 7 8 9 Change Tempo
  23. Scheduler Change Tempo Events Timeline 1 2 3 4 5

    6 7 8 9 1 2 3 4 5 6 7 8 9
  24. Scheduler Events Timeline Interval 1 Interval 2 Interval 3 Interval

    4 1 2 3 4 5 6 7 8 9
  25. Scheduler Events Timeline Interval 1 Interval 2 Interval 3 1

    2 3 4 5 6 7 8 9
  26. Scheduler Events Timeline Interval 1 Interval 2 Interval 3 Interval

    4 1 2 3 4 5 6 7 8 9
  27. Scheduler Events Timeline Interval 1 Interval 2 Interval 3 1

    2 3 4 5 6 7 8 9 Lookahead time
  28. Visualisation

  29. Visualizer

  30. (Same old JS Timer problems - precision) + Relayout +

    painting + composition
  31. SVG / Div

  32. Canvas

  33. Canvas on Main thread

  34. Better than other solutions but the delay was still there.

    @ritz078
  35. Offscreen Canvas

  36. transferControlToOffscreen

  37. The main thread has transferred this to worker and has

    no control over it anymore.
  38. In worker, we get the canvas.

  39. None
  40. T

  41. Offscreen canvas

  42. Other optimisations…

  43. • Tried to use useLayoutEffect and useEffect effectively. • Avoided

    too many nested children. • Passed props to far off children using context. • Didn’t use any state management library.
  44. Paint

  45. But…

  46. Connect to a MIDI Device • Read Mode • Write

    Mode
  47.  https://github.com/ritz078/raaga

  48. Roadmap • Better support for mobile devices • Support MusicXML

    • Display Sheet music • Record music and export MIDI • Browser Compatibility
  49. Acknowledgements The icons have been designed by Freepik from Flaticon.

    The MIDI file (royalty free) belongs to Partners In Rhyme carbon.now.sh for code snipped screenshots
  50. Thank You ritz078