Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Beats on the web - A journey into performance

Ritesh Kumar
September 27, 2019

Beats on the web - A journey into performance

React is a brilliant JavaScript library for building simple and complex user interfaces. It is quite performant when used correctly but there are times when the library alone is not enough to achieve what we want. This talk focuses on using the power of React and the platform together to build a fast and performant music application. It will involve using the different threads, web features and React in sync. I will be talking about the journey of converting an idea into a product and different decisions I had to take both in regards to design and implementation.

Ritesh Kumar

September 27, 2019
Tweet

More Decks by Ritesh Kumar

Other Decks in Programming

Transcript

  1. Beats on the web
    Ritesh Kumar
    Front-end, pspdfkit
    @ritz078

    View full-size slide

  2. Music can’t be delayed

    View full-size slide

  3. Percussions / Beats
    Instruments

    View full-size slide

  4. • Loading and Parsing MIDI files
    • Scheduling notes
    • Visualization sync

    View full-size slide

  5. JavaScript is single threaded

    View full-size slide

  6. Use worker in main thread

    View full-size slide

  7. Load File & parse

    View full-size slide

  8. Instruments
    Percussion

    View full-size slide

  9. Scheduling Notes

    View full-size slide

  10. The JS timers (setTimeout, setInterval) call can be
    delayed if the thread is blocked.

    View full-size slide

  11. JS Clock has a maximum precision of 1ms

    View full-size slide

  12. 44.1 KHz
    Analog audio is recorded by sampling it 44.1
    times per ms
    Sampling Frequency

    View full-size slide

  13. In case of inactive tabs, JS Clocks are throttled in background

    View full-size slide

  14. performance.now()

    View full-size slide

  15. performance.now()
    Different value for different threads

    View full-size slide

  16. AudioContext

    View full-size slide

  17. Scheduler
    Events Timeline
    1 2 3 4 5 6 7 8 9
    1 2 3 4 5 6 7 8 9
    Change Tempo

    View full-size slide

  18. Scheduler
    Change Tempo
    Events Timeline
    1 2 3 4 5 6 7 8 9
    1 2 3 4 5 6 7 8 9

    View full-size slide

  19. Scheduler
    Events Timeline
    Interval 1
    Interval 2
    Interval 3
    Interval 4
    1 2 3 4 5 6 7 8 9

    View full-size slide

  20. Scheduler
    Events Timeline
    Interval 1
    Interval 2
    Interval 3
    1 2 3 4 5 6 7 8 9

    View full-size slide

  21. Scheduler
    Events Timeline
    Interval 1
    Interval 2
    Interval 3
    Interval 4
    1 2 3 4 5 6 7 8 9

    View full-size slide

  22. Scheduler
    Events Timeline
    Interval 1
    Interval 2
    Interval 3
    1 2 3 4 5 6 7 8 9
    Lookahead time

    View full-size slide

  23. Visualisation

    View full-size slide

  24. (Same old JS Timer problems - precision)
    +
    Relayout + painting + composition

    View full-size slide

  25. Canvas on Main thread

    View full-size slide

  26. Better than other solutions but the delay was still there.
    @ritz078

    View full-size slide

  27. Offscreen Canvas

    View full-size slide

  28. transferControlToOffscreen

    View full-size slide

  29. The main thread has transferred this to worker and has no
    control over it anymore.

    View full-size slide

  30. In worker, we get
    the canvas.

    View full-size slide

  31. Offscreen canvas

    View full-size slide

  32. Other optimisations…

    View full-size slide

  33. • Tried to use useLayoutEffect and useEffect effectively.
    • Avoided too many nested children.
    • Passed props to far off children using context.
    • Didn’t use any state management library.

    View full-size slide

  34. Connect to a MIDI Device
    • Read Mode
    • Write Mode

    View full-size slide


  35. https://github.com/ritz078/raaga

    View full-size slide

  36. Roadmap
    • Better support for mobile devices
    • Support MusicXML
    • Display Sheet music
    • Record music and export MIDI
    • Browser Compatibility

    View full-size slide

  37. Acknowledgements
    The icons have been designed by Freepik from Flaticon.
    The MIDI file (royalty free) belongs to Partners In Rhyme
    carbon.now.sh for code snipped screenshots

    View full-size slide

  38. Thank You
    ritz078

    View full-size slide