Beats on the web - A journey into performance

React is a brilliant JavaScript library for building simple and complex user interfaces. It is quite performant when used correctly but there are times when the library alone is not enough to achieve what we want. This talk focuses on using the power of React and the platform together to build a fast and performant music application. It will involve using the different threads, web features and React in sync. I will be talking about the journey of converting an idea into a product and different decisions I had to take both in regards to design and implementation.

Transcript

1. Beats on the web Ritesh Kumar Front-end, pspdfkit @ritz078
2. @ritz078
3. Music can’t be delayed
5. MIDI
6. Percussions / Beats Instruments
8. • Loading and Parsing MIDI files • Scheduling notes • Visualization sync
9. JavaScript is single threaded
11. Use worker in main thread
12. Load File & parse
13. Instruments Percussion
14. Scheduling Notes
15. The JS timers (setTimeout, setInterval) call can be delayed if the thread is blocked.
16. JS Clock has a maximum precision of 1ms
17. 44.1 KHz Analog audio is recorded by sampling it 44.1 times per ms Sampling Frequency
18. In case of inactive tabs, JS Clocks are throttled in background
19. performance.now()
20. performance.now() Different value for different threads
21. AudioContext
22. Scheduler Events Timeline 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 Change Tempo
23. Scheduler Change Tempo Events Timeline 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9
24. Scheduler Events Timeline Interval 1 Interval 2 Interval 3 Interval 4 1 2 3 4 5 6 7 8 9
25. Scheduler Events Timeline Interval 1 Interval 2 Interval 3 1 2 3 4 5 6 7 8 9
26. Scheduler Events Timeline Interval 1 Interval 2 Interval 3 Interval 4 1 2 3 4 5 6 7 8 9
27. Scheduler Events Timeline Interval 1 Interval 2 Interval 3 1 2 3 4 5 6 7 8 9 Lookahead time
28. Visualisation
29. Visualizer
30. (Same old JS Timer problems - precision) + Relayout + painting + composition
31. SVG / Div
32. Canvas
33. Canvas on Main thread
34. Better than other solutions but the delay was still there. @ritz078
35. Offscreen Canvas
36. transferControlToOffscreen
37. The main thread has transferred this to worker and has no control over it anymore.
38. In worker, we get the canvas.
40. T
41. Offscreen canvas
42. Other optimisations…
43. • Tried to use useLayoutEffect and useEffect effectively. • Avoided too many nested children. • Passed props to far off children using context. • Didn’t use any state management library.
44. Paint
45. But…
46. Connect to a MIDI Device • Read Mode • Write Mode
47. 
     https://github.com/ritz078/raaga
48. Roadmap • Better support for mobile devices • Support MusicXML • Display Sheet music • Record music and export MIDI • Browser Compatibility
49. Acknowledgements The icons have been designed by Freepik from Flaticon. The MIDI file (royalty free) belongs to Partners In Rhyme carbon.now.sh for code snipped screenshots
50. Thank You ritz078