Musical Creativity in the Browser

CascadiaJS 2020 lightning talk

Transcript

1. Musical Creativity in the Browser Scott Ammon Senior Software Engineer @ Slalom Build
2. Why music and the web? +
3. Musical Instrument Digital Interface How to connect How to communicate
4. Web MIDI API
5. Web MIDI API message { "type": "midimessage", "timeStamp": "1369.3500000517815", "data": [144, 62, 95], "target": { "connection": "open", "id": "input-0", "name": "Arturia MiniLab mkII", "state": "connected" } }
6. { "type": "midimessage", "timeStamp": "1369.3500000517815", "data": [144, 62, 95], "target": { "connection": "open", "id": "input-0", "name": "Arturia MiniLab mkII", "state": "connected" } } event that occurred Web MIDI API message
7. { "type": "midimessage", "timeStamp": "1369.3500000517815", "data": [144, 62, 95], "target": { "connection": "open", "id": "input-0", "name": "Arturia MiniLab mkII", "state": "connected" } } note-on (the event type) Web MIDI API message
8. { "type": "midimessage", "timeStamp": "1369.3500000517815", "data": [144, 62, 95], "target": { "connection": "open", "id": "input-0", "name": "Arturia MiniLab mkII", "state": "connected" } } note that was pressed (middle C in this case) Web MIDI API message
9. { "type": "midimessage", "timeStamp": "1369.3500000517815", "data": [144, 62, 95], "target": { "connection": "open", "id": "input-0", "name": "Arturia MiniLab mkII", "state": "connected" } } velocity (how fast the key was struck) Web MIDI API message
10. Part 1 A keyboard clone
11. Pitch Modulation Keys
12. Keyboard Clone const MidiDataContext = React.createContext({ deviceName: null, keyData: [], pitch: 64, modulation: 0, errors: null });
13. Keyboard Clone const MidiDataContext = React.createContext({ deviceName: null, keyData: [], pitch: 64, modulation: 0, errors: null });
14. Keyboard demo will be cut in here
15. Part 2 everyone gets a keyboard!
16. MIDI over WebSockets
17. WebSockets demo will be cut in here
18. Part 3 Drum visualizer
19. Drums to SVGs Web MIDI API
20. export class HiHat extends Instrument { makeShape = (velocity) => { } } Making an instrument svg
21. export class HiHat extends Instrument { makeShape = (velocity) => { const radius = velocity * 5; this.rotationRadians += .1; const xPos = Math.cos(this.rotationRadians) * radius + 400; const yPos = Math.sin(this.rotationRadians) * radius + 400; //code removed for clarity } } Making an instrument svg
22. export class HiHat extends Instrument { makeShape = (velocity) => { // angle calculation removed for clarity return ( <div className="hiHatFixed" key={Math.random()}> <svg className="hiHat" height="100%" width="100%"> <line x1="50%" y1="50%" x2={xPos} y2={yPos}/> </svg> </div> ); } } Making an instrument svg
23. Warning: This demo contains a visualization with flashing shapes and light
24. Drum demo will be cut in here
25. recap MIDI - the communication protocol Web MIDI API – implements MIDI protocol in the browser Go forth and create!
26. Thank You - CascadiaJS Team - W3C Audio Working Group (creators of the Web MIDI API) - Anyone who has lived near me and been subjected to loud noises Web MIDI API spec: https://www.w3.org/TR/webmidi/ Repos used in this presentation: https://github.com/scott-ammon