The Lost Art of MIDI – Bringing <bgsound> Back to the Web

Slide 1

Slide 1 text

The Lost Art of MIDI Bringing Back to the Web Image Credit: hexeosis.tumblr.com

Slide 2

Slide 2 text

No content

Slide 3

Slide 3 text

Source: cameronsworld.net

Slide 4

Slide 4 text

Source: toastytech.com/evil/

Slide 5

Slide 5 text

Source: spacejam.com

Slide 6

Slide 6 text

Source: fogcam.org

Slide 7

Slide 7 text

Source: zombo.com

Slide 8

Slide 8 text

Slide 9

Slide 9 text

Image Credit: Legend of Zelda: Ocarina of Time 3D

Slide 10

Slide 10 text

No content

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

What even is a MIDI file? Image Credit: hexeosis.tumblr.com

Slide 13

Slide 13 text

Musical Instrument Digital Interface

Slide 14

Slide 14 text

Prophet-600

Slide 15

Slide 15 text

• 1971 – FTP • 1974 – TCP • 1982 – MIDI • 1991 – HTTP • 1995 – SSH • 2001 – BitTorrent • 2009 – Bitcoin • 2015 – HTTP/2

Slide 16

Slide 16 text

MIDI Messages Command Meaning 0x80 Note-off 0x90 Note-on ... ...

Slide 17

Slide 17 text

90 3C 40 -----------------> time • 90 = Note on • 3C = Which key? • 40 = How hard was it pressed?

Slide 18

Slide 18 text

General MIDI • 128 notes (~10 octaves) • 16 channels • 128 programs (instrument sounds)

Slide 19

Slide 19 text

1-8 Piano 9-16 Chromatic Percussion 17-24 Organ 25-32 Guitar 33-40 Bass 41-48 Strings 49-56 Ensemble 57-64 Brass 65-72 Reed 73-80 Pipe 81-88 Synth Lead 89-96 Synth Pad 97-104 Synth Effects 105-112 Ethnic 113-120 Percussive 121-128 Sound Effects

Slide 20

Slide 20 text

Pianos 1. Acoustic Grand Piano 2. Bright Acoustic Piano 3. Electric Grand Piano 4. Honky-tonk Piano 5. Electric Piano 1 6. Electric Piano 2 7. Harpsichord 8. Clavi

Slide 21

Slide 21 text

No content

Slide 22

Slide 22 text

No content

Slide 23

Slide 23 text

Where else is MIDI used? Image Credit: hexeosis.tumblr.com

Slide 24

Slide 24 text

No content

Slide 25

Slide 25 text

No content

Slide 26

Slide 26 text

No content

Slide 27

Slide 27 text

No content

Slide 28

Slide 28 text

No content

Slide 29

Slide 29 text

No content

Slide 30

Slide 30 text

No content

Slide 31

Slide 31 text

No content

Slide 32

Slide 32 text

No content

Slide 33

Slide 33 text

WebAssembly Image Credit: hexeosis.tumblr.com

Slide 34

Slide 34 text

No content

Slide 35

Slide 35 text

static void load_instrument(MidSong *song, ...) { ... if (song->ifp == NULL) { DEBUG_MSG("Instrument `%s' can't be found.\n", name); // Added for JavaScript port song->load_requests[song->load_request_count] = strdup(name); song->load_request_count += 1; ... }

Slide 36

Slide 36 text

// Added for JavaScript port extern int mid_get_load_request_count(MidSong *song) { return song->load_request_count; } extern char *mid_get_load_request(MidSong *song, int index) { return song->load_requests[index]; }

Slide 37

Slide 37 text

File system // Make directory lib.FS.mkdir('new-directory') // Write ﬁles lib.FS.writeFile('myﬁle.txt', buffer)

Slide 38

Slide 38 text

Pointers // Allocate memory const ptr = lib._malloc(byteLength) // Free memory lib._free(ptr) // Convert a `char*` to a JavaScript string const str = lib.Pointer_stringify(ptr)

Slide 39

Slide 39 text

No content

Slide 40

Slide 40 text

function loadSong (buffer) { const bufferPtr = lib._malloc(buffer.byteLength) lib.HEAPU8.set(buffer, bufferPtr) const iStreamPtr = lib._mid_istream_open_mem(bufferPtr, buffer.byteLength) const songPtr = lib._mid_song_load(iStreamPtr) return songPtr }

Slide 41

Slide 41 text

let songPtr = loadSong(buffer) const instruments = getMissingInstruments(songPtr) if (instruments.length > 0) { await Promise.all( instruments.map(instrument => fetchInstrument(instrument)) ) lib._mid_song_free(songPtr) songPtr = loadSong(buffer) } lib._mid_song_start(songPtr)

Slide 42

Slide 42 text

function getMissingInstruments (songPtr) { const missingCount = lib._mid_get_load_request_count(songPtr) const missingInstruments = [] for (let i = 0; i < missingCount; i++) { const instrumentPtr = lib._mid_get_load_request(songPtr, i) const instrument = lib.Pointer_stringify(instrumentPtr) missingInstruments.push(instrument) } return missingInstruments }

Slide 43

Slide 43 text

async function fetchInstrument (instrument) { const response = await fetch(`${urlPreﬁx}/${instrument}`) const buf = await response.arrayBuffer() lib.FS.writeFile(instrument, buf) }

Slide 44

Slide 44 text

Web Audio Image Credit: hexeosis.tumblr.com

Slide 45

Slide 45 text

const audioContext = new AudioContext() const node = audioContext.createScriptProcessor(...) node.addEventListener('audioprocess', onAudioProcess) node.connect(audioContext.destination)

Slide 46

Slide 46 text

Slide 47

Slide 47 text

function readMidiData () { const byteCount = lib._mid_song_read_wave(songPtr, buffer, ...) array.set( lib.HEAP16.subarray(bufferPtr / 2, (bufferPtr + byteCount) / 2) ) return byteCount / BYTES_PER_SAMPLE // sampleCount }

Slide 48

Slide 48 text

npm install timidity

Slide 49

Slide 49 text

const Timidity = require('timidity') const player = new Timidity() player.load('/my-ﬁle.mid') player.play() player.on('timeupdate', seconds => { console.log(seconds) })

Slide 50

Slide 50 text

Web Components Image Credit: hexeosis.tumblr.com

Slide 51

Slide 51 text

Make it a Web Component Before After

Slide 52

Slide 52 text

class BgSound extends HTMLElement { connectedCallback () { this.player = new Timidity() this.player.load(this.src) this.player.play() } disconnectedCallback () { this.player.destroy() } } window.customElements.deﬁne('bg-sound', BgSound)

Slide 53

Slide 53 text

npm install bg-sound

Slide 54

Slide 54 text

DEMO

Slide 55

Slide 55 text

No content

Slide 56

Slide 56 text

No content

Slide 57

Slide 57 text

The Mythical Man-Month The programmer, like the poet, works only slightly removed from pure thought-stuff. He builds castles in the air, from air, creating by exertion of the imagination. Few media of creation are so flexible, so easy to polish and rework, so readily capable of realizing grand conceptual structures. — Fred Brooks