The Lost Art of MIDI – Bringing <bgsound> Back to the Web

In the days of Geocities and Angelfire, a quirky HTML tag called ⟨bgsound⟩ enabled sound files to play in the background of webpages. Usually, these files were in the MIDI format. What a glorious era that was! Sadly, ⟨bgsound⟩ has been removed from browsers and MIDI is obscure and hard to play back. In this talk, we'll bring MIDI and ⟨bgsound⟩ back from the dead using WebAssembly, Emscripten, Web Audio, and Web Components. When we're finished, you'll be able to give your webpages the 90's treatment in a modern, standards-compliant way!

Links from the talk:

Timidity – Play MIDI files in the browser w/ Web Audio, WebAssembly, and libtimidity
https://github.com/feross/timidity

bg-sound – Web Component to emulate the old-school HTML element
https://github.com/feross/bg-sound

BitMidi – Listen to free MIDI songs, download the best MIDI files, and share the best MIDIs on the web
https://bitmidi.com

Feross's Blog
https://feross.org

Feross's Twitter
https://twitter.com/feross

Transcript

1. The Lost Art of MIDI Bringing <bgsound> Back to the Web Image Credit: hexeosis.tumblr.com
3. Source: cameronsworld.net
4. Source: toastytech.com/evil/
5. Source: spacejam.com
6. Source: fogcam.org
7. Source: zombo.com
8. <bgsound src="tetris.mid">
9. Image Credit: Legend of Zelda: Ocarina of Time 3D
12. What even is a MIDI file? Image Credit: hexeosis.tumblr.com
13. Musical Instrument Digital Interface
14. Prophet-600
15. • 1971 – FTP • 1974 – TCP • 1982 – MIDI • 1991 – HTTP • 1995 – SSH • 2001 – BitTorrent • 2009 – Bitcoin • 2015 – HTTP/2
16. MIDI Messages Command Meaning 0x80 Note-off 0x90 Note-on ... ...
17. 90 3C 40 -----------------> time • 90 = Note on • 3C = Which key? • 40 = How hard was it pressed?
18. General MIDI • 128 notes (~10 octaves) • 16 channels • 128 programs (instrument sounds)
19. 1-8 Piano 9-16 Chromatic Percussion 17-24 Organ 25-32 Guitar 33-40 Bass 41-48 Strings 49-56 Ensemble 57-64 Brass 65-72 Reed 73-80 Pipe 81-88 Synth Lead 89-96 Synth Pad 97-104 Synth Effects 105-112 Ethnic 113-120 Percussive 121-128 Sound Effects
20. Pianos 1. Acoustic Grand Piano 2. Bright Acoustic Piano 3. Electric Grand Piano 4. Honky-tonk Piano 5. Electric Piano 1 6. Electric Piano 2 7. Harpsichord 8. Clavi
23. Where else is MIDI used? Image Credit: hexeosis.tumblr.com
33. WebAssembly Image Credit: hexeosis.tumblr.com
35. static void load_instrument(MidSong *song, ...) { ... if (song->ifp == NULL) { DEBUG_MSG("Instrument `%s' can't be found.\n", name); // Added for JavaScript port song->load_requests[song->load_request_count] = strdup(name); song->load_request_count += 1; ... }
36. // Added for JavaScript port extern int mid_get_load_request_count(MidSong *song) { return song->load_request_count; } extern char *mid_get_load_request(MidSong *song, int index) { return song->load_requests[index]; }
37. File system // Make directory lib.FS.mkdir('new-directory') // Write files lib.FS.writeFile('myfile.txt', buffer)
38. Pointers // Allocate memory const ptr = lib._malloc(byteLength) // Free memory lib._free(ptr) // Convert a `char*` to a JavaScript string const str = lib.Pointer_stringify(ptr)
40. function loadSong (buffer) { const bufferPtr = lib._malloc(buffer.byteLength) lib.HEAPU8.set(buffer, bufferPtr) const iStreamPtr = lib._mid_istream_open_mem(bufferPtr, buffer.byteLength) const songPtr = lib._mid_song_load(iStreamPtr) return songPtr }
41. let songPtr = loadSong(buffer) const instruments = getMissingInstruments(songPtr) if (instruments.length > 0) { await Promise.all( instruments.map(instrument => fetchInstrument(instrument)) ) lib._mid_song_free(songPtr) songPtr = loadSong(buffer) } lib._mid_song_start(songPtr)
42. function getMissingInstruments (songPtr) { const missingCount = lib._mid_get_load_request_count(songPtr) const missingInstruments = [] for (let i = 0; i < missingCount; i++) { const instrumentPtr = lib._mid_get_load_request(songPtr, i) const instrument = lib.Pointer_stringify(instrumentPtr) missingInstruments.push(instrument) } return missingInstruments }
43. async function fetchInstrument (instrument) { const response = await fetch(`${urlPrefix}/${instrument}`) const buf = await response.arrayBuffer() lib.FS.writeFile(instrument, buf) }
44. Web Audio Image Credit: hexeosis.tumblr.com
45. const audioContext = new AudioContext() const node = audioContext.createScriptProcessor(...) node.addEventListener('audioprocess', onAudioProcess) node.connect(audioContext.destination)
46. function onAudioProcess (event) { const sampleCount = readMidiData() const output0 = event.outputBuffer.getChannelData(0) const output1 = event.outputBuffer.getChannelData(1) for (let i = 0; i < sampleCount; i++) { output0[i] = array[i * 2] / 0x7FFF output1[i] = array[i * 2 + 1] / 0x7FFF } for (let i = sampleCount; i < BUFFER_SIZE; i++) { output0[i] = 0 output1[i] = 0 } }
47. function readMidiData () { const byteCount = lib._mid_song_read_wave(songPtr, buffer, ...) array.set( lib.HEAP16.subarray(bufferPtr / 2, (bufferPtr + byteCount) / 2) ) return byteCount / BYTES_PER_SAMPLE // sampleCount }
48. npm install timidity
49. const Timidity = require('timidity') const player = new Timidity() player.load('/my-file.mid') player.play() player.on('timeupdate', seconds => { console.log(seconds) })
50. Web Components Image Credit: hexeosis.tumblr.com
51. Make it a Web Component Before <bgsound src="sound.mid"> After <bg-sound src="sound.mid"></bg-sound> <script src="bg-sound.min.js"></script>
52. class BgSound extends HTMLElement { connectedCallback () { this.player = new Timidity() this.player.load(this.src) this.player.play() } disconnectedCallback () { this.player.destroy() } } window.customElements.define('bg-sound', BgSound)
53. npm install bg-sound
54. DEMO
57. The Mythical Man-Month The programmer, like the poet, works only slightly removed from pure thought-stuff. He builds castles in the air, from air, creating by exertion of the imagination. Few media of creation are so flexible, so easy to polish and rework, so readily capable of realizing grand conceptual structures. — Fred Brooks
59. Thanks! @feross feross.org