The Lost Art of MIDI – Bringing <bgsound> Back to the Web

The Lost Art of MIDI – Bringing <bgsound> Back to the Web

In the days of Geocities and Angelfire, a quirky HTML tag called ⟨bgsound⟩ enabled sound files to play in the background of webpages. Usually, these files were in the MIDI format. What a glorious era that was! Sadly, ⟨bgsound⟩ has been removed from browsers and MIDI is obscure and hard to play back. In this talk, we'll bring MIDI and ⟨bgsound⟩ back from the dead using WebAssembly, Emscripten, Web Audio, and Web Components. When we're finished, you'll be able to give your webpages the 90's treatment in a modern, standards-compliant way!

Links from the talk:

Timidity – Play MIDI files in the browser w/ Web Audio, WebAssembly, and libtimidity
https://github.com/feross/timidity

bg-sound – Web Component to emulate the old-school HTML element
https://github.com/feross/bg-sound

BitMidi – Listen to free MIDI songs, download the best MIDI files, and share the best MIDIs on the web
https://bitmidi.com

Feross's Blog
https://feross.org

Feross's Twitter
https://twitter.com/feross

B498d33041627b07726dc29c28f02df7?s=128

Feross Aboukhadijeh

November 16, 2018
Tweet

Transcript

  1. The Lost Art of MIDI Bringing <bgsound> Back to the

    Web Image Credit: hexeosis.tumblr.com
  2. None
  3. Source: cameronsworld.net

  4. Source: toastytech.com/evil/

  5. Source: spacejam.com

  6. Source: fogcam.org

  7. Source: zombo.com

  8. <bgsound src="tetris.mid">

  9. Image Credit: Legend of Zelda: Ocarina of Time 3D

  10. None
  11. None
  12. What even is a MIDI file? Image Credit: hexeosis.tumblr.com

  13. Musical Instrument Digital Interface

  14. Prophet-600

  15. • 1971 – FTP • 1974 – TCP • 1982

    – MIDI • 1991 – HTTP • 1995 – SSH • 2001 – BitTorrent • 2009 – Bitcoin • 2015 – HTTP/2
  16. MIDI Messages Command Meaning 0x80 Note-off 0x90 Note-on ... ...

  17. 90 3C 40 -----------------> time • 90 = Note on

    • 3C = Which key? • 40 = How hard was it pressed?
  18. General MIDI • 128 notes (~10 octaves) • 16 channels

    • 128 programs (instrument sounds)
  19. 1-8 Piano 9-16 Chromatic Percussion 17-24 Organ 25-32 Guitar 33-40

    Bass 41-48 Strings 49-56 Ensemble 57-64 Brass 65-72 Reed 73-80 Pipe 81-88 Synth Lead 89-96 Synth Pad 97-104 Synth Effects 105-112 Ethnic 113-120 Percussive 121-128 Sound Effects
  20. Pianos 1. Acoustic Grand Piano 2. Bright Acoustic Piano 3.

    Electric Grand Piano 4. Honky-tonk Piano 5. Electric Piano 1 6. Electric Piano 2 7. Harpsichord 8. Clavi
  21. None
  22. None
  23. Where else is MIDI used? Image Credit: hexeosis.tumblr.com

  24. None
  25. None
  26. None
  27. None
  28. None
  29. None
  30. None
  31. None
  32. None
  33. WebAssembly Image Credit: hexeosis.tumblr.com

  34. None
  35. static void load_instrument(MidSong *song, ...) { ... if (song->ifp ==

    NULL) { DEBUG_MSG("Instrument `%s' can't be found.\n", name); // Added for JavaScript port song->load_requests[song->load_request_count] = strdup(name); song->load_request_count += 1; ... }
  36. // Added for JavaScript port extern int mid_get_load_request_count(MidSong *song) {

    return song->load_request_count; } extern char *mid_get_load_request(MidSong *song, int index) { return song->load_requests[index]; }
  37. File system // Make directory lib.FS.mkdir('new-directory') // Write files lib.FS.writeFile('myfile.txt',

    buffer)
  38. Pointers // Allocate memory const ptr = lib._malloc(byteLength) // Free

    memory lib._free(ptr) // Convert a `char*` to a JavaScript string const str = lib.Pointer_stringify(ptr)
  39. None
  40. function loadSong (buffer) { const bufferPtr = lib._malloc(buffer.byteLength) lib.HEAPU8.set(buffer, bufferPtr)

    const iStreamPtr = lib._mid_istream_open_mem(bufferPtr, buffer.byteLength) const songPtr = lib._mid_song_load(iStreamPtr) return songPtr }
  41. let songPtr = loadSong(buffer) const instruments = getMissingInstruments(songPtr) if (instruments.length

    > 0) { await Promise.all( instruments.map(instrument => fetchInstrument(instrument)) ) lib._mid_song_free(songPtr) songPtr = loadSong(buffer) } lib._mid_song_start(songPtr)
  42. function getMissingInstruments (songPtr) { const missingCount = lib._mid_get_load_request_count(songPtr) const missingInstruments

    = [] for (let i = 0; i < missingCount; i++) { const instrumentPtr = lib._mid_get_load_request(songPtr, i) const instrument = lib.Pointer_stringify(instrumentPtr) missingInstruments.push(instrument) } return missingInstruments }
  43. async function fetchInstrument (instrument) { const response = await fetch(`${urlPrefix}/${instrument}`)

    const buf = await response.arrayBuffer() lib.FS.writeFile(instrument, buf) }
  44. Web Audio Image Credit: hexeosis.tumblr.com

  45. const audioContext = new AudioContext() const node = audioContext.createScriptProcessor(...) node.addEventListener('audioprocess',

    onAudioProcess) node.connect(audioContext.destination)
  46. function onAudioProcess (event) { const sampleCount = readMidiData() const output0

    = event.outputBuffer.getChannelData(0) const output1 = event.outputBuffer.getChannelData(1) for (let i = 0; i < sampleCount; i++) { output0[i] = array[i * 2] / 0x7FFF output1[i] = array[i * 2 + 1] / 0x7FFF } for (let i = sampleCount; i < BUFFER_SIZE; i++) { output0[i] = 0 output1[i] = 0 } }
  47. function readMidiData () { const byteCount = lib._mid_song_read_wave(songPtr, buffer, ...)

    array.set( lib.HEAP16.subarray(bufferPtr / 2, (bufferPtr + byteCount) / 2) ) return byteCount / BYTES_PER_SAMPLE // sampleCount }
  48. npm install timidity

  49. const Timidity = require('timidity') const player = new Timidity() player.load('/my-file.mid')

    player.play() player.on('timeupdate', seconds => { console.log(seconds) })
  50. Web Components Image Credit: hexeosis.tumblr.com

  51. Make it a Web Component Before <bgsound src="sound.mid"> After <bg-sound

    src="sound.mid"></bg-sound> <script src="bg-sound.min.js"></script>
  52. class BgSound extends HTMLElement { connectedCallback () { this.player =

    new Timidity() this.player.load(this.src) this.player.play() } disconnectedCallback () { this.player.destroy() } } window.customElements.define('bg-sound', BgSound)
  53. npm install bg-sound

  54. DEMO

  55. None
  56. None
  57. The Mythical Man-Month The programmer, like the poet, works only

    slightly removed from pure thought-stuff. He builds castles in the air, from air, creating by exertion of the imagination. Few media of creation are so flexible, so easy to polish and rework, so readily capable of realizing grand conceptual structures. — Fred Brooks
  58. None
  59. Thanks! @feross feross.org