GEORGE MANDIS WEB — EMAIL — TWITTER — GITHUB — BLOG — WORK — I'm a freelance web developer from the U.S. I've been developing for the web for over a decade. george.mand.is [email protected] @georgeMandis @georgemandis george.mand.is snaptortoise.com 2 / 82
GEORGE MANDIS FUN FACTS! First time visiting Ukraine Lived as digital nomad in 18 countries Born in Saudi Arabia! Long story Once unintentionally cheated running a marathon in North Korea Most known project — — come see my other talk! :-D Konami-JS 3 / 82
IT'S WHERE THE PEOPLE ARE In ~8 years Node.JS has made JavaScript a legitimate server-side contender and fundamentally changed how we develop for the web and beyond. 11 / 82
MIDI THE PROTOCOL Allows electronic musical instruments to communicate with one another Describes common instrument performance parameters — noteOn, noteOff, pitchbend, programChange and others 27 / 82
BACK TO THE QUESTION... WHY MIDI? MIDI is asynchronous (JavaScript!) MIDI talks to hardware (The IoT!) MIDI is easy to use for non-musical applications (Yay!) 32 / 82
LESS COMMON MIDI CONTROLLERS Expression pedals Breathe & bite controllers Accelerometer based controllers Tenori On Pianocade Karlax from Da Fact Bananas... (via Makey Makey) 38 / 82
ANATOMY OF A MIDI MESSAGE Every message is 3 bytes. There are only 2 types of messages: status and data. A status byte always begins with 1 and data bytes always begin with 0 That leaves 7 bits left per byte for expressing the message For a status message, 3 bits of the first byte describe the type of status message; the remaining 4 describe the channel 41 / 82
JAVASCRIPT + MIDI IN THE BROWSER Requesting access. navigator.requestMIDIAccess navigator.requestMIDIAccess().then( onMIDIInit, onMIDIFail ); let onMIDIInit = function (MIDIAccessObject) { // Allows us to start interacting with MIDIAccessObjects ... } let onMIDIFail = function () { // MIDI supported but access denied ... } 46 / 82
JAVASCRIPT + MIDI IN THE BROWSER Iterating through the MIDI inputs and attaching listeners to MIDI messages as they're received let onMIDIInit = function (MIDIAccessObject) { for (let input of midi.inputs.values()) { if (input.name === "Logidy UMI3") logidy = input; if (input.name === "PuckCC Bluetooth") puckJS = input; } logidy.onmidimessage = handleLogidyController; puckJS.onmidimessage = handlePuckController; } 47 / 82
JAVASCRIPT + MIDI IN THE BROWSER Handling a received MIDI message Our 3-Byte MIDI message shows up as an array [0, 0, 0] let handlePuckController = function(event) { data = event.data; if (data[1] == 80) {...} } 48 / 82
JAVASCRIPT + MIDI IN THE BROWSER Libraries to make life a little easier WebMIDI.js WebMidi.enable(function (err) { logidy = WebMidi.getInputByname("Logidy UMI3"); puck = WebMidi.getInputByname("PuckCC Bluetooth"); puck.addListener("noteon", "all", function(event) { let data = e.data; }); logidy.addListener("noteon", "all", function(event) { let data = e.data; }); }); 49 / 82
JAVASCRIPT + MIDI IN THE BROWSER Libraries to make life a little easier MIDIUtils.js puck.addListener("noteon", "all", function(event) { let frequency = MIDIUtils.noteNumberToFrequency(e.data[1]) let noteName = MIDIUtils.noteNumberToName(e.data[1]) let noteNumberFromName = MIDIUtils.noteNameToNoteNumber("A-4") let noteNumberFromFreq = MIDIUtils.frequencyToNoteNumber(440) }); 50 / 82
JAVASCRIPT + MIDI ON THE SERVER What's different and what's the same We don't need to request permission Latency can be more of an issue in some contexts Reading & writing MIDI files is easier (more packages) Some of the frontend libraries work on the server 52 / 82
JAVASCRIPT + MIDI ON THE SERVER Basic setup for talking to and listening to a controller const midi = require("midi"); let input = new midi.input(); input.on('message', function(deltaTime, message) { console.log(message, deltaTime); }); input.openPort(0); 53 / 82
WEB SYMPHONY II SHARING PARTS OF MIDI FILE UNDER THE HOOD Express Socket.io (WebSockets) MIDIUtils.js midifile midievents HTTP://ODESSAJS.MAND.IS/WS2 59 / 82
THE MIDI POETRY BOX HOW IT WORKED Pick a random video from Pick a random book from Choose 4 random sentences from that book to make a "poem" Display the poem on top of the video as it plays at a randomized playback speed archive.org gutenberg.org 63 / 82
MIDI PIXEL ART UNDER THE HOOD Native WebMIDI implementation! var x = noteNumber & 0x0f; var y = (noteNumber & 0xf0) >> 4; DRAW SIMPLE IMAGES ON A LAUNCHPAD 73 / 82
georgemandis
uhooi
niftycorp
trapezoid
huffon
antyadev
deepu105
prof18
hiroyaonoe
ivanova1991
nrslib
tomokusaba
ojun9
nonsquared
afnizarnur
searls
smashingmag
jacobian
brad_frost
reverentgeek
addyosmani
lynnandtonic
dougneiner
samanthasiow
productmarketing