What for, Where and How to Adopt MIDI 2.0 @COSCUP 2022 atsushieno

@atsushieno and MIDI... /me had been playing with MIDI since 199x. using MML (Music Macro Language) to MIDI compiler to create *.mid files atsushieno/managed-midi: "the truly cross-platform .NET MIDI API", and co. atsushieno/ktmidi: switched to Kotlin, supports MIDI 2.0 atsushieno/augene-ng: MML to MIDI 2.0 to DAW (tracktion) compiler wrote a Japanese indie tech book about MIDI 2.0, IIRC world's first.

What is MIDI? MIDI = Musical Instrument Digital Interface ( = commonly used) MIDI 1.0 was standardized in 1983, then MIDI 2.0 in 2020

MIDI 1.0 was about... General MIDI: collection of instruments indexed by program number source

MIDI 1.0 was about... MIDI messages: instructions such as note-on, note-off, control change and pitchbend source

MIDI devices and cables later: USB, BLE MIDI 1.0 was about...

MIDI 1.0 was about... Standard MIDI: format of MIDI music file (SMF) It was a popular format to distribute music before MP3 became popular. They were even ded on WWW. (source)

How do we compose music on PC in 2022? In 2020s we mostly use DAWs, and they don't save songs in .mid file! We use audio plugins (VST / AU / LV2 / ...) for instruments and effect plugins

Why audio plugins? We need unlimited kind of instruments and effect controllers beyond MIDI We need same (identical) audio outputs across computers We need finer parameters than 7-bit data ranges We want to pass raw audio samples (or whatever) to those instruments.

Is MIDI 1.0 gone? YES and NO... The basic song structures in DAWs remain similar to SMF: - We have tracks Also, meta event tracks - We have MIDI-like events (optionally with loops)

Is MIDI 1.0 gone? Some music composers distribute SMF as additional contents.

Is MIDI 1.0 gone? DAWs can import and export SMF too.

DAW and MIDI We use pianorolls and/or (rhythm) step sequencers. We also use MIDI keyboard to add notes onto the bound track. BUT, those audio plugins do not (necessarily) receive MIDI events directly. Every plugin format has their own "events" e.g. VST3 Note On/Off MIDI VST3

DAWs usually support multiple audio plugin formats in consistent manner. DAWs store some abstract sequence, and convert them to the target event. DAW and MIDI VST3 LV2 audio audio audio

Audio Plugins: the World is Divided Audio plugin products are often built and released for multiple targets (Win/Mac/..., VST/AU/...) Event inputs for multiple targets: needs abstraction too VST3 events AU events LV2 events audio audio audio

In the wake of MIDI 2.0... Is MIDI 1.0 good for the abstract sequence? It's not expressive enough... But MIDI 2.0 is here! - Specification finalized in early 2020 - Has 32-bit fine parameters, per-note expressions and controllers, etc... The specification can be downloaded at AMEI https://tinyurl.com/midi-2-spec (or at MMA, but they require user registration)

What are the MIDI 2.0 specification components? MIDI-CI, MIDI 2.0 Property Exchange, Profile Configuration CI: bi-directional messaging. (Pairing one-way MIDI conns. like UDP in HTTP/3.) PE: based on MIDI-CI, manage MIDI device properties: "get", "set", "query" etc. PC: MIDI devices may have profiles such as "Organ profile", "E.Piano profile" etc. MIDI 2.0 UMP (Universal MIDI Packet) Some other specifications e.g. USB MIDI 2.0 protocol

MIDI 2.0 UMP: brand new message format "programmer friendly" fixed size: 4 bytes, 8 bytes, or 16 bytes Timestamps are there too (i.e. SMF event delta time equivalents) 00h 4 bytes Utility messages (timestamps) - 10h 4 bytes System Realtime or System Common messages F1h-FFh in MIDI 1.0 20h 4 bytes MIDI 1.0 Channel Voice messages 80h-E0h in MIDI 1.0 30h 8 bytes Sysex7 messages F0h in MIDI 1.0 40h 8 bytes MIDI 2.0 channel voice messages - 50h 16 bytes Sysex8 and Mixed Data Set messages -

MIDI 2.0 channel voice messages 00h Per-Note RPN (Registered Controller) 80h Note Off 10h Per-Note NRPN (Assignable Controller) 90h Note On 20h RPN (Registered Controller) A0h Polyphonic Pressure 30h NRPN (Assignable Controller) B0h Control Change 40h Relative RPN (Registered Controller) C0h Program Change 50h Relative NRPN (Assignable Controller) D0h Channel Pressure 60h Per-Note Pitch Bend E0h Pitch Bend 70h (unused) F0h Per-Note Management

MIDI 2.0 channel voice messages note on (90h) / off (80h): has 16-bits velocity, extra attribute (16 bits) MIDI 2.0 supports 256 channels, 16 channels * 16 groups The attribute can be anything. "pitch 7.9" can be specified for microtonal note (7-bit semitones, 9-bit microtone)

MIDI 2.0 channel voice messages control change (B0h): has 32-bit data ... and it can be per-note controller (00h)

Note: they are not new features in audio world VST3 and AU supported per-note expressions and controllers. MIDI 1.0 was capable of partially supporting: - microtonal tunings (i.e. beyond 12-tones equal temperament) => MTS - pitch bend that does not apply to ALL notes on the channel => MPE MIDI 2.0 made them "standard".

Are We MIDI 2.0 Yet? Platform support: Only on Mac, iOS, and Android (USB-MIDI 2.0 only) "Where can I buy MIDI 2.0 ?" (devices) Roland A-88MkII ? Erae Touch ? If there is no MIDI 2.0 device yet, can we even use it? → still YES in theory (source)

MIDI 2.0 on MIDI 1.0: Promote to UMPs using MIDI-CI (1) You can use MIDI-CI to promote the protocol to MIDI 2.0 UMPs. MIDI-CI: universal system exclusive messages (MIDI 1.0 compatible) (MIDI-CI v1.1 spec. P.23)

MIDI 2.0 on MIDI 1.0: Promote to UMPs using MIDI-CI (2) Remember earlier discussion on audio plugins? DAW events can be represented using MIDI 2.0 UMPs! MIDI AU MIDI

MIDI 2.0 on MIDI 1.0: Promote to UMPs using MIDI-CI (2) Remember earlier discussion on audio plugins? DAW events can be represented using MIDI 2.0 UMPs! MIDI 2.0 AU Set New Protocol MIDI 2.0

Audio Plugins and MIDI 2.0 AudioUnit: we'll still have to map UMPs to AU-specific events. But CoreMIDI supports MIDI-CI and UMP, so native UMP support may be matter of time. vst3sdk is reportedly capable of map most of MIDI 2.0 UMPs to VST3 events. CLAP plugin (very new) can deal with MIDI 2.0 natively. JUCE framework: supports UMP processing. vNext will support MIDI 2.0 devices. The plugin base (AudioProcessor) receives raw MIDI buffer so UMPs could fit there. Developers are going to explore MIDI 2.0 applicability here.

MIDI 2.0 software synthesizers? Maybe? It should be possible too

Summary MIDI 2.0 is modern upgrade from MIDI 1.0, paying attention to backward compat. Limited platform support, no instrument devices, but that doesn't matter. Audio plugins APIs are similarly featureful, but MIDI 2.0 made them standard. Audio plugins are (getting) ready to deal with UMPs, so it's worth to learn!

Thanks! Questions?