Slide 1

Slide 1 text

What for, Where and How to Adopt MIDI 2.0 @COSCUP 2022 atsushieno

Slide 2

Slide 2 text

@atsushieno and MIDI... /me had been playing with MIDI since 199x. using MML (Music Macro Language) to MIDI compiler to create *.mid files atsushieno/managed-midi: "the truly cross-platform .NET MIDI API", and co. atsushieno/ktmidi: switched to Kotlin, supports MIDI 2.0 atsushieno/augene-ng: MML to MIDI 2.0 to DAW (tracktion) compiler wrote a Japanese indie tech book about MIDI 2.0, IIRC world's first.

Slide 3

Slide 3 text

What is MIDI? MIDI = Musical Instrument Digital Interface ( = commonly used) MIDI 1.0 was standardized in 1983, then MIDI 2.0 in 2020

Slide 4

Slide 4 text

MIDI 1.0 was about... General MIDI: collection of instruments indexed by program number source

Slide 5

Slide 5 text

MIDI 1.0 was about... MIDI messages: instructions such as note-on, note-off, control change and pitchbend source

Slide 6

Slide 6 text

MIDI devices and cables later: USB, BLE MIDI 1.0 was about...

Slide 7

Slide 7 text

MIDI 1.0 was about... Standard MIDI: format of MIDI music file (SMF) It was a popular format to distribute music before MP3 became popular. They were even ded on WWW. (source)

Slide 8

Slide 8 text

How do we compose music on PC in 2022? In 2020s we mostly use DAWs, and they don't save songs in .mid file! We use audio plugins (VST / AU / LV2 / ...) for instruments and effect plugins

Slide 9

Slide 9 text

Why audio plugins? We need unlimited kind of instruments and effect controllers beyond MIDI We need same (identical) audio outputs across computers We need finer parameters than 7-bit data ranges We want to pass raw audio samples (or whatever) to those instruments.

Slide 10

Slide 10 text

Is MIDI 1.0 gone? YES and NO... The basic song structures in DAWs remain similar to SMF: - We have tracks Also, meta event tracks - We have MIDI-like events (optionally with loops)

Slide 11

Slide 11 text

Is MIDI 1.0 gone? Some music composers distribute SMF as additional contents.

Slide 12

Slide 12 text

Is MIDI 1.0 gone? DAWs can import and export SMF too.

Slide 13

Slide 13 text

DAW and MIDI We use pianorolls and/or (rhythm) step sequencers. We also use MIDI keyboard to add notes onto the bound track. BUT, those audio plugins do not (necessarily) receive MIDI events directly. Every plugin format has their own "events" e.g. VST3 Note On/Off MIDI VST3

Slide 14

Slide 14 text

DAWs usually support multiple audio plugin formats in consistent manner. DAWs store some abstract sequence, and convert them to the target event. DAW and MIDI VST3 LV2 audio audio audio

Slide 15

Slide 15 text

Audio Plugins: the World is Divided Audio plugin products are often built and released for multiple targets (Win/Mac/..., VST/AU/...) Event inputs for multiple targets: needs abstraction too VST3 events AU events LV2 events audio audio audio

Slide 16

Slide 16 text

In the wake of MIDI 2.0... Is MIDI 1.0 good for the abstract sequence? It's not expressive enough... But MIDI 2.0 is here! - Specification finalized in early 2020 - Has 32-bit fine parameters, per-note expressions and controllers, etc... The specification can be downloaded at AMEI (or at MMA, but they require user registration)

Slide 17

Slide 17 text

What are the MIDI 2.0 specification components? MIDI-CI, MIDI 2.0 Property Exchange, Profile Configuration CI: bi-directional messaging. (Pairing one-way MIDI conns. like UDP in HTTP/3.) PE: based on MIDI-CI, manage MIDI device properties: "get", "set", "query" etc. PC: MIDI devices may have profiles such as "Organ profile", "E.Piano profile" etc. MIDI 2.0 UMP (Universal MIDI Packet) Some other specifications e.g. USB MIDI 2.0 protocol

Slide 18

Slide 18 text

MIDI 2.0 UMP: brand new message format "programmer friendly" fixed size: 4 bytes, 8 bytes, or 16 bytes Timestamps are there too (i.e. SMF event delta time equivalents) 00h 4 bytes Utility messages (timestamps) - 10h 4 bytes System Realtime or System Common messages F1h-FFh in MIDI 1.0 20h 4 bytes MIDI 1.0 Channel Voice messages 80h-E0h in MIDI 1.0 30h 8 bytes Sysex7 messages F0h in MIDI 1.0 40h 8 bytes MIDI 2.0 channel voice messages - 50h 16 bytes Sysex8 and Mixed Data Set messages -

Slide 19

Slide 19 text

MIDI 2.0 channel voice messages 00h Per-Note RPN (Registered Controller) 80h Note Off 10h Per-Note NRPN (Assignable Controller) 90h Note On 20h RPN (Registered Controller) A0h Polyphonic Pressure 30h NRPN (Assignable Controller) B0h Control Change 40h Relative RPN (Registered Controller) C0h Program Change 50h Relative NRPN (Assignable Controller) D0h Channel Pressure 60h Per-Note Pitch Bend E0h Pitch Bend 70h (unused) F0h Per-Note Management

Slide 20

Slide 20 text

MIDI 2.0 channel voice messages note on (90h) / off (80h): has 16-bits velocity, extra attribute (16 bits) MIDI 2.0 supports 256 channels, 16 channels * 16 groups The attribute can be anything. "pitch 7.9" can be specified for microtonal note (7-bit semitones, 9-bit microtone)

Slide 21

Slide 21 text

MIDI 2.0 channel voice messages control change (B0h): has 32-bit data ... and it can be per-note controller (00h)

Slide 22

Slide 22 text

Note: they are not new features in audio world VST3 and AU supported per-note expressions and controllers. MIDI 1.0 was capable of partially supporting: - microtonal tunings (i.e. beyond 12-tones equal temperament) => MTS - pitch bend that does not apply to ALL notes on the channel => MPE MIDI 2.0 made them "standard".

Slide 23

Slide 23 text

Are We MIDI 2.0 Yet? Platform support: Only on Mac, iOS, and Android (USB-MIDI 2.0 only) "Where can I buy MIDI 2.0 ?" (devices) Roland A-88MkII ? Erae Touch ? If there is no MIDI 2.0 device yet, can we even use it? → still YES in theory (source)

Slide 24

Slide 24 text

MIDI 2.0 on MIDI 1.0: Promote to UMPs using MIDI-CI (1) You can use MIDI-CI to promote the protocol to MIDI 2.0 UMPs. MIDI-CI: universal system exclusive messages (MIDI 1.0 compatible) (MIDI-CI v1.1 spec. P.23)

Slide 25

Slide 25 text

MIDI 2.0 on MIDI 1.0: Promote to UMPs using MIDI-CI (2) Remember earlier discussion on audio plugins? DAW events can be represented using MIDI 2.0 UMPs! MIDI AU MIDI

Slide 26

Slide 26 text

MIDI 2.0 on MIDI 1.0: Promote to UMPs using MIDI-CI (2) Remember earlier discussion on audio plugins? DAW events can be represented using MIDI 2.0 UMPs! MIDI 2.0 AU Set New Protocol MIDI 2.0

Slide 27

Slide 27 text

Audio Plugins and MIDI 2.0 AudioUnit: we'll still have to map UMPs to AU-specific events. But CoreMIDI supports MIDI-CI and UMP, so native UMP support may be matter of time. vst3sdk is reportedly capable of map most of MIDI 2.0 UMPs to VST3 events. CLAP plugin (very new) can deal with MIDI 2.0 natively. JUCE framework: supports UMP processing. vNext will support MIDI 2.0 devices. The plugin base (AudioProcessor) receives raw MIDI buffer so UMPs could fit there. Developers are going to explore MIDI 2.0 applicability here.

Slide 28

Slide 28 text

MIDI 2.0 software synthesizers? Maybe? It should be possible too

Slide 29

Slide 29 text

Summary MIDI 2.0 is modern upgrade from MIDI 1.0, paying attention to backward compat. Limited platform support, no instrument devices, but that doesn't matter. Audio plugins APIs are similarly featureful, but MIDI 2.0 made them standard. Audio plugins are (getting) ready to deal with UMPs, so it's worth to learn!

Slide 30

Slide 30 text

Thanks! Questions?