Tobi Wulff: Control All Your Programs with MIDI Controllers

Tobi Wulff: Control All Your Programs with MIDI Controllers

= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
Tobi Wulff:
Control All Your Programs with MIDI Controllers
= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
@ Kiwi PyCon 2014 - Saturday, 13 Sep 2014 - Track 1
http://kiwi.pycon.org/

**Audience level**

Intermediate

**Description**

MIDI devices are commonly used to provide input to audio applications, e.g. to play a synthesizer via a keyboard or to control a mixer in an audio workstation. This talk will show you tools for receiving input and processing messages, and how to control all of your programs with minimal configuration effort, the advantage being that not every single program has to be made natively MIDI aware.

**Abstract**

This talk will give a quick overview of the MIDI protocol and controllers which are commonly used to provide input to audio applications. These controllers can have the form of a musical keyboard, a mixer with knobs, sliders and buttons, or a combination of those. Some other variants use touch-sensitive trackpads or look like a drum kit. Then, the talk will go over the following tools: Python (duh), mididings (to receive and manipulate MIDI messages), xte (send keystrokes and mouse events to the Linux Xorg server) and some other necessary programs such as jackd to virtually connected MIDI inputs and outputs. I'll describe the steps needed to bring all those tools together and make it extensible to (hopefully, eventually) work on other OS as well.

mididings provides a Domain Specific Language (DSL) for MIDI processing which will be introduced as part of this talk. If you are doing anything with MIDI devices (and other external devices apart from keyboards) and feel like they pose some limitations on you (for example not supporting different customisable button assignments on the device), this will be of interest to you. xte will be used in a very basic way but if you haven't heard of it and want to send events to your X server, this is a great program. The possibilities are endless and we haven't even combined all the tools yet. This is about to change in the next step of the talk. A live demo/video will demonstrate how it all comes together into one beautiful piece of ingenuity and software engineering.

Finally, I'll discuss pros and cons of this solution versus other possible methods including native MIDI device support for various programs.

**YouTube**

https://www.youtube.com/watch?v=p2CsRZoatc8

6b880a0b67fac54c42c77fe70d97334d?s=128

New Zealand Python User Group

September 13, 2014
Tweet

Transcript

  1. 3.

    The Good: Standardised Protocol No worries about OS drivers Lots

    of connections and daisy chaining Reusability Libraries for Arduino, and more Cheap MIDI­to­USB adapters
  2. 11.

    The Protocol and Theory Controllers generate notes and controlling messages

    Software can generate or process (play) messages Synthesisers can play MIDI messages => Keyboards are controller + synthesisers
  3. 13.

    MIDI Messages Can be sent on specific channels: 0(all) ­

    15 Data is in the range of 0­127 On/off usually done with ≤ 63 and ≥ 64 Control Change [Control, Value] Program Change Note On/Off [Note, Speed] Pitch Bend SysEx Panic! Note Off for all notes
  4. 15.

    Quick Excursion Linux Audio: basic layer (ALSA) Real­time and routing

    with Jack (jackd) Mididings can connect to both of those
  5. 21.

    A small Mididings Example f r o m m i

    d i d i n g s i m p o r t r u n , P r o c e s s , N O T E O N d e f i n v e r t ( e v e n t ) : i f e v e n t . c h a n n e l = = 1 a n d e v e n t . t y p e = = N O T E O N : e v e n t . v e l o c i t y = 1 2 7 - e v e n t . v e l o c i t y r e t u r n e v e n t r u n ( P r o c e s s ( i n v e r t ) )
  6. 22.

    Domain Specific Languages "A domain­specific language (DSL) is a computer

    language specialized to a particular application domain."
  7. 23.

    Mididings as a DSL p i a n o =

    O u t p u t ( ' s a m p l e r ' , 1 , 1 ) # p o r t , c h , p r g s t r i n g s = O u t p u t ( ' s y n t h ' , 2 ) # p o r t , c h , p r g C h a n n e l F i l t e r ( 1 ) > > K e y S p l i t ( ' c 3 ' , p i a n o , s t r i n g s )
  8. 24.

    Mididings as a DSL F i l t e r

    ( C T R L ) > > C h a n n e l ( 2 ) F i l t e r ( N O T E O N ) > > T r a n s p o s e ( - 1 2 ) > > o r c h e s t r a / / b e l l s { C T R L : C h a n n e l ( 2 ) , N O T E O N : T r a n s p o s e ( - 1 2 ) > > [ o r c h e s t r a , b e l l s ] }
  9. 25.

    Mididings as a DSL ( ( C h a n

    n e l F i l t e r ( 1 ) & ( C t r l F i l t e r ( 2 3 ) | C t r l F i l t e r ( 4 2 ) ) ) & ~ C t r l V a l u e F i l t e r ( l o w e r = 5 0 , u p p e r = 1 0 0 ) ) > > C h a n n e l ( 2 )
  10. 26.

    Mididings as a DSL ( # o n l y

    u s e c h a n n e l 1 ( C h a n n e l F i l t e r ( 1 ) & # C C # 2 3 o r 4 2 ( C t r l F i l t e r ( 2 3 ) | C t r l F i l t e r ( 4 2 ) ) ) & # s l i d e r n o t b e t w . 5 0 a n d 1 0 0 ~ C t r l V a l u e F i l t e r ( l o w e r = 5 0 , u p p e r = 1 0 0 ) ) # s e n d T H O S E m e s s a g e s t o c h . 2 > > C h a n n e l ( 2 )
  11. 28.

    CTRL specific functions C t r l M a p

    ( ) # c h a n g e s C C # C t r l R a n g e ( ) # c h a n g e s v a l u e C t r l C u r v e ( ) # a p p l y a c u r v e t o v a l u e s Should use those more
  12. 29.

    Ok, back to controlling GUIs 1. Collect incoming MIDI messages

    (filtered) 2. Grab collected events and process 3. Turn event into key press (or something else) 4. Send off key press
  13. 30.

    Collect Main process acting as a producer adding items to

    the event queue. Overwrite if same event happens again. r u n ( P r o c e s s ( c o l l e c t ) ) d e f c o l l e c t ( s e l f , m i d i ) : w i t h s e l f . _ e v e n t s _ l o c k : s e l f . _ c o l l e c t ( m i d i )
  14. 31.

    Grab Second thread acting as a consumer taking items from

    the event queue. i m p o r t t h r e a d i n g w i t h s e l f . _ l o c k : s e l f . _ e v e n t s . i t e m s ( )
  15. 32.

    Key Press Map MIDI message to shortcut action object: M

    A P = { ( 0 , C T R L , 1 1 ) : ( ' W i n d o w ' , S h o r t c u t A c t i o n ( N O _ M O D , ' o ' ) ) , . . . } c l a s s S h o r t c u t A c t i o n ( o b j e c t ) : d e f _ _ i n i t _ _ ( s e l f , m o d i f i e r s , k e y , f u n c = N o n e ) : . . . d e f r u n ( s e l f , e x e c u t o r , v a l u e ) : . . . c l a s s S h o r t c u t A c t i o n S e q u e n c e ( o b j e c t ) : . . .
  16. 33.

    Send Will only go to foreground window (unless special global

    shortcut). f r o m s u b p r o c e s s i m p o r t P o p e n , P I P E k e y _ s e q u e n c e = [ ' k e y d o w n C o n t r o l _ L ' , ' k e y o ' , ' k e y u p C o n t r o l _ L ' , ' s t r 1 . 3 3 ' ] P o p e n ( [ ' x t e ' ] , s t d i n = P I P E ) . c o m m u n i c a t e ( i n p u t = k e y _ s e q u e n c e )
  17. 34.

    The Future OSC will probably replace MIDI: mididings ­> S

    e n d O S C Extend to send to other OS (not just using xte) More flexibility for other programs (GIMP, ...) Mouse etc macros Feedback (MIDI from PC to controller)
  18. 35.

    References For more theory: www.midi.org This program: github.com/tobiw/midi2gui Mididings: (part

    of most Linux package managers) das.nasophon.de Xte: xautomation.sf.net
  19. 36.

    Contact Me T w i t t e r :

    t o b i a s w u l f f G i t h u b a n d I R C : t o b i w J a b b e r a n d E m a i l : t o b i ( a ) s w u l f f . d e