Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Tobi Wulff: Control All Your Programs with MIDI Controllers

Tobi Wulff: Control All Your Programs with MIDI Controllers

= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
Tobi Wulff:
Control All Your Programs with MIDI Controllers
= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
@ Kiwi PyCon 2014 - Saturday, 13 Sep 2014 - Track 1
http://kiwi.pycon.org/

**Audience level**

Intermediate

**Description**

MIDI devices are commonly used to provide input to audio applications, e.g. to play a synthesizer via a keyboard or to control a mixer in an audio workstation. This talk will show you tools for receiving input and processing messages, and how to control all of your programs with minimal configuration effort, the advantage being that not every single program has to be made natively MIDI aware.

**Abstract**

This talk will give a quick overview of the MIDI protocol and controllers which are commonly used to provide input to audio applications. These controllers can have the form of a musical keyboard, a mixer with knobs, sliders and buttons, or a combination of those. Some other variants use touch-sensitive trackpads or look like a drum kit. Then, the talk will go over the following tools: Python (duh), mididings (to receive and manipulate MIDI messages), xte (send keystrokes and mouse events to the Linux Xorg server) and some other necessary programs such as jackd to virtually connected MIDI inputs and outputs. I'll describe the steps needed to bring all those tools together and make it extensible to (hopefully, eventually) work on other OS as well.

mididings provides a Domain Specific Language (DSL) for MIDI processing which will be introduced as part of this talk. If you are doing anything with MIDI devices (and other external devices apart from keyboards) and feel like they pose some limitations on you (for example not supporting different customisable button assignments on the device), this will be of interest to you. xte will be used in a very basic way but if you haven't heard of it and want to send events to your X server, this is a great program. The possibilities are endless and we haven't even combined all the tools yet. This is about to change in the next step of the talk. A live demo/video will demonstrate how it all comes together into one beautiful piece of ingenuity and software engineering.

Finally, I'll discuss pros and cons of this solution versus other possible methods including native MIDI device support for various programs.

**YouTube**

https://www.youtube.com/watch?v=p2CsRZoatc8

6b880a0b67fac54c42c77fe70d97334d?s=128

New Zealand Python User Group

September 13, 2014
Tweet

More Decks by New Zealand Python User Group

Other Decks in Programming

Transcript

  1. Control All Your Programs With MIDI Controllers Tobi Wulff KiwiPyCon

    2014
  2. MIDI Controllers

  3. The Good: Standardised Protocol No worries about OS drivers Lots

    of connections and daisy chaining Reusability Libraries for Arduino, and more Cheap MIDI­to­USB adapters
  4. MIDI Controllers Mixer boards Pads Keyboards (Piano) Even drum kits

  5. Lighting Console www.alg­uk.com

  6. Keyboards www.openmusiclabs.com

  7. Keyboards www.flickr.com (Stephen Drake)

  8. Drums musicradar.com

  9. Drums performing­musician.com

  10. MIDI Controller Components Sliders, Buttons, Potis, Rotary Encoders, Keyboard, Trackpads,

    ... www.bauerindustries.com
  11. The Protocol and Theory Controllers generate notes and controlling messages

    Software can generate or process (play) messages Synthesisers can play MIDI messages => Keyboards are controller + synthesisers
  12. MIDI Messages www.sebkhachott.net

  13. MIDI Messages Can be sent on specific channels: 0(all) ­

    15 Data is in the range of 0­127 On/off usually done with ≤ 63 and ≥ 64 Control Change [Control, Value] Program Change Note On/Off [Note, Speed] Pitch Bend SysEx Panic! Note Off for all notes
  14. MIDI Messages midi­protocol.wikispaces.com

  15. Quick Excursion Linux Audio: basic layer (ALSA) Real­time and routing

    with Jack (jackd) Mididings can connect to both of those
  16. Audio Connections

  17. MIDI Connections

  18. The Target: Darktable

  19. The Target: Darktable Alternative: API

  20. Introducing: mididings Can receive, process and send MIDI messages das.nasophon.de

  21. A small Mididings Example f r o m m i

    d i d i n g s i m p o r t r u n , P r o c e s s , N O T E O N d e f i n v e r t ( e v e n t ) : i f e v e n t . c h a n n e l = = 1 a n d e v e n t . t y p e = = N O T E O N : e v e n t . v e l o c i t y = 1 2 7 - e v e n t . v e l o c i t y r e t u r n e v e n t r u n ( P r o c e s s ( i n v e r t ) )
  22. Domain Specific Languages "A domain­specific language (DSL) is a computer

    language specialized to a particular application domain."
  23. Mididings as a DSL p i a n o =

    O u t p u t ( ' s a m p l e r ' , 1 , 1 ) # p o r t , c h , p r g s t r i n g s = O u t p u t ( ' s y n t h ' , 2 ) # p o r t , c h , p r g C h a n n e l F i l t e r ( 1 ) > > K e y S p l i t ( ' c 3 ' , p i a n o , s t r i n g s )
  24. Mididings as a DSL F i l t e r

    ( C T R L ) > > C h a n n e l ( 2 ) F i l t e r ( N O T E O N ) > > T r a n s p o s e ( - 1 2 ) > > o r c h e s t r a / / b e l l s { C T R L : C h a n n e l ( 2 ) , N O T E O N : T r a n s p o s e ( - 1 2 ) > > [ o r c h e s t r a , b e l l s ] }
  25. Mididings as a DSL ( ( C h a n

    n e l F i l t e r ( 1 ) & ( C t r l F i l t e r ( 2 3 ) | C t r l F i l t e r ( 4 2 ) ) ) & ~ C t r l V a l u e F i l t e r ( l o w e r = 5 0 , u p p e r = 1 0 0 ) ) > > C h a n n e l ( 2 )
  26. Mididings as a DSL ( # o n l y

    u s e c h a n n e l 1 ( C h a n n e l F i l t e r ( 1 ) & # C C # 2 3 o r 4 2 ( C t r l F i l t e r ( 2 3 ) | C t r l F i l t e r ( 4 2 ) ) ) & # s l i d e r n o t b e t w . 5 0 a n d 1 0 0 ~ C t r l V a l u e F i l t e r ( l o w e r = 5 0 , u p p e r = 1 0 0 ) ) # s e n d T H O S E m e s s a g e s t o c h . 2 > > C h a n n e l ( 2 )
  27. das.nasophon.de

  28. CTRL specific functions C t r l M a p

    ( ) # c h a n g e s C C # C t r l R a n g e ( ) # c h a n g e s v a l u e C t r l C u r v e ( ) # a p p l y a c u r v e t o v a l u e s Should use those more
  29. Ok, back to controlling GUIs 1. Collect incoming MIDI messages

    (filtered) 2. Grab collected events and process 3. Turn event into key press (or something else) 4. Send off key press
  30. Collect Main process acting as a producer adding items to

    the event queue. Overwrite if same event happens again. r u n ( P r o c e s s ( c o l l e c t ) ) d e f c o l l e c t ( s e l f , m i d i ) : w i t h s e l f . _ e v e n t s _ l o c k : s e l f . _ c o l l e c t ( m i d i )
  31. Grab Second thread acting as a consumer taking items from

    the event queue. i m p o r t t h r e a d i n g w i t h s e l f . _ l o c k : s e l f . _ e v e n t s . i t e m s ( )
  32. Key Press Map MIDI message to shortcut action object: M

    A P = { ( 0 , C T R L , 1 1 ) : ( ' W i n d o w ' , S h o r t c u t A c t i o n ( N O _ M O D , ' o ' ) ) , . . . } c l a s s S h o r t c u t A c t i o n ( o b j e c t ) : d e f _ _ i n i t _ _ ( s e l f , m o d i f i e r s , k e y , f u n c = N o n e ) : . . . d e f r u n ( s e l f , e x e c u t o r , v a l u e ) : . . . c l a s s S h o r t c u t A c t i o n S e q u e n c e ( o b j e c t ) : . . .
  33. Send Will only go to foreground window (unless special global

    shortcut). f r o m s u b p r o c e s s i m p o r t P o p e n , P I P E k e y _ s e q u e n c e = [ ' k e y d o w n C o n t r o l _ L ' , ' k e y o ' , ' k e y u p C o n t r o l _ L ' , ' s t r 1 . 3 3 ' ] P o p e n ( [ ' x t e ' ] , s t d i n = P I P E ) . c o m m u n i c a t e ( i n p u t = k e y _ s e q u e n c e )
  34. The Future OSC will probably replace MIDI: mididings ­> S

    e n d O S C Extend to send to other OS (not just using xte) More flexibility for other programs (GIMP, ...) Mouse etc macros Feedback (MIDI from PC to controller)
  35. References For more theory: www.midi.org This program: github.com/tobiw/midi2gui Mididings: (part

    of most Linux package managers) das.nasophon.de Xte: xautomation.sf.net
  36. Contact Me T w i t t e r :

    t o b i a s w u l f f G i t h u b a n d I R C : t o b i w J a b b e r a n d E m a i l : t o b i ( a ) s w u l f f . d e