Music Theory and Performance Analysis with Sebastian and Czerny

This talk will discuss two open source projects for using Python for music analysis. Sebastian focuses on music theory while Czerny focuses on performance (particularly keyboard playing).

Transcript

1. Music Theory and
   Performance Analysis with
   Sebastian and Czerny
   James Tauber
   @jtauber
   

   View Slide

2. The Idea
   The Components
   The Score
   The Performance
   The Recording
   Lilypond
   MIDI
   WAV
   Sebastian
   Czerny
   

   View Slide

3. Sebastian
   

   View Slide

4. Mathematical:
   Symbolic more than
   Numerical
   

   View Slide

5. repetitive
   hierarchical
   sequential + parallel
   factorable
   underspecified
   Modeling Music
   

   View Slide

6. Point of Departure
   [
   {PITCH: 3, OCTAVE: 4, DURATION: 16},
   {PITCH: 2, OCTAVE: 4, DURATION: 8}
   ]
   

   View Slide

7. Three Blind Mice
   mice =
   (seq1 * 2) +
   (seq2 + seq2a) +
   (seq2 + seq2b) +
   ((seq3 + seq3a) * 2) +
   (seq3 + seq3b) +
   seq1
   

   View Slide

8. Three Types of Sequence
   HSeq — Horizontal / Sequential
   VSeq — Vertical / Parallel
   OSeq — Offset-based
   

   View Slide

9. HSeq
   each element follows the previous
   concatenate(next_seq)
   repeat(count)
   __add__
   __mul__
   A = B * 2 + C
   

   View Slide

10. VSeq
    each element is coincident
    merge(parallel_seq)
    __floordiv__
    A = B // C
    

    View Slide

11. OSeq
    factory function that returns a class
    using the given offset attribute and
    duration attribute
    def OSeq(offset_attr,
    duration_attr)
    has operators of both HSeq and VSeq
    

    View Slide

12. All Sequences
    __init__(*elements)
    __getitem__ & __len__
    __eq__ & __ne__
    transform(func)
    __or__
    A = B | transpose(5) | reverse()
    

    View Slide

13. Three Types of Sequence
    HSeq — Horizontal / Sequential
    VSeq — Vertical / Parallel
    OSeq — Offset-based
    

    View Slide

14. Points
    handling underspecification with
    dictionaries
    unify(other)
    __mod__
    {PITCH: 3, OCTAVE: 4} % {DURATION: 16}
    

    View Slide

15. Pitch
    note names
    two numbers or one?
    MIDI pitch
    Hewlett’s base-40 system
    my own system
    just have transforms for conversion
    

    View Slide

16. Tauber’s Pitch System
    D is 0 and each successor is a fifth higher
    modulo octave (e.g. A is 1, G is -1)
    semitone above is x-5
    augment (semitone above, same letter) is x+7
    modifiers:
    ((val+3)-((val+3)%7))/7
    major scale:
    [tonic + i for i in
    [0, 2, 4, -1, 1, 3, 5]]
    

    View Slide

17. Lilypond
    from sebastian.lilypond.interp import parse
    seq1 = parse("e4. d c r")
    seq2 = parse("g4. f4 f8 e4.")
    seq2a = parse("r4.")
    seq2b = parse("r4 g8")
    seq3 = parse("c'4 c'8 b a b c'4 g8 g4")
    seq3a = parse("g8")
    seq3b = parse("f8")
    

    View Slide

18. Lilypond
    seq1 = parse("e4. d c r")
    [{'midi_pitch': 52, 'offset_64': 0,
    'duration_64': 24}, {'midi_pitch': 50,
    'offset_64': 24, 'duration_64': 24},
    {'midi_pitch': 48, 'offset_64': 48,
    'duration_64': 24}, {'offset_64': 96}]
    

    View Slide

19. MIDI
    library for parsing and generating MIDI files
    from sebastian.midi import write_midi
    write_midi.write("mice.mid", [mice])
    

    View Slide

20. Alberti
    common classical figuration of a triad
    a function from a triad to a sequence of
    notes
    great example of factorization and
    underspecification
    

    View Slide

21. Alberti
    def alberti(triad):
    return HSeq([
    triad[0], triad[2], triad[1], triad[2]
    ])
    

    View Slide

22. Alberti
    root_triad = VSeq(
    Point(degree=1), Point(degree=3),
    Point(degree=5))
    quaver = Point({DURATION_64: 8})
    alberti_seq =
    OSequence(
    alberti(root_triad) * 16 | add(quaver)
    )
    

    View Slide

23. Alberti
    C_major = Key("C", major_scale)
    seq = alberti_seq |
    degree_in_key_with_octave(C_major, 5) |
    midi_pitch()
    write_midi.write("alberti.mid", [seq])
    

    View Slide

24. What’s Next?
    a lot more examples
    better use of MIDI input
    control over instruments in MIDI
    output
    nesting of sequences (for hierarchy)
    the notion of a “head” point
    

    View Slide

25. Sebastian
    

    View Slide

26. github.com/
    jtauber/
    sebastian
    

    View Slide

27. Czerny
    

    View Slide

28. What’s the Difference
    between the “Score” and
    the “Performance”
    

    View Slide

29. Positive vs Negative
    Differences
    study expression the performer is
    adding
    what errors were made? what
    inconsistencies or unevenness?
    

    View Slide

30. Representing the
    Performance
    a sequence of events
    MIDI?
    

    View Slide

31. Representing the Score
    just another sequence of events
    but this time desired events
    and with less information
    

    View Slide

32. C major scale
    ascending then
    descending over one
    octave
    represent as sequence
    of note + duration
    60 8
    62 4
    64 4
    65 4
    67 4
    69 4
    71 4
    72 8
    71 4
    69 4
    67 4
    65 4
    64 4
    62 4
    60 8
    

    View Slide

33. MIDI Performance
    4d54 6864 0000 0006 0000 0001 01e0 4d54
    726b 0000 00cd 00ff 0308 2364 6566 6175
    6c74 00ff 0413 5374 6569 6e77 6179 2050
    6961 6e6f 2048 616c 6c00 ff58 0404 0218
    0800 ff59 0200 0000 ff54 0521 0000 0000
    00ff 5103 07a1 2089 6d90 3c4f 832b 903e
    3a2c 903c 0081 3290 3e00 0390 403f 8137
    9040 001f 9041 4181 3e90 4100 0790 4344
    812d 9043 002a 9045 4d81 2090 4500 2690
    473b 814b 9047 0018 9048 5583 2690 4800
    0b90 473b 8148 9047 0006 9045 3b81 5990
    433d 0490 4500 8132 9043 0022 9041 3981
    2c90 4100 2090 4038 810e 9040 0035 903e
    3081 1490 3e00 4190 3c42 831c 903c 0000
    ff2f 00
    

    View Slide

34. Performance Events
    offset in “ticks”
    MIDI pitch
    velocity
    0 velocity indicates
    note off
    1261 60 79
    1688 62 58
    1732 60 0
    1910 62 0
    1913 64 63
    2096 64 0
    2127 65 65
    2317 65 0
    2324 67 68
    2497 67 0
    2539 69 77
    2699 69 0
    2737 71 59
    2940 71 0
    2964 72 85
    3386 72 0
    ...
    

    View Slide

35. Performance Events
    convert from
    note start + note end
    to just
    start + duration
    1261 60 79 471
    1688 62 58 222
    1913 64 63 183
    2127 65 65 190
    2324 67 68 173
    2539 69 77 160
    2737 71 59 203
    2964 72 85 422
    3397 71 59 200
    3603 69 59 221
    3820 67 61 182
    4036 65 57 172
    4240 64 56 142
    4435 62 48 148
    4648 60 66 412
    

    View Slide

36. We Want to Diff
    1261 60 79 471
    1688 62 58 222
    1913 64 63 183
    2127 65 65 190
    2324 67 68 173
    2539 69 77 160
    2737 71 59 203
    2964 72 85 422
    3397 71 59 200
    3603 69 59 221
    3820 67 61 182
    4036 65 57 172
    4240 64 56 142
    4435 62 48 148
    4648 60 66 412
    60 8
    62 4
    64 4
    65 4
    67 4
    69 4
    71 4
    72 8
    71 4
    69 4
    67 4
    65 4
    64 4
    62 4
    60 8
    vs
    

    View Slide

37. Want to Diff
    to diff, you must first align
    alignment is a dynamic programming
    problem generally relying on notion of
    scoring the relative cost of adding,
    deletion or changing items
    the “change” score can be very domain
    specific
    Needleman-Wunsch algorithm
    

    View Slide

38. “Change” Score
    just consider note value: +1 if same, 0 if
    different
    or: +1 if same, +0.5 if a tone or semi-
    tone apart, 0 if more
    duration?
    wrong octave?
    

    View Slide

39. Result
    ((60, 8), (1261, 60, 79, 471))
    ((62, 4), (1688, 62, 58, 222))
    ((64, 4), (1913, 64, 63, 183))
    ((65, 4), (2127, 65, 65, 190))
    ((67, 4), (2324, 67, 68, 173))
    ((69, 4), (2539, 69, 77, 160))
    ((71, 4), (2737, 71, 59, 203))
    ((72, 8), (2964, 72, 85, 422))
    ((71, 4), (3397, 71, 59, 200))
    ((69, 4), (3603, 69, 59, 221))
    ((67, 4), (3820, 67, 61, 182))
    ((65, 4), (4036, 65, 57, 172))
    ((64, 4), (4240, 64, 56, 142))
    ((62, 4), (4435, 62, 48, 148))
    ((60, 4), (4648, 60, 66, 412))
    

    View Slide

40. Visualizing Mistakes
    

    View Slide

41. Czerny
    

    View Slide

42. github.com/
    jtauber/
    czerny
    

    View Slide

43. Music Theory and
    Performance Analysis with
    Sebastian and Czerny
    James Tauber
    @jtauber
    

    View Slide