Automatic Construction of Interactive Machine Improvisation Scenarios from Audio Recordings

Transcript

1. Automatic Construction of Interactive Machine Improvisation Scenarios from Audio Recordings Jaime Arias, Myriam Desainte-Catherine and Shlomo Dubnov Université de Bordeaux, LaBRI, UMR 5800 Inria - Bordeaux Sud-Ouest Les Cycles Arts et Sciences du SCRIME June 24, 2016
2. Motivation • PyOracle: https://gitlab.com/himito/PyOracle_I-score • i-score: https://github.com/himito/i-score • VMO-Score: https://himito.github.io/vmo_i-score_generator Shlomo Dubnov - University of California, San Diego - CREL (2016) Automatic Construction of Interactive Machine Improvisation Scenarios from Audio Recordings 1/16 1/16
3. Motivation Overview of the system Pre-Recorded Audio Variable Markov Oracle (VMO) Audio Segmentation Audio Oracle SNAKES Petri Net Model input output output input output Configuration File output (a) Generation of the structure Petri Net Model Offline Improviser output input input Audio Synthesizer Audio File output input Audio Oracle input Petri Net Parameters Actions Oracle Parameters Performer Controls Configuration File Oracle Regions Sequence Audio Buffer input (b) Offline improvisation Figure: Overview Shlomo Dubnov - University of California, San Diego - CREL (2016) Automatic Construction of Interactive Machine Improvisation Scenarios from Audio Recordings 2/16 2/16
4. Motivation Overview of the system Pre-Recorded Audio Variable Markov Oracle (VMO) Audio Segmentation Audio Oracle SNAKES Petri Net Model input output output input output Configuration File output (a) Generation of the structure Petri Net Model Audio Oracle i-score Petri Net Parameters Configuration File PyOracle input input input Real-time Performer Controls input input inout Real-time Audio Output Output (b) Real-time improvisation Figure: Overview Shlomo Dubnov - University of California, San Diego - CREL (2016) Automatic Construction of Interactive Machine Improvisation Scenarios from Audio Recordings 3/16 3/16
5. Composition
6. Composition Audio Recording Figure: Audio recording: Philou. Shlomo Dubnov - University of California, San Diego - CREL (2016) Automatic Construction of Interactive Machine Improvisation Scenarios from Audio Recordings 4/16 4/16
7. Composition Segmentation Figure: Segmentation Shlomo Dubnov - University of California, San Diego - CREL (2016) Automatic Construction of Interactive Machine Improvisation Scenarios from Audio Recordings 5/16 5/16
8. Composition Petri Net G D C t (a) The marking before firing the enabled transition t. G D C t (b) The marking after firing transition t, where t is disabled. Figure: Illustration of a firing rule in a Petri Net. Shlomo Dubnov - University of California, San Diego - CREL (2016) Automatic Construction of Interactive Machine Improvisation Scenarios from Audio Recordings 6/16 6/16
9. Composition Petri Net Figure: Segmentation end init t1 t0 t2 t3 t4 t5 t6 t7 t8 t9 t10 t11 Figure: Timed Petri Net Shlomo Dubnov - University of California, San Diego - CREL (2016) Automatic Construction of Interactive Machine Improvisation Scenarios from Audio Recordings 7/16 7/16
10. Composition Parameters of the Petri Net 1 # file: configuration.yml 2 3 conditions: 4 - transition : 't0' 5 time-min : 0.0 6 time-max : 3.0 7 condition : '/device/key == 10' 8 9 - transition : 't1' 10 ... init t0 [0.0, 3.0] /device/key == 10 Shlomo Dubnov - University of California, San Diego - CREL (2016) Automatic Construction of Interactive Machine Improvisation Scenarios from Audio Recordings 8/16 8/16
11. Offline Improvisation
12. Offline Improvisation Specification of actions 1 # file: configuration.yml 2 3 actions: 4 - address : '/volume/sensor/pos_x' 5 value : 10 6 time : 250 7 8 - address : ... Shlomo Dubnov - University of California, San Diego - CREL (2016) Automatic Construction of Interactive Machine Improvisation Scenarios from Audio Recordings 9/16 9/16
13. Offline Improvisation Adding environment init t0 /device/key == 10 [0.0, 3.0] Shlomo Dubnov - University of California, San Diego - CREL (2016) Automatic Construction of Interactive Machine Improvisation Scenarios from Audio Recordings 10/16 10/16
14. Offline Improvisation Adding environment init env t0 (a,v,t) /device/key == 10 [0.0, 3.0] Shlomo Dubnov - University of California, San Diego - CREL (2016) Automatic Construction of Interactive Machine Improvisation Scenarios from Audio Recordings 11/16 11/16
15. Offline Improvisation Adding environment init env t0 (a,v,t) (a == /device/key) ∧ (v == 10) ∧ (t == get_time()) [0.0, 3.0] Shlomo Dubnov - University of California, San Diego - CREL (2016) Automatic Construction of Interactive Machine Improvisation Scenarios from Audio Recordings 12/16 12/16
16. Offline Improvisation Adding environment init env t0 (a,v,t) te (a,v,t) get_time() > t (a == /device/key) ∧ (v == 10) ∧ (t == get_time()) [0.0, 3.0] Shlomo Dubnov - University of California, San Diego - CREL (2016) Automatic Construction of Interactive Machine Improvisation Scenarios from Audio Recordings 13/16 13/16
17. Real-time Improvisation
18. Real-time Improvisation The inter-media sequencer i-score score / Initial Transition T10 T6 T5 T8 T4 T2 T1 T9 T7 T11 Popolari61oreweed40 Loop pattern snozzle24ambier35 Loop pattern pronger64inurbane46 Loop pattern spouty57carthame37 Loop pattern T5 T6 T10 T1 T2 T4 T8 T11 T7 T9 T3 scissel86thermo23 Loop pattern T3 (a) i-score end init t1 t0 t2 t3 t4 t5 t6 t7 t8 t9 t10 t11 (b) Petri net Figure: i-score representation. Shlomo Dubnov - University of California, San Diego - CREL (2016) Automatic Construction of Interactive Machine Improvisation Scenarios from Audio Recordings 14/16 14/16
19. Real-time Improvisation The inter-media sequencer i-score score / Initial Transition T10 T6 T5 T8 T4 T2 T1 Popolari61oreweed40 Loop pattern snozzle24ambier35 Loop pattern pronger64inurbane46 Loop pattern T5 T6 T10 T1 T2 T4 T8 (a) i-score end init t1 t0 t2 t3 t4 t5 t6 t7 t8 t9 t10 t11 (b) Petri net Figure: i-score representation. Shlomo Dubnov - University of California, San Diego - CREL (2016) Automatic Construction of Interactive Machine Improvisation Scenarios from Audio Recordings 15/16 15/16
20. Concluding Remarks
21. Concluding Remarks Shlomo Dubnov - University of California, San Diego - CREL (2016) Automatic Construction of Interactive Machine Improvisation Scenarios from Audio Recordings 16/16 16/16
22. Thank you for your attention!
23. Automatic Construction of Interactive Machine Improvisation Scenarios from Audio Recordings Jaime Arias, Myriam Desainte-Catherine and Shlomo Dubnov Université de Bordeaux, LaBRI, UMR 5800 Inria - Bordeaux Sud-Ouest Les Cycles Arts et Sciences du SCRIME June 24, 2016