Myriam Desainte-Catherine, Carlos Olarte, and Camilo Rueda Laboratoire Bordelais de Recherche en Informatique (LaBRI) Université de Bordeaux 5th International Conference on Mathematics and Computation in Music (MCM 2015) United Kingdom, June 2015 1
formalism for composing and interpreting interactive multimedia scenarios. • Some applications of IS: ◦ Live performances ◦ Museum installations ◦ Plastic art installations †A. Allombert. Aspects Temporels d’un Système de Partitions Musicales Interactives pour la Composition et l’Exécution. PhD thesis, Université de Bordeaux, 2009 Arias, Desainte-Catherine, Olarte, and Rueda (LaBRI) Foundations for Reliable and Flexible Interactive Multimedia Scores 1/13 1/13
the software i-score†. • A Hierarchical Time Stream Petri Net (HTSPN) model represents and executes the partially ordered set of events. †Website: http://www.i-score.org/ Arias, Desainte-Catherine, Olarte, and Rueda (LaBRI) Foundations for Reliable and Flexible Interactive Multimedia Scores 2/13 2/13
that are not supported by the IS model†: ◦ Flexible control structures such as conditionals. ◦ Automatic verification of scenarios in order to avoid abnormal behaviors. • The specification of some behaviors could be cumbersome due to the horizontal time-line of i-score. †T. De la Hogue, P. Baltazar, M. Desainte-Catherine, J. Chao, and C. Bossut. OSSIA : Open Scenario System for Interactive Applications. In Journées d’Informatique Musicale, pages 78–84, Bourges, 2014 Arias, Desainte-Catherine, Olarte, and Rueda (LaBRI) Foundations for Reliable and Flexible Interactive Multimedia Scores 3/13 3/13
the specification and execution of interactive multimedia scenarios endowed with a tree-based operational semantics and a declarative interpretation in intuitionistic linear logic. Arias, Desainte-Catherine, Olarte, and Rueda (LaBRI) Foundations for Reliable and Flexible Interactive Multimedia Scores 4/13 4/13
WaitFromEnd(A,5,5) ∧ WaitFromEnd(B,3,3) WaitFromEnd(C.D,0,∞) ∧ WaitFromEnd(C.E,0,∞) Conditionals C start: stop: WaitFromStart(C,1,1) WaitFromStart(C.E,2,2) Conditionals Messages start: stop: /sound/2 on /sound/2 off E B A D Arias, Desainte-Catherine, Olarte, and Rueda (LaBRI) Foundations for Reliable and Flexible Interactive Multimedia Scores 7/13 7/13
3 5 Times B start: stop: 1 3 Times A Arias, Desainte-Catherine, Olarte, and Rueda (LaBRI) Foundations for Reliable and Flexible Interactive Multimedia Scores 8/13 8/13
3 5 Times B start: stop: 1 3 Times A start: stop: 8 ⊥ Times C start(S,C,8) Arias, Desainte-Catherine, Olarte, and Rueda (LaBRI) Foundations for Reliable and Flexible Interactive Multimedia Scores 8/13 8/13
3 5 Times B start: stop: 1 3 Times A start: stop: 8 14 Times C stop(S,C,14) Arias, Desainte-Catherine, Olarte, and Rueda (LaBRI) Foundations for Reliable and Flexible Interactive Multimedia Scores 8/13 8/13
I, t⟩ |= cs(n) ⟨St, O⟩I,t S −→P ⟨start(St, p, t), O ∪ {ms(n)}⟩I,t S where n = targetP(p) RSTOP p ∈ canStop(S) ⟨P, S, I, t⟩ |= ce(n) ⟨St, O⟩I,t S −→P ⟨stop(St, p, t), O ∪ {me(n)}⟩I,t S where n = targetP(p) RTIME ⟨S, ∅⟩I,t S −→∗ P ⟨S′, O⟩I,t S ̸ −→P ⟨S, t⟩ I,O =⇒P ⟨S′, t + 1⟩ Arias, Desainte-Catherine, Olarte, and Rueda (LaBRI) Foundations for Reliable and Flexible Interactive Multimedia Scores 9/13 9/13
I, t⟩ |= cs(n) ⟨St, O⟩I,t S −→P ⟨start(St, p, t), O ∪ {ms(n)}⟩I,t S where n = targetP(p) palive(S) = {p | p ∈ L(S) ∧ te(targetS (p)) = ⊥} 0 ⊥ 8 ⊥ 3 5 1 3 C A B palive (S) Arias, Desainte-Catherine, Olarte, and Rueda (LaBRI) Foundations for Reliable and Flexible Interactive Multimedia Scores 9/13 9/13
I, t⟩ |= cs(n) ⟨St, O⟩I,t S −→P ⟨start(St, p, t), O ∪ {ms(n)}⟩I,t S where n = targetP(p) palive(S) = {p | p ∈ L(S) ∧ te(targetS (p)) = ⊥} canStart(P, S) △ = {p | pparent ∈ palive(S) ∧ p ∈ Children(pparent )} \ L(S) C A B E D canStart(P, S) 0 ⊥ 8 ⊥ 3 5 1 3 C A B palive (S) Arias, Desainte-Catherine, Olarte, and Rueda (LaBRI) Foundations for Reliable and Flexible Interactive Multimedia Scores 9/13 9/13
I, t⟩ |= cs(n) ⟨St, O⟩I,t S −→P ⟨start(St, p, t), O ∪ {ms(n)}⟩I,t S where n = targetP(p) cs (targetP (C.E)) = WaitFromStart(C, 1, 1) ⟨P, S, I, t⟩ | = WaitFromStart(p, t1, t2) iff ∃n · n ∈ V (S) ∧ n = targetS (p) ∧ t1 ≤ t − ts (n) ≤ t2 0 ⊥ 8 ⊥ 3 5 1 3 C A B t = 9 Arias, Desainte-Catherine, Olarte, and Rueda (LaBRI) Foundations for Reliable and Flexible Interactive Multimedia Scores 9/13 9/13
I, t⟩ |= cs(n) ⟨St, O⟩I,t S −→P ⟨start(St, p, t), O ∪ {ms(n)}⟩I,t S where n = targetP(p) RSTOP p ∈ canStop(S) ⟨P, S, I, t⟩ |= ce(n) ⟨St, O⟩I,t S −→P ⟨stop(St, p, t), O ∪ {me(n)}⟩I,t S where n = targetP(p) RTIME ⟨S, ∅⟩I,t S −→∗ P ⟨S′, O⟩I,t S ̸ −→P ⟨S, t⟩ I,O =⇒P ⟨S′, t + 1⟩ Arias, Desainte-Catherine, Olarte, and Rueda (LaBRI) Foundations for Reliable and Flexible Interactive Multimedia Scores 9/13 9/13
Formulas are seen as resources, e.g, c ⊗ c ̸−→ c. Subexponentials: meanings for !aF • “F holds in location a” (e.g., in a given structure) • “F holds with certain modality a” (e.g., in a time-unit t). • Subexp. are ordered in a poset. • The ordering determines the provability relation, e.g.,: !aF −→!bF iff b ⪯ a • We then have a better control of resources in proofs. 0+ 0 1+ 1 2+ 2 3+ 3 … where t is defined as t TI t.i t.o t.s.scenario t.s.A t.s.B t.s.C t.s.D t.s.E t.p TP †J. Girard. Linear logic. Theor. Comput. Sci., 50:1–102, 1987 Arias, Desainte-Catherine, Olarte, and Rueda (LaBRI) Foundations for Reliable and Flexible Interactive Multimedia Scores 10/13 10/13
encoding from ReactiveIS programs into SELL formulas. Adequacy P reaches a state s iff the sequent [[P]] −→ [[s]] is provable in SELL. This opens the possibility of reasoning about IS by using well established techniques in proof theory! We can verify properties such as: • Is it possible for the structure A and B to be played concurrently ? • Is the execution of A always preceded by an execution of B ? • Will all the structures be eventually played ? Arias, Desainte-Catherine, Olarte, and Rueda (LaBRI) Foundations for Reliable and Flexible Interactive Multimedia Scores 11/13 11/13
for the specification, verification and interpretation of interactive scores. • ReactiveIS extends the full capacity of temporal organization of IS and also it allows the specification of conditionals. • The operational semantics based on labelled trees are simpler and more intuitive than the current model in HTSPN of i-score. • We presented a declarative interpretation of ReactiveIS programs as formulas in ILL with subexponentials that is adequate. Arias, Desainte-Catherine, Olarte, and Rueda (LaBRI) Foundations for Reliable and Flexible Interactive Multimedia Scores 12/13 12/13
of the semantics. • Front-end for the specification of properties. • Extension for loops. Arias, Desainte-Catherine, Olarte, and Rueda (LaBRI) Foundations for Reliable and Flexible Interactive Multimedia Scores 13/13 13/13
Myriam Desainte-Catherine, Carlos Olarte, and Camilo Rueda Laboratoire Bordelais de Recherche en Informatique (LaBRI) Université de Bordeaux 5th International Conference on Mathematics and Computation in Music (MCM 2015) United Kingdom, June 2015 1
CONCUR 2013, Buenos Aires, Argentina, August 27-30, 2013. Proceedings, volume 8052 of Lecture Notes in Computer Science, 2013. Springer. ISBN 978-3-642-40183-1. A. Allombert. Aspects Temporels d’un Système de Partitions Musicales Interactives pour la Composition et l’Exécution. PhD thesis, Université de Bordeaux, 2009. A. Allombert, R. Marczak, M. Desainte-Catherine, P. Baltazar, and GarnierLaurent. Virage : Designing An Interactive Intermedia Sequencer From Users Requirements And Theoretical Background. In International Computer Music Conference, 2010. J.-M. Andreoli. Logic programming with focusing proofs in linear logic. J. Log. Comput., 2(3):297–347, 1992.
Kreowski, S. Kuske, D. Plump, A. Schürr, and G. Taentzer. Graph transformation for specification and programming. Sci. Comput. Program., 34(1):1–54, 1999. doi: 10.1016/S0167-6423(98)00023-9. URL http://dx.doi.org/10.1016/S0167-6423(98)00023-9. J. Arias, M. Desainte-Catherine, C. Olarte, and C. Rueda. Foundations for reliable and flexible interactive multimedia scores. Technical report, Labri, University of Bordeaux, Mar. 2015. V. Danos, J. Joinet, and H. Schellinx. The structure of exponentials: Uncovering the dynamics of linear logic proofs. In Gottlob et al. [1993], pages 159–171. ISBN 3-540-57184-1. doi: 10.1007/BFb0022564. URL http://dx.doi.org/10.1007/BFb0022564.
Chao, and C. Bossut. OSSIA : Open Scenario System for Interactive Applications. In Journées d’Informatique Musicale, pages 78–84, Bourges, 2014. M. Desainte-Catherine, A. Allombert, and G. Assayag. Towards a hybrid temporal paradigm for musical composition and performance: The case of musical interpretation. Computer Music Journal, 37(2):61–72, 2013. doi: 10.1162/COMJ_a_00179. URL http://dx.doi.org/10.1162/COMJ_a_00179. J. Girard. Linear logic. Theor. Comput. Sci., 50:1–102, 1987. G. Gottlob, A. Leitsch, and D. Mundici, editors. Computational Logic and Proof Theory, Third Kurt Gödel Colloquium, KGC’93, Brno, Czech Republic, August 24-27, 1993, Proceedings, volume 713 of Lecture Notes in Computer Science, 1993. Springer. ISBN 3-540-57184-1.
semantics of an orchestration language. In Engineering Theories of Software Intensive Systems, volume 195 of NATO Science Series II: Mathematics, Physics and Chemistry, pages 331–350. Springer, 2005. ISBN 978-1-4020-3530-2. doi: 10.1007/1-4020-3532-2_11. R. Marczak, M. Desainte-Catherine, and A. Allombert. Real-time temporal control of musical processes. In The Third International Conferences on Advances in Multimedia, MMEDIA 2011, pages 12–17, 2011. ISBN 978-1-61208-129-8. G. D. Michelis and M. Diaz, editors. Application and Theory of Petri Nets 1995, 16th International Conference, Turin, Italy, June 26-30, 1995, Proceedings, volume 935 of Lecture Notes in Computer Science, 1995. Springer. ISBN 3-540-60029-9.
proof system for modalities in concurrent constraint programming. In CONCUR DBL [2013], pages 410–424. ISBN 978-3-642-40183-1. C. Olarte and C. Rueda. A Declarative Language for Dynamic Multimedia Interaction Systems. In E. Chew, A. Childs, and C.-H. Chuan, editors, Mathematics and Computation in Music, volume 38 of Communications in Computer and Information Science, pages 218–227. Springer Berlin Heidelberg, Berlin, Heidelberg, 2009. ISBN 978-3-642-02393-4. doi: 10.1007/978-3-642-02394-1. URL http://www.springerlink.com/index/10.1007/ 978-3-642-02394-1http: //link.springer.com/10.1007/978-3-642-02394-1.
time stream petri net: A model for hypermedia systems. In Michelis and Diaz [1995], pages 451–470. ISBN 3-540-60029-9. doi: 10.1007/3-540-60029-9_54. URL http://dx.doi.org/10.1007/3-540-60029-9_54. M. Toro, M. Desainte-Catherine, and C. Rueda. Formal semantics for interactive music scores: a framework to design, specify properties and execute interactive scenarios. Journal of Mathematics and Music, 8 (1):93–112, 2014. doi: 10.1080/17459737.2013.870610. D. A. Tran, K. A. Hua, and K. Vu. Videograph: A graphical object-based model for representing and querying video data. In ER, pages 383–396, 2000. doi: 10.1007/3-540-45393-8_28. URL http://dx.doi.org/10.1007/3-540-45393-8_28.