A tree-based Operational Semantics for Interactive Multimedia Scores

Talk presented at the seminar on formal methods and languages.

Transcript

1. A Tree-Based Operational Semantics for Interactive Multimedia Scores Jaime ARIAS, Myriam DESAINTE-CATHERINE, Carlos OLARTE, and Camilo RUEDA Seminars on Formal Methods and Languages Inria Grenoble - Rhône-Alpes October 2017
2. Static Score Play 2 =
3. Interactive Scores 3 Les Baltazars. Nocturne nº2 - Tumbleweed During the performance of a scenario, the performer has the possibility to influence its execution by triggering interactive points (IPs). IPs are defined by the composer during the composition and allow for agogic modifications (i.e., the possibility to change the start and stop times of TOs during execution ). In this sense, a performance constitutes an instance of a finite set of possible scenarios that share the same temporal properties.
4. Interactive Scores 4 i-score - https://ossia.io
5. Interactive Scores 5 f f f f T D F S
6. Interactive Scores 6 The underlying model of i-score does not support: 1. Flexible control structures (e.g., conditionals, loops) 2. Mechanisms for the automatic verification Several researchers have made many efforts to overcome these limitations: • Petri nets: Allombert 2009 • Process calculi: Olarte 2009, Toro 2014 but there are no practical solutions for the automatic verification and real-time performance, and the models cannot be straightforwardly implemented or extended.
7. Outline 1. ReactiveML 1.1. Syntax 1.2. Tree Representation 1.3. Operational Semantics 1.4. Logical Characterization 2. Concluding Remarks 7
8. ReactiveIS 8 https://github.com/himito/reactiveIS
9. Syntax 9
10. Syntax 10
11. Conditions 11 WaitFromStart(WH,1,1)
12. Program (Score) Tree 12
13. Program (Score) Tree 13
14. State Tree 14
15. Tree Homomorphism 15
16. State Tree Operation: Start 16
17. State Tree Operation: Stop 17
18. Operational Semantics 18 P T S T C I C O C S H
19. Operational Semantics 19
20. Operational Semantics 20
21. Operational Semantics 21
22. Operational Semantics 22
23. Operational Semantics 23
24. Properties of the SOS 24
25. Logical Characterization 25 L L S (SELL)
26. Logical Characterization This opens the possibility of reasoning about interactive scores by using well established techniques in proof theory. We can verify, for example, if all the structures will be eventually played. 26
27. Concluding Remarks 27
28. Summary • New programming language for the specification, verification and interpretation of interactive scores. • ReactiveIS extends the full capacity of temporal organization of i-score. • It is possible to extend the language for the specification of conditionals and loops. • The operational semantics based on labelled trees are simpler and more intuitive than the current model in HTSPN of i-score. • The declarative interpretation of ReactiveIS programs as formulas in SELL is adequate. 28
29. Future Work • Coq or Maude for proving the correctness of the semantics. • Automatic checking of bound times of execution. • Front-end for the specification of properties. • Extension for conditional and loops. 29
30. Thank you for your attention 30
31. References 1. Arias, J., Desainte-Catherine, M., Olarte, C., & Rueda, C. (2015). Foundations for Reliable and Flexible Interactive Multimedia Scores. In T. Collins, D. Meredith, & A. Volk (Eds.), 5th International Conference on Mathematics and Computation in Music, MCM 2015, London, UK, June 22-25, 2015 (Vol. 9110, pp. 29–41). Springer. https://doi.org/10.1007/978-3-319-20603-5_3 2. Allombert, A. (2009). Aspects Temporels d’un Système de Partitions Musicales Interactives pour la Composition et l’Exécution. Université de Bordeaux. Retrieved from http://ori-oai.u-bordeaux1.fr/pdf/2009/ALLOMBERT_ANTOINE_2009.pdf 3. Olarte, C., & Rueda, C. (2009). A Declarative Language for Dynamic Multimedia Interaction Systems. In E. Chew, A. Childs, & C.-H. Chuan (Eds.), 2nd International Conference on Mathematics and Computation in Music, MCM 2009, New Haven, CT, USA, June 19-22, 2009 (Vol. 38, pp. 218–227). Springer Berlin Heidelberg. https://doi.org/10.1007/978-3-642-02394-1_20 4. Toro, M., Desainte-Catherine, M., & Rueda, C. (2014). Formal semantics for interactive music scores: a framework to design, specify properties and execute interactive scenarios. Journal of Mathematics and Music, 8(1), 93–112. https://doi.org/10.1080/17459737.2013.870610 31