A tree-based Operational Semantics for Interactive Multimedia Scores

Transcript

1. A Tree-Based Operational Semantics for Interactive Multimedia Scores Jaime ARIAS,

   Myriam DESAINTE-CATHERINE, Carlos OLARTE, and Camilo RUEDA Seminars on Formal Methods and Languages Inria Grenoble - Rhône-Alpes October 2017

2. Static Score Play 2 =

3. Interactive Scores 3 Les Baltazars. Nocturne nº2 - Tumbleweed During

   the performance of a scenario, the performer has the possibility to influence its execution by triggering interactive points (IPs). IPs are defined by the composer during the composition and allow for agogic modifications (i.e., the possibility to change the start and stop times of TOs during execution ). In this sense, a performance constitutes an instance of a finite set of possible scenarios that share the same temporal properties.

4. Interactive Scores 4 i-score - https://ossia.io

5. Interactive Scores 5 f f f f T D F

   S

6. Interactive Scores 6 The underlying model of i-score does not

   support: 1. Flexible control structures (e.g., conditionals, loops) 2. Mechanisms for the automatic verification Several researchers have made many efforts to overcome these limitations: • Petri nets: Allombert 2009 • Process calculi: Olarte 2009, Toro 2014 but there are no practical solutions for the automatic verification and real-time performance, and the models cannot be straightforwardly implemented or extended.

7. Outline 1. ReactiveML 1.1. Syntax 1.2. Tree Representation 1.3. Operational

   Semantics 1.4. Logical Characterization 2. Concluding Remarks 7

8. ReactiveIS 8 https://github.com/himito/reactiveIS

9. Syntax 9

10. Syntax 10

11. Conditions 11 WaitFromStart(WH,1,1)

12. Program (Score) Tree 12

13. Program (Score) Tree 13

14. State Tree 14

15. Tree Homomorphism 15

16. State Tree Operation: Start 16

17. State Tree Operation: Stop 17

18. Operational Semantics 18 P T S T C I C

    O C S H

19. Operational Semantics 19

20. Operational Semantics 20

21. Operational Semantics 21

22. Operational Semantics 22

23. Operational Semantics 23

24. Properties of the SOS 24

25. Logical Characterization 25 L L S (SELL)

26. Logical Characterization This opens the possibility of reasoning about interactive

    scores by using well established techniques in proof theory. We can verify, for example, if all the structures will be eventually played. 26

27. Concluding Remarks 27

28. Summary • New programming language for the specification, verification and

    interpretation of interactive scores. • ReactiveIS extends the full capacity of temporal organization of i-score. • It is possible to extend the language for the specification of conditionals and loops. • The operational semantics based on labelled trees are simpler and more intuitive than the current model in HTSPN of i-score. • The declarative interpretation of ReactiveIS programs as formulas in SELL is adequate. 28

29. Future Work • Coq or Maude for proving the correctness

    of the semantics. • Automatic checking of bound times of execution. • Front-end for the specification of properties. • Extension for conditional and loops. 29

30. Thank you for your attention 30

31. References 1. Arias, J., Desainte-Catherine, M., Olarte, C., & Rueda,

    C. (2015). Foundations for Reliable and Flexible Interactive Multimedia Scores. In T. Collins, D. Meredith, & A. Volk (Eds.), 5th International Conference on Mathematics and Computation in Music, MCM 2015, London, UK, June 22-25, 2015 (Vol. 9110, pp. 29–41). Springer. https://doi.org/10.1007/978-3-319-20603-5_3 2. Allombert, A. (2009). Aspects Temporels d’un Système de Partitions Musicales Interactives pour la Composition et l’Exécution. Université de Bordeaux. Retrieved from http://ori-oai.u-bordeaux1.fr/pdf/2009/ALLOMBERT_ANTOINE_2009.pdf 3. Olarte, C., & Rueda, C. (2009). A Declarative Language for Dynamic Multimedia Interaction Systems. In E. Chew, A. Childs, & C.-H. Chuan (Eds.), 2nd International Conference on Mathematics and Computation in Music, MCM 2009, New Haven, CT, USA, June 19-22, 2009 (Vol. 38, pp. 218–227). Springer Berlin Heidelberg. https://doi.org/10.1007/978-3-642-02394-1_20 4. Toro, M., Desainte-Catherine, M., & Rueda, C. (2014). Formal semantics for interactive music scores: a framework to design, specify properties and execute interactive scenarios. Journal of Mathematics and Music, 8(1), 93–112. https://doi.org/10.1080/17459737.2013.870610 31