A tree-based Operational Semantics for Interactive Multimedia Scores

Slide 1

Slide 1 text

A Tree-Based Operational Semantics for Interactive Multimedia Scores Jaime ARIAS, Myriam DESAINTE-CATHERINE, Carlos OLARTE, and Camilo RUEDA Seminars on Formal Methods and Languages Inria Grenoble - Rhône-Alpes October 2017

Slide 2

Slide 2 text

Static Score Play 2 =

Slide 3

Slide 3 text

Interactive Scores 3 Les Baltazars. Nocturne nº2 - Tumbleweed During the performance of a scenario, the performer has the possibility to influence its execution by triggering interactive points (IPs). IPs are defined by the composer during the composition and allow for agogic modifications (i.e., the possibility to change the start and stop times of TOs during execution ). In this sense, a performance constitutes an instance of a finite set of possible scenarios that share the same temporal properties.

Slide 4

Slide 4 text

Interactive Scores 4 i-score - https://ossia.io

Slide 5

Slide 5 text

Interactive Scores 5 f f f f T D F S

Slide 6

Slide 6 text

Interactive Scores 6 The underlying model of i-score does not support: 1. Flexible control structures (e.g., conditionals, loops) 2. Mechanisms for the automatic verification Several researchers have made many efforts to overcome these limitations: ● Petri nets: Allombert 2009 ● Process calculi: Olarte 2009, Toro 2014 but there are no practical solutions for the automatic verification and real-time performance, and the models cannot be straightforwardly implemented or extended.

Slide 7

Slide 7 text

Outline 1. ReactiveML 1.1. Syntax 1.2. Tree Representation 1.3. Operational Semantics 1.4. Logical Characterization 2. Concluding Remarks 7

Slide 8

Slide 8 text

ReactiveIS 8 https://github.com/himito/reactiveIS

Slide 9

Slide 9 text

Syntax 9

Slide 10

Slide 10 text

Syntax 10

Slide 11

Slide 11 text

Conditions 11 WaitFromStart(WH,1,1)

Slide 12

Slide 12 text

Program (Score) Tree 12

Slide 13

Slide 13 text

Program (Score) Tree 13

Slide 14

Slide 14 text

State Tree 14

Slide 15

Slide 15 text

Tree Homomorphism 15

Slide 16

Slide 16 text

State Tree Operation: Start 16

Slide 17

Slide 17 text

State Tree Operation: Stop 17

Slide 18

Slide 18 text

Operational Semantics 18 P T S T C I C O C S H

Slide 19

Slide 19 text

Operational Semantics 19

Slide 20

Slide 20 text

Operational Semantics 20

Slide 21

Slide 21 text

Operational Semantics 21

Slide 22

Slide 22 text

Operational Semantics 22

Slide 23

Slide 23 text

Operational Semantics 23

Slide 24

Slide 24 text

Properties of the SOS 24

Slide 25

Slide 25 text

Logical Characterization 25 L L S (SELL)

Slide 26

Slide 26 text

Logical Characterization This opens the possibility of reasoning about interactive scores by using well established techniques in proof theory. We can verify, for example, if all the structures will be eventually played. 26

Slide 27

Slide 27 text

Concluding Remarks 27

Slide 28

Slide 28 text

Summary ● New programming language for the specification, verification and interpretation of interactive scores. ● ReactiveIS extends the full capacity of temporal organization of i-score. ● It is possible to extend the language for the specification of conditionals and loops. ● The operational semantics based on labelled trees are simpler and more intuitive than the current model in HTSPN of i-score. ● The declarative interpretation of ReactiveIS programs as formulas in SELL is adequate. 28

Slide 29

Slide 29 text

Future Work ● Coq or Maude for proving the correctness of the semantics. ● Automatic checking of bound times of execution. ● Front-end for the specification of properties. ● Extension for conditional and loops. 29

Slide 30

Slide 30 text

Thank you for your attention 30

Slide 31

Slide 31 text

References 1. Arias, J., Desainte-Catherine, M., Olarte, C., & Rueda, C. (2015). Foundations for Reliable and Flexible Interactive Multimedia Scores. In T. Collins, D. Meredith, & A. Volk (Eds.), 5th International Conference on Mathematics and Computation in Music, MCM 2015, London, UK, June 22-25, 2015 (Vol. 9110, pp. 29–41). Springer. https://doi.org/10.1007/978-3-319-20603-5_3 2. Allombert, A. (2009). Aspects Temporels d’un Système de Partitions Musicales Interactives pour la Composition et l’Exécution. Université de Bordeaux. Retrieved from http://ori-oai.u-bordeaux1.fr/pdf/2009/ALLOMBERT_ANTOINE_2009.pdf 3. Olarte, C., & Rueda, C. (2009). A Declarative Language for Dynamic Multimedia Interaction Systems. In E. Chew, A. Childs, & C.-H. Chuan (Eds.), 2nd International Conference on Mathematics and Computation in Music, MCM 2009, New Haven, CT, USA, June 19-22, 2009 (Vol. 38, pp. 218–227). Springer Berlin Heidelberg. https://doi.org/10.1007/978-3-642-02394-1_20 4. Toro, M., Desainte-Catherine, M., & Rueda, C. (2014). Formal semantics for interactive music scores: a framework to design, specify properties and execute interactive scenarios. Journal of Mathematics and Music, 8(1), 93–112. https://doi.org/10.1080/17459737.2013.870610 31