A tree-based Operational Semantics for Interactive Multimedia Scores

Talk presented at the seminar on formal methods and languages.

Transcript

1. A Tree-Based Operational Semantics for
   Interactive Multimedia Scores
   Jaime ARIAS, Myriam DESAINTE-CATHERINE, Carlos OLARTE, and Camilo RUEDA
   Seminars on Formal Methods and Languages
   Inria Grenoble - Rhône-Alpes
   October 2017
   

   View Slide

2. Static Score
   Play
   2
   =
   

   View Slide

3. Interactive Scores
   3
   Les Baltazars. Nocturne nº2 - Tumbleweed
   During the performance of a scenario, the performer
   has the possibility to influence its execution by
   triggering interactive points (IPs).
   IPs are defined by the composer during the
   composition and allow for agogic modifications (i.e.,
   the possibility to change the start and stop times of
   TOs during execution ).
   In this sense, a performance constitutes an instance of
   a finite set of possible scenarios that share the same
   temporal properties.
   

   View Slide

4. Interactive Scores
   4
   i-score - https://ossia.io
   

   View Slide

5. Interactive Scores
   5
   f
   f
   f
   f
   T D F S
   

   View Slide

6. Interactive Scores
   6
   The underlying model of i-score does not support:
   1. Flexible control structures (e.g., conditionals, loops)
   2. Mechanisms for the automatic verification
   Several researchers have made many efforts to overcome these limitations:
   ● Petri nets: Allombert 2009
   ● Process calculi: Olarte 2009, Toro 2014
   but there are no practical solutions for the automatic verification and real-time performance, and the models
   cannot be straightforwardly implemented or extended.
   

   View Slide

7. Outline 1. ReactiveML
   1.1. Syntax
   1.2. Tree Representation
   1.3. Operational Semantics
   1.4. Logical Characterization
   2. Concluding Remarks
   7
   

   View Slide

8. ReactiveIS
   8
   https://github.com/himito/reactiveIS
   

   View Slide

9. Syntax
   9
   

   View Slide

10. Syntax
    10
    

    View Slide

11. Conditions
    11
    WaitFromStart(WH,1,1)
    

    View Slide

12. Program (Score) Tree
    12
    

    View Slide

13. Program (Score) Tree
    13
    

    View Slide

14. State Tree
    14
    

    View Slide

15. Tree Homomorphism
    15
    

    View Slide

16. State Tree Operation: Start
    16
    

    View Slide

17. State Tree Operation: Stop
    17
    

    View Slide

18. Operational Semantics
    18
    P
    T
    S
    T
    C
    I
    C
    O
    C
    S H
    

    View Slide

19. Operational Semantics
    19
    

    View Slide

20. Operational Semantics
    20
    

    View Slide

21. Operational Semantics
    21
    

    View Slide

22. Operational Semantics
    22
    

    View Slide

23. Operational Semantics
    23
    

    View Slide

24. Properties of the SOS
    24
    

    View Slide

25. Logical Characterization
    25
    L L S (SELL)
    

    View Slide

26. Logical Characterization
    This opens the possibility of reasoning about interactive scores by using well established techniques in
    proof theory.
    We can verify, for example, if all the structures will be eventually played.
    26
    

    View Slide

27. Concluding
    Remarks
    27
    

    View Slide

28. Summary
    ● New programming language for the specification, verification and interpretation of interactive
    scores.
    ● ReactiveIS extends the full capacity of temporal organization of i-score.
    ● It is possible to extend the language for the specification of conditionals and loops.
    ● The operational semantics based on labelled trees are simpler and more intuitive than the current
    model in HTSPN of i-score.
    ● The declarative interpretation of ReactiveIS programs as formulas in SELL is adequate.
    28
    

    View Slide

29. Future Work
    ● Coq or Maude for proving the correctness of the semantics.
    ● Automatic checking of bound times of execution.
    ● Front-end for the specification of properties.
    ● Extension for conditional and loops.
    29
    

    View Slide

30. Thank you for your attention
    30
    

    View Slide

31. References
    1. Arias, J., Desainte-Catherine, M., Olarte, C., & Rueda, C. (2015). Foundations for Reliable and Flexible Interactive
    Multimedia Scores. In T. Collins, D. Meredith, & A. Volk (Eds.), 5th International Conference on Mathematics and
    Computation in Music, MCM 2015, London, UK, June 22-25, 2015 (Vol. 9110, pp. 29–41). Springer.
    https://doi.org/10.1007/978-3-319-20603-5_3
    2. Allombert, A. (2009). Aspects Temporels d’un Système de Partitions Musicales Interactives pour la Composition
    et l’Exécution. Université de Bordeaux. Retrieved from
    http://ori-oai.u-bordeaux1.fr/pdf/2009/ALLOMBERT_ANTOINE_2009.pdf
    3. Olarte, C., & Rueda, C. (2009). A Declarative Language for Dynamic Multimedia Interaction Systems. In E. Chew,
    A. Childs, & C.-H. Chuan (Eds.), 2nd International Conference on Mathematics and Computation in Music, MCM
    2009, New Haven, CT, USA, June 19-22, 2009 (Vol. 38, pp. 218–227). Springer Berlin Heidelberg.
    https://doi.org/10.1007/978-3-642-02394-1_20
    4. Toro, M., Desainte-Catherine, M., & Rueda, C. (2014). Formal semantics for interactive music scores: a
    framework to design, specify properties and execute interactive scenarios. Journal of Mathematics and Music,
    8(1), 93–112. https://doi.org/10.1080/17459737.2013.870610
    31
    

    View Slide