A Tree-Based Operational Semantics for Interactive Multimedia Scores

1.

A Tree-Based Operational Semantics for Interactive Multimedia Scores Jaime Arias,
Myriam Desainte-Catherine and Camilo Rueda Laboratoire Bordelais de Recherche en Informatique (LaBRI) Université de Bordeaux MUSICAL Project October, 2014 1

2.

This talk is about … Definition of the syntax and
the operational semantics of a programming language for the specification and interpretation of interactive scores. Arias, Desainte-Catherine and Rueda A Tree-Based Operational Semantics for Interactive Multimedia Scores 1/32 1/32

3.

Outline Overview Syntax Program State Operational Semantics Conclusions Arias, Desainte-Catherine
and Rueda A Tree-Based Operational Semantics for Interactive Multimedia Scores 2/32 2/32

4.


5.

Overview Program State hS, !, i Temporal Transition hS, !,
i Instantaneous Transition The Synchrony Hypothesis Arias, Desainte-Catherine and Rueda A Tree-Based Operational Semantics for Interactive Multimedia Scores 4/32 4/32

6.

Overview Program State hS, !, i Temporal Transition hS, !,
i Instantaneous Transition The Synchrony Hypothesis ReactiveIS It’s very intuitive Arias, Desainte-Catherine and Rueda A Tree-Based Operational Semantics for Interactive Multimedia Scores 4/32 4/32

7.


8.

Grammar ⟨score⟩ ::= <box><score> | <process><score> | <empty> ⟨box⟩ ::=
"box" (<params> <score>) ⟨process⟩ ::= "process" (<params> <msg> <msg>) ⟨params⟩ ::= <name> <condition> <condition> ⟨condition⟩ ::= "wait" (<box-event> <time> <duration>) | "event" <msg> | "and" (<condition> <condition>) | "or" (<condition> <condition>) ⟨box-event⟩ ::= "start" <name> | "end" <name> ⟨duration⟩ ::= "finite" <time> | "infinite" Arias, Desainte-Catherine and Rueda A Tree-Based Operational Semantics for Interactive Multimedia Scores 6/32 6/32

9.

Grammar Example Arias, Desainte-Catherine and Rueda A Tree-Based Operational Semantics
for Interactive Multimedia Scores 7/32 7/32

10.


11.

Program ��
�� Arias, Desainte-Catherine and Rueda A Tree-Based Operational Semantics for Interactive Multimedia Scores 9/32 9/32

12.

Program ��
�� Camilo said Arias, Desainte-Catherine and Rueda A Tree-Based Operational Semantics for Interactive Multimedia Scores 9/32 9/32

13.

Program Definition (Condition System) A condition system is a pair
(Σ, ∆) where Σ is a signature of constants, functions and predicate symbols, and ∆ is a first-order theory over Σ (i.e., a set of first-order sentences over Σ having at least one model). Definition (Condition) Given a condition system (Σ, ∆), let C be the set of formulas (conditions) built over Σ with the set of distinguished predicates P = {WaitFromStart, WaitFromEnd, EndScenario, WaitEvent} and the grammar F, G, ... := true | A | F ∧ G | F ∨ G where A is an atomic formula, and the symbols ∧, ∨ and true denote logical conjunction, disjunction and the always true predicate, respectively. Arias, Desainte-Catherine and Rueda A Tree-Based Operational Semantics for Interactive Multimedia Scores 10/32 10/32

14.

Program Let N be an infinite set of nodes, B
the set of labels representing the names of TOs, and M the set of messages. Definition (Program Tree) A program is an edge-labelled tree P = ⟨N, E, ℓ, m, r⟩ where • N ⊆ N is the set of nodes, • E ⊆ N × B × N is the set of edges, • ℓ : N → C × C is a total function representing the start and the end conditions of the node, • m : N ⇀ M × M is a partial function representing the messages for starting and stopping an external application, and • r ∈ N is the root of the tree. Arias, Desainte-Catherine and Rueda A Tree-Based Operational Semantics for Interactive Multimedia Scores 11/32 11/32

15.


16.

State ��
� �� Arias, Desainte-Catherine and Rueda A Tree-Based Operational Semantics for Interactive Multimedia Scores 13/32 13/32

17.

State Carlos said Why do you define two trees ?
We can reuse the program tree and define partial functions. �� Arias, Desainte-Catherine and Rueda A Tree-Based Operational Semantics for Interactive Multimedia Scores 13/32 13/32

18.

State Definition (State Tree) An execution state is an edge-labelled
tree S = ⟨N, E, ℓ, r⟩ where • N, E and r are defined as in the Program Tree definition, • ℓ : N → Z+ × Z⊥ is a total function giving, for each node, its starting and ending times. Arias, Desainte-Catherine and Rueda A Tree-Based Operational Semantics for Interactive Multimedia Scores 14/32 14/32

19.

State Operation on state trees Dates start: 0 stop: !
Dates start: 3 stop: ! B Dates start: 1 stop: 3 A Arias, Desainte-Catherine and Rueda A Tree-Based Operational Semantics for Interactive Multimedia Scores 15/32 15/32

20.

Dates start: 3 stop: ! B Dates start: 1 stop: 3 A Elaine said How do you stop a process? Arias, Desainte-Catherine and Rueda A Tree-Based Operational Semantics for Interactive Multimedia Scores 15/32 15/32

21.

Dates start: 3 stop: ! B Dates start: 1 stop: 3 A Dates start: 0 stop: ! Dates start: 3 stop: 5 B Dates start: 1 stop: 3 A stop(S,B,5) Elaine said How do you stop a process? Arias, Desainte-Catherine and Rueda A Tree-Based Operational Semantics for Interactive Multimedia Scores 15/32 15/32

22.

Dates start: 3 stop: ! B Dates start: 1 stop: 3 A Dates start: 0 stop: ! Dates start: 3 stop: 5 B Dates start: 1 stop: 3 A stop(S,B,5) Elaine said How do you start a process? Arias, Desainte-Catherine and Rueda A Tree-Based Operational Semantics for Interactive Multimedia Scores 15/32 15/32

23.

Dates start: 3 stop: ! B Dates start: 1 stop: 3 A Dates start: 0 stop: ! Dates start: 3 stop: 5 B Dates start: 1 stop: 3 A stop(S,B,5) Dates start: 0 stop: ! Dates start: 8 stop: ! C Dates start: 3 stop: 5 B Dates start: 1 stop: 3 A start(S,C,8) Elaine said How do you start a process? Arias, Desainte-Catherine and Rueda A Tree-Based Operational Semantics for Interactive Multimedia Scores 15/32 15/32

24.

State Operation on state trees Domain Restriction The domain restriction
A ◁ − s is a subset of the relation s that contains all of the pairs whose first element is not in A. More formally A ◁ − s = { x → y | x → y ∈ s ∧ x ̸∈ A} Relation Overwrite The relational overwrite t ◁ − s is equal to t except all entries in t whose first element is in the domain of s are replaced by the corresponding entries in s. More formally, t ◁ − s = s ∪ (dom(s) ◁ − t) Arias, Desainte-Catherine and Rueda A Tree-Based Operational Semantics for Interactive Multimedia Scores 16/32 16/32

25.

State Operation on state trees Stopping a TO For a
path p ∈ L(S) and a time t ∈ Z+, stop(S, p, t) = ⟨N, E, ℓ ◁ − {n → (ts(n), t) | n ∈ k}, r⟩ where k = { n | n ∈ D(target(p)) ∧ te(n) = ⊥ } Starting a TO For a path p ∈ L(S) and a time t ∈ Z+, start(S, p, t) = ⟨N ∪ {n1}, E ∪ {n b − → n1}, ℓ ∪ {n1 → (t, ⊥)}, r⟩ where n1 / ∈ N, n = target(up(p)), b = last(p) Arias, Desainte-Catherine and Rueda A Tree-Based Operational Semantics for Interactive Multimedia Scores 17/32 17/32

26.


27.

Operational Semantics Configuration hP, S, I, O, ti Arias, Desainte-Catherine

28.

Operational Semantics Configuration hP, S, I, O, ti Program Tree
Arias, Desainte-Catherine and Rueda A Tree-Based Operational Semantics for Interactive Multimedia Scores 19/32 19/32

29.

Current State Tree Arias, Desainte-Catherine and Rueda A Tree-Based Operational Semantics for Interactive Multimedia Scores 19/32 19/32

30.

Current State Tree Current Input Arias, Desainte-Catherine and Rueda A Tree-Based Operational Semantics for Interactive Multimedia Scores 19/32 19/32

31.

Current State Tree Current Output Current Input Arias, Desainte-Catherine and Rueda A Tree-Based Operational Semantics for Interactive Multimedia Scores 19/32 19/32

32.

Current State Tree Current Output Current Input Current Time Arias, Desainte-Catherine and Rueda A Tree-Based Operational Semantics for Interactive Multimedia Scores 19/32 19/32

33.

Operational Semantics Definition (Semantics) We say that a configuration γ
satisfies a condition F in the underlying condition system if ⟨P, S, I, O, t⟩ |= F, where ⟨P, S, I, O, t⟩ |= true ⟨P, S, I, O, t⟩ |= WaitFromStart(p, t1, t2) iff ∃n · n ∈ V (S) ∧ n = target(p) ∧ t1 ≤ t − ts (n) ≤ t2 ⟨P, S, I, O, t⟩ |= WaitFromEnd(p, t1, t2) iff ∃n · n ∈ V (S) ∧ n = target(p) ∧ te(n) ̸= ⊥ ∧ t1 ≤ t − te(n) ≤ t2 ⟨P, S, I, O, t⟩ |= EndScenario iff ∀p · p ∈ out(root(P)) ⇒ te(target(p)) ̸= ⊥ ⟨P, S, I, O, t⟩ |= WaitEvent(e) iff e ∈ I ⟨P, S, I, O, t⟩ |= F ∧ G iff ⟨P, S, I, O, t⟩ |= F and ⟨P, S, I, O, t⟩ |= G ⟨P, S, I, O, t⟩ |= F ∨ G iff ⟨P, S, I, O, t⟩ |= F or ⟨P, S, I, O, t⟩ |= G Arias, Desainte-Catherine and Rueda A Tree-Based Operational Semantics for Interactive Multimedia Scores 20/32 20/32

34.

Operational Semantics RSTART p ∈ canStart(S, P) ⟨P, S, I,
O, t⟩ |= cs (target(p)) ⟨P, S, I, O, t⟩ −→ ⟨P, start(S, p, t), I, O ∪ {ms (target(p))}, t⟩ RSTOP p ∈ canStop(S) ⟨P, S, I, O, t⟩ |= ce(target(p)) ⟨P, S, I, O, t⟩ −→ ⟨P, stop(S, p, t), I, O ∪ {me(target(p))}, t⟩ RTIME ⟨P, S, I, ∅, t⟩ −→∗ ⟨P, S′, I, O, t⟩ ̸ −→ ⟨P, S, t⟩ I,O =⇒ ⟨P, S′, t + 1⟩ Table: Rules for the internal reduction −→ and the observable reduction =⇒. Notation γ ̸ −→ means that there is no γ′ such that γ −→ γ′. Arias, Desainte-Catherine and Rueda A Tree-Based Operational Semantics for Interactive Multimedia Scores 21/32 21/32

35.

Operational Semantics Start rule RSTART p ∈ canStart(S, P) ⟨P,
S, I, O, t⟩ |= cs (target(p)) ⟨P, S, I, O, t⟩ −→ ⟨P, start(S, p, t), I, O ∪ {ms (target(p))}, t⟩ Arias, Desainte-Catherine and Rueda A Tree-Based Operational Semantics for Interactive Multimedia Scores 22/32 22/32

36.

S, I, O, t⟩ |= cs (target(p)) ⟨P, S, I, O, t⟩ −→ ⟨P, start(S, p, t), I, O ∪ {ms (target(p))}, t⟩ Salim said Which nodes can start? Arias, Desainte-Catherine and Rueda A Tree-Based Operational Semantics for Interactive Multimedia Scores 22/32 22/32

37.

S, I, O, t⟩ |= cs (target(p)) ⟨P, S, I, O, t⟩ −→ ⟨P, start(S, p, t), I, O ∪ {ms (target(p))}, t⟩ Salim said Which nodes can start? Friki said canStart(S, P) b = L(P) \ L(S) Arias, Desainte-Catherine and Rueda A Tree-Based Operational Semantics for Interactive Multimedia Scores 22/32 22/32

38.

Operational Semantics Stop rule RSTOP p ∈ canStop(S) ⟨P, S,
I, O, t⟩ |= ce(target(p)) ⟨P, S, I, O, t⟩ −→ ⟨P, stop(S, p, t), I, O ∪ {me(target(p))}, t⟩ Arias, Desainte-Catherine and Rueda A Tree-Based Operational Semantics for Interactive Multimedia Scores 23/32 23/32

39.

I, O, t⟩ |= ce(target(p)) ⟨P, S, I, O, t⟩ −→ ⟨P, stop(S, p, t), I, O ∪ {me(target(p))}, t⟩ Salim said Which nodes can stop? Arias, Desainte-Catherine and Rueda A Tree-Based Operational Semantics for Interactive Multimedia Scores 23/32 23/32

40.

I, O, t⟩ |= ce(target(p)) ⟨P, S, I, O, t⟩ −→ ⟨P, stop(S, p, t), I, O ∪ {me(target(p))}, t⟩ Salim said Which nodes can stop? canStop ( S ) b = { p | p 2 L ( S ) ^ te( target ( p )) = ?} Friki said Arias, Desainte-Catherine and Rueda A Tree-Based Operational Semantics for Interactive Multimedia Scores 23/32 23/32

41.

Operational Semantics Time rule RTIME ⟨P, S, I, ∅, t⟩
−→∗ ⟨P, S′, I, O, t⟩ ̸ −→ ⟨P, S, t⟩ I,O =⇒ ⟨P, S′, t + 1⟩ Arias, Desainte-Catherine and Rueda A Tree-Based Operational Semantics for Interactive Multimedia Scores 24/32 24/32

42.

Operational Semantics Time rule RTIME ⟨P, S, I, ∅, t⟩
−→∗ ⟨P, S′, I, O, t⟩ ̸ −→ ⟨P, S, t⟩ I,O =⇒ ⟨P, S′, t + 1⟩ Arias, Desainte-Catherine and Rueda A Tree-Based Operational Semantics for Interactive Multimedia Scores 24/32 24/32

43.

Example ��
�� Arias, Desainte-Catherine and Rueda A Tree-Based Operational Semantics for Interactive Multimedia Scores 25/32 25/32

44.

Example ��
�� Time 0 Time 1 Time 2 Time 3 Time 4 "/light/1 on" "/light/2 on" "/light/1 off" Time 5 Time 6 Time 9 Time 10 "/light/1 off" "/sound/2 on" Time 11 Time 12 Time 13 Time 14 Time 15 "/mouse 1" "/sound/1 on" Time 16 "/sound/1 off" "/sound/2 off" Arias, Desainte-Catherine and Rueda A Tree-Based Operational Semantics for Interactive Multimedia Scores 25/32 25/32

45.


46.

Conclusions • ReactiveIS is a programming language for the specification
and execution of interactive scores. • Tree structures are used to represent programs and execution states. • Trees provide a clear and intuitive representation of scores and the evolution of their execution. • Conditions defined as formulas of first-order logic allow to define more complex temporal relations in scores. • The operational semantics can be easily implemented in a programming language. Arias, Desainte-Catherine and Rueda A Tree-Based Operational Semantics for Interactive Multimedia Scores 27/32 27/32

47.

Future Work • We plan to add control structures (i.e.,
conditionals and loops) to the operational semantics. • We plan to take advantage of the formal semantics to develop techniques and tools for the verification of scores (i.e., the playability of the score and the maximum number of multimedia processes executed simultaneously). Arias, Desainte-Catherine and Rueda A Tree-Based Operational Semantics for Interactive Multimedia Scores 28/32 28/32

48.

Thank you!

49.

A Tree-Based Operational Semantics for Interactive Multimedia Scores Jaime Arias,
Myriam Desainte-Catherine and Camilo Rueda Laboratoire Bordelais de Recherche en Informatique (LaBRI) Université de Bordeaux MUSICAL Project October, 2014 1

A Tree-Based Operational Semantics for Interactive Multimedia Scores

A Tree-Based Operational Semantics for Interactive Multimedia Scores

Want more?

Transcript