Towards an Automatic Verification of Interactive Scores and their Real-Time Performance

Transcript

1. Towards an Automatic Verification of Interactive Scores and their Real-Time Performance Jaime Arias, Myriam Desainte-Catherine and Camilo Rueda (Carlos Olarte and Sylvain Salvati) Journées INEDIT – Bordeaux, France Mars 2015 1 
   
2. (Open) Interactive Multimedia Scenarios (i-score) Texture Structure Rigid Interval Semi-Flexible Interval Interaction Point OSC Messages Multimedia Processes
3. (Open) Interactive Multimedia Scenarios (i-score) Conditionals Loops Data Stream Some Limitations Visual Representation Automatic Verification
4. Overview (2013 – present) MUSICAL PoSET OSSIA
5. Overview (2013 – present) •J. Arias, M. Guzmán, and C. Olarte. A symbolic model for timed concurrent constraint programming. In 9th Workshop on Logical and Semantic Frameworks, with Applications, LSFA 2014, Brasília D.F., Brazil, September 8-9, 2014, 2014. Proceedings in Electronic Notes in Theoretical Computer Science. NTCC (SyMoN) •J. Arias, M. Desainte-Catherine, and C. Rueda. Modeling data processing for interactive scores using colored petri nets. In 14th International Conference on Application of Concurrency to System Design, ACSD 2014, Tunis, Tunisia, June 23-27, 2014, pages 186–195. IEEE, 2014 Colored Petri Nets (CPN Tools) •J. Arias, M. Desainte-Catherine, S. Salvati, and C. Rueda. Executing hierarchical interactive scores in ReactiveML. In Journées d’Informatique Musicale, JIM 2014, Bourges, France, May 21-23, 2014, pages 25–34, 2014 ReactiveML + INScore •J. Arias, M. Desainte-Catherine, C. Olarte, and C. Rueda. Foundations for reliable and flexible interactive multimedia scores. Submitted to the Fifth Biennial International Conference on Mathematics and Computation in Music, MCM 2015. ReactiveIS •J. Arias, M. Desainte-Catherine, and C. Rueda. Exploiting parallelism in FPGAs for the real-time interpretation of interactive multimedia scores. Submitted to Journées d’Informatique Musicale, JIM 2015. FPGAs •J. Arias, M. Desainte-Catherine, and C. Rueda. A framework for composition, verification and real- time performance of multimedia interactive scenarios. Submitted to the 15th International Conference on Application of Concurrency to System Design, ACSD 2015. Timed Automata (UPPAAL)
6. A Non-Deterministic Temporal Concurrent Constraint Programming (NTCC) Specification Event Structures Specification & Execution NTCC Verification Model Checking * M. Toro Bermúdez, M. Desainte-Catherine and C. Rueda. Formal semantics for interactive music scores: A framework to design, specify properties and execute interactive scenarios. Journal of Mathematics and Music, Volume 8, Issue 1: 93-112, 2014.
7. Colored Petri Nets (CPN Tools) Declarations colset TIME = time; colset UNIT = unit; colset INT = int; colset DATA = int; colset DURATION = TIME timed; colset FILE = product INT*DATA; var f_dur : TIME; var n_max : INT; var n : INT; var f : DATA; TIME frame duration start f_dur f_dur get frame DURATION f_dur f_dur@+f_dur max num INT n_max n_max file FILE (n,f) next frame INT n+1 n receive frame FILE [n <= n_max] (n,f) n_max n_max 1 if (n = n_max) then 1`() else empty end EOF R E A D if (n = n_max) then 1`() else empty output FILE 5 1,5 2,9 3,6 4,1 (n,f) (n,f) 4 TIME = 0 start TIME frame durat ion max num INT outp ut FILE end EOF file FILE Hierarchy Time Data SML
8. Colored Petri Nets (CPN Tools) • Data Streams • Conditionals and loops • Buffering • Expresive • Verification tools (CPN Tools) • Complex Nets • Very Static • Implementation
9. Synchronous Approach The Synchrony Hypothesis Signal Broadcasting Concurrency … … … Logical Time
10. Synchronous Approach Interpreter RML Observer RML Pure Data / MAX/MSP OSC Messages INScore Events OSC Messages OSC Client OSC Server OSC Messages Box 1 Box 2 Rel 1 Rel2 … … 2 4 3 5 1 6 0 r2 r1 r3 r4 r5 r6 r7
11. Synchronous Approach
12. Specific Domain Programming Language Program State hS, !, i Temporal Transition hS, !, i Instantaneous Transition The Synchrony Hypothesis * Nigam, V., Olarte, C., Pimentel, E.: A general proof system for modalities in concurrent constraint programming. In: CONCUR. LNCS , vol. 8052, pp. 410– 424. Springer (2013) Logic Characterization as formulas in intuitionistic linear logic (ILL) with subexponentials (SELL)*.
13. Specific Domain Programming Language
          
                     
      !    
      !
    
          
            " " #$   %&$'%"   %&$'%" (( 
          
            " " #$   %&$'%   %&$'% (( 
          
           
     " " #$   % %"   % %" (( 
          
     "   ) %       
       #$   % %   % % ((  A B C D E Dates start: 0 stop: ! Dates start: 3 stop: ! B Dates start: 1 stop: 3 A f P S f f A B C E r1 r2 r3 r4 r5 r6 r7 r8 r9 S [1,1] [3,3] [5,5] [3,3] [1,1] [2,5] D “/mouse 1” time 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 0
14. Framework IS Composition IS2UPP Verification Property Checking C/C++ Code VHDL/System Verilog Code UPP2C UPP2HDL Interpretation
15. TimedAutomata Finite-state machine Multiple Clocks Process 1 Process 2 c1 c1 c2 c2 Communication Channels (CCS) Network of TA x = 10 x = 30 t = 1 Shared Variables
16. TimedAutomata temporal relation 1 temporal relation 2 equivalent temporal relation min max min max min max interaction point enabled time (s) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
17. TimedAutomata 0% 30% 100% 50% Brightness 1 2 3 4 5 6 p0 p1 p2 p3 p4 p5 p6
18. TimedAutomata
19. True Parallel Implementation (FPGAs)
20. True Parallel Implementation (FPGAs) Next State State Register Output Logic Inputs Outputs Mealy FSM Local Variables Clock Variables Global Clock Inputs Outputs
21. True Parallel Implementation (FPGAs)