Upgrade to Pro — share decks privately, control downloads, hide ads and more …

A Framework for Composition, Verification and Real-Time Performance of Multimedia Interactive Scenarios

A Framework for Composition, Verification and Real-Time Performance of Multimedia Interactive Scenarios

Jaime Arias Almeida

June 01, 2015
Tweet

More Decks by Jaime Arias Almeida

Other Decks in Research

Transcript

  1. A Framework for Composition, Verification and
    Real-Time Performance of Multimedia Interactive
    Scenarios
    Jaime Arias, Myriam Desainte-Catherine, and Camilo Rueda
    Laboratoire Bordelais de Recherche en Informatique (LaBRI)
    Université de Bordeaux
    15th International Conference on Application of Concurrency to
    System Design (ACSD 2015)
    Belgium, June 2015
    1

    View Slide

  2. Motivation
    Interactive Multimedia Scenarios
    • Interactive Scores† (IS) is a formalism for composing and interpreting
    interactive multimedia scenarios.
    • Some applications of IS:
    ◦ Live performances.
    ◦ Museum installations.
    ◦ Plastic art installations.
    †A. Allombert. Aspects Temporels d’un Système de Partitions Musicales Interactives pour la
    Composition et l’Exécution. Ph.D. thesis, Université de Bordeaux, 2009
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 1/13
    1/13

    View Slide

  3. Motivation
    Interactive Multimedia Scenarios
    • Currently, IS is implemented in the software i-score†.
    • A Hierarchical Time Stream Petri Net (HTSPN) model represents
    and executes the partially ordered set of events.
    †Website: http://www.i-score.org/
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 2/13
    2/13

    View Slide

  4. Motivation
    Interactive Multimedia Scenarios
    • Some applications demand two features that are not supported by
    the IS model†:
    ◦ Flexible control structures such as branching behavior.
    ◦ Automatic verification of scenarios in order to avoid abnormal
    behaviors.
    • Multimedia applications perform real-time, compute-intensive, and
    data-intensive tasks that sometimes the performance of standard
    computers is not sufficient.
    • Most of the multimedia applications are executed on architectures
    and operating systems that do not provide low-latency and real-time
    performance (e.g., i-score).
    †T. De la Hogue, P. Baltazar, M. Desainte-Catherine, J. Chao, and C. Bossut. OSSIA : Open
    Scenario System for Interactive Applications. In Journées d’Informatique Musicale, pages 78–84,
    Bourges, 2014
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 3/13
    3/13

    View Slide

  5. Motivation
    Field Programmable Gate Arrays (FPGAs)
    • FPGAs have risen over the last years and at the same time their cost
    have been reduced.
    • FPGAs have already been used with success in many different
    industrial applications†:
    ◦ Aerospace
    ◦ Automotive
    ◦ Medical
    ◦ Video and audio processing
    †J. Rodriguez-Andina, M. Moure, and M. Valdes. Features, Design Tools, and Application
    Domains of FPGAs. IEEE Transactions on Industrial Electronics, 54(4):1810–1823, Aug. 2007. ISSN
    0278-0046. doi: 10.1109/TIE.2007.898279
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 4/13
    4/13

    View Slide

  6. Motivation
    Field Programmable Gate Arrays (FPGAs)
    • FPGAs offer the following benefits†:
    ◦ Reconfigurability
    ◦ High-level design
    ◦ Physical parallelism
    ◦ High-speed
    ◦ Reliability
    ◦ Re-use
    D Q
    Q
    Look-up Table
    (LUT)
    Flip Flop
    Mux
    Clock
    I0
    I1
    I2
    I3
    Out
    Configurable Logic Blocks (CLBs)
    Programmable Interconnection
    Network
    Configurable Input/Output Blocks (IOBs)
    †R. Dubey. Introduction to Embedded System Design Using Field Programmable Gate Arrays.
    Springer London, London, 2009
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 5/13
    5/13

    View Slide

  7. Motivation
    Field Programmable Gate Arrays (FPGAs)
    (a) Performance comparison. (b) Cost/performance comparison.
    Figure: A comparison between FPGA and DSP platforms for real-time video
    applications*.
    *L. Langlois. Multirate digital signal processing for high-speed data converters for high-speed
    data converters. Xcell Journal, 73:50–53, Q4 2010
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 6/13
    6/13

    View Slide

  8. This talk is about …
    A Timed Automata semantics for the specification and automatic
    verification of interactive multimedia scenarios with branching behavior,
    and their real-time execution on FPGAs.
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 7/13
    7/13

    View Slide

  9. Outline
    i-score scenario
    IS2UPP
    UPPAAL model
    Property
    checking
    UPP2C UPP2HDL
    C/C++ code
    VHDL/System
    Verilog code
    Composition
    Verification
    Interpretation

    View Slide

  10. Outline
    i-score scenario
    IS2UPP
    UPPAAL model
    Property
    checking
    UPP2C UPP2HDL
    C/C++ code
    VHDL/System
    Verilog code
    Composition
    Verification
    Interpretation

    View Slide

  11. Interactive Scores
    An overview
    • An interactive scenario is composed of:
    ◦ Temporal objects
    • Textures → Multimedia processes
    • Structures → Hierarchy
    ◦ Temporal relations
    • Synchronization → ∆min
    = ∆max
    = 0
    • Rigid relations → ∆min
    = ∆max > 0
    • flexible relations → ∆min
    = 0 ∧ ∆max
    = ∞
    • Semi-flexible relations → ∆min
    ̸= ∆max
    ∧ ∆max
    ̸= ∞
    ◦ Interaction points
    • The system maintains the temporal relations during execution.
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 6/13
    6/13

    View Slide

  12. Interactive Scores
    An overview
    time (s)
    0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
    Texture B
    Texture A
    Structure C
    Structure D
    Texture F
    Texture G
    Texture E
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 7/13
    7/13

    View Slide

  13. Outline
    i-score scenario
    IS2UPP
    UPPAAL model
    Property
    checking
    UPP2C UPP2HDL
    C/C++ code
    VHDL/System
    Verilog code
    Composition
    Verification
    Interpretation

    View Slide

  14. Timed Automata Model
    Temporal Relation
    t ≤ γ0
    t ≤ γ1
    idle wait_min
    skipped finished
    killed
    flexible semi flexible
    t ≥ 0, event_s?, {t = 0}
    true, skip_p?, ∅
    true, skip!, {t = 0} true, event_e2!, {t = 0}
    true, kill!, {t = 0}
    true, kill_p?, ∅
    true, kill_p?, ∅
    true, event_i?, ∅
    true, kill_p?, ∅
    true, event_i?, ∅
    t =
    γ1
    , τ, ∅
    true, kill_p?, ∅
    t = γ0 ∧ finite = false, event_e1!, ∅
    t =
    γ0

    finite =
    true, event_e1
    !, ∅
    time (s)
    0 1 2 3 4 5 6
    Texture A Texture B
    γ0 = 8
    γ1 = 24
    finite = true
    t = 0
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 7/13
    7/13

    View Slide

  15. Timed Automata Model
    Temporal Relation
    t ≤ γ0
    t ≤ γ1
    idle wait_min
    skipped finished
    killed
    flexible semi flexible
    t ≥ 0, event_s?, {t = 0}
    true, skip_p?, ∅
    true, skip!, {t = 0} true, event_e2!, {t = 0}
    true, kill!, {t = 0}
    true, kill_p?, ∅
    true, kill_p?, ∅
    true, event_i?, ∅
    true, kill_p?, ∅
    true, event_i?, ∅
    t =
    γ1
    , τ, ∅
    true, kill_p?, ∅
    t = γ0 ∧ finite = false, event_e1!, ∅
    t =
    γ0

    finite =
    true, event_e1
    !, ∅
    time (s)
    0 1 2 3 4 5 6
    Texture A Texture B
    γ0 = 8
    γ1 = 24
    finite = true
    t = 4
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 7/13
    7/13

    View Slide

  16. Timed Automata Model
    Temporal Relation
    t ≤ γ0
    t ≤ γ1
    idle wait_min
    skipped finished
    killed
    flexible semi flexible
    t ≥ 0, event_s?, {t = 0}
    true, skip_p?, ∅
    true, skip!, {t = 0} true, event_e2!, {t = 0}
    true, kill!, {t = 0}
    true, kill_p?, ∅
    true, kill_p?, ∅
    true, event_i?, ∅
    true, kill_p?, ∅
    true, event_i?, ∅
    t =
    γ1
    , τ, ∅
    true, kill_p?, ∅
    t = γ0 ∧ finite = false, event_e1!, ∅
    t =
    γ0

    finite =
    true, event_e1
    !, ∅
    time (s)
    0 1 2 3 4 5 6
    Texture A Texture B
    γ0 = 8
    γ1 = 24
    finite = true
    t = 8
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 7/13
    7/13

    View Slide

  17. Timed Automata Model
    Temporal Relation
    t ≤ γ0
    t ≤ γ1
    idle wait_min
    skipped finished
    killed
    flexible semi flexible
    t ≥ 0, event_s?, {t = 0}
    true, skip_p?, ∅
    true, skip!, {t = 0} true, event_e2!, {t = 0}
    true, kill!, {t = 0}
    true, kill_p?, ∅
    true, kill_p?, ∅
    true, event_i?, ∅
    true, kill_p?, ∅
    true, event_i?, ∅
    t =
    γ1
    , τ, ∅
    true, kill_p?, ∅
    t = γ0 ∧ finite = false, event_e1!, ∅
    t =
    γ0

    finite =
    true, event_e1
    !, ∅
    time (s)
    0 1 2 3 4 5 6
    Texture A Texture B
    γ0 = 8
    γ1 = 24
    finite = true
    t = 12
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 7/13
    7/13

    View Slide

  18. Timed Automata Model
    Temporal Relation
    t ≤ γ0
    t ≤ γ1
    idle wait_min
    skipped finished
    killed
    flexible semi flexible
    t ≥ 0, event_s?, {t = 0}
    true, skip_p?, ∅
    true, skip!, {t = 0} true, event_e2!, {t = 0}
    true, kill!, {t = 0}
    true, kill_p?, ∅
    true, kill_p?, ∅
    true, event_i?, ∅
    true, kill_p?, ∅
    true, event_i?, ∅
    t =
    γ1
    , τ, ∅
    true, kill_p?, ∅
    t = γ0 ∧ finite = false, event_e1!, ∅
    t =
    γ0

    finite =
    true, event_e1
    !, ∅
    time (s)
    0 1 2 3 4 5 6
    Texture A Texture B
    γ0 = 8
    γ1 = 24
    finite = true
    t = 16
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 7/13
    7/13

    View Slide

  19. Timed Automata Model
    Temporal Relation
    t ≤ γ0
    t ≤ γ1
    idle wait_min
    skipped finished
    killed
    flexible semi flexible
    t ≥ 0, event_s?, {t = 0}
    true, skip_p?, ∅
    true, skip!, {t = 0} true, event_e2!, {t = 0}
    true, kill!, {t = 0}
    true, kill_p?, ∅
    true, kill_p?, ∅
    true, event_i?, ∅
    true, kill_p?, ∅
    true, event_i?, ∅
    t =
    γ1
    , τ, ∅
    true, kill_p?, ∅
    t = γ0 ∧ finite = false, event_e1!, ∅
    t =
    γ0

    finite =
    true, event_e1
    !, ∅
    time (s)
    0 1 2 3 4 5 6
    Texture A Texture B
    γ0 = 8
    γ1 = 24
    finite = true
    t = 20
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 7/13
    7/13

    View Slide

  20. Timed Automata Model
    Temporal Relation
    t ≤ γ0
    t ≤ γ1
    idle wait_min
    skipped finished
    killed
    flexible semi flexible
    t ≥ 0, event_s?, {t = 0}
    true, skip_p?, ∅
    true, skip!, {t = 0} true, event_e2!, {t = 0}
    true, kill!, {t = 0}
    true, kill_p?, ∅
    true, kill_p?, ∅
    true, event_i?, ∅
    true, kill_p?, ∅
    true, event_i?, ∅
    t =
    γ1
    , τ, ∅
    true, kill_p?, ∅
    t = γ0 ∧ finite = false, event_e1!, ∅
    t =
    γ0

    finite =
    true, event_e1
    !, ∅
    time (s)
    0 1 2 3 4 5 6
    Texture A Texture B
    γ0 = 8
    γ1 = 24
    finite = true
    t = 0
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 7/13
    7/13

    View Slide

  21. Timed Automata Model
    Temporal Relation
    t ≤ γ0
    t ≤ γ1
    idle wait_min
    skipped finished
    killed
    flexible semi flexible
    t ≥ 0, event_s?, {t = 0}
    true, skip_p?, ∅
    true, skip!, {t = 0} true, event_e2!, {t = 0}
    true, kill!, {t = 0}
    true, kill_p?, ∅
    true, kill_p?, ∅
    true, event_i?, ∅
    true, kill_p?, ∅
    true, event_i?, ∅
    t =
    γ1
    , τ, ∅
    true, kill_p?, ∅
    t = γ0 ∧ finite = false, event_e1!, ∅
    t =
    γ0

    finite =
    true, event_e1
    !, ∅
    time (s)
    0 1 2 3 4 5 6
    Texture A Texture B
    γ0 = 8
    γ1 = 24
    finite = true
    t = 4
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 7/13
    7/13

    View Slide

  22. Timed Automata Model
    Temporal Relation
    t ≤ γ0
    t ≤ γ1
    idle wait_min
    skipped finished
    killed
    flexible semi flexible
    t ≥ 0, event_s?, {t = 0}
    true, skip_p?, ∅
    true, skip!, {t = 0} true, event_e2!, {t = 0}
    true, kill!, {t = 0}
    true, kill_p?, ∅
    true, kill_p?, ∅
    true, event_i?, ∅
    true, kill_p?, ∅
    true, event_i?, ∅
    t =
    γ1
    , τ, ∅
    true, kill_p?, ∅
    t = γ0 ∧ finite = false, event_e1!, ∅
    t =
    γ0

    finite =
    true, event_e1
    !, ∅
    time (s)
    0 1 2 3 4 5 6
    Texture A Texture B
    γ0 = 8
    γ1 = 24
    finite = true
    t = 8
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 7/13
    7/13

    View Slide

  23. Timed Automata Model
    Temporal Relation
    t ≤ γ0
    t ≤ γ1
    idle wait_min
    skipped finished
    killed
    flexible semi flexible
    t ≥ 0, event_s?, {t = 0}
    true, skip_p?, ∅
    true, skip!, {t = 0} true, event_e2!, {t = 0}
    true, kill!, {t = 0}
    true, kill_p?, ∅
    true, kill_p?, ∅
    true, event_i?, ∅
    true, kill_p?, ∅
    true, event_i?, ∅
    t =
    γ1
    , τ, ∅
    true, kill_p?, ∅
    t = γ0 ∧ finite = false, event_e1!, ∅
    t =
    γ0

    finite =
    true, event_e1
    !, ∅
    time (s)
    0 1 2 3 4 5 6
    Texture A Texture B
    γ0 = 8
    γ1 = 24
    finite = true
    t = 12
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 7/13
    7/13

    View Slide

  24. Timed Automata Model
    Temporal Relation
    t ≤ γ0
    t ≤ γ1
    idle wait_min
    skipped finished
    killed
    flexible semi flexible
    t ≥ 0, event_s?, {t = 0}
    true, skip_p?, ∅
    true, skip!, {t = 0} true, event_e2!, {t = 0}
    true, kill!, {t = 0}
    true, kill_p?, ∅
    true, kill_p?, ∅
    true, event_i?, ∅
    true, kill_p?, ∅
    true, event_i?, ∅
    t =
    γ1
    , τ, ∅
    true, kill_p?, ∅
    t = γ0 ∧ finite = false, event_e1!, ∅
    t =
    γ0

    finite =
    true, event_e1
    !, ∅
    time (s)
    0 1 2 3 4 5 6
    Texture A Texture B
    γ0 = 8
    γ1 = 24
    finite = true
    t = 16
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 7/13
    7/13

    View Slide

  25. Timed Automata Model
    Temporal Relation
    t ≤ γ0
    t ≤ γ1
    idle wait_min
    skipped finished
    killed
    flexible semi flexible
    t ≥ 0, event_s?, {t = 0}
    true, skip_p?, ∅
    true, skip!, {t = 0} true, event_e2!, {t = 0}
    true, kill!, {t = 0}
    true, kill_p?, ∅
    true, kill_p?, ∅
    true, event_i?, ∅
    true, kill_p?, ∅
    true, event_i?, ∅
    t =
    γ1
    , τ, ∅
    true, kill_p?, ∅
    t = γ0 ∧ finite = false, event_e1!, ∅
    t =
    γ0

    finite =
    true, event_e1
    !, ∅
    time (s)
    0 1 2 3 4 5 6
    Texture A Texture B
    γ0 = 8
    γ1 = 24
    finite = true
    t = 20
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 7/13
    7/13

    View Slide

  26. Timed Automata Model
    Temporal Relation
    t ≤ γ0
    t ≤ γ1
    idle wait_min
    skipped finished
    killed
    flexible semi flexible
    t ≥ 0, event_s?, {t = 0}
    true, skip_p?, ∅
    true, skip!, {t = 0} true, event_e2!, {t = 0}
    true, kill!, {t = 0}
    true, kill_p?, ∅
    true, kill_p?, ∅
    true, event_i?, ∅
    true, kill_p?, ∅
    true, event_i?, ∅
    t =
    γ1
    , τ, ∅
    true, kill_p?, ∅
    t = γ0 ∧ finite = false, event_e1!, ∅
    t =
    γ0

    finite =
    true, event_e1
    !, ∅
    time (s)
    0 1 2 3 4 5 6
    Texture A Texture B
    γ0 = 8
    γ1 = 24
    finite = true
    t = 24
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 7/13
    7/13

    View Slide

  27. Timed Automata Model
    Temporal Relation
    t ≤ γ0
    t ≤ γ1
    idle wait_min
    skipped finished
    killed
    flexible semi flexible
    t ≥ 0, event_s?, {t = 0}
    true, skip_p?, ∅
    true, skip!, {t = 0} true, event_e2!, {t = 0}
    true, kill!, {t = 0}
    true, kill_p?, ∅
    true, kill_p?, ∅
    true, event_i?, ∅
    true, kill_p?, ∅
    true, event_i?, ∅
    t =
    γ1
    , τ, ∅
    true, kill_p?, ∅
    t = γ0 ∧ finite = false, event_e1!, ∅
    t =
    γ0

    finite =
    true, event_e1
    !, ∅
    time (s)
    0 1 2 3 4 5 6
    Texture A Texture B
    γ0 = 8
    γ1 = 24
    finite = true
    t = 0
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 7/13
    7/13

    View Slide

  28. Timed Automata Model
    Temporal Relation
    t ≤ γ0
    t ≤ γ1
    idle wait_min
    skipped finished
    killed
    flexible semi flexible
    t ≥ 0, event_s?, {t = 0}
    true, skip_p?, ∅
    true, skip!, {t = 0} true, event_e2!, {t = 0}
    true, kill!, {t = 0}
    true, kill_p?, ∅
    true, kill_p?, ∅
    true, event_i?, ∅
    true, kill_p?, ∅
    true, event_i?, ∅
    t =
    γ1
    , τ, ∅
    true, kill_p?, ∅
    t = γ0 ∧ finite = false, event_e1!, ∅
    t =
    γ0

    finite =
    true, event_e1
    !, ∅
    time (s)
    0 1 2 3 4 5 6
    Texture A Texture B
    γ0 = 8
    γ1 = 24
    finite = true
    t = 0
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 7/13
    7/13

    View Slide

  29. Timed Automata Model
    Temporal Relation
    t ≤ γ0
    t ≤ γ1
    idle wait_min
    skipped finished
    killed
    flexible semi flexible
    t ≥ 0, event_s?, {t = 0}
    true, skip_p?, ∅
    true, skip!, {t = 0} true, event_e2!, {t = 0}
    true, kill!, {t = 0}
    true, kill_p?, ∅
    true, kill_p?, ∅
    true, event_i?, ∅
    true, kill_p?, ∅
    true, event_i?, ∅
    t =
    γ1
    , τ, ∅
    true, kill_p?, ∅
    t = γ0 ∧ finite = false, event_e1!, ∅
    t =
    γ0

    finite =
    true, event_e1
    !, ∅
    time (s)
    0 1 2 3 4 5 6
    Texture A Texture B
    γ0 = 8
    γ1 = 24
    finite = true
    t = 0
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 7/13
    7/13

    View Slide

  30. Timed Automata Model
    Temporal Relation
    t ≤ γ0
    t ≤ γ1
    idle wait_min
    skipped finished
    killed
    flexible semi flexible
    t ≥ 0, event_s?, {t = 0}
    true, skip_p?, ∅
    true, skip!, {t = 0} true, event_e2!, {t = 0}
    true, kill!, {t = 0}
    true, kill_p?, ∅
    true, kill_p?, ∅
    true, event_i?, ∅
    true, kill_p?, ∅
    true, event_i?, ∅
    t =
    γ1
    , τ, ∅
    true, kill_p?, ∅
    t = γ0 ∧ finite = false, event_e1!, ∅
    t =
    γ0

    finite =
    true, event_e1
    !, ∅
    time (s)
    0 1 2 3 4 5 6
    Texture A Texture B
    γ0 = 8
    γ1 = 24
    finite = true
    t = 0
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 7/13
    7/13

    View Slide

  31. Timed Automata Model
    Temporal Relation
    t ≤ γ0
    t ≤ γ1
    idle wait_min
    skipped finished
    killed
    flexible semi flexible
    t ≥ 0, event_s?, {t = 0}
    true, skip_p?, ∅
    true, skip!, {t = 0} true, event_e2!, {t = 0}
    true, kill!, {t = 0}
    true, kill_p?, ∅
    true, kill_p?, ∅
    true, event_i?, ∅
    true, kill_p?, ∅
    true, event_i?, ∅
    t =
    γ1
    , τ, ∅
    true, kill_p?, ∅
    t = γ0 ∧ finite = false, event_e1!, ∅
    t =
    γ0

    finite =
    true, event_e1
    !, ∅
    time (s)
    0 1 2 3 4 5 6
    Texture A Texture B
    γ0 = 8
    γ1 = 24
    finite = true
    t = 0
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 7/13
    7/13

    View Slide

  32. Timed Automata Model
    Handling Intervals
    idle
    error
    finished
    true, event_s2 , ∅
    true, kill_p?, ∅
    true, skip_p?, {counter + +}
    true, event_s1 ?, {counter + +, skip_v =
    false}
    counter < n, τ, ∅
    counter = n ∧ skip_v = false, event_e!, ∅
    counter = n ∧ skip_v = true, skip!, ∅
    time
    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
    min max
    TR 2
    min max
    TR 1
    min max
    interaction point
    n = 2
    skipv = true
    counter = 0
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 8/13
    8/13

    View Slide

  33. Timed Automata Model
    Handling Intervals
    idle
    error
    finished
    true, event_s2 , ∅
    true, kill_p?, ∅
    true, skip_p?, {counter + +}
    true, event_s1 ?, {counter + +, skip_v =
    false}
    counter < n, τ, ∅
    counter = n ∧ skip_v = false, event_e!, ∅
    counter = n ∧ skip_v = true, skip!, ∅
    time
    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
    min max
    TR 2
    min max
    TR 1
    min max
    interaction point
    n = 2
    skipv = true
    counter = 0
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 8/13
    8/13

    View Slide

  34. Timed Automata Model
    Handling Intervals
    idle
    error
    finished
    true, event_s2 , ∅
    true, kill_p?, ∅
    true, skip_p?, {counter + +}
    true, event_s1 ?, {counter + +, skip_v =
    false}
    counter < n, τ, ∅
    counter = n ∧ skip_v = false, event_e!, ∅
    counter = n ∧ skip_v = true, skip!, ∅
    time
    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
    min max
    TR 2
    min max
    TR 1
    min max
    interaction point
    n = 2
    skipv = true
    counter = 0
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 8/13
    8/13

    View Slide

  35. Timed Automata Model
    Handling Intervals
    idle
    error
    finished
    true, event_s2 , ∅
    true, kill_p?, ∅
    true, skip_p?, {counter + +}
    true, event_s1 ?, {counter + +, skip_v =
    false}
    counter < n, τ, ∅
    counter = n ∧ skip_v = false, event_e!, ∅
    counter = n ∧ skip_v = true, skip!, ∅
    time
    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
    min max
    TR 2
    min max
    TR 1
    min max
    interaction point
    n = 2
    skipv = true
    counter = 0
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 8/13
    8/13

    View Slide

  36. Timed Automata Model
    Handling Intervals
    idle
    error
    finished
    true, event_s2 , ∅
    true, kill_p?, ∅
    true, skip_p?, {counter + +}
    true, event_s1 ?, {counter + +, skip_v =
    false}
    counter < n, τ, ∅
    counter = n ∧ skip_v = false, event_e!, ∅
    counter = n ∧ skip_v = true, skip!, ∅
    time
    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
    min max
    TR 2
    min max
    TR 1
    min max
    interaction point
    n = 2
    skipv = true
    counter = 0
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 8/13
    8/13

    View Slide

  37. Timed Automata Model
    Handling Intervals
    idle
    error
    finished
    true, event_s2 , ∅
    true, kill_p?, ∅
    true, skip_p?, {counter + +}
    true, event_s1 ?, {counter + +, skip_v =
    false}
    counter < n, τ, ∅
    counter = n ∧ skip_v = false, event_e!, ∅
    counter = n ∧ skip_v = true, skip!, ∅
    time
    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
    min max
    TR 2
    min max
    TR 1
    min max
    interaction point
    n = 2
    skipv = true
    counter = 0
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 8/13
    8/13

    View Slide

  38. Timed Automata Model
    Handling Intervals
    idle
    error
    finished
    true, event_s2 , ∅
    true, kill_p?, ∅
    true, skip_p?, {counter + +}
    true, event_s1 ?, {counter + +, skip_v =
    false}
    counter < n, τ, ∅
    counter = n ∧ skip_v = false, event_e!, ∅
    counter = n ∧ skip_v = true, skip!, ∅
    time
    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
    min max
    TR 2
    min max
    TR 1
    min max
    interaction point
    n = 2
    skipv = true
    counter = 0
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 8/13
    8/13

    View Slide

  39. Timed Automata Model
    Handling Intervals
    idle
    error
    finished
    true, event_s2 , ∅
    true, kill_p?, ∅
    true, skip_p?, {counter + +}
    true, event_s1 ?, {counter + +, skip_v =
    false}
    counter < n, τ, ∅
    counter = n ∧ skip_v = false, event_e!, ∅
    counter = n ∧ skip_v = true, skip!, ∅
    time
    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
    min max
    TR 2
    min max
    TR 1
    min max
    interaction point
    n = 2
    skipv = true
    counter = 0
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 8/13
    8/13

    View Slide

  40. Timed Automata Model
    Handling Intervals
    idle
    error
    finished
    true, event_s2 , ∅
    true, kill_p?, ∅
    true, skip_p?, {counter + +}
    true, event_s1 ?, {counter + +, skip_v =
    false}
    counter < n, τ, ∅
    counter = n ∧ skip_v = false, event_e!, ∅
    counter = n ∧ skip_v = true, skip!, ∅
    time
    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
    min max
    TR 2
    min max
    TR 1
    min max
    interaction point
    n = 2
    skipv = false
    counter = 1
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 8/13
    8/13

    View Slide

  41. Timed Automata Model
    Handling Intervals
    idle
    error
    finished
    true, event_s2 , ∅
    true, kill_p?, ∅
    true, skip_p?, {counter + +}
    true, event_s1 ?, {counter + +, skip_v =
    false}
    counter < n, τ, ∅
    counter = n ∧ skip_v = false, event_e!, ∅
    counter = n ∧ skip_v = true, skip!, ∅
    time
    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
    min max
    TR 2
    min max
    TR 1
    min max
    interaction point
    n = 2
    skipv = false
    counter = 1
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 8/13
    8/13

    View Slide

  42. Timed Automata Model
    Handling Intervals
    idle
    error
    finished
    true, event_s2 , ∅
    true, kill_p?, ∅
    true, skip_p?, {counter + +}
    true, event_s1 ?, {counter + +, skip_v =
    false}
    counter < n, τ, ∅
    counter = n ∧ skip_v = false, event_e!, ∅
    counter = n ∧ skip_v = true, skip!, ∅
    time
    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
    min max
    TR 2
    min max
    TR 1
    min max
    interaction point
    n = 2
    skipv = false
    counter = 1
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 8/13
    8/13

    View Slide

  43. Timed Automata Model
    Handling Intervals
    idle
    error
    finished
    true, event_s2 , ∅
    true, kill_p?, ∅
    true, skip_p?, {counter + +}
    true, event_s1 ?, {counter + +, skip_v =
    false}
    counter < n, τ, ∅
    counter = n ∧ skip_v = false, event_e!, ∅
    counter = n ∧ skip_v = true, skip!, ∅
    time
    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
    min max
    TR 2
    min max
    TR 1
    min max
    interaction point
    n = 2
    skipv = false
    counter = 2
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 8/13
    8/13

    View Slide

  44. Timed Automata Model
    Handling Intervals
    idle
    error
    finished
    true, event_s2 , ∅
    true, kill_p?, ∅
    true, skip_p?, {counter + +}
    true, event_s1 ?, {counter + +, skip_v =
    false}
    counter < n, τ, ∅
    counter = n ∧ skip_v = false, event_e!, ∅
    counter = n ∧ skip_v = true, skip!, ∅
    time
    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
    min max
    TR 2
    min max
    TR 1
    min max
    interaction point
    n = 2
    skipv = false
    counter = 2
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 8/13
    8/13

    View Slide

  45. Timed Automata Model
    Handling Intervals
    idle
    error
    finished
    true, event_s2 , ∅
    true, kill_p?, ∅
    true, skip_p?, {counter + +}
    true, event_s1 ?, {counter + +, skip_v =
    false}
    counter < n, τ, ∅
    counter = n ∧ skip_v = false, event_e!, ∅
    counter = n ∧ skip_v = true, skip!, ∅
    time
    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
    min max
    TR 2
    min max
    TR 1
    min max
    interaction point
    n = 2
    skipv = false
    counter = 2
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 8/13
    8/13

    View Slide

  46. Timed Automata Model
    Handling Intervals
    idle
    error
    finished
    true, event_s2 , ∅
    true, kill_p?, ∅
    true, skip_p?, {counter + +}
    true, event_s1 ?, {counter + +, skip_v =
    false}
    counter < n, τ, ∅
    counter = n ∧ skip_v = false, event_e!, ∅
    counter = n ∧ skip_v = true, skip!, ∅
    time
    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
    min max
    TR 2
    min max
    TR 1
    min max
    interaction point
    n = 2
    skipv = false
    counter = 2
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 8/13
    8/13

    View Slide

  47. Timed Automata Model
    Handling Intervals
    idle
    error
    finished
    true, event_s2 , ∅
    true, kill_p?, ∅
    true, skip_p?, {counter + +}
    true, event_s1 ?, {counter + +, skip_v =
    false}
    counter < n, τ, ∅
    counter = n ∧ skip_v = false, event_e!, ∅
    counter = n ∧ skip_v = true, skip!, ∅
    time
    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
    min max
    TR 2
    min max
    TR 1
    min max
    interaction point
    n = 2
    skipv = false
    counter = 2
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 8/13
    8/13

    View Slide

  48. Timed Automata Model
    Handling Intervals
    idle
    error
    finished
    true, event_s2 , ∅
    true, kill_p?, ∅
    true, skip_p?, {counter + +}
    true, event_s1 ?, {counter + +, skip_v =
    false}
    counter < n, τ, ∅
    counter = n ∧ skip_v = false, event_e!, ∅
    counter = n ∧ skip_v = true, skip!, ∅
    time
    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
    min max
    TR 2
    min max
    TR 1
    min max
    interaction point
    n = 2
    skipv = false
    counter = 2
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 8/13
    8/13

    View Slide

  49. Timed Automata Model
    Handling Intervals
    idle
    error
    finished
    true, event_s2 , ∅
    true, kill_p?, ∅
    true, skip_p?, {counter + +}
    true, event_s1 ?, {counter + +, skip_v =
    false}
    counter < n, τ, ∅
    counter = n ∧ skip_v = false, event_e!, ∅
    counter = n ∧ skip_v = true, skip!, ∅
    time
    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
    min max
    TR 2
    min max
    TR 1
    min max
    interaction point
    n = 2
    skipv = false
    counter = 2
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 8/13
    8/13

    View Slide

  50. Timed Automata Model
    Handling Intervals
    idle
    error
    finished
    true, event_s2 , ∅
    true, kill_p?, ∅
    true, skip_p?, {counter + +}
    true, event_s1 ?, {counter + +, skip_v =
    false}
    counter < n, τ, ∅
    counter = n ∧ skip_v = false, event_e!, ∅
    counter = n ∧ skip_v = true, skip!, ∅
    time
    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
    min max
    TR 2
    min max
    TR 1
    min max
    interaction point
    n = 2
    skipv = false
    counter = 2
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 8/13
    8/13

    View Slide

  51. Timed Automata Model
    Interaction Points
    idle
    finished
    skipped enabled
    timeout
    cond = true
    true, skip_p?, ∅ en = false, event_e?, ∅
    true, event_s?, {en = true}
    en
    =
    true, event_e?, ∅
    true, kill_p?, ∅
    true, skip!, ∅
    true, kill_p?, ∅
    urg = true, event_t!, ∅
    urg = false, skip!, ∅
    true, event?, {cond = condition()}
    true, event_t!, {en = false}
    true, event_e!, ∅
    time
    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
    min max
    TR 2
    min max
    TR 1
    min max
    interaction point
    urg =true
    en =true
    cond =false
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 9/13
    9/13

    View Slide

  52. Timed Automata Model
    Interaction Points
    idle
    finished
    skipped enabled
    timeout
    cond = true
    true, skip_p?, ∅ en = false, event_e?, ∅
    true, event_s?, {en = true}
    en
    =
    true, event_e?, ∅
    true, kill_p?, ∅
    true, skip!, ∅
    true, kill_p?, ∅
    urg = true, event_t!, ∅
    urg = false, skip!, ∅
    true, event?, {cond = condition()}
    true, event_t!, {en = false}
    true, event_e!, ∅
    time
    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
    min max
    TR 2
    min max
    TR 1
    min max
    interaction point
    urg =true
    en =true
    cond =false
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 9/13
    9/13

    View Slide

  53. Timed Automata Model
    Interaction Points
    idle
    finished
    skipped enabled
    timeout
    cond = true
    true, skip_p?, ∅ en = false, event_e?, ∅
    true, event_s?, {en = true}
    en
    =
    true, event_e?, ∅
    true, kill_p?, ∅
    true, skip!, ∅
    true, kill_p?, ∅
    urg = true, event_t!, ∅
    urg = false, skip!, ∅
    true, event?, {cond = condition()}
    true, event_t!, {en = false}
    true, event_e!, ∅
    time
    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
    min max
    TR 2
    min max
    TR 1
    min max
    interaction point
    urg =true
    en =true
    cond =false
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 9/13
    9/13

    View Slide

  54. Timed Automata Model
    Interaction Points
    idle
    finished
    skipped enabled
    timeout
    cond = true
    true, skip_p?, ∅ en = false, event_e?, ∅
    true, event_s?, {en = true}
    en
    =
    true, event_e?, ∅
    true, kill_p?, ∅
    true, skip!, ∅
    true, kill_p?, ∅
    urg = true, event_t!, ∅
    urg = false, skip!, ∅
    true, event?, {cond = condition()}
    true, event_t!, {en = false}
    true, event_e!, ∅
    time
    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
    min max
    TR 2
    min max
    TR 1
    min max
    interaction point
    urg =true
    en =true
    cond =false
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 9/13
    9/13

    View Slide

  55. Timed Automata Model
    Interaction Points
    idle
    finished
    skipped enabled
    timeout
    cond = true
    true, skip_p?, ∅ en = false, event_e?, ∅
    true, event_s?, {en = true}
    en
    =
    true, event_e?, ∅
    true, kill_p?, ∅
    true, skip!, ∅
    true, kill_p?, ∅
    urg = true, event_t!, ∅
    urg = false, skip!, ∅
    true, event?, {cond = condition()}
    true, event_t!, {en = false}
    true, event_e!, ∅
    time
    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
    min max
    TR 2
    min max
    TR 1
    min max
    interaction point
    urg =true
    en =true
    cond =false
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 9/13
    9/13

    View Slide

  56. Timed Automata Model
    Interaction Points
    idle
    finished
    skipped enabled
    timeout
    cond = true
    true, skip_p?, ∅ en = false, event_e?, ∅
    true, event_s?, {en = true}
    en
    =
    true, event_e?, ∅
    true, kill_p?, ∅
    true, skip!, ∅
    true, kill_p?, ∅
    urg = true, event_t!, ∅
    urg = false, skip!, ∅
    true, event?, {cond = condition()}
    true, event_t!, {en = false}
    true, event_e!, ∅
    time
    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
    min max
    TR 2
    min max
    TR 1
    min max
    interaction point
    urg =true
    en =true
    cond =false
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 9/13
    9/13

    View Slide

  57. Timed Automata Model
    Interaction Points
    idle
    finished
    skipped enabled
    timeout
    cond = true
    true, skip_p?, ∅ en = false, event_e?, ∅
    true, event_s?, {en = true}
    en
    =
    true, event_e?, ∅
    true, kill_p?, ∅
    true, skip!, ∅
    true, kill_p?, ∅
    urg = true, event_t!, ∅
    urg = false, skip!, ∅
    true, event?, {cond = condition()}
    true, event_t!, {en = false}
    true, event_e!, ∅
    time
    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
    min max
    TR 2
    min max
    TR 1
    min max
    interaction point
    urg =true
    en =true
    cond =false
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 9/13
    9/13

    View Slide

  58. Timed Automata Model
    Interaction Points
    idle
    finished
    skipped enabled
    timeout
    cond = true
    true, skip_p?, ∅ en = false, event_e?, ∅
    true, event_s?, {en = true}
    en
    =
    true, event_e?, ∅
    true, kill_p?, ∅
    true, skip!, ∅
    true, kill_p?, ∅
    urg = true, event_t!, ∅
    urg = false, skip!, ∅
    true, event?, {cond = condition()}
    true, event_t!, {en = false}
    true, event_e!, ∅
    time
    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
    min max
    TR 2
    min max
    TR 1
    min max
    interaction point
    urg =true
    en =true
    cond =false
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 9/13
    9/13

    View Slide

  59. Timed Automata Model
    Interaction Points
    idle
    finished
    skipped enabled
    timeout
    cond = true
    true, skip_p?, ∅ en = false, event_e?, ∅
    true, event_s?, {en = true}
    en
    =
    true, event_e?, ∅
    true, kill_p?, ∅
    true, skip!, ∅
    true, kill_p?, ∅
    urg = true, event_t!, ∅
    urg = false, skip!, ∅
    true, event?, {cond = condition()}
    true, event_t!, {en = false}
    true, event_e!, ∅
    time
    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
    min max
    TR 2
    min max
    TR 1
    min max
    interaction point
    urg =true
    en =true
    cond =false
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 9/13
    9/13

    View Slide

  60. Timed Automata Model
    Multimedia Processes
    Brightness
    100%
    50%
    30%
    0%
    p0
    p1
    p2
    p3
    p4
    p5
    p6
    ∆1 ∆2 ∆3 ∆4 ∆5 ∆6
    typedef struct {
    int value;
    int offset;
    } parameter_t;
    parameter_t process_brightness[7] = {
    {0,0}, {30,5}, {50,2}, ... };
    idle
    finished
    wait
    t ≤ mp[i].offset
    t ≥ 0, start?, {t = 0, i = 0}
    true, kill_p?, {t = 0}
    true, skip_p?, {t = 0}
    true, kill_p?, {t =
    0}
    true, stop?, {t =
    0}
    t =
    m
    p[i].offset ∧
    i =
    lim
    it, send!,
    {data
    =
    m
    p[i].value,t =
    0}
    t = mp[i].offset ∧ i < limit, send!,
    {t = 0, data = mp[i].value, i + +}
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 10/13
    10/13

    View Slide

  61. Outline
    i-score scenario
    IS2UPP
    UPPAAL model
    Property
    checking
    UPP2C UPP2HDL
    C/C++ code
    VHDL/System
    Verilog code
    Composition
    Verification
    Interpretation

    View Slide

  62. Verification
    Proving some properties…
    Some important properties are:
    • Is a temporal constraint always satisfied?
    • Are all temporal relations always satisfied?
    • Is a multimedia ressource accessed by two or more processes?
    • Will the scenario finish?
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 10/13
    10/13

    View Slide

  63. Outline
    i-score scenario
    IS2UPP
    UPPAAL model
    Property
    checking
    UPP2C UPP2HDL
    C/C++ code
    VHDL/System
    Verilog code
    Composition
    Verification
    Interpretation

    View Slide

  64. Real-time Performance
    Executing IS on FPGAs
    Next
    State
    State
    Register
    Output
    Logic
    Inputs
    Outputs
    Mealy FSM
    Local
    Variables
    Clock
    Variables
    Global Clock
    Inputs Outputs
    Figure: Block diagram of the hardware implementation.†
    †Y. Jiang, H. Zhang, Z. Li, Y. Deng, X. Song, M. Gu, and J. Sun. Design and Optimization of
    Multiclocked Embedded Systems Using Formal Techniques. IEEE Transactions on Industrial
    Electronics, 62(2):1270–1278, 2015
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 10/13
    10/13

    View Slide

  65. Real-time Performance
    Executing IS on FPGAs
    Figure: Simulation of our example.
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 11/13
    11/13

    View Slide

  66. Concluding Remarks
    Summary
    • We presented a novel framework for composition, verification and
    real-time performance of interactive scenarios.
    • Compilers have been developed for the automatic construction of
    the UPPAAL models.
    • Synthesizable SystemVerilog templates of the UPPAAL models have
    been developed for their execution on FPGAs.
    • To the best of our knowledge, none of the existing models of IS
    supports the automatic verification of scenarios or provides an
    implementation that guarantees their real-time execution.
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 12/13
    12/13

    View Slide

  67. Concluding Remarks
    Future work
    • Front-end for composers in order to specify the properties.
    • Validate the SystemVerilog code using the formal specification
    language SVA (SystemVerilog Assertions)
    • Integration of a Fast Ethernet module in order to provide a low-rate
    and reliable communication.
    • Synthesize DSP programs into FPGAs (e.g., programs written in
    FAUST – http://faust.grame.fr).
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 13/13
    13/13

    View Slide

  68. Concluding Remarks
    Future work
    • Front-end for composers in order to specify the properties.
    • Validate the SystemVerilog code using the formal specification
    language SVA† (SystemVerilog Assertions)
    • Integration of a Fast Ethernet module in order to provide a low-rate
    and reliable communication.
    • Synthesize DSP programs into FPGAs (e.g., programs written in
    FAUST – http://faust.grame.fr).
    †Y. Jiang, H. Zhang, Z. Li, Y. Deng, X. Song, M. Gu, and J. Sun. Design and Optimization of
    Multiclocked Embedded Systems Using Formal Techniques. IEEE Transactions on Industrial
    Electronics, 62(2):1270–1278, 2015
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 13/13
    13/13

    View Slide

  69. Concluding Remarks
    Future work
    • Front-end for composers in order to specify the properties.
    • Validate the SystemVerilog code using the formal specification
    language SVA (SystemVerilog Assertions)
    • Integration of a Fast Ethernet module in order to provide a low-rate
    and reliable communication†.
    • Synthesize DSP programs into FPGAs (e.g., programs written in
    FAUST – http://faust.grame.fr).
    †R. Aviziensis, A. Freed, T. Suzuki, and D. Wessel. Scalable Connectivity Processor for
    Computer Music Performance Systems. In International Computer Music Conference, 2000
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 13/13
    13/13

    View Slide

  70. Concluding Remarks
    Future work
    • Front-end for composers in order to specify the properties.
    • Validate the SystemVerilog code using the formal specification
    language SVA (SystemVerilog Assertions)
    • Integration of a Fast Ethernet module in order to provide a low-rate
    and reliable communication.
    • Synthesize DSP programs into FPGAs (e.g., programs written in
    FAUST† – http://faust.grame.fr).
    †R. Trausmuth, C. Dusek, and Y. Orlarey. Using FAUST for FPGA Programming. In 9th
    International Conference on Digital Audio Effects, pages 18–20, 2006
    Arias, Desainte-Catherine, and Rueda (LaBRI) A Framework for Composition, Verification and Real-Time Performance of IS 13/13
    13/13

    View Slide

  71. Thank you for your attention!
    [email protected]

    View Slide

  72. A Framework for Composition, Verification and
    Real-Time Performance of Multimedia Interactive
    Scenarios
    Jaime Arias, Myriam Desainte-Catherine, and Camilo Rueda
    Laboratoire Bordelais de Recherche en Informatique (LaBRI)
    Université de Bordeaux
    15th International Conference on Application of Concurrency to
    System Design (ACSD 2015)
    Belgium, June 2015
    1

    View Slide

  73. References
    Application and Theory of Petri Nets 1995, 16th International Conference,
    Turin, Italy, June 26-30, 1995, Proceedings, volume 935 of LNCS, 1995.
    Springer. ISBN 3-540-60029-9.
    A. Allombert. Aspects Temporels d’un Système de Partitions Musicales
    Interactives pour la Composition et l’Exécution. Ph.D. thesis, Université
    de Bordeaux, 2009.
    A. Allombert, R. Marczak, M. Desainte-Catherine, P. Baltazar, and
    GarnierLaurent. VIRAGE : Designing An Interactive Intermedia
    Sequencer From Users Requirements And Theoretical Background.
    In International Computer Music Conference, 2010.
    R. Alur and D. L. Dill. A theory of timed automata. Theoretical Computer
    Science, 126(2):183–235, Apr. 1994.
    R. Alur, T. A. Henzinger, and M. Y. Vardi. Parametric real-time
    reasoning. In Proceedings of the 25th annual ACM symposium on Theory
    of computing, pages 592–601. ACM Press, 1993.

    View Slide

  74. References
    J. Arias, M. Desainte-Catherine, and C. Rueda. Modelling Data
    Processing for Interactive Scores Using Coloured Petri Nets. In 14th
    International Conference On Applications Of Concurrency To System
    Design, pages 186–195, 2014.
    R. Aviziensis, A. Freed, T. Suzuki, and D. Wessel. Scalable Connectivity
    Processor for Computer Music Performance Systems. In International
    Computer Music Conference, 2000.
    G. Behrmann, A. David, and K. G. Larsen. A Tutorial on Uppaal. In
    Formal Methods for the Design of Real-Time Systems, pages 200–236.
    Springer Berlin Heidelberg, Berlin, Heidelberg, Jan. 2004.
    E. Cerny, S. Dudani, J. Havlicek, and D. Korchemny. The Power of
    Assertions in SystemVerilog. 2010.

    View Slide

  75. References
    T. De la Hogue, P. Baltazar, M. Desainte-Catherine, J. Chao, and
    C. Bossut. OSSIA : Open Scenario System for Interactive
    Applications. In Journées d’Informatique Musicale, pages 78–84,
    Bourges, 2014.
    M. Desainte-Catherine, A. Allombert, and G. Assayag. Towards a hybrid
    temporal paradigm for musical composition and performance: The
    case of musical interpretation. Computer Music Journal, 37(2):61–72,
    2013.
    J. S. Dong, Y. Liu, J. Sun, and X. Zhang. Towards Verification of
    Computation Orchestration. Formal Aspects of Computing, 26(4):
    729–759, 2014.
    R. Dubey. Introduction to Embedded System Design Using Field
    Programmable Gate Arrays. Springer London, London, 2009.

    View Slide

  76. References
    J. Echeveste, A. Cont, J.-L. Giavitto, and F. Jacquemard. Operational
    Semantics of a Domain Specific Language for Real Time
    Musician–Computer Interaction. Discrete Event Dynamic Systems, 23
    (4):343–383, Dec. 2013.
    S. Hauck and A. DeHon. Reconfigurable Computing: The Theory and
    Practice of FPGA-Based Computation. San Francisco, CA, USA, 2007.
    K. Jensen and L. M. Kristensen. Coloured Petri Nets: Modelling and
    Validation of Concurrent Systems. Springer Berlin Heidelberg, 2009.
    Y. Jiang, H. Zhang, Z. Li, Y. Deng, X. Song, M. Gu, and J. Sun. Design and
    Optimization of Multiclocked Embedded Systems Using Formal
    Techniques. IEEE Transactions on Industrial Electronics, 62(2):
    1270–1278, 2015.
    L. Langlois. Multirate digital signal processing for high-speed data
    converters for high-speed data converters. Xcell Journal, 73:50–53,
    Q4 2010.

    View Slide

  77. References
    K. G. Larsen, P. Pettersson, and W. Yi. Uppaal in a nutshell. International
    Journal on Software Tools for Technology Transfer, 1(1-2):134–152, Dec.
    1997.
    E. A. Lee. The problem with threads. Computer, 39(5):33–42, May 2006.
    ISSN 0018-9162.
    R. Marczak, M. Desainte-Catherine, and A. Allombert. Real-time
    temporal control of musical processes. In The Third International
    Conferences on Advances in Multimedia, pages 12–17, 2011. ISBN
    978-1-61208-129-8.
    R. Milner. Communication and concurrency. Prentice Hall, 1989. ISBN
    978-0-13-115007-2.
    E. Monmasson, L. Idkhajine, M. N. Cirstea, I. Bahri, A. Tisan, and M. W.
    Naouar. FPGAs in Industrial Control Applications. IEEE Transactions
    on Industrial Informatics, 7(2):224–243, 2011. ISSN 1551-3203. doi:
    10.1109/TII.2011.2123908.

    View Slide

  78. References
    C. Olarte and C. Rueda. A Declarative Language for Dynamic
    Multimedia Interaction Systems. In Mathematics and Computation in
    Music, volume 38, pages 218–227. Springer Berlin Heidelberg, 2009.
    J. Rodriguez-Andina, M. Moure, and M. Valdes. Features, Design Tools,
    and Application Domains of FPGAs. IEEE Transactions on Industrial
    Electronics, 54(4):1810–1823, Aug. 2007. ISSN 0278-0046. doi:
    10.1109/TIE.2007.898279.
    P. Sénac, P. de Saqui-Sannes, and R. Willrich. Hierarchical time stream
    petri net: A model for hypermedia systems. In Application and Theory
    of Petri Nets DBL [1995], pages 451–470. ISBN 3-540-60029-9.
    M. Toro, M. Desainte-Catherine, and C. Rueda. Formal semantics for
    interactive music scores: a framework to design, specify properties
    and execute interactive scenarios. Journal of Mathematics and Music, 8
    (1):93–112, 2014. ISSN 1745-9737.

    View Slide

  79. References
    R. Trausmuth, C. Dusek, and Y. Orlarey. Using FAUST for FPGA
    Programming. In 9th International Conference on Digital Audio Effects,
    pages 18–20, 2006.

    View Slide