Multimedia Scores* Jaime Arias, Myriam Desainte-Catherine, and Camilo Rueda Laboratoire Bordelais de Recherche en Informatique (LaBRI) Université de Bordeaux Journées d’Informatique Musicale (JIM 2015) Montreal - Canada, May 2015 1 *Supported by the ANR Project OSSIA and SCRIME
formalism for composing and executing interactive multimedia scenarios. (a) Live-art performances. (b) Interactive museum installations. (c) Plastic art installations. Figure: Some applications of IS †Antoine Allombert. Aspects Temporels d’un Système de Partitions Musicales Interactives pour la Composition et l’Exécution. PhD thesis, Université de Bordeaux, 2009 Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 1/23 1/23
the software i-score†. Texture Structure Rigid Interval Semi-Flexible Interval Interaction Point OSC Messages Multimedia Processes Figure: Graphical interface of i-score †I-score website: http://i-score.org/ Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 2/23 2/23
real-time tasks ◦ compute-intensive tasks ◦ data-intensive tasks • Sometimes the performance of standard computers is not sufficient. • Most of the multimedia applications are executed on architectures and operating systems that do not provide low-latency and real-time performance. • I-score is implemented using threads†: non-determinism and unreliability. †Edward A. Lee. The problem with threads. Computer, 39(5):33–42, May 2006 Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 3/23 3/23
over the last years and at the same time their cost have been reduced. • FPGAs have already been used with success in many different industrial applications: ◦ Aerospace ◦ Automotive ◦ Medical ◦ Video and audio processing †Réseau de Portes Programmables in situ Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 4/23 4/23
over the last years and at the same time their cost have been reduced. • FPGAs have already been used with success in many different industrial applications†: ◦ Aerospace ◦ Automotive ◦ Medical ◦ Video and audio processing †J.J. Rodriguez-Andina, M.J. Moure, and M.D. Valdes. Features, Design Tools, and Application Domains of FPGAs. IEEE Transactions on Industrial Electronics, 54(4):1810–1823, August 2007 Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 4/23 4/23
Eyre. BDTI study certifies high-level synthesis flows for DSP-centric FPGA design. Xcell Journal, 71:12–17, Q2 2010 Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 6/23 6/23
signal processing for high-speed data converters for high-speed data converters. Xcell Journal, 73:50–53, Q4 2010 Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 7/23 7/23
parallel implementation on FPGAs of the execution model of interactive multimedia scenarios. *David Wessel and Matthew Wright. Problems and prospects for intimate musical control of computers. In Ivan Poupyrev, Michael J. Lyons, Sidney Fels, and Tina Blaine, editors, New Interfaces for Musical Expression, NIME-01, Proceedings, Seattle, April 1-2, 2001, pages 11–14, 2001 Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 8/23 8/23
4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 Texture B Texture A Structure C Structure D Texture F Texture G Texture E Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 10/23 10/23
our clock signal: frequency divider. cycles = Tclockscore TclockFPGA Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 11/23 11/23
our clock signal: frequency divider. cycles = Tclockscore TclockFPGA = 400ns 10ns Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 11/23 11/23
our clock signal: frequency divider. cycles = Tclockscore TclockFPGA = 400ns 10ns = 20 cycles Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 11/23 11/23
our clock signal: frequency divider. cycles = Tclockscore TclockFPGA = 400ns 10ns = 20 cycles Figure: Frequency divider. Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 11/23 11/23
temporal relation: ◦ It is defined between two points. ◦ Defines a precedence relation. ◦ Defines an interval of time. • An interaction point allows for agogic modifications. Texture A Texture B Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 12/23 12/23
temporal relation: ◦ It is defined between two points. ◦ Defines a precedence relation. ◦ Defines an interval of time. • An interaction point allows for agogic modifications. Texture A Texture B end A start B Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 12/23 12/23
temporal relation: ◦ It is defined between two points. ◦ Defines a precedence relation. ◦ Defines an interval of time. • An interaction point allows for agogic modifications. Texture A Texture B end A start B B after A Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 12/23 12/23
temporal relation: ◦ It is defined between two points. ◦ Defines a precedence relation. ◦ Defines an interval of time. • An interaction point allows for agogic modifications. Texture A Texture B end A start B B after A min max Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 12/23 12/23
temporal relation: ◦ It is defined between two points. ◦ Defines a precedence relation. ◦ Defines an interval of time. • An interaction point allows for agogic modifications. Texture A Texture B end A start B B after A min max interaction point Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 12/23 12/23
0 1 2 3 4 5 6 Texture A Texture B timer 1 timer 2 – – idle new wait min. wait max. end eventstart eventmin eventext /eventend eventmax /eventend Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 13/23 13/23
0 1 2 3 4 5 6 Texture A Texture B timer 1 timer 2 – – idle new wait min. wait max. end eventstart eventmin eventext /eventend eventmax /eventend Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 13/23 13/23
0 1 2 3 4 5 6 Texture A Texture B timer 1 timer 2 – – idle new wait min. wait max. end eventstart eventmin eventext /eventend eventmax /eventend Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 13/23 13/23
0 1 2 3 4 5 6 Texture A Texture B timer 1 timer 2 – – idle new wait min. wait max. end eventstart eventmin eventext /eventend eventmax /eventend Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 13/23 13/23
0 1 2 3 4 5 6 Texture A Texture B timer 1 timer 2 – – idle new wait min. wait max. end eventstart eventmin eventext /eventend eventmax /eventend Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 13/23 13/23
0 1 2 3 4 5 6 Texture A Texture B timer 1 timer 2 – – idle new wait min. wait max. end eventstart eventmin eventext /eventend eventmax /eventend Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 13/23 13/23
0 1 2 3 4 5 6 Texture A Texture B timer 1 timer 2 – – idle new wait min. wait max. end eventstart eventmin eventext /eventend eventmax /eventend Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 13/23 13/23
0 1 2 3 4 5 6 Texture A Texture B timer 1 timer 2 8ms 24ms idle new wait min. wait max. end eventstart eventmin eventext /eventend eventmax /eventend Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 13/23 13/23
0 1 2 3 4 5 6 Texture A Texture B timer 1 timer 2 4ms 20ms idle new wait min. wait max. end eventstart eventmin eventext /eventend eventmax /eventend Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 13/23 13/23
0 1 2 3 4 5 6 Texture A Texture B timer 2 timer 1 0ms 16ms idle new wait min. wait max. end eventstart eventmin eventext /eventend eventmax /eventend Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 13/23 13/23
0 1 2 3 4 5 6 Texture A Texture B timer 1 timer 2 0ms 12ms idle new wait min. wait max. end eventstart eventmin eventext /eventend eventmax /eventend Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 13/23 13/23
0 1 2 3 4 5 6 Texture A Texture B timer 1 timer 2 0ms 8ms idle new wait min. wait max. end eventstart eventmin eventext /eventend eventmax /eventend Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 13/23 13/23
0 1 2 3 4 5 6 Texture A Texture B timer 1 timer 2 0ms 4ms idle new wait min. wait max. end eventstart eventmin eventext /eventend eventmax /eventend Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 13/23 13/23
0 1 2 3 4 5 6 Texture A Texture B timer 1 timer 2 0ms 0ms idle new wait min. wait max. end eventstart eventmin eventext /eventend eventmax /eventend Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 13/23 13/23
0 1 2 3 4 5 6 Texture A Texture B timer 1 timer 2 0ms 0ms idle new wait min. wait max. end eventstart eventmin eventext /eventend eventmax /eventend Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 13/23 13/23
0 1 2 3 4 5 6 Texture A Texture B timer 1 timer 2 0ms 0ms idle new wait min. wait max. end eventstart eventmin eventext /eventend eventmax /eventend Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 13/23 13/23
0 1 2 3 4 5 6 Texture A Texture B timer 1 timer 2 0ms 0ms idle new wait min. wait max. end eventstart eventmin eventext /eventend eventmax /eventend Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 13/23 13/23
0 1 2 3 4 5 6 Texture A Texture B timer 1 timer 2 0ms 0ms idle new wait min. wait max. end eventstart eventmin eventext /eventend eventmax /eventend Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 13/23 13/23
a temporal relation but it has an attached multimedia process. Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 14/23 14/23
one or more temporal relations in order to define the starting time of a temporal object. Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 15/23 15/23
4 5 6 7 8 9 10 11 12 13 14 15 16 min max TR 2 min max TR 1 min max interaction point Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 16/23 16/23
4 5 6 7 8 9 10 11 12 13 14 15 16 min max TR 2 min max TR 1 min max interaction point idle new wait min. wait max. end eventstart eventmin eventext /eventend eventmax /eventend – – timer 1 timer 2 Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 16/23 16/23
4 5 6 7 8 9 10 11 12 13 14 15 16 min max TR 2 min max TR 1 min max interaction point idle new wait min. update wait max. end eventstart eventmin eventext /eventend eventmax /eventend eventstart – – timer 1 timer 2 Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 16/23 16/23
4 5 6 7 8 9 10 11 12 13 14 15 16 min max TR 2 min max TR 1 min max interaction point idle new wait min. update wait max. end eventstart eventmin eventext /eventend eventmax /eventend eventstart – – timer 1 timer 2 Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 16/23 16/23
4 5 6 7 8 9 10 11 12 13 14 15 16 min max TR 2 min max TR 1 min max interaction point idle new wait min. update wait max. end eventstart eventmin eventext /eventend eventmax /eventend eventstart – – timer 1 timer 2 Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 16/23 16/23
4 5 6 7 8 9 10 11 12 13 14 15 16 min max TR 2 min max TR 1 min max interaction point idle new wait min. update wait max. end eventstart eventmin eventext /eventend eventmax /eventend eventstart – – timer 1 timer 2 Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 16/23 16/23
4 5 6 7 8 9 10 11 12 13 14 15 16 min max TR 2 min max TR 1 min max interaction point idle new wait min. update wait max. end eventstart eventmin eventext /eventend eventmax /eventend eventstart 5ms 10ms timer 1 timer 2 Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 16/23 16/23
4 5 6 7 8 9 10 11 12 13 14 15 16 min max TR 2 min max TR 1 min max interaction point idle new wait min. update wait max. end eventstart eventmin eventext /eventend eventmax /eventend eventstart 4ms 9ms timer 1 timer 2 Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 16/23 16/23
4 5 6 7 8 9 10 11 12 13 14 15 16 min max TR 2 min max TR 1 min max interaction point idle new wait min. update wait max. end eventstart eventmin eventext /eventend eventmax /eventend eventstart 3ms 8ms timer 1 timer 2 max(3, 5) = 3 min(8, 10) = 8 Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 16/23 16/23
4 5 6 7 8 9 10 11 12 13 14 15 16 min max TR 2 min max TR 1 min max interaction point idle new wait min. update wait max. end eventstart eventmin eventext /eventend eventmax /eventend eventstart 5ms 8ms timer 1 timer 2 Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 16/23 16/23
4 5 6 7 8 9 10 11 12 13 14 15 16 min max TR 2 min max TR 1 min max interaction point idle new wait min. update wait max. end eventstart eventmin eventext /eventend eventmax /eventend eventstart 4ms 7ms timer 1 timer 2 Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 16/23 16/23
4 5 6 7 8 9 10 11 12 13 14 15 16 min max TR 2 min max TR 1 min max interaction point idle new wait min. update wait max. end eventstart eventmin eventext /eventend eventmax /eventend eventstart 3ms 6ms timer 1 timer 2 Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 16/23 16/23
4 5 6 7 8 9 10 11 12 13 14 15 16 min max TR 2 min max TR 1 min max interaction point idle new wait min. update wait max. end eventstart eventmin eventext /eventend eventmax /eventend eventstart 2ms 5ms timer 1 timer 2 Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 16/23 16/23
4 5 6 7 8 9 10 11 12 13 14 15 16 min max TR 2 min max TR 1 min max interaction point idle new wait min. update wait max. end eventstart eventmin eventext /eventend eventmax /eventend eventstart 1ms 4ms timer 1 timer 2 Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 16/23 16/23
4 5 6 7 8 9 10 11 12 13 14 15 16 min max TR 2 min max TR 1 min max interaction point idle new wait min. update wait max. end eventstart eventmin eventext /eventend eventmax /eventend eventstart 0ms 3ms timer 2 timer 1 Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 16/23 16/23
4 5 6 7 8 9 10 11 12 13 14 15 16 min max TR 2 min max TR 1 min max interaction point idle new wait min. update wait max. end eventstart eventmin eventext /eventend eventmax /eventend eventstart 0ms 2ms timer 1 timer 2 Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 16/23 16/23
4 5 6 7 8 9 10 11 12 13 14 15 16 min max TR 2 min max TR 1 min max interaction point idle new wait min. update wait max. end eventstart eventmin eventext /eventend eventmax /eventend eventstart 0ms 1ms timer 1 timer 2 Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 16/23 16/23
4 5 6 7 8 9 10 11 12 13 14 15 16 min max TR 2 min max TR 1 min max interaction point idle new wait min. update wait max. end eventstart eventmin eventext /eventend eventmax /eventend eventstart 0ms 0ms timer 1 timer 2 Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 16/23 16/23
4 5 6 7 8 9 10 11 12 13 14 15 16 min max TR 2 min max TR 1 min max interaction point idle new wait min. update wait max. end eventstart eventmin eventext /eventend eventmax /eventend eventstart 0ms 0ms timer 1 timer 2 Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 16/23 16/23
4 5 6 7 8 9 10 11 12 13 14 15 16 min max TR 2 min max TR 1 min max interaction point idle new wait min. update wait max. end eventstart eventmin eventext /eventend eventmax /eventend eventstart 0ms 0ms timer 1 timer 2 Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 16/23 16/23
4 5 6 7 8 9 10 11 12 13 14 15 16 min max TR 2 min max TR 1 min max interaction point idle new wait min. update wait max. end eventstart eventmin eventext /eventend eventmax /eventend eventstart 0ms 0ms timer 1 timer 2 Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 16/23 16/23
objects and defines its own temporal organization. • A multimedia scenario is a structure (root). • We can model a structure as a temporal relation. • The end of a structure defines the end of its children. Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 17/23 17/23
objects and defines its own temporal organization. • A multimedia scenario is a structure (root). • We can model a structure as a temporal relation. • The end of a structure defines the end of its children. idle new wait min. update wait max. end eventstart eventm in event ext /event end event m ax /event end eventstart Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 17/23 17/23
objects and defines its own temporal organization. • A multimedia scenario is a structure (root). • We can model a structure as a temporal relation. • The end of a structure defines the end of its children. idle new wait min. update wait max. end eventstart eventm in event ext /event end event m ax /event end eventstart kill/killed kill/killed kill/killed kill/killed kill/killed Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 17/23 17/23
◦ with IP: 1. its maximum duration has elapsed. 2. the interaction point has been triggered. ◦ normal: 1. the maximum duration is infinity. 2. its minimum duration has elapsed. 3. its children have stopped. Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 18/23 18/23
◦ with IP: 1. its maximum duration has elapsed. 2. the interaction point has been triggered. ◦ normal: 1. the maximum duration is infinity. 2. its minimum duration has elapsed. 3. its children have stopped. Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 18/23 18/23
◦ with IP: 1. its maximum duration has elapsed. 2. the interaction point has been triggered. ◦ normal: 1. the maximum duration is infinity. 2. its minimum duration has elapsed. 3. its children have stopped. eventmin eventend1 eventend2 eventendn eventext Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 18/23 18/23
F Texture G Texture E triggered at 21 s 2 s 1 s 4 s ∆ = 6 s ∆ = 14 s ∆ = 10 s ∆ = 4 s Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 20/23 20/23
F Texture G Texture E triggered at 21 s triggered at 31 s 2 s 1 s 4 s ∆ = 6 s ∆ = 14 s ∆ = 10 s ∆ = 4 s Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 20/23 20/23
IS. • Low-latency and real-time performance. • Reaction to events is almost instant. • It is not affected by the complex behavior of operating systems (e.g., interrupt handling). Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 22/23 22/23
(e.g., FAUST†). • Implement a Fast Ethernet module to provide a reliable and low-rate communication between a FPGA and external applications running on standard computers. • Compile the Timed Automata semantics of IS into FPGA. • Distributed extension. †Robert Trausmuth, Christian Dusek, and Yann Orlarey. Using FAUST for FPGA Programming. In Proceedings of the 9th International Conference on Digital Audio Effects, pages 18–20, 2006 Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 23/23 23/23
(e.g., FAUST). • Implement a Fast Ethernet module to provide a reliable and low-rate communication between a FPGA and external applications running on standard computers†. • Compile the Timed Automata semantics of IS into FPGA. • Distributed extension. †Rimas Aviziensis, Adrian Freed, Takahiko Suzuki, and David Wessel. Scalable Connectivity Processor for Computer Music Performance Systems. In Proceedings of the International Computer Music Conference, 2000 Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 23/23 23/23
(e.g., FAUST). • Implement a Fast Ethernet module to provide a reliable and low-rate communication between a FPGA and external applications running on standard computers. • Compile the Timed Automata semantics of IS into FPGA† [Done]. • Distributed extension. †Jaime Arias, Myriam Desainte-Catherine, and Camilo Rueda. A framework for composition, verification and real-time performance of multimedia interactive scenarios. In 15th International Conference on Application of Concurrency to System Design (ACSD’15), 2015. To appear Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 23/23 23/23
(e.g., FAUST). • Implement a Fast Ethernet module to provide a reliable and low-rate communication between a FPGA and external applications running on standard computers. • Compile the Timed Automata semantics of IS into FPGA. • Distributed extension†. †Frank Opitz, Edris Sahak, and Bernd Schwarz. Accelerating distributed computing with FPGAs. Xcell Journal, 79:20–27, Q2 2012 Arias, Desainte-Catherine, and Rueda (LaBRI) Exploiting Parallelism in FPGAs for the Real-Time Interpretation of IS 23/23 23/23
Multimedia Scores* Jaime Arias, Myriam Desainte-Catherine, and Camilo Rueda Laboratoire Bordelais de Recherche en Informatique (LaBRI) Université de Bordeaux Journées d’Informatique Musicale (JIM 2015) Montreal - Canada, May 2015 1 *Supported by the ANR Project OSSIA and SCRIME
Interactives pour la Composition et l’Exécution. PhD thesis, Université de Bordeaux, 2009. Jaime Arias, Myriam Desainte-Catherine, and Camilo Rueda. A framework for composition, verification and real-time performance of multimedia interactive scenarios. In 15th International Conference on Application of Concurrency to System Design (ACSD’15), 2015. To appear. Rimas Aviziensis, Adrian Freed, Takahiko Suzuki, and David Wessel. Scalable Connectivity Processor for Computer Music Performance Systems. In Proceedings of the International Computer Music Conference, 2000.
synthesis flows for DSP-centric FPGA design. Xcell Journal, 71:12–17, Q2 2010. Rahul Dubey. Introduction to Embedded System Design Using Field Programmable Gate Arrays. Springer London, London, 2009. Luc Langlois. Multirate digital signal processing for high-speed data converters for high-speed data converters. Xcell Journal, 73:50–53, Q4 2010. Edward A. Lee. The problem with threads. Computer, 39(5):33–42, May 2006. Frank Opitz, Edris Sahak, and Bernd Schwarz. Accelerating distributed computing with FPGAs. Xcell Journal, 79:20–27, Q2 2012.
Tools, and Application Domains of FPGAs. IEEE Transactions on Industrial Electronics, 54(4):1810–1823, August 2007. Robert Trausmuth, Christian Dusek, and Yann Orlarey. Using FAUST for FPGA Programming. In Proceedings of the 9th International Conference on Digital Audio Effects, pages 18–20, 2006. David Wessel and Matthew Wright. Problems and prospects for intimate musical control of computers. In Ivan Poupyrev, Michael J. Lyons, Sidney Fels, and Tina Blaine, editors, New Interfaces for Musical Expression, NIME-01, Proceedings, Seattle, April 1-2, 2001, pages 11–14, 2001.