VISP Testbed - A Toolkit for Modeling and Evaluating Resource Provisioning Algorithms for Stream Processing Applications

Presentation at the ZEUS workshop 2017 in Lugano, Switzerland.

Transcript

1. VISP TESTBED A Toolkit for Modeling and Evaluating Resource Provisioning Algorithms for Stream Processing Applications Christoph Hochreiner
2. Motivation
3. Stream Processing Application 3 Temperature Filter (1) Transform Data (3) Calculate OEE (4) Update Dashboard (5) Analyze Temperature (2) Production Data Sensor Temperature Sensor Stream Processing Operator Data Transmission Manufacturing Machine
4. Stream Processing Application Deployment 4 Temperature Filter (1) Transform Data (3) Calculate OEE (4) Update Dashboard (5) Analyze Temperature (2) Production Da Temperature Stream Proce Operator Data Transmis Manufacturin Machine Incoming Data Time 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 3 4 3 4 3 4 1 2 5 3 4 1 Stream Processing Operator Computational Resources
5. Stream Processing Application Deployment 4 Temperature Filter (1) Transform Data (3) Calculate OEE (4) Update Dashboard (5) Analyze Temperature (2) Production Da Temperature Stream Proce Operator Data Transmis Manufacturin Machine Incoming Data Time 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 3 4 3 4 3 4 1 2 5 3 4 1 Stream Processing Operator Computational Resources
6. Stream Processing Application Deployment 5 Temperature Filter (1) Transform Data (3) Calculate OEE (4) Update Dashboard (5) Analyze Temperature (2) Production Da Temperature Stream Proce Operator Data Transmis Manufacturin Machine Incoming Data Time 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 3 4 3 4 3 4 1 2 5 3 4 1 Stream Processing Operator Computational Resources
7. Stream Processing Application Deployment 5 Temperature Filter (1) Transform Data (3) Calculate OEE (4) Update Dashboard (5) Analyze Temperature (2) Production Da Temperature Stream Proce Operator Data Transmis Manufacturin Machine Incoming Data Time 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 3 4 3 4 3 4 1 2 5 3 4 1 Stream Processing Operator Computational Resources
8. Stream Processing Application Deployment 6 Temperature Filter (1) Transform Data (3) Calculate OEE (4) Update Dashboard (5) Analyze Temperature (2) Production Da Temperature Stream Proce Operator Data Transmis Manufacturin Machine Incoming Data Time 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 3 4 3 4 3 4 1 2 5 3 4 1 Stream Processing Operator Computational Resources
9. Stream Processing Application Deployment 6 Temperature Filter (1) Transform Data (3) Calculate OEE (4) Update Dashboard (5) Analyze Temperature (2) Production Da Temperature Stream Proce Operator Data Transmis Manufacturin Machine Incoming Data Time 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 3 4 3 4 3 4 1 2 5 3 4 1 Stream Processing Operator Computational Resources
10. Steps to design resource provisioning algorithms 7 Design algorithm Evaluate algorithm Benchmark algorithm
11. VISP Testbed
12. VISP Testbed 9 Data Provider VISP Runtime Data Processing Resource Provisioning Resource Optimization QoS Monitoring Activity Tracking Reporting Graph Generation Data Aggregation KPI Calculation Monitoring Data Monitoring Data Data Stream Arrival Pattern Generation Speed Generation Duration Baseline Algorithm Custom Algorithm Topologies
13. VISP Runtime 10 Data Provider VISP Runtime Data Processing Resource Provisioning Resource Optimization QoS Monitoring Activity Tracking Reporting Graph Generation Data Aggregation KPI Calculation Monitoring Data Monitoring Data Data Stream Arrival Pattern Generation Speed Generation Duration Baseline Algorithm Custom Algorithm Topologies Data Provider VISP Runtime Data Processing Resource Provisioning Resource Optimization QoS Monitoring Activity Tracking Reporting Graph Generation Data Aggregation KPI Calculation Monitoring Data Monitoring Data Data Stream Arrival Pattern Generation Speed Generation Duration Baseline Algorithm Custom Algorithm Topologies Data Provider VISP Runtime Data Processing Resource Provisioning Resource Optimization QoS Monitoring Activity Tracking Reporting Graph Generation Data Aggregation KPI Calculation Monitoring Data Monitoring Data Data Stream Arrival Pattern Generation Speed Generation Duration Baseline Algorithm Custom Algorithm Topologies
14. VISP Runtime 10 Data Provider VISP Runtime Data Processing Resource Provisioning Resource Optimization QoS Monitoring Activity Tracking Reporting Graph Generation Data Aggregation KPI Calculation Monitoring Data Monitoring Data Data Stream Arrival Pattern Generation Speed Generation Duration Baseline Algorithm Custom Algorithm Topologies Data Provider VISP Runtime Data Processing Resource Provisioning Resource Optimization QoS Monitoring Activity Tracking Reporting Graph Generation Data Aggregation KPI Calculation Monitoring Data Monitoring Data Data Stream Arrival Pattern Generation Speed Generation Duration Baseline Algorithm Custom Algorithm Topologies Data Provider VISP Runtime Data Processing Resource Provisioning Resource Optimization QoS Monitoring Activity Tracking Reporting Graph Generation Data Aggregation KPI Calculation Monitoring Data Monitoring Data Data Stream Arrival Pattern Generation Speed Generation Duration Baseline Algorithm Custom Algorithm Topologies Process streaming data
15. VISP Runtime 10 Data Provider VISP Runtime Data Processing Resource Provisioning Resource Optimization QoS Monitoring Activity Tracking Reporting Graph Generation Data Aggregation KPI Calculation Monitoring Data Monitoring Data Data Stream Arrival Pattern Generation Speed Generation Duration Baseline Algorithm Custom Algorithm Topologies Data Provider VISP Runtime Data Processing Resource Provisioning Resource Optimization QoS Monitoring Activity Tracking Reporting Graph Generation Data Aggregation KPI Calculation Monitoring Data Monitoring Data Data Stream Arrival Pattern Generation Speed Generation Duration Baseline Algorithm Custom Algorithm Topologies Data Provider VISP Runtime Data Processing Resource Provisioning Resource Optimization QoS Monitoring Activity Tracking Reporting Graph Generation Data Aggregation KPI Calculation Monitoring Data Monitoring Data Data Stream Arrival Pattern Generation Speed Generation Duration Baseline Algorithm Custom Algorithm Topologies Route streaming data
16. VISP Runtime 10 Data Provider VISP Runtime Data Processing Resource Provisioning Resource Optimization QoS Monitoring Activity Tracking Reporting Graph Generation Data Aggregation KPI Calculation Monitoring Data Monitoring Data Data Stream Arrival Pattern Generation Speed Generation Duration Baseline Algorithm Custom Algorithm Topologies Data Provider VISP Runtime Data Processing Resource Provisioning Resource Optimization QoS Monitoring Activity Tracking Reporting Graph Generation Data Aggregation KPI Calculation Monitoring Data Monitoring Data Data Stream Arrival Pattern Generation Speed Generation Duration Baseline Algorithm Custom Algorithm Topologies Data Provider VISP Runtime Data Processing Resource Provisioning Resource Optimization QoS Monitoring Activity Tracking Reporting Graph Generation Data Aggregation KPI Calculation Monitoring Data Monitoring Data Data Stream Arrival Pattern Generation Speed Generation Duration Baseline Algorithm Custom Algorithm Topologies Provision computational resources
17. VISP Runtime 10 Data Provider VISP Runtime Data Processing Resource Provisioning Resource Optimization QoS Monitoring Activity Tracking Reporting Graph Generation Data Aggregation KPI Calculation Monitoring Data Monitoring Data Data Stream Arrival Pattern Generation Speed Generation Duration Baseline Algorithm Custom Algorithm Topologies Data Provider VISP Runtime Data Processing Resource Provisioning Resource Optimization QoS Monitoring Activity Tracking Reporting Graph Generation Data Aggregation KPI Calculation Monitoring Data Monitoring Data Data Stream Arrival Pattern Generation Speed Generation Duration Baseline Algorithm Custom Algorithm Topologies Data Provider VISP Runtime Data Processing Resource Provisioning Resource Optimization QoS Monitoring Activity Tracking Reporting Graph Generation Data Aggregation KPI Calculation Monitoring Data Monitoring Data Data Stream Arrival Pattern Generation Speed Generation Duration Baseline Algorithm Custom Algorithm Topologies Monitor activities
18. VISP Runtime 10 Data Provider VISP Runtime Data Processing Resource Provisioning Resource Optimization QoS Monitoring Activity Tracking Reporting Graph Generation Data Aggregation KPI Calculation Monitoring Data Monitoring Data Data Stream Arrival Pattern Generation Speed Generation Duration Baseline Algorithm Custom Algorithm Topologies Data Provider VISP Runtime Data Processing Resource Provisioning Resource Optimization QoS Monitoring Activity Tracking Reporting Graph Generation Data Aggregation KPI Calculation Monitoring Data Monitoring Data Data Stream Arrival Pattern Generation Speed Generation Duration Baseline Algorithm Custom Algorithm Topologies Data Provider VISP Runtime Data Processing Resource Provisioning Resource Optimization QoS Monitoring Activity Tracking Reporting Graph Generation Data Aggregation KPI Calculation Monitoring Data Monitoring Data Data Stream Arrival Pattern Generation Speed Generation Duration Baseline Algorithm Custom Algorithm Topologies Optimize resource provisioning
19. VISP Runtime - Base Line Algorithms ‣ Fixed Provisioning - Over Provisioning - Under Provisioning ‣ Threshold Based Provisioning - CPU - Memory - Queue Size 11 Data Provider VISP Runtime Data Processing Resource Provisioning Resource Optimization QoS Monitoring Activity Tracking Reporting Graph Generation Data Aggregation KPI Calculation Monitoring Data Monitoring Data Data Stream Arrival Pattern Generation Speed Generation Duration Baseline Algorithm Custom Algorithm Topologies
20. VISP Runtime - Topologies 12 Operator 1 Operator 2 Operator 4 Source Sink Operator 3 Operator 1 Operator 2 Operator 3 Source Sink Distance Speed Average Speed Aggregation Analysis Manufacturing Machine Manufacturing Machine Manufacturing Machine Calculate Performance (O3) Calculate Availability (O4) Calculate Quality (O5) Monitor Temperature (O6) Calculate OEE (O7) Filter Availability (O2) Generate Report (O9) Inform User (O8) Distribute Data (O1) Availability Sensor (S1) Production Sensor (S2) Temperature Sensor (S3) Data Provider VISP Runtime Data Processing Resource Provisioning Resource Optimization QoS Monitoring Activity Tracking Reporting Graph Generation Data Aggregation KPI Calculation Monitoring Data Monitoring Data Data Stream Arrival Pattern Generation Speed Generation Duration Baseline Algorithm Custom Algorithm Topologies
21. Data Provider Replay data streams with ‣ predefined arrival pattern at ‣ predefined generation velocities with ‣ predefined volumes and ‣ predefined message structures. 13 Data Provider VISP Runtime Data Processing Resource Provisioning Resource Optimization QoS Monitoring Activity Tracking Reporting Graph Generation Data Aggregation KPI Calculation Monitoring Data Monitoring Data Data Stream Arrival Pattern Generation Speed Generation Duration Baseline Algorithm Custom Algorithm Topologies
22. Data Provider - Arrival Pattern 14 Data Provider VISP Runtime Data Processing Resource Provisioning Resource Optimization QoS Monitoring Activity Tracking Reporting Graph Generation Data Aggregation KPI Calculation Monitoring Data Monitoring Data Data Stream Arrival Pattern Generation Speed Generation Duration Baseline Algorithm Custom Algorithm Topologies 0 5 10 15 20 0 5 10 15 20 25 30 35 Data Volume Time Real World Arrival
23. Data Provider - Arrival Pattern 15 Data Provider VISP Runtime Data Processing Resource Provisioning Resource Optimization QoS Monitoring Activity Tracking Reporting Graph Generation Data Aggregation KPI Calculation Monitoring Data Monitoring Data Data Stream Arrival Pattern Generation Speed Generation Duration Baseline Algorithm Custom Algorithm Topologies 0 5 10 15 20 0 5 10 15 20 25 30 35 Data Volume Time Constant Arrival Real World Arrival
24. Data Provider - Arrival Pattern 16 Data Provider VISP Runtime Data Processing Resource Provisioning Resource Optimization QoS Monitoring Activity Tracking Reporting Graph Generation Data Aggregation KPI Calculation Monitoring Data Monitoring Data Data Stream Arrival Pattern Generation Speed Generation Duration Baseline Algorithm Custom Algorithm Topologies 0 5 10 15 20 0 5 10 15 20 25 30 35 Data Volume Time Constant Increase Real World Arrival
25. Data Provider - Arrival Pattern 17 Data Provider VISP Runtime Data Processing Resource Provisioning Resource Optimization QoS Monitoring Activity Tracking Reporting Graph Generation Data Aggregation KPI Calculation Monitoring Data Monitoring Data Data Stream Arrival Pattern Generation Speed Generation Duration Baseline Algorithm Custom Algorithm Topologies 0 5 10 15 20 0 5 10 15 20 25 30 35 Data Volume Time Pulsating Arrival Real World Arrival
26. Data Provider - Arrival Pattern 18 Data Provider VISP Runtime Data Processing Resource Provisioning Resource Optimization QoS Monitoring Activity Tracking Reporting Graph Generation Data Aggregation KPI Calculation Monitoring Data Monitoring Data Data Stream Arrival Pattern Generation Speed Generation Duration Baseline Algorithm Custom Algorithm Topologies 0 5 10 15 20 0 5 10 15 20 25 30 35 Data Volume Time Spikes Real World Arrival
27. Data Provider - Arrival Pattern 19 Data Provider VISP Runtime Data Processing Resource Provisioning Resource Optimization QoS Monitoring Activity Tracking Reporting Graph Generation Data Aggregation KPI Calculation Monitoring Data Monitoring Data Data Stream Arrival Pattern Generation Speed Generation Duration Baseline Algorithm Custom Algorithm Topologies 0 5 10 15 20 0 5 10 15 20 25 30 35 Data Volume Time Constant Arrival Constant Increase Pulsating Arrival Spikes Real World Arrival
28. Data Provider - Message Structures ‣ Simple messages to trigger data processing events ‣ GPS time-series from taxis ‣ Sensor data from manufacturing machines 20 Data Provider VISP Runtime Data Processing Resource Provisioning Resource Optimization QoS Monitoring Activity Tracking Reporting Graph Generation Data Aggregation KPI Calculation Monitoring Data Monitoring Data Data Stream Arrival Pattern Generation Speed Generation Duration Baseline Algorithm Custom Algorithm Topologies
29. Reporting ‣ Aggregate information from monitoring data ‣ Generate KPIs ‣ Generate figures 21 Data Provider VISP Runtime Data Processing Resource Provisioning Resource Optimization QoS Monitoring Activity Tracking Reporting Graph Generation Data Aggregation KPI Calculation Monitoring Data Monitoring Data Data Stream Arrival Pattern Generation Speed Generation Duration Baseline Algorithm Custom Algorithm Topologies
30. Reporting 22 Data Provider VISP Runtime Data Processing Resource Provisioning Resource Optimization QoS Monitoring Activity Tracking Reporting Graph Generation Data Aggregation KPI Calculation Monitoring Data Monitoring Data Data Stream Arrival Pattern Generation Speed Generation Duration Baseline Algorithm Custom Algorithm Topologies
31. Conclusion
32. Conclusion VISP Testbed provides a toolkit ‣to replay data streams and reproduce evaluation results ‣to benchmark custom resource provisioning algorithm against existing ones 24
33. Outlook
34. Outlook ‣Increase benchmark algorithm library ‣Increase topology library and stream processing operator library ‣Integrate topologies and data from the DEBS Grand Challenges ‣Provide the VISP Testbed as a Service 26
35. Q & A Christoph Hochreiner c.hochreiner@infosys.tuwien.ac.at https://github.com/visp-streaming/