Reproducible Distributed Experiments on Cloud

This is a project for creating a reproducible experiment for comparing Apache Flink and Apache Spark
Batch Processing.
https://github.com/karamel-lab/batch-processing-comparison

Transcript

1. Reproducible distributed experiments on cloud vs Shelan Perera Ashansa Perera Kamal Hakimzadeh
2. “ Reproducing experiments with minimal effort
3. Spark and Flink ▷ Batch Processing vs. Stream Processing ▷ Micro Batching vs. Natural Data Flow ▷ Good fit for scalable deployment in the cloud
4. Motivation ▷ Validate Performance claims ▷ Take off deployment overhead ▷ Design reproducible experiments
5. Karamel => “Framework for reproducible distributed experiments”
6. Benchmark - Batch Teragen - To generate data (Hadoop) Terasort - Benchmarking Algorithm (Spark, Flink)
7. To make Dongwon Kim’s comparison reproducible. http://www.slideshare.net/ssuser6bb12d/a- comparative-performance-evaluation-of-apache-flink
8. 1 Namenode ⇒ Master (Low processing ) 2 Worker nodes ⇒ Slaves (High processing ) Our Deployment
9. EC2 Slave EC2 Master Deployment Hadoop Name Node Spark Master Flink Job Manager Spark Worker Flink Task Manager Hadoop Data Node Karamel x 2 Karamel Config
10. Configuration Master / Namenode 2.6 4 16 80 Slave / Worker 2.5 16 122 1600 CPU (GHz) No of vCPUs Memory (GB) Storage :SSD (GB) (m3.xlarge) (i2.4xlarge)
11. Experiment Hadoop MR : Teragen HDFS Spark/Flink : Terasort 200/ 400/ 600 GB
12. Results Batch Processing
13. Application Performance
14. Flink 1.5 x Faster than Spark
15. ▷ Spark : Does not overlap stages ▷ Flink : Do pipelining Mainly because...
16. Collectl- Monitor • Tool used to collect and draw results. • https://github. com/shelan/collectl- monitoring
17. System Performance -CPU (%)
18. System Performance -Memory (GB)
19. System Performance -Disk (MB/s)
20. System Performance -Network (KB/s)
21. Load Balancing -Workers (CPU %)
22. Load Balancing -Workers (CPU %)
23. Outcome ▷ Performance Comparison Results ▷ Karamel experiments to reproduce the same results with minimal effort
24. How not to reproduce “our problems”
25. EC2 claims 800 GB disks, But Disk File system (DF) does shows only 30GB. If you are using I2 or R3 instances you should create a file system and partition disks manually.
26. Large Spark or Flink Batch applications can fail with not enough disk space Configure Flink temp directory and Spark local directory to a partition with at least enough space to store the total input.
27. Reproducing experiments on EC2 may cost you a lot Spot instances which allow to reduce the cost by 10x is also supported by Karamel
28. IncompatibleClassChangeError when running StreamBench built for MR2 on hadoop2.x No explicitly defined dependencies for previous versions, but one of the dependencies (mahout) had internal references to hadoop1.x jar
29. Summary ▷ Introducing reproducible experiments on cloud ▷ Performance Comparison of Spark and Flink ▷ Reproducible experiments are available online (https://github.com/karamel-lab)
30. Thanks ..!!