Benchmarking Public Clouds With Cloud Workbench - Do You Really Know What You Are Paying For?

Transcript

1. software evolution & architecture lab University of Zurich, Switzerland Benchmarking Public Clouds With Cloud Workbench Do You Really Know What You Are Paying For? Dr. Philipp Leitner @xLeitix
2. Joint Work with …
3. Capacity planning in public clouds is hard
4. What cloud provider should I choose? Should I go for many small or few large instances? General-purpose or *-optimized? Pay for better IOPS or not? …………… ➡ Need for Benchmarking Common Questions
5. Capacity planning in public clouds is hard
6. Capacity planning in public clouds is hard 20 30 40 0 20 40 60 Measurement Runtime [h] IO Bandwidth [Mb/s] Instance 9097 Instance 14704 Leitner and Cito, TOIT 2016 http://dl.acm.org/citation.cfm?id=2885497
7. https://segment.com/blog/the-million-dollar-eng-problem/
8. Code: https://github.com/ sealuzh/cloud- workbench Demo: https:// www.youtube.com/ watch?v=0yGFGvHvobk Cloud Workbench Scheuner et al., CloudCom 2014 http://ieeexplore.ieee.org/document/7037674/
9. A Simple CWB Benchmark (Optional) Step 1: Write Chef Cookbook
10. A Simple CWB Benchmark Step 1I: Define IaaS config and schedule
11. A Simple CWB Benchmark Step 1II: Execute and download result CSV
12. Selected Results I - Microbenchmarks Leitner and Cito, TOIT 2016 http://dl.acm.org/citation.cfm?id=2885497
13. Selected Results II - Web Application Performance Davatz et al., CCGrid 2017 https://www.merlin.uzh.ch/contributionDocument/download/9780
14. Selected Results II - Web Application Performance Davatz et al., CCGrid 2017 https://www.merlin.uzh.ch/contributionDocument/download/9780
15. software evolution & architecture lab Dr. Philipp Leitner University of Zurich, Switzerland Some insights from benchmarking EC2, GCE, Azure, and Softlayer More info: Philipp Leitner and Jürgen Cito. 2016. Patterns in the Chaos — A Study of Performance Variation and Predictability in Public IaaS Clouds. ACM Trans. Internet Technol. 16, 3, Article 15 (April 2016), 23 pages. DOI: http://dx.doi.org/10.1145/2885497 Philipp Leitner @xLeitix Main contacts: Joel Scheuner @joe4dev Christian Davatz, Christian Inzinger, Joel Scheuner, Philipp Leitner (2017). An Approach and Case Study of Cloud Instance Type Selection for Multi-Tier Web Applications. In Proceedings of the 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid) Joel Scheuner, Philipp Leitner, Jürgen Cito, Harald Gall (2014). Cloud WorkBench - Infrastructure-as-Code Based Cloud Benchmarking. In Proceedings of the 6th IEEE International Conference on Cloud Computing Technology and Science (CloudCom'14) Philipp Leitner, Joel Scheuner (2015). Bursting With Possibilities – an Empirical Study of Credit-Based Bursting Cloud Instance Types. In Proceedings of the 8th IEEE/ACM International Conference on Utility and Cloud Computing (UCC)