Performance Stability of Public Clouds

Talk given at VECS (automotive industry conference in Gothenburg)

Transcript

1. Performance Stability of (Public) Clouds Philipp Leitner philipp.leitner@chalmers.se @xLeitix
2. Chalmers !2 Cloud Computing Image Credit: https://www.networkworld.com/article/3195527/did-cloud-kill-backup.html
3. Chalmers !3 Some disclaimers before we get started …. Image Credit: https://thenounproject.com/term/exclamation-mark/
4. Chalmers !4 Image Credit: https://nordicapis.com/living-in-the-cloud-stack-understanding-saas-paas-and-iaas-apis/
5. Chalmers !5 Cloud Usage in Automotive Source (Accenture): https://www.accenture.com/t20150914T170053__w__/us-en/_acnmedia/Accenture/Conversion-Assets/DotCom/Documents/Global/PDF/ Industries_18/Accenture-Cloud-Automative-PoV.pdf
6. Chalmers !6 Cloud Usage in Automotive Source (Accenture): https://www.accenture.com/t20150914T170053__w__/us-en/_acnmedia/Accenture/Conversion-Assets/DotCom/Documents/Global/PDF/ Industries_18/Accenture-Cloud-Automative-PoV.pdf
7. Chalmers !7 (One) Challenge for Cloud Adoption in Automotive: Predictability Image Credit: http://chittagongit.com
8. Chalmers !8 Predictability Do I know what I will get? Do I get the same thing every time? Image Credit: http://chittagongit.com
9. Chalmers !9 Aside: Cloud Instance Types (“flavors”) Image Credit (Rightscale): https://www.rightscale.com/about-cloud-management/cloud-cost-optimization/cloud-pricing-comparison
10. Chalmers !10 Predictability Inter-Instance Intra-Instance
11. Chalmers !11 Predictability Inter-Instance Intra-Instance
12. Chalmers !12 Predictability Inter-Instance Intra-Instance
13. Chalmers !13 Source (Leitner and Cito): https://arxiv.org/pdf/1411.2429.pdf
14. Chalmers !14 Relative Standard Deviations Benchmarks of identical instances Source (Leitner and Cito): https://arxiv.org/pdf/1411.2429.pdf (anno ~ 2015)
15. Chalmers !15 Recent Results (unpublished data) (Feb 2019) 2015
16. Chalmers !16 Instance Runtime (unpublished data) (Feb 2019) 2015 3.5 4.0 4.5 5.0 5.5 0 20 40 60 Benchmark Runtime [h] Benchmark Value Continuous io azure D2s
17. Chalmers !17 Changes Over the Years (mean of all measurements)
18. Chalmers !18 2015 2019 CPU 8.1 3.6 - 55% Changes Over the Years (mean of all measurements)
19. Chalmers !19 2015 2019 CPU 8.1 3.6 - 55% MEM 12.6 6.5 - 48% Changes Over the Years (mean of all measurements)
20. Chalmers !20 2015 2019 CPU 8.1 3.6 - 55% MEM 12.6 6.5 - 48% IO 38.6 15.9 - 59% Changes Over the Years (mean of all measurements)
21. Chalmers !21 What has changed?
22. Chalmers !22 For IO: multi-tenancy For CPU: hardware heterogeneity Traditional Reasons for Lack of Predictability
23. Chalmers !23 Reason 0: More commitment to predictable performance levels and transparency
24. Chalmers !24 Reason 1: Move towards guaranteed hardware
25. Chalmers !25 (anno ~ 2015) Heterogenous Hardware? Source (Leitner and Cito): https://arxiv.org/pdf/1411.2429.pdf
26. Chalmers !26 (anno ~ 2015) Heterogenous Hardware? Source (Leitner and Cito): https://arxiv.org/pdf/1411.2429.pdf (now) (Largely) guaranteed hardware
27. Chalmers !27 Reason 2: Move towards SLAs and credit systems over best-effort delivery
28. Chalmers !28 (anno ~ 2015) Best-Effort Delivery? Source (Leitner and Cito): https://arxiv.org/pdf/1411.2429.pdf
29. Chalmers !29 (anno ~ 2015) Best-Effort Delivery? Source (Leitner and Cito): https://arxiv.org/pdf/1411.2429.pdf (unpublished data) (now) 0 5 10 15 20 25 0 50 100 150 200 Benchmark Runtime [h] Benchmark Value c5−large / IO
30. Chalmers !30 Credit Models - General Idea Resources are distributed fairly between tenants based on usage tokens Available for: CPU (in case of shared CPU instance types) IO (some providers)
31. Chalmers !31 Credit Models at Runtime Source (Leitner and Scheuner): https://www.zora.uzh.ch/id/eprint/112940/
32. Chalmers !32 Summary Public clouds are not all that unpredictable (anymore)
33. Chalmers !33 Summary Public clouds are not all that unpredictable (anymore) … useful even for workloads sensitive to performance variation … but it’s still virtualized infrastructure
34. Chalmers !34 Summary Public clouds are not all that unpredictable (anymore) New developments have changed the game: Specialized hardware, credit models, provisioned IOPS
35. Chalmers !35 Cloud Workbench Tool for scheduling cloud experiments Code: https://github.com/sealuzh/cloud-workbench Demo: https://www.youtube.com/watch? v=0yGFGvHvobk
36. Chalmers !36 Questions? Source: https://dilbert.com/strip/2008-05-08