Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Performance Stability of Public Clouds
Search
xLeitix
April 03, 2019
Research
1
74
Performance Stability of Public Clouds
Talk given at VECS (automotive industry conference in Gothenburg)
xLeitix
April 03, 2019
Tweet
Share
More Decks by xLeitix
See All by xLeitix
Presentation WASP Software Technology Cluster 2025
xleitix
0
100
2024_uzh_collo.pdf
xleitix
0
33
CrossFit: Fine-Grained Benchmarking of Serverless Application Performance Across Cloud Providers
xleitix
0
250
Unit testing performance using code microbenchmarks - how far are we?
xleitix
0
400
Developer-Targeted Performance Engineering (ZHAW Colloquium)
xleitix
0
77
Developer-Targeted Performance Engineering
xleitix
0
220
Cachematic – Automatic Invalidation in Application-Level Caching Systems
xleitix
0
120
AWS Lambda and #serverless. What’s all the fuzz about?
xleitix
1
610
Performance Testing in a Public Cloud - How bad is it really?
xleitix
0
440
Other Decks in Research
See All in Research
Panopticon: Advancing Any-Sensor Foundation Models for Earth Observation
satai
3
220
論文紹介:Not All Tokens Are What You Need for Pretraining
kosuken
0
200
AI in Enterprises - Java and Open Source to the Rescue
ivargrimstad
0
580
2025年度人工知能学会全国大会チュートリアル講演「深層基盤モデルの数理」
taiji_suzuki
25
19k
CVPR2025論文紹介:Unboxed
murakawatakuya
0
180
単施設でできる臨床研究の考え方
shuntaros
0
3k
EcoWikiRS: Learning Ecological Representation of Satellite Images from Weak Supervision with Species Observation and Wikipedia
satai
3
250
Combinatorial Search with Generators
kei18
0
950
長期・短期メモリを活用したエージェントの個別最適化
isidaitc
0
170
Submeter-level land cover mapping of Japan
satai
3
410
snlp2025_prevent_llm_spikes
takase
0
360
診断前の病歴テキストを対象としたLLMによるエンティティリンキング精度検証
hagino3000
1
150
Featured
See All Featured
Speed Design
sergeychernyshev
32
1.2k
Site-Speed That Sticks
csswizardry
11
890
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
kevinliebholz
34
6.1k
Typedesign – Prime Four
hannesfritz
42
2.8k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
49
3.1k
Rebuilding a faster, lazier Slack
samanthasiow
84
9.2k
GitHub's CSS Performance
jonrohan
1032
470k
A Modern Web Designer's Workflow
chriscoyier
697
190k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
54
3k
Raft: Consensus for Rubyists
vanstee
139
7.1k
A Tale of Four Properties
chriscoyier
160
23k
Art, The Web, and Tiny UX
lynnandtonic
303
21k
Transcript
Performance Stability of (Public) Clouds Philipp Leitner
[email protected]
@xLeitix
Chalmers !2 Cloud Computing Image Credit: https://www.networkworld.com/article/3195527/did-cloud-kill-backup.html
Chalmers !3 Some disclaimers before we get started …. Image
Credit: https://thenounproject.com/term/exclamation-mark/
Chalmers !4 Image Credit: https://nordicapis.com/living-in-the-cloud-stack-understanding-saas-paas-and-iaas-apis/
Chalmers !5 Cloud Usage in Automotive Source (Accenture): https://www.accenture.com/t20150914T170053__w__/us-en/_acnmedia/Accenture/Conversion-Assets/DotCom/Documents/Global/PDF/ Industries_18/Accenture-Cloud-Automative-PoV.pdf
Chalmers !6 Cloud Usage in Automotive Source (Accenture): https://www.accenture.com/t20150914T170053__w__/us-en/_acnmedia/Accenture/Conversion-Assets/DotCom/Documents/Global/PDF/ Industries_18/Accenture-Cloud-Automative-PoV.pdf
Chalmers !7 (One) Challenge for Cloud Adoption in Automotive: Predictability
Image Credit: http://chittagongit.com
Chalmers !8 Predictability Do I know what I will get?
Do I get the same thing every time? Image Credit: http://chittagongit.com
Chalmers !9 Aside: Cloud Instance Types (“flavors”) Image Credit (Rightscale):
https://www.rightscale.com/about-cloud-management/cloud-cost-optimization/cloud-pricing-comparison
Chalmers !10 Predictability Inter-Instance Intra-Instance
Chalmers !11 Predictability Inter-Instance Intra-Instance
Chalmers !12 Predictability Inter-Instance Intra-Instance
Chalmers !13 Source (Leitner and Cito): https://arxiv.org/pdf/1411.2429.pdf
Chalmers !14 Relative Standard Deviations Benchmarks of identical instances Source
(Leitner and Cito): https://arxiv.org/pdf/1411.2429.pdf (anno ~ 2015)
Chalmers !15 Recent Results (unpublished data) (Feb 2019) 2015
Chalmers !16 Instance Runtime (unpublished data) (Feb 2019) 2015 3.5
4.0 4.5 5.0 5.5 0 20 40 60 Benchmark Runtime [h] Benchmark Value Continuous io azure D2s
Chalmers !17 Changes Over the Years (mean of all measurements)
Chalmers !18 2015 2019 CPU 8.1 3.6 - 55% Changes
Over the Years (mean of all measurements)
Chalmers !19 2015 2019 CPU 8.1 3.6 - 55% MEM
12.6 6.5 - 48% Changes Over the Years (mean of all measurements)
Chalmers !20 2015 2019 CPU 8.1 3.6 - 55% MEM
12.6 6.5 - 48% IO 38.6 15.9 - 59% Changes Over the Years (mean of all measurements)
Chalmers !21 What has changed?
Chalmers !22 For IO: multi-tenancy For CPU: hardware heterogeneity Traditional
Reasons for Lack of Predictability
Chalmers !23 Reason 0: More commitment to predictable performance levels
and transparency
Chalmers !24 Reason 1: Move towards guaranteed hardware
Chalmers !25 (anno ~ 2015) Heterogenous Hardware? Source (Leitner and
Cito): https://arxiv.org/pdf/1411.2429.pdf
Chalmers !26 (anno ~ 2015) Heterogenous Hardware? Source (Leitner and
Cito): https://arxiv.org/pdf/1411.2429.pdf (now) (Largely) guaranteed hardware
Chalmers !27 Reason 2: Move towards SLAs and credit systems
over best-effort delivery
Chalmers !28 (anno ~ 2015) Best-Effort Delivery? Source (Leitner and
Cito): https://arxiv.org/pdf/1411.2429.pdf
Chalmers !29 (anno ~ 2015) Best-Effort Delivery? Source (Leitner and
Cito): https://arxiv.org/pdf/1411.2429.pdf (unpublished data) (now) 0 5 10 15 20 25 0 50 100 150 200 Benchmark Runtime [h] Benchmark Value c5−large / IO
Chalmers !30 Credit Models - General Idea Resources are distributed
fairly between tenants based on usage tokens Available for: CPU (in case of shared CPU instance types) IO (some providers)
Chalmers !31 Credit Models at Runtime Source (Leitner and Scheuner):
https://www.zora.uzh.ch/id/eprint/112940/
Chalmers !32 Summary Public clouds are not all that unpredictable
(anymore)
Chalmers !33 Summary Public clouds are not all that unpredictable
(anymore) … useful even for workloads sensitive to performance variation … but it’s still virtualized infrastructure
Chalmers !34 Summary Public clouds are not all that unpredictable
(anymore) New developments have changed the game: Specialized hardware, credit models, provisioned IOPS
Chalmers !35 Cloud Workbench Tool for scheduling cloud experiments Code:
https://github.com/sealuzh/cloud-workbench Demo: https://www.youtube.com/watch? v=0yGFGvHvobk
Chalmers !36 Questions? Source: https://dilbert.com/strip/2008-05-08