Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Performance Stability of Public Clouds
Search
xLeitix
April 03, 2019
Research
1
71
Performance Stability of Public Clouds
Talk given at VECS (automotive industry conference in Gothenburg)
xLeitix
April 03, 2019
Tweet
Share
More Decks by xLeitix
See All by xLeitix
2024_uzh_collo.pdf
xleitix
0
28
CrossFit: Fine-Grained Benchmarking of Serverless Application Performance Across Cloud Providers
xleitix
0
230
Unit testing performance using code microbenchmarks - how far are we?
xleitix
0
390
Developer-Targeted Performance Engineering (ZHAW Colloquium)
xleitix
0
74
Developer-Targeted Performance Engineering
xleitix
0
220
Cachematic – Automatic Invalidation in Application-Level Caching Systems
xleitix
0
110
AWS Lambda and #serverless. What’s all the fuzz about?
xleitix
1
600
Performance Testing in a Public Cloud - How bad is it really?
xleitix
0
440
Performance Testing of and in the Cloud
xleitix
1
390
Other Decks in Research
See All in Research
さくらインターネット研究所 アップデート2025年
matsumoto_r
PRO
0
660
3D Gaussian Splattingによる高効率な新規視点合成技術とその応用
muskie82
5
2.7k
ノンパラメトリック分布表現を用いた位置尤度場周辺化によるRTK-GNSSの整数アンビギュイティ推定
aoki_nosse
0
320
最適化と機械学習による問題解決
mickey_kubo
0
140
[輪講] SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features
nk35jk
2
560
90 分で学ぶ P 対 NP 問題
e869120
18
7.6k
CSP: Self-Supervised Contrastive Spatial Pre-Training for Geospatial-Visual Representations
satai
3
220
LLM-as-a-Judge: 文章をLLMで評価する@教育機関DXシンポ
k141303
3
830
EOGS: Gaussian Splatting for Efficient Satellite Image Photogrammetry
satai
4
290
RapidPen: AIエージェントによるペネトレーションテスト 初期侵入全自動化の研究
laysakura
0
1.6k
なめらかなシステムと運用維持の終わらぬ未来 / dicomo2025_coherently_fittable_system
monochromegane
0
620
Submeter-level land cover mapping of Japan
satai
3
120
Featured
See All Featured
Adopting Sorbet at Scale
ufuk
77
9.5k
GitHub's CSS Performance
jonrohan
1031
460k
Fashionably flexible responsive web design (full day workshop)
malarkey
407
66k
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
kevinliebholz
34
5.9k
Measuring & Analyzing Core Web Vitals
bluesmoon
7
510
How STYLIGHT went responsive
nonsquared
100
5.6k
Fantastic passwords and where to find them - at NoRuKo
philnash
51
3.3k
Into the Great Unknown - MozCon
thekraken
40
1.9k
The MySQL Ecosystem @ GitHub 2015
samlambert
251
13k
Building Flexible Design Systems
yeseniaperezcruz
328
39k
Principles of Awesome APIs and How to Build Them.
keavy
126
17k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
48
2.9k
Transcript
Performance Stability of (Public) Clouds Philipp Leitner
[email protected]
@xLeitix
Chalmers !2 Cloud Computing Image Credit: https://www.networkworld.com/article/3195527/did-cloud-kill-backup.html
Chalmers !3 Some disclaimers before we get started …. Image
Credit: https://thenounproject.com/term/exclamation-mark/
Chalmers !4 Image Credit: https://nordicapis.com/living-in-the-cloud-stack-understanding-saas-paas-and-iaas-apis/
Chalmers !5 Cloud Usage in Automotive Source (Accenture): https://www.accenture.com/t20150914T170053__w__/us-en/_acnmedia/Accenture/Conversion-Assets/DotCom/Documents/Global/PDF/ Industries_18/Accenture-Cloud-Automative-PoV.pdf
Chalmers !6 Cloud Usage in Automotive Source (Accenture): https://www.accenture.com/t20150914T170053__w__/us-en/_acnmedia/Accenture/Conversion-Assets/DotCom/Documents/Global/PDF/ Industries_18/Accenture-Cloud-Automative-PoV.pdf
Chalmers !7 (One) Challenge for Cloud Adoption in Automotive: Predictability
Image Credit: http://chittagongit.com
Chalmers !8 Predictability Do I know what I will get?
Do I get the same thing every time? Image Credit: http://chittagongit.com
Chalmers !9 Aside: Cloud Instance Types (“flavors”) Image Credit (Rightscale):
https://www.rightscale.com/about-cloud-management/cloud-cost-optimization/cloud-pricing-comparison
Chalmers !10 Predictability Inter-Instance Intra-Instance
Chalmers !11 Predictability Inter-Instance Intra-Instance
Chalmers !12 Predictability Inter-Instance Intra-Instance
Chalmers !13 Source (Leitner and Cito): https://arxiv.org/pdf/1411.2429.pdf
Chalmers !14 Relative Standard Deviations Benchmarks of identical instances Source
(Leitner and Cito): https://arxiv.org/pdf/1411.2429.pdf (anno ~ 2015)
Chalmers !15 Recent Results (unpublished data) (Feb 2019) 2015
Chalmers !16 Instance Runtime (unpublished data) (Feb 2019) 2015 3.5
4.0 4.5 5.0 5.5 0 20 40 60 Benchmark Runtime [h] Benchmark Value Continuous io azure D2s
Chalmers !17 Changes Over the Years (mean of all measurements)
Chalmers !18 2015 2019 CPU 8.1 3.6 - 55% Changes
Over the Years (mean of all measurements)
Chalmers !19 2015 2019 CPU 8.1 3.6 - 55% MEM
12.6 6.5 - 48% Changes Over the Years (mean of all measurements)
Chalmers !20 2015 2019 CPU 8.1 3.6 - 55% MEM
12.6 6.5 - 48% IO 38.6 15.9 - 59% Changes Over the Years (mean of all measurements)
Chalmers !21 What has changed?
Chalmers !22 For IO: multi-tenancy For CPU: hardware heterogeneity Traditional
Reasons for Lack of Predictability
Chalmers !23 Reason 0: More commitment to predictable performance levels
and transparency
Chalmers !24 Reason 1: Move towards guaranteed hardware
Chalmers !25 (anno ~ 2015) Heterogenous Hardware? Source (Leitner and
Cito): https://arxiv.org/pdf/1411.2429.pdf
Chalmers !26 (anno ~ 2015) Heterogenous Hardware? Source (Leitner and
Cito): https://arxiv.org/pdf/1411.2429.pdf (now) (Largely) guaranteed hardware
Chalmers !27 Reason 2: Move towards SLAs and credit systems
over best-effort delivery
Chalmers !28 (anno ~ 2015) Best-Effort Delivery? Source (Leitner and
Cito): https://arxiv.org/pdf/1411.2429.pdf
Chalmers !29 (anno ~ 2015) Best-Effort Delivery? Source (Leitner and
Cito): https://arxiv.org/pdf/1411.2429.pdf (unpublished data) (now) 0 5 10 15 20 25 0 50 100 150 200 Benchmark Runtime [h] Benchmark Value c5−large / IO
Chalmers !30 Credit Models - General Idea Resources are distributed
fairly between tenants based on usage tokens Available for: CPU (in case of shared CPU instance types) IO (some providers)
Chalmers !31 Credit Models at Runtime Source (Leitner and Scheuner):
https://www.zora.uzh.ch/id/eprint/112940/
Chalmers !32 Summary Public clouds are not all that unpredictable
(anymore)
Chalmers !33 Summary Public clouds are not all that unpredictable
(anymore) … useful even for workloads sensitive to performance variation … but it’s still virtualized infrastructure
Chalmers !34 Summary Public clouds are not all that unpredictable
(anymore) New developments have changed the game: Specialized hardware, credit models, provisioned IOPS
Chalmers !35 Cloud Workbench Tool for scheduling cloud experiments Code:
https://github.com/sealuzh/cloud-workbench Demo: https://www.youtube.com/watch? v=0yGFGvHvobk
Chalmers !36 Questions? Source: https://dilbert.com/strip/2008-05-08