Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Performance Stability of Public Clouds
Search
xLeitix
April 03, 2019
Research
1
65
Performance Stability of Public Clouds
Talk given at VECS (automotive industry conference in Gothenburg)
xLeitix
April 03, 2019
Tweet
Share
More Decks by xLeitix
See All by xLeitix
CrossFit: Fine-Grained Benchmarking of Serverless Application Performance Across Cloud Providers
xleitix
0
150
Unit testing performance using code microbenchmarks - how far are we?
xleitix
0
340
Developer-Targeted Performance Engineering (ZHAW Colloquium)
xleitix
0
64
Developer-Targeted Performance Engineering
xleitix
0
200
Cachematic – Automatic Invalidation in Application-Level Caching Systems
xleitix
0
100
AWS Lambda and #serverless. What’s all the fuzz about?
xleitix
1
530
Performance Testing in a Public Cloud - How bad is it really?
xleitix
0
420
Performance Testing of and in the Cloud
xleitix
1
370
(Mis-)Adventures in Performance Engineering
xleitix
1
78
Other Decks in Research
See All in Research
#SRE論文紹介 Detection is Better Than Cure: A Cloud Incidents Perspective V. Ganatra et. al., ESEC/FSE’23
yuukit
3
950
[Human-AI Decision Making勉強会] 説明の更新はユーザにどのような影響をもたらすか
okoso
1
310
訓練データ作成のためのCloudCompareを利用した点群の手動ラベリング
kentaitakura
0
930
動物倫理学ことはじめ:人間以外の動物との倫理的な付き合い方を考える
takeshit_m
0
350
SSII2024 [OS3] 基盤モデル(オープニング)
ssii
PRO
0
280
SSII2024 [PD] SSII、次の30年への期待
ssii
PRO
2
1.3k
SSII2024 [OS1] 画像認識におけるモデル・データの共進化
ssii
PRO
0
380
自動運転・AIシステムの問題を賢く探す・賢く直す / Smart Search & Repair Techniques for Automated Driving Systems and AI Systems
ishikawafyu
0
140
初めての研究発表を成功させよう! スライド作成の基本
ayaco0
10
4.1k
独立成分分析を用いた埋め込み表現の視覚的な理解
momoseoyama
3
770
Harnessing the Power of Vicinity-Informed Analysis for Classification under Covariate Shift
nanofi
3
230
"多様な推薦"はユーザーの目にどう映るか
kuri8ive
3
260
Featured
See All Featured
Scaling GitHub
holman
458
140k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
29
2.5k
Building Adaptive Systems
keathley
34
2k
In The Pink: A Labor of Love
frogandcode
139
22k
Making Projects Easy
brettharned
111
5.7k
The Language of Interfaces
destraynor
151
23k
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
13
430
Building Applications with DynamoDB
mza
89
5.8k
Building Better People: How to give real-time feedback that sticks.
wjessup
357
18k
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
90
47k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
360
22k
Build The Right Thing And Hit Your Dates
maggiecrowley
28
2.2k
Transcript
Performance Stability of (Public) Clouds Philipp Leitner
[email protected]
@xLeitix
Chalmers !2 Cloud Computing Image Credit: https://www.networkworld.com/article/3195527/did-cloud-kill-backup.html
Chalmers !3 Some disclaimers before we get started …. Image
Credit: https://thenounproject.com/term/exclamation-mark/
Chalmers !4 Image Credit: https://nordicapis.com/living-in-the-cloud-stack-understanding-saas-paas-and-iaas-apis/
Chalmers !5 Cloud Usage in Automotive Source (Accenture): https://www.accenture.com/t20150914T170053__w__/us-en/_acnmedia/Accenture/Conversion-Assets/DotCom/Documents/Global/PDF/ Industries_18/Accenture-Cloud-Automative-PoV.pdf
Chalmers !6 Cloud Usage in Automotive Source (Accenture): https://www.accenture.com/t20150914T170053__w__/us-en/_acnmedia/Accenture/Conversion-Assets/DotCom/Documents/Global/PDF/ Industries_18/Accenture-Cloud-Automative-PoV.pdf
Chalmers !7 (One) Challenge for Cloud Adoption in Automotive: Predictability
Image Credit: http://chittagongit.com
Chalmers !8 Predictability Do I know what I will get?
Do I get the same thing every time? Image Credit: http://chittagongit.com
Chalmers !9 Aside: Cloud Instance Types (“flavors”) Image Credit (Rightscale):
https://www.rightscale.com/about-cloud-management/cloud-cost-optimization/cloud-pricing-comparison
Chalmers !10 Predictability Inter-Instance Intra-Instance
Chalmers !11 Predictability Inter-Instance Intra-Instance
Chalmers !12 Predictability Inter-Instance Intra-Instance
Chalmers !13 Source (Leitner and Cito): https://arxiv.org/pdf/1411.2429.pdf
Chalmers !14 Relative Standard Deviations Benchmarks of identical instances Source
(Leitner and Cito): https://arxiv.org/pdf/1411.2429.pdf (anno ~ 2015)
Chalmers !15 Recent Results (unpublished data) (Feb 2019) 2015
Chalmers !16 Instance Runtime (unpublished data) (Feb 2019) 2015 3.5
4.0 4.5 5.0 5.5 0 20 40 60 Benchmark Runtime [h] Benchmark Value Continuous io azure D2s
Chalmers !17 Changes Over the Years (mean of all measurements)
Chalmers !18 2015 2019 CPU 8.1 3.6 - 55% Changes
Over the Years (mean of all measurements)
Chalmers !19 2015 2019 CPU 8.1 3.6 - 55% MEM
12.6 6.5 - 48% Changes Over the Years (mean of all measurements)
Chalmers !20 2015 2019 CPU 8.1 3.6 - 55% MEM
12.6 6.5 - 48% IO 38.6 15.9 - 59% Changes Over the Years (mean of all measurements)
Chalmers !21 What has changed?
Chalmers !22 For IO: multi-tenancy For CPU: hardware heterogeneity Traditional
Reasons for Lack of Predictability
Chalmers !23 Reason 0: More commitment to predictable performance levels
and transparency
Chalmers !24 Reason 1: Move towards guaranteed hardware
Chalmers !25 (anno ~ 2015) Heterogenous Hardware? Source (Leitner and
Cito): https://arxiv.org/pdf/1411.2429.pdf
Chalmers !26 (anno ~ 2015) Heterogenous Hardware? Source (Leitner and
Cito): https://arxiv.org/pdf/1411.2429.pdf (now) (Largely) guaranteed hardware
Chalmers !27 Reason 2: Move towards SLAs and credit systems
over best-effort delivery
Chalmers !28 (anno ~ 2015) Best-Effort Delivery? Source (Leitner and
Cito): https://arxiv.org/pdf/1411.2429.pdf
Chalmers !29 (anno ~ 2015) Best-Effort Delivery? Source (Leitner and
Cito): https://arxiv.org/pdf/1411.2429.pdf (unpublished data) (now) 0 5 10 15 20 25 0 50 100 150 200 Benchmark Runtime [h] Benchmark Value c5−large / IO
Chalmers !30 Credit Models - General Idea Resources are distributed
fairly between tenants based on usage tokens Available for: CPU (in case of shared CPU instance types) IO (some providers)
Chalmers !31 Credit Models at Runtime Source (Leitner and Scheuner):
https://www.zora.uzh.ch/id/eprint/112940/
Chalmers !32 Summary Public clouds are not all that unpredictable
(anymore)
Chalmers !33 Summary Public clouds are not all that unpredictable
(anymore) … useful even for workloads sensitive to performance variation … but it’s still virtualized infrastructure
Chalmers !34 Summary Public clouds are not all that unpredictable
(anymore) New developments have changed the game: Specialized hardware, credit models, provisioned IOPS
Chalmers !35 Cloud Workbench Tool for scheduling cloud experiments Code:
https://github.com/sealuzh/cloud-workbench Demo: https://www.youtube.com/watch? v=0yGFGvHvobk
Chalmers !36 Questions? Source: https://dilbert.com/strip/2008-05-08