Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Performance Stability of Public Clouds
Search
xLeitix
April 03, 2019
Research
1
71
Performance Stability of Public Clouds
Talk given at VECS (automotive industry conference in Gothenburg)
xLeitix
April 03, 2019
Tweet
Share
More Decks by xLeitix
See All by xLeitix
2024_uzh_collo.pdf
xleitix
0
28
CrossFit: Fine-Grained Benchmarking of Serverless Application Performance Across Cloud Providers
xleitix
0
230
Unit testing performance using code microbenchmarks - how far are we?
xleitix
0
390
Developer-Targeted Performance Engineering (ZHAW Colloquium)
xleitix
0
74
Developer-Targeted Performance Engineering
xleitix
0
220
Cachematic – Automatic Invalidation in Application-Level Caching Systems
xleitix
0
110
AWS Lambda and #serverless. What’s all the fuzz about?
xleitix
1
600
Performance Testing in a Public Cloud - How bad is it really?
xleitix
0
440
Performance Testing of and in the Cloud
xleitix
1
390
Other Decks in Research
See All in Research
引力・斥力を制御可能なランダム部分集合の確率分布
wasyro
0
180
SSII2025 [TS1] 光学・物理原理に基づく深層画像生成
ssii
PRO
4
3.7k
業界横断 副業・兼業者の実態調査
fkske
0
190
20250605_新交通システム推進議連_熊本都市圏「車1割削減、渋滞半減、公共交通2倍」から考える地方都市交通政策
trafficbrain
0
510
NLP2025参加報告会 LT資料
hargon24
1
320
[輪講] SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features
nk35jk
2
560
RapidPen: AIエージェントによるペネトレーションテスト 初期侵入全自動化の研究
laysakura
0
1.6k
Adaptive Experimental Design for Efficient Average Treatment Effect Estimation and Treatment Choice
masakat0
0
130
ASSADS:ASMR動画に合わせて撫でられる感覚を提示するシステムの開発と評価 / ec75-shimizu
yumulab
1
390
Towards a More Efficient Reasoning LLM: AIMO2 Solution Summary and Introduction to Fast-Math Models
analokmaus
2
230
利用シーンを意識した推薦システム〜SpotifyとAmazonの事例から〜
kuri8ive
1
210
NLP Colloquium
junokim
1
160
Featured
See All Featured
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
194
16k
Stop Working from a Prison Cell
hatefulcrawdad
271
21k
Facilitating Awesome Meetings
lara
54
6.4k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
29
1.8k
Large-scale JavaScript Application Architecture
addyosmani
512
110k
Optimizing for Happiness
mojombo
379
70k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
48
2.9k
The World Runs on Bad Software
bkeepers
PRO
69
11k
Visualization
eitanlees
146
16k
Code Reviewing Like a Champion
maltzj
524
40k
Designing for Performance
lara
610
69k
VelocityConf: Rendering Performance Case Studies
addyosmani
332
24k
Transcript
Performance Stability of (Public) Clouds Philipp Leitner
[email protected]
@xLeitix
Chalmers !2 Cloud Computing Image Credit: https://www.networkworld.com/article/3195527/did-cloud-kill-backup.html
Chalmers !3 Some disclaimers before we get started …. Image
Credit: https://thenounproject.com/term/exclamation-mark/
Chalmers !4 Image Credit: https://nordicapis.com/living-in-the-cloud-stack-understanding-saas-paas-and-iaas-apis/
Chalmers !5 Cloud Usage in Automotive Source (Accenture): https://www.accenture.com/t20150914T170053__w__/us-en/_acnmedia/Accenture/Conversion-Assets/DotCom/Documents/Global/PDF/ Industries_18/Accenture-Cloud-Automative-PoV.pdf
Chalmers !6 Cloud Usage in Automotive Source (Accenture): https://www.accenture.com/t20150914T170053__w__/us-en/_acnmedia/Accenture/Conversion-Assets/DotCom/Documents/Global/PDF/ Industries_18/Accenture-Cloud-Automative-PoV.pdf
Chalmers !7 (One) Challenge for Cloud Adoption in Automotive: Predictability
Image Credit: http://chittagongit.com
Chalmers !8 Predictability Do I know what I will get?
Do I get the same thing every time? Image Credit: http://chittagongit.com
Chalmers !9 Aside: Cloud Instance Types (“flavors”) Image Credit (Rightscale):
https://www.rightscale.com/about-cloud-management/cloud-cost-optimization/cloud-pricing-comparison
Chalmers !10 Predictability Inter-Instance Intra-Instance
Chalmers !11 Predictability Inter-Instance Intra-Instance
Chalmers !12 Predictability Inter-Instance Intra-Instance
Chalmers !13 Source (Leitner and Cito): https://arxiv.org/pdf/1411.2429.pdf
Chalmers !14 Relative Standard Deviations Benchmarks of identical instances Source
(Leitner and Cito): https://arxiv.org/pdf/1411.2429.pdf (anno ~ 2015)
Chalmers !15 Recent Results (unpublished data) (Feb 2019) 2015
Chalmers !16 Instance Runtime (unpublished data) (Feb 2019) 2015 3.5
4.0 4.5 5.0 5.5 0 20 40 60 Benchmark Runtime [h] Benchmark Value Continuous io azure D2s
Chalmers !17 Changes Over the Years (mean of all measurements)
Chalmers !18 2015 2019 CPU 8.1 3.6 - 55% Changes
Over the Years (mean of all measurements)
Chalmers !19 2015 2019 CPU 8.1 3.6 - 55% MEM
12.6 6.5 - 48% Changes Over the Years (mean of all measurements)
Chalmers !20 2015 2019 CPU 8.1 3.6 - 55% MEM
12.6 6.5 - 48% IO 38.6 15.9 - 59% Changes Over the Years (mean of all measurements)
Chalmers !21 What has changed?
Chalmers !22 For IO: multi-tenancy For CPU: hardware heterogeneity Traditional
Reasons for Lack of Predictability
Chalmers !23 Reason 0: More commitment to predictable performance levels
and transparency
Chalmers !24 Reason 1: Move towards guaranteed hardware
Chalmers !25 (anno ~ 2015) Heterogenous Hardware? Source (Leitner and
Cito): https://arxiv.org/pdf/1411.2429.pdf
Chalmers !26 (anno ~ 2015) Heterogenous Hardware? Source (Leitner and
Cito): https://arxiv.org/pdf/1411.2429.pdf (now) (Largely) guaranteed hardware
Chalmers !27 Reason 2: Move towards SLAs and credit systems
over best-effort delivery
Chalmers !28 (anno ~ 2015) Best-Effort Delivery? Source (Leitner and
Cito): https://arxiv.org/pdf/1411.2429.pdf
Chalmers !29 (anno ~ 2015) Best-Effort Delivery? Source (Leitner and
Cito): https://arxiv.org/pdf/1411.2429.pdf (unpublished data) (now) 0 5 10 15 20 25 0 50 100 150 200 Benchmark Runtime [h] Benchmark Value c5−large / IO
Chalmers !30 Credit Models - General Idea Resources are distributed
fairly between tenants based on usage tokens Available for: CPU (in case of shared CPU instance types) IO (some providers)
Chalmers !31 Credit Models at Runtime Source (Leitner and Scheuner):
https://www.zora.uzh.ch/id/eprint/112940/
Chalmers !32 Summary Public clouds are not all that unpredictable
(anymore)
Chalmers !33 Summary Public clouds are not all that unpredictable
(anymore) … useful even for workloads sensitive to performance variation … but it’s still virtualized infrastructure
Chalmers !34 Summary Public clouds are not all that unpredictable
(anymore) New developments have changed the game: Specialized hardware, credit models, provisioned IOPS
Chalmers !35 Cloud Workbench Tool for scheduling cloud experiments Code:
https://github.com/sealuzh/cloud-workbench Demo: https://www.youtube.com/watch? v=0yGFGvHvobk
Chalmers !36 Questions? Source: https://dilbert.com/strip/2008-05-08