Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Big Data in the Cloud - Welcome to cost oriented design
Search
Arnon Rotem-Gal-Oz
PRO
March 13, 2016
Technology
0
8
Big Data in the Cloud - Welcome to cost oriented design
Cost consideration and design decisions when dealing with Big Data
Arnon Rotem-Gal-Oz
PRO
March 13, 2016
Tweet
Share
More Decks by Arnon Rotem-Gal-Oz
See All by Arnon Rotem-Gal-Oz
Brownfield Architecture transformations
arnonrgo
PRO
0
44
Software architecture 101
arnonrgo
PRO
0
660
Apache Spark - Overview
arnonrgo
PRO
0
32
Taking ML to production - a journey
arnonrgo
PRO
0
93
Topics in Distributed Systems
arnonrgo
PRO
0
17
Docker & Kubernetes
arnonrgo
PRO
0
10
Data Security @ the personal level
arnonrgo
PRO
0
13
Microservices it's deja vu all over again
arnonrgo
PRO
0
10
Big Data Overview
arnonrgo
PRO
0
7
Other Decks in Technology
See All in Technology
自動化と効率化のためにGitHub Actionsを使いこなそう
devops_vtj
4
470
composer dump-autoloadを「なんとなく使う」から「理解して使う」になる
akinoriakatsuka
0
220
AWSの生成AIサービス Amazon Bedrock入門!(2024/6月版)
minorun365
PRO
3
300
Janus
bkuhlmann
1
540
AI Risk Solution Study
tkhresk
0
110
AWS IAMのアンチパターン/AWSが考える最低権限実現へのアプローチ概略
htan
3
230
Okioに愛を込めて
ryunen344
3
660
アプリケーションが 正しく動作するということ - 自動テスト編 / Automated Testing
soudai
4
760
The Path to Zero Touch Production
ramimac
0
860
WebサーバーとPHP実行方式を きちんと理解してPHPランタイムを 適切に使い分ける - PHPカンファレンス福岡2024
ysknsid25
2
120
feature flag と OpenTelemetry
biwashi
4
860
私たちはなぜ事業責任者にならないといけないのか ~Webサービスを作る茨の道~
gimupop
4
760
Featured
See All Featured
Stop Working from a Prison Cell
hatefulcrawdad
266
19k
Done Done
chrislema
178
15k
Product Roadmaps are Hard
iamctodd
PRO
46
10k
Intergalactic Javascript Robots from Outer Space
tanoku
266
26k
Why You Should Never Use an ORM
jnunemaker
PRO
51
8.8k
What the flash - Photography Introduction
edds
64
11k
Teambox: Starting and Learning
jrom
129
8.5k
Pencils Down: Stop Designing & Start Developing
hursman
117
11k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
228
16k
A Modern Web Designer's Workflow
chriscoyier
689
190k
Bash Introduction
62gerente
606
210k
Being A Developer After 40
akosma
68
580k
Transcript
Big data in the cloud welcome to cost oriented
design Arnon Rotem-Gal-Oz Chief Data Officer
Cost oriented design Cheap (but hopefully cheaper) Caveat Emptor
None
security performance availability correctness scalability solution cost
“Traditional” design for cost • POC • (choose technologies, understand
constraints) • Size needed hardware • purchase HW + SW • Optimize
Amdocs TeraScale Source: https://www.youtube.com/watch?v=0N2KhuMKAxM
Optimize & maximize utilization
Big data in the cloud
None
Big Data Big Data
None
Analytic Database Vertica
security performance availability correctness scalability solution initial cost
security performance availability correctness scalability Initial cost Variable running costs
Dedicated Servers vs. S3 with EC2
Spot instances
security performance availability correctness scalability Initial cost Continuous cost optimization
variable cost
Analytics database (take 2) Redshift, TokuMX
vs.
Google BigQuery
What do users actually need? “You can’t always get what
you want, But if you try, sometimes, you might find you get what you need”The rolling stones
Initial cost variable cost continuous cost optimization
“COST ORIENTATION” YOU KEEP USING THESE WORDS I DON’T THINK
THEY MEAN WHAT YOU THINK THEY MEAN
None