Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
How we learned to love the Data Center Operatin...
Search
DevOps Pro
June 01, 2016
Technology
0
190
How we learned to love the Data Center Operating System
by Saulius Valatka
DevOps Pro Vilnius 2016
DevOps Pro
June 01, 2016
Tweet
Share
More Decks by DevOps Pro
See All by DevOps Pro
Practical Application Logging and Monitoring
devopspro
0
110
Google Cloud Platform for DevOps Professionals
devopspro
0
290
Dev vs Ops transformation to Agility DevOps culture
devopspro
1
240
Using Test Automation for Continuous Integration, Delivery and Quality
devopspro
0
90
How the hell do I run my microservices in production, and will it scale?
devopspro
0
130
Accelerating innovation through AWS Cloud technology
devopspro
0
89
The Field Guide to Understanding Declarative Systems
devopspro
0
82
DevOps is the answer! What was the question again?
devopspro
0
150
From Monolith to Docker Distributed Applications
devopspro
0
250
Other Decks in Technology
See All in Technology
開発者体験を定量的に把握する手法と活用事例
ham0215
0
160
あなたが人生で成功するための5つの普遍的法則 #jawsug #jawsdays2025 / 20250301 HEROZ
yoshidashingo
2
500
Postman AI Agent Builderで AI Agentic workflow のプロトタイピング / Prototyping AI Agentic Workflow with Postman AI Agent Builder
yokawasa
0
200
OSSの実装を参考にBedrockエージェントを作る
moritalous
2
370
最近のSRE支援ニーズ考察 | sogaoh's LT @ Road to SRE NEXT@札幌
sogaoh
PRO
1
170
マネコン操作いらず! TerraformでAWSインフラのコーディングに入門しよう
minorun365
PRO
2
280
How to benefit from the latest Keycloak features
ahus1
0
100
自分のやることに価値を見出だせるようになり、挑戦する勇気をもらったベイトソンの考え / Scrum Fest Fukuoka 2025
bonbon0605
0
180
いまからでも遅くない!コンテナでWebアプリを動かしてみよう!コンテナハンズオン編
nomu
0
200
JAWS FESTA 2024「バスロケ」GPS×サーバーレスの開発と運用の舞台裏/jawsfesta2024-bus-gps-serverless
ma2shita
3
420
AWSではじめる Web APIテスト実践ガイド / A practical guide to testing Web APIs on AWS
yokawasa
8
840
2025/3/1 公共交通オープンデータデイ2025
morohoshi
0
130
Featured
See All Featured
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
134
33k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
666
120k
Adopting Sorbet at Scale
ufuk
75
9.2k
Building a Modern Day E-commerce SEO Strategy
aleyda
38
7.1k
What's in a price? How to price your products and services
michaelherold
244
12k
No one is an island. Learnings from fostering a developers community.
thoeni
21
3.2k
It's Worth the Effort
3n
184
28k
Fashionably flexible responsive web design (full day workshop)
malarkey
406
66k
Rebuilding a faster, lazier Slack
samanthasiow
80
8.9k
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
kevinliebholz
30
4.6k
For a Future-Friendly Web
brad_frost
176
9.6k
Designing for Performance
lara
605
68k
Transcript
HOW WE LEARNED TO LOVE THE DATA CENTER OPERATING SYSTEM
SAULIUS VALATKA / ADFORM
Online Advertising Full Stack Platform
Online Advertising Full Stack Platform Realtime “smart“ ads Forecasting, fraud
detection, etc.
Online Advertising Full Stack Platform Realtime “smart“ ads Forecasting, fraud
detection, etc. 1mln QPS under 100ms 1TB daily data
EVOLUTION
MIDDLE AGES ctrtrain.ec2-aws.com test2.ec2-aws.com modelling.ec2-aws.com
THE TORTURE # yum install python R libboost-3.12 $ scp
script.R test.aws.com:/script.R # crontab -e
THE TORTURE # yum install python R libboost-3.12 $ scp
script.R test.aws.com:/script.R # crontab -e “strange, worked on my machine …”
RENAISSANCE ctrtrain.ec2-aws.com test2.ec2-aws.com worker-1.adform.com worker-2.adform.com ab34na3n ar2afga3n
None
CONTAINERIZE ! self contained artifacts isolated runtime basically no overhead
unified deployment
BUT WAIT … what about configuration ?
The twelve-factor app stores config in environment variables Env vars
are easy to change between deploys without changing any code There is little chance of them being checked into the code repo accidentally They are a language- and OS-agnostic standard
BUT WAIT … where do I log ? and what
about metrics ?
None
MODERN ERA e34sadf ab34na3n af4f5a4r aafde33a fa45daws faes4fa3 aaf444a2 fas3rfa4
None
MARATHON the init of the DCOS constraints deployment { "id":
“my-nginx", "container": { "type": "DOCKER", "docker": { "image": "nginx:1.7.7", "network": "BRIDGE", } }, "instances": 1, "cpus": 0.5, "mem": 128 }
SPRINT the exec of he DCOS will be open sourced
scheduler to follow!
MANAGING RESOURCES how much memory do I really need ?
MANAGING RESOURCES how much memory do I really need ?
and CPUs ? what does 0.5 CPUs mean anyway ?
MANAGING RESOURCES how much memory do I really need ?
and CPUs ? what does 0.5 CPUs mean anyway ? and what happens with the network ?
None
ISOLATION cgroups: cpu cpuset memory blkio net_cls
NETWORK ISOLATION Layer 3 routing software defined networks
ATOMIC AGE a4faw3f 4afsdgg asdf4faf se4faw aw3d3ff g4aefgsd 5gsdgr54s a4rff4afa
4f4qaf4
SERVICE DISCOVERY where is my app ? how do I
reach it ? won’t containers conflict about ports ?
MARATHON-LB
PERSISTENCE so .. where do I store my data ?
on the host ? won’t it disappear ?
PERSISTENCE / /opt/app/cache /var/lib/docker/devicemapper /var/lib/mesos/slave/volumes
PERSISTENCE / /opt/app/cache /opt/app/profile /var/lib/docker/devicemapper /var/lib/mesos/slave/volumes /mnt/sdc network block storage
FUTURE PLANS DC/OS IP per container Containerize all the things
@adforminsider