Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
How we learned to love the Data Center Operatin...
Search
DevOps Pro
June 01, 2016
Technology
270
0
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
How we learned to love the Data Center Operating System
by Saulius Valatka
DevOps Pro Vilnius 2016
DevOps Pro
June 01, 2016
Transcript
33 slides
HOW WE LEARNED TO LOVE THE DATA CENTER OPERATING SYSTEM
SAULIUS VALATKA / ADFORM
Online Advertising Full Stack Platform
Online Advertising Full Stack Platform Realtime “smart“ ads Forecasting, fraud
detection, etc.
Online Advertising Full Stack Platform Realtime “smart“ ads Forecasting, fraud
detection, etc. 1mln QPS under 100ms 1TB daily data
EVOLUTION
MIDDLE AGES ctrtrain.ec2-aws.com test2.ec2-aws.com modelling.ec2-aws.com
THE TORTURE # yum install python R libboost-3.12 $ scp
script.R test.aws.com:/script.R # crontab -e
THE TORTURE # yum install python R libboost-3.12 $ scp
script.R test.aws.com:/script.R # crontab -e “strange, worked on my machine …”
RENAISSANCE ctrtrain.ec2-aws.com test2.ec2-aws.com worker-1.adform.com worker-2.adform.com ab34na3n ar2afga3n
None
CONTAINERIZE ! self contained artifacts isolated runtime basically no overhead
unified deployment
BUT WAIT … what about configuration ?
The twelve-factor app stores config in environment variables Env vars
are easy to change between deploys without changing any code There is little chance of them being checked into the code repo accidentally They are a language- and OS-agnostic standard
BUT WAIT … where do I log ? and what
about metrics ?
None
MODERN ERA e34sadf ab34na3n af4f5a4r aafde33a fa45daws faes4fa3 aaf444a2 fas3rfa4
None
MARATHON the init of the DCOS constraints deployment { "id":
“my-nginx", "container": { "type": "DOCKER", "docker": { "image": "nginx:1.7.7", "network": "BRIDGE", } }, "instances": 1, "cpus": 0.5, "mem": 128 }
SPRINT the exec of he DCOS will be open sourced
scheduler to follow!
MANAGING RESOURCES how much memory do I really need ?
MANAGING RESOURCES how much memory do I really need ?
and CPUs ? what does 0.5 CPUs mean anyway ?
MANAGING RESOURCES how much memory do I really need ?
and CPUs ? what does 0.5 CPUs mean anyway ? and what happens with the network ?
None
ISOLATION cgroups: cpu cpuset memory blkio net_cls
NETWORK ISOLATION Layer 3 routing software defined networks
ATOMIC AGE a4faw3f 4afsdgg asdf4faf se4faw aw3d3ff g4aefgsd 5gsdgr54s a4rff4afa
4f4qaf4
SERVICE DISCOVERY where is my app ? how do I
reach it ? won’t containers conflict about ports ?
MARATHON-LB
PERSISTENCE so .. where do I store my data ?
on the host ? won’t it disappear ?
PERSISTENCE / /opt/app/cache /var/lib/docker/devicemapper /var/lib/mesos/slave/volumes
PERSISTENCE / /opt/app/cache /opt/app/profile /var/lib/docker/devicemapper /var/lib/mesos/slave/volumes /mnt/sdc network block storage
FUTURE PLANS DC/OS IP per container Containerize all the things
@adforminsider
More Decks by DevOps Pro
See All by DevOps Pro
Practical Application Logging and Monitoring
devopspro
0
130
Google Cloud Platform for DevOps Professionals
devopspro
0
310
Dev vs Ops transformation to Agility DevOps culture
devopspro
1
270
Using Test Automation for Continuous Integration, Delivery and Quality
devopspro
0
120
How the hell do I run my microservices in production, and will it scale?
devopspro
0
150
Accelerating innovation through AWS Cloud technology
devopspro
0
120
The Field Guide to Understanding Declarative Systems
devopspro
0
120
DevOps is the answer! What was the question again?
devopspro
0
180
From Monolith to Docker Distributed Applications
devopspro
0
270
Other Decks in Technology
See All in Technology
「正しいデータ」の定義をAI エージェントとつくるGoogleADKによるデータ品質定義の実践
takaha4k
0
110
Is the Website Dying?
miriamschwab
0
130
E2E Test というハーネス — AI時代の自動テスト民主化
daishukojima
10
6k
クラウドでも使えるNetApp -現場目線での設計とクラウドバックアップの活用
ryutakondo
1
310
さようならSSH鍵、こんにちはゼロトラスト ― AIフレンドリーで快適な自宅鯖SSH
atalie2m
14
7.5k
マネジメントされる側の仕事は「文句」を言うこと
riyaamemiya
7
6k
[DBRE Summit 26] DBRE はデータベース運用をどう変えたのか
_awache
0
130
AWS re:Invent 2026 FSI Pre-Event Networking Night
junjikoide
0
170
AIエージェントのアーキテクチャ設計
nsatoh
0
480
[関西Kaggler会 20261113] 続・コミュニティコンペのススメ
nejumi
0
200
New Relicのアラート対応をClaude Codeで自動修正するワークフローを作った話
manabuseki
0
430
飯から始まるデータエンジニア
sansantech
PRO
0
140
Featured
See All Featured
Building a A Zero-Code AI SEO Workflow
portentint
PRO
0
840
Unsuck your backbone
ammeep
672
58k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
287
14k
WCS-LA-2024
lcolladotor
0
1k
Writing Fast Ruby
sferik
630
63k
Fantastic passwords and where to find them - at NoRuKo
philnash
53
4k
Abbi's Birthday
coloredviolet
5
11k
AI: The stuff that nobody shows you
jnunemaker
PRO
12
1.3k
KATA
mclloyd
PRO
35
16k
What’s in a name? Adding method to the madness
productmarketing
PRO
24
4.2k
Future Trends and Review - Lecture 12 - Web Technologies (1019888BNR)
signer
PRO
0
3.8k
The World Runs on Bad Software
bkeepers
PRO
72
12k