Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Docker In Production - A War Story
Search
Seb
July 23, 2015
Technology
1
290
Docker In Production - A War Story
Docker Meetup Vancouver - 23 July 2015
Seb
July 23, 2015
Tweet
Share
More Decks by Seb
See All by Seb
Double Click - Continue Building Better CLIs
elbaschid
0
340
I Can Be A Speaker, So Can You
elbaschid
0
200
Click - PyCaribbean 2017 - Puerto Rico
elbaschid
0
320
Conferencing - Engineering Meeting
elbaschid
1
41
Show & Tell - PyCon US 2016 Summary
elbaschid
1
74
Click: A Pleasure To Write, A Pleasure To Use
elbaschid
0
500
Hunting for Treasure in Django
elbaschid
1
510
Moby & The Beanstalk
elbaschid
1
420
Hunting For Treasure In Django
elbaschid
0
160
Other Decks in Technology
See All in Technology
Compose Compiler Metricsを使った実践的なコードレビュー
tomorrowkey
1
210
MapLibreとAmazon Location Service
dayjournal
1
140
On Your Data を超えていく!
hirotomotaguchi
2
640
Google Cloud の AI を支える裏側のインフラを垣間見る!
maroon1st
0
330
Cracking the KubeCon CfP
inductor
2
210
KubeCon EU 2024 Recap “Kubernetes Policy Time Machine: Where to Next?”
ryysud
0
190
2024/4/26 コンピュータ歴史博物館解説告知
toshi_atsumi
0
210
アクセシビリティを考慮したUI/CSSフレームワーク・ライブラリ選定
yajihum
2
990
プラットフォームってつくることより計測することが重要なんじゃないかという話 / Platform Engineering Meetup #8
taishin
0
330
複雑な構成要素を持つUIとの向き合い方 〜新・支出グラフでの実例〜 / B43 TECH TALK
nakamuuu
0
140
Tableau事例紹介 / Tableau Case Study of Eureka
kazuya_araki_tokyo
1
180
エンジニアのキャリアをちょっと楽しくする3本の軸/Three Pillars to Make an Engineer's Career More Enjoyable
kwappa
0
2.6k
Featured
See All Featured
Why You Should Never Use an ORM
jnunemaker
PRO
51
8.6k
Large-scale JavaScript Application Architecture
addyosmani
504
110k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
244
20k
Design by the Numbers
sachag
274
18k
The Pragmatic Product Professional
lauravandoore
25
5.8k
Java REST API Framework Comparison - PWX 2021
mraible
PRO
18
6.9k
A Tale of Four Properties
chriscoyier
151
22k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
274
13k
Rails Girls Zürich Keynote
gr2m
91
13k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
34
8.9k
Reflections from 52 weeks, 52 projects
jeffersonlam
345
19k
What the flash - Photography Introduction
edds
64
11k
Transcript
Docker in Production A War Story
Kyle Young github.com/kyoung @ksgyoung Sebastian Vetter github.com/elbaschid @elbaschid Who We
Are
Rackspace Setup • Many many Rackspace servers • A lot
of tooling around legacy infrastructure • Running a lot of services ourselves: RabbitMQ, MySQL (in replication), and more
None
None
New Setup • AWS Beanstalk • Docker • Quay.io: Docker
Registry • External services: RDS, ElastiCache, hosted RabbitMQ
None
None
Development Environment • How are we running the dev env
◦ docker-machine ◦ docker-compose ◦ much bash
docker-compose • Container per service • Full async setup •
Controlled by our own CLI docker ├── cli │ ├── bootstrap.sh │ ├── circle-run-tests-in-parallel.sh │ ├── cleanup.sh │ ├── deploy.sh │ ├── dev.sh │ ├── hotfix.sh │ └── promote.sh
Deployment • Testing on Circle CI • Auto-deploy to Staging
• ./portal.sh promote
None
Problems
Beanstalk Tooling
Beanstalk Tooling • Very basic tooling around Beanstalk • Needs
a bit of a nudge some of the times • Modifying AMI using .ebextensions
Auto-Mounting Volumes
Auto-Mounting Volumes • Having VOLUME in Dockerfile • Creates auto-mounted
AUFS volume • Doesn’t get cleaned up • Don’t use VOLUME
Containers & Images
Containers & Images • Failed deployments due to lack of
disk space • Pulling images adds data • No automatic cleanup • Adding some cleanup commands to . ebextensions
Centralized Logging
Centralized Logging • Docker logs to stdout & stderr •
Beanstalk pulls them into log files • Forwarding to syslog is tricky ◦ dynamic naming of log files: /var/log/eb-docker/.../6fb0153a0b99-stdouterr.log ◦ old version of rsyslog that doesn't allow globbing
Solution • Using gliderlabs’ logspout: https://github. com/gliderlabs/logspout • Best of
both worlds: ◦ Beanstalk logging ◦ syslog forwarding
Monitoring
Monitoring with Datadog • Better insight: ◦ containers running ◦
CPU & memory usage ◦ triggering alerts • Using the dd-agent container from Datadog https://github.com/DataDog/dd-agent
Monitoring with Datadog
Migrations
Database and migrations • Currently has to be run manually
• Better strategy needed long-term But: • No sophisticated migration strategy • No high-traffic, high-performance service
The Future
Possible Improvements • AWS ECS for more efficient containers •
Possibly investigating "Project Orca" at some point (maybe)
Questions?