Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Docker In Production - A War Story
Search
Seb
July 23, 2015
Technology
1
310
Docker In Production - A War Story
Docker Meetup Vancouver - 23 July 2015
Seb
July 23, 2015
Tweet
Share
More Decks by Seb
See All by Seb
Double Click - Continue Building Better CLIs
elbaschid
0
440
I Can Be A Speaker, So Can You
elbaschid
0
300
Click - PyCaribbean 2017 - Puerto Rico
elbaschid
0
420
Conferencing - Engineering Meeting
elbaschid
1
44
Show & Tell - PyCon US 2016 Summary
elbaschid
1
100
Click: A Pleasure To Write, A Pleasure To Use
elbaschid
0
620
Hunting for Treasure in Django
elbaschid
1
670
Moby & The Beanstalk
elbaschid
1
490
Hunting For Treasure In Django
elbaschid
0
170
Other Decks in Technology
See All in Technology
shake-upを科学する
rsakata
7
1k
無理しない AI 活用サービス / #jazug
koudaiii
0
100
united airlines ™®️ USA Contact Numbers: Complete 2025 Support Guide
flyunitedhelp
1
470
マルチプロダクト環境におけるSREの役割 / SRE NEXT 2025 lunch session
sugamasao
1
730
AWS 怖い話 WAF編 @fillz_noh #AWSStartup #AWSStartup_Kansai
fillznoh
0
130
PHPからはじめるコンピュータアーキテクチャ / From Scripts to Silicon: A Journey Through the Layers of Computing
tomzoh
2
120
How to Quickly Call American Airlines®️ U.S. Customer Care : Full Guide
flyaahelpguide
0
240
Contract One Engineering Unit 紹介資料
sansan33
PRO
0
6.9k
今だから言えるセキュリティLT_Wordpress5.7.2未満を一斉アップデートせよ
cuebic9bic
2
170
セキュアな社内Dify運用と外部連携の両立 ~AIによるAPIリスク評価~
zozotech
PRO
0
120
推し書籍📚 / Books and a QA Engineer
ak1210
0
140
CDKコード品質UP!ナイスな自作コンストラクタを作るための便利インターフェース
harukasakihara
2
240
Featured
See All Featured
How STYLIGHT went responsive
nonsquared
100
5.6k
What’s in a name? Adding method to the madness
productmarketing
PRO
23
3.5k
Visualization
eitanlees
146
16k
GraphQLとの向き合い方2022年版
quramy
49
14k
Building Adaptive Systems
keathley
43
2.7k
Gamification - CAS2011
davidbonilla
81
5.4k
Why You Should Never Use an ORM
jnunemaker
PRO
58
9.5k
The Pragmatic Product Professional
lauravandoore
35
6.7k
The Illustrated Children's Guide to Kubernetes
chrisshort
48
50k
A designer walks into a library…
pauljervisheath
207
24k
4 Signs Your Business is Dying
shpigford
184
22k
Intergalactic Javascript Robots from Outer Space
tanoku
271
27k
Transcript
Docker in Production A War Story
Kyle Young github.com/kyoung @ksgyoung Sebastian Vetter github.com/elbaschid @elbaschid Who We
Are
Rackspace Setup • Many many Rackspace servers • A lot
of tooling around legacy infrastructure • Running a lot of services ourselves: RabbitMQ, MySQL (in replication), and more
None
None
New Setup • AWS Beanstalk • Docker • Quay.io: Docker
Registry • External services: RDS, ElastiCache, hosted RabbitMQ
None
None
Development Environment • How are we running the dev env
◦ docker-machine ◦ docker-compose ◦ much bash
docker-compose • Container per service • Full async setup •
Controlled by our own CLI docker ├── cli │ ├── bootstrap.sh │ ├── circle-run-tests-in-parallel.sh │ ├── cleanup.sh │ ├── deploy.sh │ ├── dev.sh │ ├── hotfix.sh │ └── promote.sh
Deployment • Testing on Circle CI • Auto-deploy to Staging
• ./portal.sh promote
None
Problems
Beanstalk Tooling
Beanstalk Tooling • Very basic tooling around Beanstalk • Needs
a bit of a nudge some of the times • Modifying AMI using .ebextensions
Auto-Mounting Volumes
Auto-Mounting Volumes • Having VOLUME in Dockerfile • Creates auto-mounted
AUFS volume • Doesn’t get cleaned up • Don’t use VOLUME
Containers & Images
Containers & Images • Failed deployments due to lack of
disk space • Pulling images adds data • No automatic cleanup • Adding some cleanup commands to . ebextensions
Centralized Logging
Centralized Logging • Docker logs to stdout & stderr •
Beanstalk pulls them into log files • Forwarding to syslog is tricky ◦ dynamic naming of log files: /var/log/eb-docker/.../6fb0153a0b99-stdouterr.log ◦ old version of rsyslog that doesn't allow globbing
Solution • Using gliderlabs’ logspout: https://github. com/gliderlabs/logspout • Best of
both worlds: ◦ Beanstalk logging ◦ syslog forwarding
Monitoring
Monitoring with Datadog • Better insight: ◦ containers running ◦
CPU & memory usage ◦ triggering alerts • Using the dd-agent container from Datadog https://github.com/DataDog/dd-agent
Monitoring with Datadog
Migrations
Database and migrations • Currently has to be run manually
• Better strategy needed long-term But: • No sophisticated migration strategy • No high-traffic, high-performance service
The Future
Possible Improvements • AWS ECS for more efficient containers •
Possibly investigating "Project Orca" at some point (maybe)
Questions?