Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Docker In Production - A War Story
Search
Seb
July 23, 2015
Technology
1
300
Docker In Production - A War Story
Docker Meetup Vancouver - 23 July 2015
Seb
July 23, 2015
Tweet
Share
More Decks by Seb
See All by Seb
Double Click - Continue Building Better CLIs
elbaschid
0
390
I Can Be A Speaker, So Can You
elbaschid
0
250
Click - PyCaribbean 2017 - Puerto Rico
elbaschid
0
380
Conferencing - Engineering Meeting
elbaschid
1
41
Show & Tell - PyCon US 2016 Summary
elbaschid
1
93
Click: A Pleasure To Write, A Pleasure To Use
elbaschid
0
560
Hunting for Treasure in Django
elbaschid
1
590
Moby & The Beanstalk
elbaschid
1
450
Hunting For Treasure In Django
elbaschid
0
170
Other Decks in Technology
See All in Technology
tenntennはなんでnewmoにnew社したの? - YAPC::Hakodate 2024
tenntenn
PRO
0
290
【shownet.conf_】ShowNet伝送改めShowNet APN 2024
shownet
PRO
0
450
Castor - Le Task Runner PHP qui simplifie votre Workflow
lyrixx
1
280
テクニカルライターのチームで「目標」をどう決めたか / MVV for a Team of Technical Writers
lycorptech_jp
PRO
3
100
ラブグラフ紹介資料 〜プロダクト解体新書〜 / Lovegraph Product Deck
lovegraph
0
14k
ゼロから実装まで!機械学習入門
natsuki0726
0
190
OPENLOGI Company Profile for engineer
hr01
1
12k
Deno Deploy で Web Cache API を 使えるようになったので試した知見
toranoana
1
110
【完全版】Dify - LINE Bot連携 考え方と実用テクニック
uezo
2
430
ADRを運用して3年経った僕らの現在地
onk
PRO
13
5.7k
Webセキュリティのあるきかた
akiym
32
11k
入社半年(合計1年)でGoogle Cloud 認定を全冠した秘訣🤫
risatube
1
230
Featured
See All Featured
Practical Orchestrator
shlominoach
186
10k
The Cult of Friendly URLs
andyhume
77
6k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
167
49k
A Modern Web Designer's Workflow
chriscoyier
692
190k
How to Think Like a Performance Engineer
csswizardry
16
1k
Producing Creativity
orderedlist
PRO
341
39k
Typedesign – Prime Four
hannesfritz
39
2.3k
The World Runs on Bad Software
bkeepers
PRO
65
11k
Designing for humans not robots
tammielis
249
25k
ParisWeb 2013: Learning to Love: Crash Course in Emotional UX Design
dotmariusz
110
6.9k
Unsuck your backbone
ammeep
668
57k
Speed Design
sergeychernyshev
22
490
Transcript
Docker in Production A War Story
Kyle Young github.com/kyoung @ksgyoung Sebastian Vetter github.com/elbaschid @elbaschid Who We
Are
Rackspace Setup • Many many Rackspace servers • A lot
of tooling around legacy infrastructure • Running a lot of services ourselves: RabbitMQ, MySQL (in replication), and more
None
None
New Setup • AWS Beanstalk • Docker • Quay.io: Docker
Registry • External services: RDS, ElastiCache, hosted RabbitMQ
None
None
Development Environment • How are we running the dev env
◦ docker-machine ◦ docker-compose ◦ much bash
docker-compose • Container per service • Full async setup •
Controlled by our own CLI docker ├── cli │ ├── bootstrap.sh │ ├── circle-run-tests-in-parallel.sh │ ├── cleanup.sh │ ├── deploy.sh │ ├── dev.sh │ ├── hotfix.sh │ └── promote.sh
Deployment • Testing on Circle CI • Auto-deploy to Staging
• ./portal.sh promote
None
Problems
Beanstalk Tooling
Beanstalk Tooling • Very basic tooling around Beanstalk • Needs
a bit of a nudge some of the times • Modifying AMI using .ebextensions
Auto-Mounting Volumes
Auto-Mounting Volumes • Having VOLUME in Dockerfile • Creates auto-mounted
AUFS volume • Doesn’t get cleaned up • Don’t use VOLUME
Containers & Images
Containers & Images • Failed deployments due to lack of
disk space • Pulling images adds data • No automatic cleanup • Adding some cleanup commands to . ebextensions
Centralized Logging
Centralized Logging • Docker logs to stdout & stderr •
Beanstalk pulls them into log files • Forwarding to syslog is tricky ◦ dynamic naming of log files: /var/log/eb-docker/.../6fb0153a0b99-stdouterr.log ◦ old version of rsyslog that doesn't allow globbing
Solution • Using gliderlabs’ logspout: https://github. com/gliderlabs/logspout • Best of
both worlds: ◦ Beanstalk logging ◦ syslog forwarding
Monitoring
Monitoring with Datadog • Better insight: ◦ containers running ◦
CPU & memory usage ◦ triggering alerts • Using the dd-agent container from Datadog https://github.com/DataDog/dd-agent
Monitoring with Datadog
Migrations
Database and migrations • Currently has to be run manually
• Better strategy needed long-term But: • No sophisticated migration strategy • No high-traffic, high-performance service
The Future
Possible Improvements • AWS ECS for more efficient containers •
Possibly investigating "Project Orca" at some point (maybe)
Questions?