Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Docker In Production - A War Story
Search
Seb
July 23, 2015
Technology
1
300
Docker In Production - A War Story
Docker Meetup Vancouver - 23 July 2015
Seb
July 23, 2015
Tweet
Share
More Decks by Seb
See All by Seb
Double Click - Continue Building Better CLIs
elbaschid
0
370
I Can Be A Speaker, So Can You
elbaschid
0
220
Click - PyCaribbean 2017 - Puerto Rico
elbaschid
0
360
Conferencing - Engineering Meeting
elbaschid
1
41
Show & Tell - PyCon US 2016 Summary
elbaschid
1
86
Click: A Pleasure To Write, A Pleasure To Use
elbaschid
0
530
Hunting for Treasure in Django
elbaschid
1
550
Moby & The Beanstalk
elbaschid
1
440
Hunting For Treasure In Django
elbaschid
0
170
Other Decks in Technology
See All in Technology
CEL(Common Expression Language)で書いた条件にマッチしたIAM Policyを見つける / iam-policy-finder
fujiwara3
0
710
AIエージェントを現場に導入する目線とは
masahiro_nishimi
1
1.5k
What if...? 처음부터 다시 LLM 어플리케이션을 개발한다면
huffon
0
1k
たくさん本を読んだけど 1年後には綺麗サッパリ!を乗り越えて 学習の鬼になるぞ👹
yum3
0
160
DevIO2024_レガシー運用からの脱却 -クラウド活用の実践事例とベストプラクティス-
jun2882
0
210
LINE WORKSへ簡単通知!Incoming Webhookアプリの紹介
mmclsntr
0
110
[2024最新版]AWS Control Towerを使ったセキュアなマルチアカウント環境の作り方
hiashisan
0
270
[I/O Extended Android 2024] What`s new in Android 2024
kyeongwan
0
220
推薦システムを本番導入する上で一番優先すべきだったこと~NewsPicks記事推薦機能の改善事例を元に~
morinota
0
120
エンジニア向け会社紹介資料
caddi_eng
14
220k
エンジニアの生存戦略 〜クラウド潮流の経験から紐解く技術トレンドのメカニズムと乗りこなし方〜
shimy
9
1.9k
Luupの開発組織におけるインシデントマネジメントの変遷 ver.RoadtoSRENEXT2024
grimoh
1
270
Featured
See All Featured
Code Reviewing Like a Champion
maltzj
517
39k
Unsuck your backbone
ammeep
666
57k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
24
1.8k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
360
22k
Why Our Code Smells
bkeepers
PRO
332
56k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
44
4.7k
Optimizing for Happiness
mojombo
373
69k
Ruby is Unlike a Banana
tanoku
96
10k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
277
13k
Designing on Purpose - Digital PM Summit 2013
jponch
113
6.6k
Building Flexible Design Systems
yeseniaperezcruz
323
37k
Pencils Down: Stop Designing & Start Developing
hursman
118
11k
Transcript
Docker in Production A War Story
Kyle Young github.com/kyoung @ksgyoung Sebastian Vetter github.com/elbaschid @elbaschid Who We
Are
Rackspace Setup • Many many Rackspace servers • A lot
of tooling around legacy infrastructure • Running a lot of services ourselves: RabbitMQ, MySQL (in replication), and more
None
None
New Setup • AWS Beanstalk • Docker • Quay.io: Docker
Registry • External services: RDS, ElastiCache, hosted RabbitMQ
None
None
Development Environment • How are we running the dev env
◦ docker-machine ◦ docker-compose ◦ much bash
docker-compose • Container per service • Full async setup •
Controlled by our own CLI docker ├── cli │ ├── bootstrap.sh │ ├── circle-run-tests-in-parallel.sh │ ├── cleanup.sh │ ├── deploy.sh │ ├── dev.sh │ ├── hotfix.sh │ └── promote.sh
Deployment • Testing on Circle CI • Auto-deploy to Staging
• ./portal.sh promote
None
Problems
Beanstalk Tooling
Beanstalk Tooling • Very basic tooling around Beanstalk • Needs
a bit of a nudge some of the times • Modifying AMI using .ebextensions
Auto-Mounting Volumes
Auto-Mounting Volumes • Having VOLUME in Dockerfile • Creates auto-mounted
AUFS volume • Doesn’t get cleaned up • Don’t use VOLUME
Containers & Images
Containers & Images • Failed deployments due to lack of
disk space • Pulling images adds data • No automatic cleanup • Adding some cleanup commands to . ebextensions
Centralized Logging
Centralized Logging • Docker logs to stdout & stderr •
Beanstalk pulls them into log files • Forwarding to syslog is tricky ◦ dynamic naming of log files: /var/log/eb-docker/.../6fb0153a0b99-stdouterr.log ◦ old version of rsyslog that doesn't allow globbing
Solution • Using gliderlabs’ logspout: https://github. com/gliderlabs/logspout • Best of
both worlds: ◦ Beanstalk logging ◦ syslog forwarding
Monitoring
Monitoring with Datadog • Better insight: ◦ containers running ◦
CPU & memory usage ◦ triggering alerts • Using the dd-agent container from Datadog https://github.com/DataDog/dd-agent
Monitoring with Datadog
Migrations
Database and migrations • Currently has to be run manually
• Better strategy needed long-term But: • No sophisticated migration strategy • No high-traffic, high-performance service
The Future
Possible Improvements • AWS ECS for more efficient containers •
Possibly investigating "Project Orca" at some point (maybe)
Questions?