Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Docker In Production - A War Story
Search
Seb
July 23, 2015
Technology
1
300
Docker In Production - A War Story
Docker Meetup Vancouver - 23 July 2015
Seb
July 23, 2015
Tweet
Share
More Decks by Seb
See All by Seb
Double Click - Continue Building Better CLIs
elbaschid
0
400
I Can Be A Speaker, So Can You
elbaschid
0
270
Click - PyCaribbean 2017 - Puerto Rico
elbaschid
0
390
Conferencing - Engineering Meeting
elbaschid
1
42
Show & Tell - PyCon US 2016 Summary
elbaschid
1
96
Click: A Pleasure To Write, A Pleasure To Use
elbaschid
0
580
Hunting for Treasure in Django
elbaschid
1
610
Moby & The Beanstalk
elbaschid
1
470
Hunting For Treasure In Django
elbaschid
0
170
Other Decks in Technology
See All in Technology
【re:Invent 2024 アプデ】 Prompt Routing の紹介
champ
0
140
kargoの魅力について伝える
magisystem0408
0
200
Qiita埋め込み用スライド
naoki_0531
0
860
OpenAIの蒸留機能(Model Distillation)を使用して運用中のLLMのコストを削減する取り組み
pharma_x_tech
4
540
GitHub Copilot のテクニック集/GitHub Copilot Techniques
rayuron
23
11k
第3回Snowflake女子会_LT登壇資料(合成データ)_Taro_CCCMK
tarotaro0129
0
180
マイクロサービスにおける容易なトランザクション管理に向けて
scalar
0
110
NW-JAWS #14 re:Invent 2024(予選落ち含)で 発表された推しアップデートについて
nagisa53
0
250
AWS re:Invent 2024で発表された コードを書く開発者向け機能について
maruto
0
180
ブラックフライデーで購入したPixel9で、Gemini Nanoを動かしてみた
marchin1989
1
520
オプトインカメラ:UWB測位を応用したオプトイン型のカメラ計測
matthewlujp
0
170
re:Invent をおうちで楽しんでみた ~CloudWatch のオブザーバビリティ機能がスゴい!/ Enjoyed AWS re:Invent from Home and CloudWatch Observability Feature is Amazing!
yuj1osm
0
120
Featured
See All Featured
Building Better People: How to give real-time feedback that sticks.
wjessup
365
19k
How to Ace a Technical Interview
jacobian
276
23k
KATA
mclloyd
29
14k
jQuery: Nuts, Bolts and Bling
dougneiner
61
7.5k
Intergalactic Javascript Robots from Outer Space
tanoku
270
27k
Rails Girls Zürich Keynote
gr2m
94
13k
Unsuck your backbone
ammeep
669
57k
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
8
1.2k
Rebuilding a faster, lazier Slack
samanthasiow
79
8.7k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
28
9.1k
StorybookのUI Testing Handbookを読んだ
zakiyama
27
5.3k
It's Worth the Effort
3n
183
28k
Transcript
Docker in Production A War Story
Kyle Young github.com/kyoung @ksgyoung Sebastian Vetter github.com/elbaschid @elbaschid Who We
Are
Rackspace Setup • Many many Rackspace servers • A lot
of tooling around legacy infrastructure • Running a lot of services ourselves: RabbitMQ, MySQL (in replication), and more
None
None
New Setup • AWS Beanstalk • Docker • Quay.io: Docker
Registry • External services: RDS, ElastiCache, hosted RabbitMQ
None
None
Development Environment • How are we running the dev env
◦ docker-machine ◦ docker-compose ◦ much bash
docker-compose • Container per service • Full async setup •
Controlled by our own CLI docker ├── cli │ ├── bootstrap.sh │ ├── circle-run-tests-in-parallel.sh │ ├── cleanup.sh │ ├── deploy.sh │ ├── dev.sh │ ├── hotfix.sh │ └── promote.sh
Deployment • Testing on Circle CI • Auto-deploy to Staging
• ./portal.sh promote
None
Problems
Beanstalk Tooling
Beanstalk Tooling • Very basic tooling around Beanstalk • Needs
a bit of a nudge some of the times • Modifying AMI using .ebextensions
Auto-Mounting Volumes
Auto-Mounting Volumes • Having VOLUME in Dockerfile • Creates auto-mounted
AUFS volume • Doesn’t get cleaned up • Don’t use VOLUME
Containers & Images
Containers & Images • Failed deployments due to lack of
disk space • Pulling images adds data • No automatic cleanup • Adding some cleanup commands to . ebextensions
Centralized Logging
Centralized Logging • Docker logs to stdout & stderr •
Beanstalk pulls them into log files • Forwarding to syslog is tricky ◦ dynamic naming of log files: /var/log/eb-docker/.../6fb0153a0b99-stdouterr.log ◦ old version of rsyslog that doesn't allow globbing
Solution • Using gliderlabs’ logspout: https://github. com/gliderlabs/logspout • Best of
both worlds: ◦ Beanstalk logging ◦ syslog forwarding
Monitoring
Monitoring with Datadog • Better insight: ◦ containers running ◦
CPU & memory usage ◦ triggering alerts • Using the dd-agent container from Datadog https://github.com/DataDog/dd-agent
Monitoring with Datadog
Migrations
Database and migrations • Currently has to be run manually
• Better strategy needed long-term But: • No sophisticated migration strategy • No high-traffic, high-performance service
The Future
Possible Improvements • AWS ECS for more efficient containers •
Possibly investigating "Project Orca" at some point (maybe)
Questions?