Moby & The Beanstalk

The popularity of Docker these days is incredible and a lot of people are already using it for their production systems. Although it makes a lot of things much simpler, it's not a silver bullet and comes with some caveats of it's own. This talk will describe the journey of my team at Mobify from a self-hosted VM to a AWS-backed web app deployed using Docker.

Transcript

1. Moby And The Beanstalk Sebastian Vetter @elbaschid
2. About Me » Back-End Engineer @ Mobify ! » Django developer since 2011 » Docker enthusiast " » elbaschid on the interwebz
3. This talk is about
4. our legacy Django app
5. porting it to Docker
6. & hosting it on Amazon Beanstalk.
7. What is Docker?
8. Docker
9. What is Beanstalk?
10. Beanstalk » Amazon PaaS Offering » Originally, Heroku-style deploys » Supports running Docker containers
11. Back To Our App
12. Our Old Stack
13. Our Problems » Fragile replication & sync » Error-prone deployments » Labour intensive maintenance
14. There Must Be A Better Way
15. Our New Stack
16. Let's Look Closer
17. Deployment
18. Automated Deployment
19. Django & Docker
20. Configuration
21. Twelve-factor App » Ephemeral systems » Configuration through environment » Explained in 12factor.net/config
22. 12-factor on Beanstalk
23. Basic 12-Factor config # settings.py import os SECRET_KEY = os.getenv('SECRET_KEY') DATABASES = { 'default': { 'NAME': os.getenv('DB_NAME'), 'HOST': os.getenv('DB_HOST'), 'USER': os.getenv('DB_USER'), 'PASSWORD': os.getenv('DB_PASSWORD'), } }
24. There Is A Better Way
25. Using django-configurations » Class-based setup » Handling & validation of env vars » Variable namespace DJANGO_
26. Example Settings from configurations import values from configurations import Configuration class Production(Configuration): SECRET_KEY = values.SecretValue() DATABASES = values.DatabaseURLValue() # shell export DJANGO_SECRET_KEY='the django pony is awesome' export DJANGO_DATABASE_URL='postgres://myuser@localhost/mydb'
27. Serving Files
28. Ephermal Systems » No persistent storage » Use cloud storage solutions » Easy to set up with Django apps
29. For File Uploads » We use S3 for storage » Django app: django-storages » Uses boto under the hood
30. Easy To Configure # settings.py INSTALLED_APPS += ['storages'] DEFAULT_FILE_STORAGE = 'storages.backends.s3boto.S3BotoStorage' AWS_ACCESS_KEY_ID = 'xxxxxxxx' AWS_SECRET_ACCESS_KEY = 'xxxxxxxx' AWS_STORAGE_BUCKET_NAME = 'mybucket'
31. For Static Files » With django-storages » As easy as adding a setting # settings.py STATICFILES_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
32. There Is A "Better" Way
33. The Way We Do It » Bake static files into images » Mount to the EC2 host » Update files on container startup » Serve via nginx
34. Challenges
35. Logging » Log files stored on EC2 » Aggregated on request (manually) » No syslog support Solution » Forward to central log server » Use gliderlabs/logspout
36. Monitoring » Limited info available » Separated by environment » No customization Solution » Use external service (e.g. DataDog) » Run DataDog/docker-dd-agent
37. DB Schema Changes » Unsolved Problem (for us) » Automatically migrating is problematic » Requires development strategy Recommended Read The Path To Smoother Migrations Nathan Duthoit
38. Benefits
39. Dev-Prod Parity
40. Easy Rollbacks
41. Faster Deploys
42. Final Thoughts » Great improvement for us » Still issues with Beanstalk » Docker is extremely young
43. It'll get better
44. Questions ?