Moby & The Beanstalk

The popularity of Docker these days is incredible and a lot of people are already using it for their production systems. Although it makes a lot of things much simpler, it's not a silver bullet and comes with some caveats of it's own. This talk will describe the journey of my team at Mobify from a self-hosted VM to a AWS-backed web app deployed using Docker.

Transcript

1. Moby And The
   Beanstalk
   Sebastian Vetter
   @elbaschid
   

   View Slide

2. About Me
   » Back-End Engineer @ Mobify !
   » Django developer since 2011
   » Docker enthusiast "
   » elbaschid on the interwebz
   

   View Slide

3. This talk is
   about
   

   View Slide

4. our legacy Django
   app
   

   View Slide

5. porting it to Docker
   

   View Slide

6. & hosting it on
   Amazon Beanstalk.
   

   View Slide

7. What is Docker?
   

   View Slide

8. Docker
   

   View Slide

9. What is Beanstalk?
   

   View Slide

10. Beanstalk
    » Amazon PaaS Offering
    » Originally, Heroku-style deploys
    » Supports running Docker containers
    

    View Slide

11. Back To Our App
    

    View Slide

12. Our Old Stack
    

    View Slide

13. Our Problems
    » Fragile replication & sync
    » Error-prone deployments
    » Labour intensive maintenance
    

    View Slide

14. There Must Be A
    Better Way
    

    View Slide

15. Our New Stack
    

    View Slide

16. Let's Look Closer
    

    View Slide

17. Deployment
    

    View Slide

18. Automated Deployment
    

    View Slide

19. Django & Docker
    

    View Slide

20. Configuration
    

    View Slide

21. Twelve-factor App
    » Ephemeral systems
    » Configuration through environment
    » Explained in 12factor.net/config
    

    View Slide

22. 12-factor on Beanstalk
    

    View Slide

23. Basic 12-Factor config
    # settings.py
    import os
    SECRET_KEY = os.getenv('SECRET_KEY')
    DATABASES = {
    'default': {
    'NAME': os.getenv('DB_NAME'),
    'HOST': os.getenv('DB_HOST'),
    'USER': os.getenv('DB_USER'),
    'PASSWORD': os.getenv('DB_PASSWORD'),
    }
    }
    

    View Slide

24. There Is A Better Way
    

    View Slide

25. Using django-configurations
    » Class-based setup
    » Handling & validation of env vars
    » Variable namespace DJANGO_
    

    View Slide

26. Example Settings
    from configurations import values
    from configurations import Configuration
    class Production(Configuration):
    SECRET_KEY = values.SecretValue()
    DATABASES = values.DatabaseURLValue()
    # shell
    export DJANGO_SECRET_KEY='the django pony is awesome'
    export DJANGO_DATABASE_URL='postgres://myuser@localhost/mydb'
    

    View Slide

27. Serving Files
    

    View Slide

28. Ephermal Systems
    » No persistent storage
    » Use cloud storage solutions
    » Easy to set up with Django apps
    

    View Slide

29. For File Uploads
    » We use S3 for storage
    » Django app: django-storages
    » Uses boto under the hood
    

    View Slide

30. Easy To Configure
    # settings.py
    INSTALLED_APPS += ['storages']
    DEFAULT_FILE_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
    AWS_ACCESS_KEY_ID = 'xxxxxxxx'
    AWS_SECRET_ACCESS_KEY = 'xxxxxxxx'
    AWS_STORAGE_BUCKET_NAME = 'mybucket'
    

    View Slide

31. For Static Files
    » With django-storages
    » As easy as adding a setting
    # settings.py
    STATICFILES_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
    

    View Slide

32. There Is A "Better"
    Way
    

    View Slide

33. The Way We Do It
    » Bake static files into images
    » Mount to the EC2 host
    » Update files on container startup
    » Serve via nginx
    

    View Slide

34. Challenges
    

    View Slide

35. Logging
    » Log files stored on EC2
    » Aggregated on request (manually)
    » No syslog support
    Solution
    » Forward to central log server
    » Use gliderlabs/logspout
    

    View Slide

36. Monitoring
    » Limited info available
    » Separated by environment
    » No customization
    Solution
    » Use external service (e.g. DataDog)
    » Run DataDog/docker-dd-agent
    

    View Slide

37. DB Schema Changes
    » Unsolved Problem (for us)
    » Automatically migrating is problematic
    » Requires development strategy
    Recommended Read
    The Path To Smoother Migrations
    Nathan Duthoit
    

    View Slide

38. Benefits
    

    View Slide

39. Dev-Prod Parity
    

    View Slide

40. Easy Rollbacks
    

    View Slide

41. Faster Deploys
    

    View Slide

42. Final Thoughts
    » Great improvement for us
    » Still issues with Beanstalk
    » Docker is extremely young
    

    View Slide

43. It'll get better
    

    View Slide

44. Questions ?
    

    View Slide