$30 off During Our Annual Pro Sale. View Details »

Moby & The Beanstalk

Seb
July 31, 2015

Moby & The Beanstalk

The popularity of Docker these days is incredible and a lot of people are already using it for their production systems. Although it makes a lot of things much simpler, it's not a silver bullet and comes with some caveats of it's own. This talk will describe the journey of my team at Mobify from a self-hosted VM to a AWS-backed web app deployed using Docker.

Seb

July 31, 2015
Tweet

More Decks by Seb

Other Decks in Technology

Transcript

  1. Moby And The
    Beanstalk
    Sebastian Vetter
    @elbaschid

    View Slide

  2. About Me
    » Back-End Engineer @ Mobify !
    » Django developer since 2011
    » Docker enthusiast "
    » elbaschid on the interwebz

    View Slide

  3. This talk is
    about

    View Slide

  4. our legacy Django
    app

    View Slide

  5. porting it to Docker

    View Slide

  6. & hosting it on
    Amazon Beanstalk.

    View Slide

  7. What is Docker?

    View Slide

  8. Docker

    View Slide

  9. What is Beanstalk?

    View Slide

  10. Beanstalk
    » Amazon PaaS Offering
    » Originally, Heroku-style deploys
    » Supports running Docker containers

    View Slide

  11. Back To Our App

    View Slide

  12. Our Old Stack

    View Slide

  13. Our Problems
    » Fragile replication & sync
    » Error-prone deployments
    » Labour intensive maintenance

    View Slide

  14. There Must Be A
    Better Way

    View Slide

  15. Our New Stack

    View Slide

  16. Let's Look Closer

    View Slide

  17. Deployment

    View Slide

  18. Automated Deployment

    View Slide

  19. Django & Docker

    View Slide

  20. Configuration

    View Slide

  21. Twelve-factor App
    » Ephemeral systems
    » Configuration through environment
    » Explained in 12factor.net/config

    View Slide

  22. 12-factor on Beanstalk

    View Slide

  23. Basic 12-Factor config
    # settings.py
    import os
    SECRET_KEY = os.getenv('SECRET_KEY')
    DATABASES = {
    'default': {
    'NAME': os.getenv('DB_NAME'),
    'HOST': os.getenv('DB_HOST'),
    'USER': os.getenv('DB_USER'),
    'PASSWORD': os.getenv('DB_PASSWORD'),
    }
    }

    View Slide

  24. There Is A Better Way

    View Slide

  25. Using django-configurations
    » Class-based setup
    » Handling & validation of env vars
    » Variable namespace DJANGO_

    View Slide

  26. Example Settings
    from configurations import values
    from configurations import Configuration
    class Production(Configuration):
    SECRET_KEY = values.SecretValue()
    DATABASES = values.DatabaseURLValue()
    # shell
    export DJANGO_SECRET_KEY='the django pony is awesome'
    export DJANGO_DATABASE_URL='postgres://myuser@localhost/mydb'

    View Slide

  27. Serving Files

    View Slide

  28. Ephermal Systems
    » No persistent storage
    » Use cloud storage solutions
    » Easy to set up with Django apps

    View Slide

  29. For File Uploads
    » We use S3 for storage
    » Django app: django-storages
    » Uses boto under the hood

    View Slide

  30. Easy To Configure
    # settings.py
    INSTALLED_APPS += ['storages']
    DEFAULT_FILE_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
    AWS_ACCESS_KEY_ID = 'xxxxxxxx'
    AWS_SECRET_ACCESS_KEY = 'xxxxxxxx'
    AWS_STORAGE_BUCKET_NAME = 'mybucket'

    View Slide

  31. For Static Files
    » With django-storages
    » As easy as adding a setting
    # settings.py
    STATICFILES_STORAGE = 'storages.backends.s3boto.S3BotoStorage'

    View Slide

  32. There Is A "Better"
    Way

    View Slide

  33. The Way We Do It
    » Bake static files into images
    » Mount to the EC2 host
    » Update files on container startup
    » Serve via nginx

    View Slide

  34. Challenges

    View Slide

  35. Logging
    » Log files stored on EC2
    » Aggregated on request (manually)
    » No syslog support
    Solution
    » Forward to central log server
    » Use gliderlabs/logspout

    View Slide

  36. Monitoring
    » Limited info available
    » Separated by environment
    » No customization
    Solution
    » Use external service (e.g. DataDog)
    » Run DataDog/docker-dd-agent

    View Slide

  37. DB Schema Changes
    » Unsolved Problem (for us)
    » Automatically migrating is problematic
    » Requires development strategy
    Recommended Read
    The Path To Smoother Migrations
    Nathan Duthoit

    View Slide

  38. Benefits

    View Slide

  39. Dev-Prod Parity

    View Slide

  40. Easy Rollbacks

    View Slide

  41. Faster Deploys

    View Slide

  42. Final Thoughts
    » Great improvement for us
    » Still issues with Beanstalk
    » Docker is extremely young

    View Slide

  43. It'll get better

    View Slide

  44. Questions ?

    View Slide