Docker Lessons from Real-World Projects

Docker Lessons from Real-World Projects

Presented at DjangoCon Europe 2017

Explores how to use Docker for complex, real-world Django applications.

895ac7b84a280f686b70e2a70a8f5cd9?s=128

Rivo Laks

April 05, 2017
Tweet

Transcript

  1. 4.

    The Problem At Thorgate, we host dozens of our projects

    A single server runs multiple projects
  2. 5.

    The Problem At Thorgate, we host dozens of our projects

    A single server runs multiple projects Dependency isolation became an issue
  3. 10.

    How to Get There? Lots of online tutorials for making

    rst steps Practical experience with bigger projects is much more scarce
  4. 12.
  5. 13.
  6. 14.
  7. 20.
  8. 24.

    The Django Service Start with o cial python image Install

    Python requirements Add application source code
  9. 25.

    The Django Service Start with o cial python image Install

    Python requirements Add application source code Use Gunicorn to serve HTTP
  10. 26.

    The Django Service Start with o cial python image Install

    Python requirements Add application source code Use Gunicorn to serve HTTP Volumes are used for data dirs This includes static assets, media and logs
  11. 28.

    The Node Service Bundles JS + CSS using Webpack Isn't

    run continuously, just once during deploy
  12. 29.

    The Node Service Bundles JS + CSS using Webpack Isn't

    run continuously, just once during deploy Basically o cial Node image + our requirements
  13. 30.

    The Node Service Bundles JS + CSS using Webpack Isn't

    run continuously, just once during deploy Basically o cial Node image + our requirements Volumes for JS/CSS source dirs
  14. 33.

    Other Services Redis, Celery, etc Can often use o cial

    image out-of-the-box Volumes for persistent data
  15. 35.

    Putting it All Together Our application has multiple services working

    in unison, communicating via network Docker Compose is “a tool for de ning and running multi-container Docker applications”
  16. 36.

    Putting it All Together Our application has multiple services working

    in unison, communicating via network Docker Compose is “a tool for de ning and running multi-container Docker applications” de ne & control multiple containers at once
  17. 37.

    Putting it All Together Our application has multiple services working

    in unison, communicating via network Docker Compose is “a tool for de ning and running multi-container Docker applications” de ne & control multiple containers at once easier networking among single app's services
  18. 38.
  19. 44.

    The Servers Almost nothing but Docker Postgres and Nginx as

    well, since we chose to leave those global
  20. 45.
  21. 48.

    Deploys docker-compose build docker-compose run --rm node npm run build

    docker-compose run --rm django ./manage.py collectstatic docker-compose run --rm django ./manage.py migrate
  22. 49.

    Deploys docker-compose build docker-compose run --rm node npm run build

    docker-compose run --rm django ./manage.py collectstatic docker-compose run --rm django ./manage.py migrate docker-compose up -d
  23. 52.

    Dev vs Production Should ideally be very similar, but some

    di erences are useful Commands are slightly di erent
  24. 53.

    Dev vs Production Should ideally be very similar, but some

    di erences are useful Commands are slightly di erent Sources not baked into images
  25. 54.

    Dev vs Production Should ideally be very similar, but some

    di erences are useful Commands are slightly di erent Sources not baked into images Postgres is run locally, along other services
  26. 55.

    Dev vs Production Should ideally be very similar, but some

    di erences are useful Commands are slightly di erent Sources not baked into images Postgres is run locally, along other services Separate Docker Compose con g le
  27. 57.

    Django App dir is mounted as volume, not baked into

    image Python dependencies still part of the image
  28. 58.

    Django App dir is mounted as volume, not baked into

    image Python dependencies still part of the image Django's runserver instead of Gunicorn
  29. 59.

    Django App dir is mounted as volume, not baked into

    image Python dependencies still part of the image Django's runserver instead of Gunicorn Separate Dockerfile
  30. 61.

    Node Image contains libraries but not our sources (same as

    production) Webpack watch mode, similar to Django's runserver
  31. 66.

    Commands docker-compose run --rm django ./manage.py runserver docker-compose run --rm

    django ./manage.py migrate Make le FTW! make docker make migrate
  32. 67.

    Commands docker-compose run --rm django ./manage.py runserver docker-compose run --rm

    django ./manage.py migrate Make le FTW! make docker make migrate Bonus: make setup
  33. 69.

    Service Discovery How to route HTTP requests to correct Django

    container Ran Nginx in container, used Docker's built-in routing
  34. 70.

    Service Discovery How to route HTTP requests to correct Django

    container Ran Nginx in container, used Docker's built-in routing But container IP can change, Nginx does DNS lookup only once
  35. 71.

    Service Discovery How to route HTTP requests to correct Django

    container Ran Nginx in container, used Docker's built-in routing But container IP can change, Nginx does DNS lookup only once Created a service monitoring Docker events and reloading Nginx
  36. 73.

    Waiting for Related Services Need to wait for Postgres to

    be available for Django wait-for-it.sh scripts
  37. 75.
  38. 76.

    Performance on non-Linux OSs On macOS / Windows, Docker containers

    run in VM Filesystem performance su ers localhost isn't localhost anymore
  39. 80.

    PyCharm Supports Python interpreter in Docker There are bugs though

    Running app inside PyCharm was tricky node_modules isn't visible to PyCharm
  40. 84.

    Problems Learning curve – you need to learn Docker You'll

    also spend time on interesting new issues
  41. 85.

    Problems Learning curve – you need to learn Docker You'll

    also spend time on interesting new issues Ongoing problems: Tooling
  42. 86.

    Problems Learning curve – you need to learn Docker You'll

    also spend time on interesting new issues Ongoing problems: Tooling Performance outside Linux
  43. 90.

    Advantages Isolation & reproducability Easier to add new services Bundling

    everything into a single entity Limiting/monitoring resource use is easier
  44. 91.

    Advantages Isolation & reproducability Easier to add new services Bundling

    everything into a single entity Limiting/monitoring resource use is easier Single-command project setup
  45. 95.

    It Depends... For legacy projects, maybe not But makes more

    sense for new projects Thorgate open-sourcing our project template
  46. 97.

    Thanks! Rivo Laks ⋅ @rivolaks ⋅ rivolaks.com CTO at Thorgate

    ⋅ thorgate.eu https://github.com/thorgate/django-project-template