Efficiently deploying Django

ChiPy: Chicago's Official Python User Group – ChiPy WebDev / DevOps SIG – Chicago, 2.6.2020.
https://www.meetup.com/_ChiPy_/events/270359645/
(video – https://www.youtube.com/watch?v=GvnR15IJnMI )

Python Croatia Meetup – Zagreb, 10.3.2020.
https://www.meetup.com/Python-Hrvatska/events/269107389/

Transcript

1. Efficiently deploying Django Dražen Lučanin https://punkrockdev.com/ @metakermit
2. So, you wrote a Django web app… • Followed something like the polls tutorial • Created your own web app • e.g. for keeping track of your family culinary recipes • everything is running nicely on http://localhost:8000 • now how do you let your users access the app?
3. … just need to deploy it • Should be easy, right…
 • hm… What’s Apache?
4. Ch. 1 Self-hosting
5. OK, let's take a step back • Core components • web server • domain • database(s) • Core actions • start everything • keep things running • Security • log errors & warnings • encryption • backups Ready to jump down the rabbit hole?
6. Web server • No, we’re not gonna use Apache • We’d go for nginx • Now a problem is nginx (or Apache) doesn’t support Django directly • python manage.py runserver ? • We need to set up a production-ready application server like uWSGI (or gunicorn) that knows how to run our Django app • Then we need to configure nginx as a passthrough for this server… except for static / media files… those need to be served by nginx
7. Connect the domain • Namecheap a decent domain registrar • A record if you have a fixed IP • CNAME if you’re forwarding to another domain • just note that CNAME on the root domain doesn’t always play nicely with MX rules • check if there is ANAME / ALIAS support • or connect your app via a subdomain like app.mydomain.com
8. Database(s) • Postgres for your relational database • You might need a task queue like Celery • Redis is a good option for the backend • Follow the tutorials and get everything set up
9. Start & keep running • So how do you start everything? • nginx, postgres and redis will start automatically via a systemd service • What about uwsgi? • You could write your custom systemd service • You could also use something like Supervisor
10. Yeah another one of those…
11. Logging & monitoring • Logging errors – https://sentry.io/ • Monitoring – https://newrelic.com/
12. Security • Oh yeah, we need to encrypt our traffic otherwise • Let’s encrypt is a free TLS certificate issuer – decent tutorial • but we need to set up the certbot • update our nginx config • set up a periodic task to auto-upgrade our certificate
13. OS upgrades • Hope you’re upgrading your OS • Ubuntu has unattended upgrades for important security patches – off by default • My little Ansible script for turning this on for my servers • https://github.com/punkrockdev/server
14. Backups?
15. uwsgi nginx namecheap letsencrypt postgres supervisor systemd ubuntu sentry newrelic redis
16. Summary • We have a bunch of services • We’re responsible for maintaining them • They will break and change over time • We need to keep our Django app configured to know where everything is • Can this be automated? • To an extent, yes – there are config management tools like Ansible
 (of Salt, Chef, Puppet) • But there are better ways…
17. Ch. 2 The twelve- factor app
18. You have a problem? • Start throwing money at it! • Platform-as-a-service • Heroku the most popular example • At $7 / month you can get a “dyno” • a thing that knows how to run your Django app • what’s under the hood? Someone else’s problem!
19. Remember all those services? • Well, in Heroku they come fully integrated • You can easily enable something like Postgres or Redis
20. A whole marketplace
21. Is there a catch? • Well, there is still a fair amount of configuration necessary • Heroku integrates all the services through the twelve- factor app methodology • But don’t worry, you’ll see it’s a good thing!
22. The twelve factors • I. Codebase – One codebase tracked in revision control, many deploys • II. Dependencies – Explicitly declare and isolate dependencies • III. Config – Store config in the environment • IV. Backing services – Treat backing services as attached resources • V. Build, release, run – Strictly separate build and run stages • VI. Processes – Execute the app as one or more stateless processes • VII. Port binding – Export services via port binding • VIII. Concurrency – Scale out via the process model • IX. Disposability – Maximize robustness with fast startup and graceful shutdown • X. Dev/prod parity – Keep development, staging, and production as similar as possible • XI. Logs – Treat logs as event streams • XII. Admin processes – Run admin/management tasks as one-off processes
23. So how do we start? • You can follow the official Heroku tutorial • You can also use my Yeoman generator • https://github.com/metakermit/generator-django-rest • Yeoman – a JavaScript-based (sssh ) code generator
24. Very quick to start npm install -g yo npm install -g generator-django-rest mkdir aviato cd aviato yo django-rest ./scripts/install.sh ./scripts/start.sh ./scripts/deploy.sh
25. … and it’s live
26. How does it work?
27. Ch. 3 The self-hosted twelve-factor app
28. Now, Heroku can get expensive • Those $7 / month can easily become $14 / month if you need periodic tasks – where you need a scheduler • If you have more than one project running these things add up • If you’re not making money from those apps, that cost could be difficult to justify • Luckily the open source world has our back!
29. Dokku • http://dokku.viewdocs.io/dokku/ • An open source Heroku alternative • Can be self-hosted on a $5 / month Digital Ocean droplet • Or a €3 / month Scaleway VM if you prefer something inside the EU • Dokku has some of the drawbacks of self-hosting, but it respects the 12-factor methodology so it’s much easier to maintain • Treat your servers as cattle, not as pets!
30. Easy from generator-django-rest dokku plugin:install https://github.com/dokku/dokku-letsencrypt.git dokku plugin:install https://github.com/dokku/dokku-postgres.git dokku plugin:install https://github.com/dokku/dokku-redis.git dokku apps:create aviato dokku postgres:create aviatodb dokku postgres:link aviatodb aviato dokku redis:create aviatoredis dokku redis:link aviatoredis aviato dokku config:set --no-restart aviato \ DOKKU_LETSENCRYPT_EMAIL=youremail@dokku.me
 dokku letsencrypt aviato dokku letsencrypt:auto-renew aviato dokku letsencrypt:cron-job --add
31. Aside – what about production production?
32. The “real” production • It’s all fun and games when you only have a small number of users (if any) • Eventually, your app might start getting serious traffic • Scaling of Django apps • a separate topic • lots of good talks on https://pyvideo.org/ (old PyCons)
33. Some pointers • Fully Docker-based deployments • Amazon ECS • Other AWS services – RDS etc. • Hosted Kubernetes solutions • Serverless • Serve a Django App from AWS Lambda • Different architecture • Are these the best approaches when you’re getting started • Probably not – you’re exploring what app your users might like • Only when you’ve found something worthwhile does it make sense to put more energy into scalability • When you do, however, find the right help – good DevOps engineers
34. Conclusion • Don’t go reinventing the wheel • The twelve-factor methodology rocks! • Heroku and Dokku rock! • Get out and have fun working on your side-projects!
35. Thanks! Dražen Lučanin https://punkrockdev.com/ @metakermit Expect slides on http://metakermit.com/talks