High Performance Django at Ten

It’s been over a decade since I co-authored High Performance Django. The book helped thousands of developers understand how to build faster, more scalable Django applications.

This talk will revisit the techniques we recommended to see what’s aged like fine wine and what's aged like, well, milk. You’ll get an inside look at how Lincoln Loop manages Django applications that handle hundreds of millions of requests per month.

Topics covered will include:

* Caching
* Database Performance
* Monitoring and Observability
* Load Testing
* Third-Party Dependencies
* Infrastructure

This talk is best suited for intermediate to advanced Django developers who are already comfortable building their own apps and have a general understanding of deployments.

Transcript

1. High Performance Django at Ten Old Tricks and New Tips Peter Baumgartner September 10, 2025 | Lincoln Loop
2. High Performance Django at Ten Eleven Old Tricks and New Tips Peter Baumgartner September 10, 2025 | Lincoln Loop
3. Peter Baumgartner • Founder at Lincoln Loop • Creator of AppPack • Co-author of High Performance Django ABOUT ME
4. HighPerformanceDjango.com
6. TIMELINE 🚀 Launch: May 12, 2014
7. TIMELINE 🚀 Launch: May 12, 2014 💰 Funded: May 15, 2014 $5k
8. TIMELINE 🚀 Launch: May 12, 2014 💰 Funded: May 15, 2014 $5k ✅ Completed: June 11, 2014 $13,190
9. TIMELINE 🚀 Launch: May 12, 2014 💰 Funded: May 15, 2014 $5k ✅ Completed: June 11, 2014 $13,190 🫠 Existential dread
10. TIMELINE 🚀 Launch: May 12, 2014 💰 Funded: May 15, 2014 $5k ✅ Completed: June 11, 2014 $13,190 🫠 Existential dread 🥵 Painful grind
11. TIMELINE 🚀 Launch: May 12, 2014 💰 Funded: May 15, 2014 $5k ✅ Completed: June 11, 2014 $13,190 🫠 Existential dread 🥵 Painful grind 😬 Imposter syndrome, more dread
12. TIMELINE 🚀 Launch: May 12, 2014 💰 Funded: May 15, 2014 $5k ✅ Completed: June 11, 2014 $13,190 🫠 Existential dread 🥵 Painful grind 😬 Imposter syndrome, more dread 🚚 Shipped: September 12, 2014
13. Was it worth it?
17. $57,514.20 Total Sales
19. Rereading in 2025
24. “ For our team at Lincoln Loop, the guiding philosophy in designing high-traffic Django sites is simplicity… Simple systems are easier to scale, easier to understand, and easier to develop. 👍
25. “ Simplicity means: 1. Using as few moving parts as possible to make it all work... 2. Choosing proven and dependable moving parts instead of the new hotness. 3. Using a proven and dependable architecture instead of blazing your own trail. 4. Deflecting traffic away from complex parts and toward fast, scalable, and simple parts. ❤
26. Boring Technology
29. • Load Balancer ◦ Open Source: HAProxy, Nginx, Varnish ◦ Commercial: Amazon ELB Elastic Load Balancer), Rackspace Cloud Load Balancer • Web Accelerator ◦ Open Source: Varnish, Nginx + Memcached ◦ Commercial: Fastly, Cloudflare • App Server: uWSGI, Gunicorn, Apache • Cache: Memcached, Redis • Database: Postgres, MySQL/MariaDB
30. • Load Balancer ◦ Open Source: HAProxy, Nginx, Varnish ◦ Commercial: Amazon ELB Elastic Load Balancer), Rackspace Cloud Load Balancer • Web Accelerator ◦ Open Source: Varnish, Nginx + Memcached ◦ Commercial: Fastly, Cloudflare • App Server: uWSGI, Gunicorn, Apache • Cache: Memcached, Redis or Valkey? • Database: Postgres, MySQL/MariaDB
32. Server based ↓ Container based
33. “ One strength of the Django ecosystem is the variety of high-quality reusable applications available on PyPI. Unfortunately, the vast majority of these applications were not written for high traffic scenarios. 👍
34. Optimization
35. “ In almost every dynamic web application, the database is the performance bottleneck. 👍
36. “ Reduce Query Counts The first and simplest reduction is to look for places where select_related and prefetch_related can be used. 👍
37. “ Missing Index The main place a missing index will hurt performance is when the WHERE clause is used on a non-indexed column(s) on a large table. 👍
38. “ Counts Database counts are notoriously slow. Avoid them whenever possible. 👍
39. “ Query Caching For many workloads, a generic query cache will give you the biggest gains with the lowest effort. There are a couple of good (well-tested, well-used) options in the Django ecosystem including Johnny Cache…
40. “ Query Caching For many some workloads, a generic query cache will give you the biggest gains with the lowest effort. There are a couple of good (well-tested, well-used) options in the Django ecosystem including Johnny Cache django-cachalot… 🤨
41. “ Do Slow Work Later The best way to make those views fast is to push the slow work to a job queue. In the Python world, Celery is the job queue of choice.
42. “ Do Slow Work Later The best way to make those views fast is to push the slow work to a job queue. In the Python world, Celery is the job queue of choice. 😐
43. “ We typically use Jenkins CI…
44. “ We typically use Jenkins CI Github Actions… 👎
46. “ Auto-scaling does not, however, work as well on stateful systems such as your database.
47. “ Auto-scaling does not, however, work as well on stateful systems such as your database. 👎
48. Self-hosted ↓ Managed
49. “ Out of the box, Postgres and MySQL are not tuned for high performance. There are lots of knobs you can turn to improve performance and the exact settings are dependent on your workload.
50. “ Just use a managed Postgres provider like Amazon RDS or Crunchy Data. Out of the box, Postgres and MySQL are not tuned for high performance. There are lots of knobs you can turn to improve performance and the exact settings are dependent on your workload. 👎
51. Observability
52. “ The de-facto open source metrics collection platform today is Graphite…and pair it with Grafana…
53. “ The de-facto open source metrics collection platform today is Graphite Prometheus…and pair it with Grafana… but I recommend you choose a hosted solution such as AWS Cloudwatch or Datadog. 👎
54. “ Alerting “Seriousˮ open source tools like Nagios and Riemann can interact with Graphite, but also come with a steep learning curve. Less mature options such as Django-based Cabot, seyren, and rearview are also available.
55. “ Alerting Just use a managed provider like Amazon Cloudwatch or Datadog. “Seriousˮ open source tools like Nagios and Riemann can interact with Graphite, but also come with a steep learning curve. Less mature options such as Django-based Cabot, seyren, and rearview are also available. 󰣻
56. “ Logging …youʼll want to use a log aggregator. Splunk and Loggly are popular commercial services, but we prefer the combination of ElasticSearch, Kibana, and Heka which provides similar functionality in an open source package. You may also see people using Logstash in place of Heka to accomplish the same task.
57. “ Logging Just use a managed provider like Amazon Cloudwatch, Sentry, or Datadog. …youʼll want to use a log aggregator. Splunk and Loggly are popular commercial services, but we prefer the combination of ElasticSearch, Kibana, and Heka which provides similar functionality in an open source package. You may also see people using Logstash in place of Heka to accomplish the same task. 😞
58. “ Error Reporting Luckily, better options for error reporting already exist. Again NewRelic is an option here, but the open source Sentry project is better suited to this task.
59. “ Error Reporting Luckily, better options for error reporting already exist. Again NewRelic is an option here, but the open source-ish Sentry project is better suited to this task. 👍
61. “ There are a number of tools available that let you generate a flood of HTTP traffic to see where and when your servers start to topple over. Apache Bench (ab) and Siege are popular and easy to use tools for basic sanity checking, but for more robust tests, Jmeter is king.
62. “ There are a number of tools available that let you generate a flood of HTTP traffic to see where and when your servers start to topple over. Apache Bench (ab) and Siege hey are popular and easy to use tools for basic sanity checking, but for more robust tests, Jmeter is king look towards Grafana k6 or Locust. 👎
63. New Topics
64. CHANNELS & ASYNC
65. CONTINUOUS DEPLOYMENT
66. REAL CONTINUOUS DEPLOYMENT
67. ROLLBACKS
68. BACKWARDS COMPATIBILITY
69. “ Our core principles when choosing a system are: • Keep it very simple • Donʼt re-invent the wheel • Go with proven and solid technologies when you can INSTAGRAM ENGINEERING
70. GET IN TOUCH! Peter Baumgartner Founder pete@lincolnloop.com @ipmb