Make your pony fly!

Tips and best practices to get your Django site to fly

Transcript

1. Make your pony fly! Benjamin Wohlwend
2. Hi, my name is Benjamin and I'm a lazy programmer.
3. Hi, my name is Benjamin and I'm a lazy programmer. I like my code to be lazy. In other words, I like my code to not do too much work.
4. Making your code not do too much work, makes your code fast(er)!
5. Let's build a StackOverflow clone but for pony owners
6. Piece of , right? class Question(models.Model): user = models.ForeignKey('auth.User') title = models.CharField(max_length=100) body = models.TextField() tags = models.ManyToManyField('Tag') class Answer(models.Model): question = models.ForeignKey(Question) user = models.ForeignKey('auth.User') title = models.CharField(max_length=100) body = models.TextField() class Tag(models.Model): name = models.CharField(max_length=30)
7. Piece of , right? class Question(models.Model): user = models.ForeignKey('auth.User') title = models.CharField(max_length=100) body = models.TextField() tags = models.ManyToManyField('Tag') class Answer(models.Model): question = models.ForeignKey(Question) user = models.ForeignKey('auth.User') body = models.TextField() class Tag(models.Model): name = models.CharField(max_length=30)
8. Piece of , right? class Question(models.Model): user = models.ForeignKey('auth.User') title = models.CharField(max_length=100) body = models.TextField() tags = models.ManyToManyField('Tag') class Answer(models.Model): question = models.ForeignKey(Question) user = models.ForeignKey('auth.User') body = models.TextField() class Tag(models.Model): name = models.CharField(max_length=30)
9. Some views... class QuestionsList(ListView): paginage_by = 20 model = Question class QuestionDetail(DetailView): model = Question
10. and templates <ul> {% for question in object_list %} <li> <h3>{{ question.title }} by {{ question.user.username }}</h3> {{ question.body|markdown }} <p>{{ question.answer.all|length }} answers</p> <p>{{ question.tags.all|join:", "}}</p> </li> {% endfor %} </ul>
11. We go live, everybody is all until...
12. Somebody links to us from ponynews (our server)
13. What happened?
14. We have a performance problem or more realistically, several of them
15. How do you even find performance issues?
16. Debug Production django-debug-toolbar django-devserver and many more server logs New Relic Opbeat ☺
17. Debug Production django-debug-toolbar django-devserver and many more server logs New Relic Opbeat ☺
18. django-debug-toolbar It's fantastic. Use it. All the time.
19. django-debug-toolbar
20. django-debug-toolbar bad news
21. django-devserver
22. We found an issue! now what?
23. The Big Three Reduce database queries Do less work Cache! And if all else fails:
24. Reducing SQL queries Django gives us two great tools: select_related()
 and prefetch_related()
25. select_related() Use it to "preload" objects related by ForeignKey a.k.a. "JOIN" class Question(models.Model): user = models.ForeignKey('auth.User') {% for question in object_list %} <li> <h3>by {{ question.user.username }}</h3>
26. {% for question in object_list %} <li> <h3>by {{ question.user.username }}</h3> </li> {% endfor %}
27. {% for question in object_list %} <li> <h3>by {{ question.user.username }}</h3> </li> {% endfor %}
28. {% for question in object_list %} <li> <h3>by {{ question.user.username }}</h3> </li> {% endfor %}
29. {% for question in object_list %} <li> <h3>by {{ question.user.username }}</h3> </li> {% endfor %}
30. {% for question in object_list %} <li> <h3>by {{ question.user.username }}</h3> </li> {% endfor %}
31. {% for question in object_list %} <li> <h3>by {{ question.user.username }}</h3> </li> {% endfor %}
32. {% for question in object_list %} <li> <h3>by {{ question.user.username }}</h3> </li> {% endfor %}
33. select_related() class QuestionsList(ListView): model = Question queryset = Question.objects.select_related('user')
34. select_related() 1 query
35. select_related() 1 query
36. prefetch_related() Use it to prefetch objects related via reverse ForeignKey or ManyToMany relationships
37. prefetch_related() Use it to prefetch objects related via reverse ForeignKey or ManyToMany relationships {% for question in object_list %} <li> <p>{{ question.tags.all|join:", " }}</p> </li> {% endfor %} class Question(models.Model): tags = models.ManyToManyField('Tag')
38. {% for question in object_list %} <li> <p>{{ question.tags.all|join:", " }}</p> </li> {% endfor %}
39. {% for question in object_list %} <li> <p>{{ question.tags.all|join:", " }}</p> </li> {% endfor %}
40. {% for question in object_list %} <li> <p>{{ question.tags.all|join:", " }}</p> </li> {% endfor %}
41. {% for question in object_list %} <li> <p>{{ question.tags.all|join:", " }}</p> </li> {% endfor %}
42. {% for question in object_list %} <li> <p>{{ question.tags.all|join:", " }}</p> </li> {% endfor %}
43. {% for question in object_list %} <li> <p>{{ question.tags.all|join:", " }}</p> </li> {% endfor %}
44. {% for question in object_list %} <li> <p>{{ question.tags.all|join:", " }}</p> </li> {% endfor %}
45. prefetch_related() class QuestionsList(ListView): model = Question queryset = Question.objects.select_related( 'user' ).prefetch_related('tags')
46. prefetch_related()
47. Do less work Duh.
48. Do less work Move work out of the request-response cycle (e.g. celery)
49. Do less work Only fetch data that you really need (e.g. use QuerySet.defer() to not load unneeded large columns)
50. Do less work Let others do the work (e.g. do calculations in the database with annotate / aggregate)
51. Do less work <p>{{ question.answers.all|length }} answers</p> We fetch all answers, just to count them. That's kinda stupid.... let the database count!
52. annotate() class QuestionsList(ListView): model = Question queryset = Question.objects.select_related( 'user' ).prefetch_related( 'tags', ).annotate(answer_count=models.Count('answer')) <p>{{ question.answer_count }} answers</p>
53. Caching a.k.a. why even bother with all the other stuff?
54. Caching Proper caching can make your site withstand thousands of concurrent users. But... caching is hard!
55. Caching Proper caching can make your site withstand thousands of concurrent users. But... caching cache invalidation is hard!
56. Caching Only start caching when there is really a need for it.
57. Let's say there is a need One of the easiest ways to start with caching is the cache_page decorator from django.views.decorators.cache import cache_page question_list = cache_page(60)(QuestionsList.as_view())
58. Further caching Template caching with {% cache %} ORM caching with johnny-cache or django-cache-machine Caching HTTP proxy like Varnish
60. In retrospect Watch out for those SQL queries! Being lazy is good, don't let your code do too much work. If everything else fails, bring out the big caching guns.
61. Thanks #teampony #bringtheponyback @piquadrat_ch