Speeding Up Django Apps

This is for people who are currently building or have finished building a Django app and are interested in minimizing load-time.

It includes optimization tips-and-tricks for:

Django ORM
Django Templates
Third party apps

Transcript

1. Paulo Alvarado @czpython SPEEDING UP
2. Hi, I’m Paulo Alvarado ✴ django CMS lead developer ✴ Web developer at ✴ Enjoy hiking and mountains
3. Speed is an essential factor
4. Topics ✴ ORM ✴ Templates ✴ Third Party Apps
5. Example Model class Hotel(models.Model):
 name = models.CharField(max_length=200)
 city = models.CharField(max_length=120)
 postcode = models.PositiveIntegerField()
 manager = models.ForeignKey( to='auth.User', blank=True, null=True ) modified_on = models.DateTimeField(auto_now=True)
 
 def __str__(self):
 return self.name
6. django ORM ✴ Prefetching ✴ Bulk create ✴ Using _id on foreign key fields ✴ Saving / updating ✴ Fetching only what you need
7. Prefetching ORM
8. Prefetching ORM In [1]: Hotel.objects.select_related(‘manager’) Out[1]: ... In [2]: Hotel.objects.prefetch_related(‘photos’) Out[2]: ...
9. Bulk Create ORM
10. Bulk Create ORM # Ran test create_v1 in 6.549 seconds # Ran test create_v2 in 0.477 seconds def create_v1(hotels): for hotel in hotels: hotel.save() def create_v2(hotels): Hotel.objects.bulk_create(hotels)
11. Using _id on Foreign Key fields
12. Using _id on Foreign Key fields ORM def has_manager(hotel): # Illustrates the usage of _id to avoid # a new query return bool(hotel.manager_id)
13. Saving / Updating ORM
14. Saving / Updating ORM def set_new_manager_v1(hotel, manager): hotel.manager = manager hotel.save() def set_new_manager_v2(hotel, manager): # Call save(update_fields) when you have full control # over the fields that change. hotel.manager = manager hotel.save(update_fields=['manager'])
15. Saving / Updating ORM def set_new_manager_v3(hotel, manager): # Call queryset.update() to avoid triggering # save signals. It will only run an SQL UPDATE. Hotel.objects.filter(pk=hotel.pk).update(manager=manager)
16. Saving / Updating ORM def set_new_manager_v4(lookup, hotel, manager): # Call queryset.update() to avoid triggering # save signals. It will only run an SQL UPDATE. lookup(pk=hotel.pk).update(manager=manager) lookup = Hotel.objects.filter for hotel, manager in hotels_and_managers: set_new_manager_v4(lookup, hotel, manager)
17. Saving / Updating ORM def update_manager_data(manager, **new_data): for field, value in new_data.items(): setattr(manager, field, value) # Not recommended as manager points to a third- # party model which might override the data # that gets written # manager.save(update_fields=list(new_data.keys())) manager.save()
18. Fetching only what you need ORM
19. Fetching only what you need ORM # Ran test print_hotel_names_v1 in 0.175117969513
 # Ran test print_hotel_names_v2 in 0.173953056335 def print_hotel_names_v1():
 for hotel in Hotel.objects.all():
 print hotel.name
 
 def print_hotel_names_v2():
 for hotel in Hotel.objects.only('name'):
 print hotel.name
20. Fetching only what you need ORM # Ran test print_hotel_names_v3 in 0.075676202774
 # Ran test print_hotel_names_v4 in 0.0628550052643 def print_hotel_names_v3():
 for data in Hotel.objects.values('name'):
 print data['name']
 
 def print_hotel_names_v4():
 for name in Hotel.objects.values_list('name', flat=True):
 print name
21. Fetching only what you need ORM 
 
 with open('hotels.csv', 'wb') as testfile:
 queryset = Hotel.objects.values_list( 'pk', 'name', 'city', ‘postcode' )
 csv_writer = csv.writer(testfile)
 csv_writer.writerow(['ID', 'Name', 'City', 'Post Code'])
 csv_writer.writerows(queryset.iterator())
22. Templates ✴ Cached template loader (projects only) ✴ Per request cached templates (reusable apps) ✴ Fragment caching ✴ More tags less template logic
23. Cached template loader Templates TEMPLATES = [{
 ... 
 'OPTIONS': {
 'loaders': [
 ('django.template.loaders.cached.Loader', [
 'django.template.loaders.filesystem.Loader',
 'django.template.loaders.app_directories.Loader',
 ]),
 ],
 }, 
 }]
24. Per request cached templates Templates 
 <h2>Hotel List</h2>
 <div class="hotels-list">
 {% for hotel in hotels %}
 {% include "hotels/includes/hotel_info.html" %}
 {% endfor %}
 </div>

25. Per request cached templates Templates <h2>Hotel List With Cached Template</h2>
 <div class="hotels-list">
 {% for hotel in hotels %}
 {% include hotel_templates.hotel_info %}
 {% endfor %}
 </div>
26. Fragment caching Templates 
 {% load cache %} <h2>Hotel List With Fragment Caching</h2>
 {# 10 minute TTL #}
 {% cache 600 "hotel_list" hotels|length %}
 <div class="hotels-list">
 {% for hotel in hotels %}
 {% include hotel_templates.hotel_info %}
 {% endfor %}
 </div>
 {% endcache %}
27. Fragment caching Templates 
 {% load cache %} <h2>Hotel List Fragment Cache Russian Doll</h2>
 {# 10 minute TTL #}
 {% cache 600 "hotel_list" hotels|length %}
 <div class="hotels-list">
 {% for hotel in hotels %}
 {# 1 day TTL #}
 {% cache 86400 "hotel_list" hotel.pk hotel.modified_on %}
 {% include hotel_templates.hotel_info %}
 {% endcache %}
 {% endfor %}
 </div>
 {% endcache %}
28. More tags Less template logic Templates {% for plugin in plugins %}
 {% include 'cms/plugin.html' %}
 {% endfor %}
29. More tags Less template logic Templates <script>
 CMS._plugins.push(['cms-plugin-{% if generic %}{{ generic.app_label }}-{{ generic.model_name }} -{% if attribute_name %}{{ attribute_name|slugify }}-{% endif %}{% endif %}{{ instance.pk|unlocalize }}', {
 type: {% if generic %}'generic'{% else %}'plugin'{% endif %},
 page_language: '{{ LANGUAGE_CODE }}',
 placeholder_id: '{{ instance.placeholder_id|unlocalize }}',
 plugin_name: '{{ instance.get_plugin_name|default:"" }}',
 plugin_type: '{{ instance.plugin_type }}',
 plugin_id: '{{ instance.pk|unlocalize }}',
 plugin_language: '{{ instance.language|default:"" }}',
 plugin_parent: '{{ instance.parent_id|unlocalize }}',
 plugin_order: '{{ instance.plugin_order }}',{% language request.toolbar.toolbar_language %}
 plugin_restriction: [{% for cls in allowed_child_classes %}"{{ cls }}"{% if not forloop.last %},{% endif %}{% endfor %}],
 plugin_parent_restriction: [{% for cls in allowed_parent_classes %}"{{ cls }}"{% if not forloop.last %},{% endif %}{% endfor %}],
 onClose: {% if refresh_page %}'REFRESH_PAGE'{% else %}{% if redirect_on_close %}'{{ redirect_on_close }}'{% else %}false{% endif %}{% endif %},
 addPluginHelpTitle: '{% trans "Add plugin to" %} {{ instance.get_plugin_name|escapejs }}',
 urls: {
 add_plugin: '{% if add_url %}{{ add_url }}{% else %}{% cms_admin_url "cms_page_add_plugin" %}{% endif %}',
 edit_plugin: '{% if edit_url %}{{ edit_url }}{% elif instance %}{% cms_admin_url "cms_page_edit_plugin" instance.pk %}{% endif %}',
 move_plugin: '{% if move_url %}{{ move_url }}{% else %}{% cms_admin_url "cms_page_move_plugin" %}{% endif %}',
 delete_plugin: '{% if delete_url %}{{ delete_url }}{% elif instance %}{% cms_admin_url "cms_page_delete_plugin" instance.pk %}{% endif %}',
 copy_plugin: '{% if copy_url %}{{ copy_url }}{% else %}{% cms_admin_url "cms_page_copy_plugins" %}{% endif %}'
 } {% endlanguage %}
 }]);
 </script>
30. More tags Less template logic Templates {% load cms_sample_tags %}
 
 {% render_plugins_js plugins 'en' %}

31. More tags Less template logic Templates PLUGIN_TOOLBAR_JS = "CMS._plugins.push(['cms-plugin-%(pk)s', %(config)s]);"
 
 def get_plugin_toolbar_js(plugin, request_language, children=None, parents=None):
 plugin_name = plugin.get_plugin_name()
 help_text = ugettext('Add plugin to %(plugin_name)s') % {'plugin_name': plugin_name}
 
 data = {
 'type': 'plugin',
 'page_language': request_language,
 'placeholder_id': text_type(plugin.placeholder_id),
 'plugin_name': force_text(plugin_name) or '',
 'plugin_parent': text_type(plugin.parent_id or ''),
 'plugin_restriction': children or [], …
 }
 return PLUGIN_TOOLBAR_JS % {'pk': plugin.pk, 'config': json.dumps(data)}
32. More tags Less template logic Templates @register.simple_tag(takes_context=False)
 def render_plugins_js(context, plugins, language):
 get_toolbar_js = functools.partial(
 get_plugin_toolbar_js,
 request_language=language,
 )
 plugin_js_output = '\n'.join(get_toolbar_js(plugin) for plugin in plugins)
 return mark_safe(plugin_js_output)
33. Third Party Apps
34. Thanks! Paulo Alvarado @czpython bit.ly/djangospeed-pr