E-Commerce with Django at Scale: Effective Performance Lessons Learned

Code available at https://github.com/npinchot/djangocon_2015_talk

Description

Two years ago our multi-million dollar E-Commerce site was processing thousands of orders per day through a performant and functional, but quickly aging, ASP classic codebase.

Now, we process those same orders, using our shiny new and Django/Python codebase. I'll take you through some performance lessons we've learned and show you how you can apply them to your own codebases.

Abstract

I'll take you through the most effective performance lessons we've learned and show you how you can implement them (with example code).

Two-pass Rendering with Class-based Views

By far, this is one of the most effective performance optimizations we have done in terms of HTTP response time.

Using class-based views, we are able to do two-pass caching. On the first pass of the view, we render everything that's not specific to the user. No AJAX calls needed to get user specific content on the page. I'll show you how.

Data Caching Strategy

I'll review how we use multiple levels of data caching to greatly improve the amount of time it takes to rebuild the entire cache.

Database Read Replicas & Failover

Read replica databases are great for performance. You've set up a few read replicas and implemented a fancy new database router which sends read queries to the read replicas (round robin) for any data that doesn't need to be up-to-the-millisecond fresh (e.g. blog posts, product descriptions).

You're sitting back and relishing in the improved performance when one of your database read replicas goes offline. Now what? I'll show you how we implemented a custom database backend to handle this gracefully.

Migration Rules

This is less of a performance optimization and more of a set of rules we try to stick to. I'll review some snafus we've had and how we avoided future production issues while keeping the site at 99% uptime.

Transcript

2. E-Commerce with Django at Scale: Effective Performance Lessons Learned
3. E-Commerce with Django at Scale: Effective Performance Lessons Learned • Database Read Replicas & Failover
4. E-Commerce with Django at Scale: Effective Performance Lessons Learned • Database Read Replicas & Failover • Data Caching Strategy
5. E-Commerce with Django at Scale: Effective Performance Lessons Learned • Database Read Replicas & Failover • Data Caching Strategy • Two-pass Rendering with Class-based Views
6. E-Commerce with Django at Scale: Effective Performance Lessons Learned • Database Read Replicas & Failover • Data Caching Strategy • Two-pass Rendering with Class-based Views • Migration Rules
7. Database Read Replicas & Failover Automatic Failover with a Custom Database Backend
8. Database Read Replicas Are Web Scale
9. SHARD even ING odd
10. PAR logs customers orders products TI TION ING
12. Database Read Replicas Django Built-in Support
13. Database Read Replicas • Example in documentation Django Built-in Support
14. Database Read Replicas • Example in documentation • Add read replicas to DATABASES Django Built-in Support
15. Database Read Replicas • Example in documentation • Add read replicas to DATABASES • Simple class added to DATABASE_ROUTERS Django Built-in Support
16. # settings.py ============================================================================= DATABASES = { 'primary': {'ENGINE': 'mysql', 'NAME': 'mydb', 'USER': 'bobby', 'PASSWORD': 'tables'}, 'replica1': {'ENGINE': 'mysql', 'NAME': 'mydb', 'USER': 'bobby', 'PASSWORD': 'tables'}, 'replica2': {'ENGINE': 'mysql', 'NAME': 'mydb', 'USER': 'bobby', 'PASSWORD': 'tables'}, } DATABASE_ROUTERS = ['PrimaryReplicaRouter'] # primary_replica_router.py =============================================================== import random class PrimaryReplicaRouter(object): def db_for_read(self, model, **hints): # Reads go to a randomly-chosen replica. return random.choice(['replica1', 'replica2']) def db_for_write(self, model, **hints): # Writes always go to primary. return 'primary' def allow_relation(self, obj1, obj2, **hints): # Relations are allowed if both objects are in primary/replica pool. db_list = ('primary', 'replica1', 'replica2') return obj1._state.db in db_list and obj2._state.db in db_list def allow_migrate(self, db, app_label, model=None, **hints): return True
17. NO FAILOVER SUPPORT
18. 502 BAD GATEWAY :( POTENTIAL CUSTOMERS :)
19. Cute puppy bro.
 Show me the code! Custom Database Backend Code Walkthrough
20. WARNING Monkey Patch Ahead
21. # mysql_failover_backend/base.py ========================================================== # Import Django standard MySQL backend base from django.db.backends.mysql import base # Import error classes import django.db; import MySQLdb # Django settings to read DATABASES setting from django.conf import settings # Handles connecting to another available DB if a read replica fails def get_new_connection(self, conn_params, **kwargs): try: # Get connection (call original get_new_connection method) conn = base.DatabaseWrapper._orig_get_new_conn(self, conn_params) except (django.db.OperationalError, MySQLdb.OperationalError): # Handle error - Find a new database raise return conn # DatabaseWrapper needs to be defined so Django can import it DatabaseWrapper = base.DatabaseWrapper # Store a reference to get_new_connection so we can call it base.DatabaseWrapper._orig_get_new_conn = base.DatabaseWrapper.get_new_connection # Monkey patch get_new_connection to our function base.DatabaseWrapper.get_new_connection = get_new_connection
22. # mysql_failover_backend/base.py ========================================================== # Import Django standard MySQL backend base from django.db.backends.mysql import base # Import error classes import django.db; import MySQLdb # Django settings to read DATABASES setting from django.conf import settings # Handles connecting to another available DB if a read replica fails def get_new_connection(self, conn_params, **kwargs): try: # Get connection (call original get_new_connection method) conn = base.DatabaseWrapper._orig_get_new_conn(self, conn_params) except (django.db.OperationalError, MySQLdb.OperationalError): # Handle error - Find a new database raise return conn # DatabaseWrapper needs to be defined so Django can import it DatabaseWrapper = base.DatabaseWrapper # Store a reference to get_new_connection so we can call it base.DatabaseWrapper._orig_get_new_conn = base.DatabaseWrapper.get_new_connection # Monkey patch get_new_connection to our function base.DatabaseWrapper.get_new_connection = get_new_connection
23. # mysql_failover_backend/base.py ========================================================== # Import Django standard MySQL backend base from django.db.backends.mysql import base # Import error classes import django.db; import MySQLdb # Django settings to read DATABASES setting from django.conf import settings # Handles connecting to another available DB if a read replica fails def get_new_connection(self, conn_params, **kwargs): try: # Get connection (call original get_new_connection method) conn = base.DatabaseWrapper._orig_get_new_conn(self, conn_params) except (django.db.OperationalError, MySQLdb.OperationalError): # Handle error - Find a new database raise return conn # DatabaseWrapper needs to be defined so Django can import it DatabaseWrapper = base.DatabaseWrapper # Store a reference to get_new_connection so we can call it base.DatabaseWrapper._orig_get_new_conn = base.DatabaseWrapper.get_new_connection # Monkey patch get_new_connection to our function base.DatabaseWrapper.get_new_connection = get_new_connection
24. # mysql_failover_backend/base.py ========================================================== # Import Django standard MySQL backend base from django.db.backends.mysql import base # Import error classes import django.db; import MySQLdb # Django settings to read DATABASES setting from django.conf import settings # Handles connecting to another available DB if a read replica fails def get_new_connection(self, conn_params, **kwargs): try: # Get connection (call original get_new_connection method) conn = base.DatabaseWrapper._orig_get_new_conn(self, conn_params) except (django.db.OperationalError, MySQLdb.OperationalError): # Handle error - Find a new database raise return conn # DatabaseWrapper needs to be defined so Django can import it DatabaseWrapper = base.DatabaseWrapper # Store a reference to get_new_connection so we can call it base.DatabaseWrapper._orig_get_new_conn = base.DatabaseWrapper.get_new_connection # Monkey patch get_new_connection to our function base.DatabaseWrapper.get_new_connection = get_new_connection
25. # mysql_failover_backend/base.py ========================================================== # Import Django standard MySQL backend base from django.db.backends.mysql import base # Import error classes import django.db; import MySQLdb # Django settings to read DATABASES setting from django.conf import settings # Handles connecting to another available DB if a read replica fails def get_new_connection(self, conn_params, **kwargs): try: # Get connection (call original get_new_connection method) conn = base.DatabaseWrapper._orig_get_new_conn(self, conn_params) except (django.db.OperationalError, MySQLdb.OperationalError): # Handle error - Find a new database raise return conn # DatabaseWrapper needs to be defined so Django can import it DatabaseWrapper = base.DatabaseWrapper # Store a reference to get_new_connection so we can call it base.DatabaseWrapper._orig_get_new_conn = base.DatabaseWrapper.get_new_connection # Monkey patch get_new_connection to our function base.DatabaseWrapper.get_new_connection = get_new_connection
26. DATABASES = { 'default': { 'NAME': 'mydb', 'USER': 'web', 'PASSWORD': 'scale', 'HOST': 'master.mysite', 'ENGINE': 'mysql_failover_backend', 'FAILOVER_RETRIES': 3, }, 'read-replica-1': { 'NAME': 'mydb', 'USER': 'web', 'PASSWORD': 'scale', 'HOST': 'rr-1.mysite', 'ENGINE': 'mysql_failover_backend', 'OPTIONS': { 'connect_timeout': 5, }, 'FAILOVER_MASTER': 'default', }, 'read-replica-2': { 'NAME': 'mydb', 'USER': 'web', 'PASSWORD': 'scale', 'HOST': 'rr-2.mysite', 'ENGINE': 'mysql_failover_backend', 'OPTIONS': { 'connect_timeout': 5, }, 'FAILOVER_MASTER': 'default', }, }
27. DATABASES = { 'default': { 'NAME': 'mydb', 'USER': 'web', 'PASSWORD': 'scale', 'HOST': 'master.mysite', 'ENGINE': 'mysql_failover_backend', 'FAILOVER_RETRIES': 3, }, 'read-replica-1': { 'NAME': 'mydb', 'USER': 'web', 'PASSWORD': 'scale', 'HOST': 'rr-1.mysite', 'ENGINE': 'mysql_failover_backend', 'OPTIONS': { 'connect_timeout': 5, }, 'FAILOVER_MASTER': 'default', }, 'read-replica-2': { 'NAME': 'mydb', 'USER': 'web', 'PASSWORD': 'scale', 'HOST': 'rr-2.mysite', 'ENGINE': 'mysql_failover_backend', 'OPTIONS': { 'connect_timeout': 5, }, 'FAILOVER_MASTER': 'default', }, }
28. DATABASES = { 'default': { 'NAME': 'mydb', 'USER': 'web', 'PASSWORD': 'scale', 'HOST': 'master.mysite', 'ENGINE': 'mysql_failover_backend', 'FAILOVER_RETRIES': 3, }, 'read-replica-1': { 'NAME': 'mydb', 'USER': 'web', 'PASSWORD': 'scale', 'HOST': 'rr-1.mysite', 'ENGINE': 'mysql_failover_backend', 'OPTIONS': { 'connect_timeout': 5, }, 'FAILOVER_MASTER': 'default', }, 'read-replica-2': { 'NAME': 'mydb', 'USER': 'web', 'PASSWORD': 'scale', 'HOST': 'rr-2.mysite', 'ENGINE': 'mysql_failover_backend', 'OPTIONS': { 'connect_timeout': 5, }, 'FAILOVER_MASTER': 'default', }, }
29. DATABASES = { 'default': { 'NAME': 'mydb', 'USER': 'web', 'PASSWORD': 'scale', 'HOST': 'master.mysite', 'ENGINE': 'mysql_failover_backend', 'FAILOVER_RETRIES': 3, }, 'read-replica-1': { 'NAME': 'mydb', 'USER': 'web', 'PASSWORD': 'scale', 'HOST': 'rr-1.mysite', 'ENGINE': 'mysql_failover_backend', 'OPTIONS': { 'connect_timeout': 5, }, 'FAILOVER_MASTER': 'default', }, 'read-replica-2': { 'NAME': 'mydb', 'USER': 'web', 'PASSWORD': 'scale', 'HOST': 'rr-2.mysite', 'ENGINE': 'mysql_failover_backend', 'OPTIONS': { 'connect_timeout': 5, }, 'FAILOVER_MASTER': 'default', }, }
30. # mysql_failover_backend/base.py ========================================================== # Import Django standard MySQL backend base from django.db.backends.mysql import base # Import error classes import django.db; import MySQLdb # Django settings to read DATABASES setting from django.conf import settings # Handles connecting to another available DB if a read replica fails def get_new_connection(self, conn_params, **kwargs): try: # Get connection (call original get_new_connection method) conn = base.DatabaseWrapper._orig_get_new_conn(self, conn_params) except (django.db.OperationalError, MySQLdb.OperationalError): # Handle error - Find a new database raise return conn # DatabaseWrapper needs to be defined so Django can import it DatabaseWrapper = base.DatabaseWrapper # Store a reference to get_new_connection so we can call it base.DatabaseWrapper._orig_get_new_conn = base.DatabaseWrapper.get_new_connection # Monkey patch get_new_connection to our function base.DatabaseWrapper.get_new_connection = get_new_connection
31. try: # Get connection (call original get_new_connection method) conn = base.DatabaseWrapper._orig_get_new_conn(self, conn_params) except (django.db.OperationalError, MySQLdb.OperationalError): # Handle error - Find a new database raise return conn
32. except (django.db.OperationalError, MySQLdb.OperationalError):
33. except (django.db.OperationalError, MySQLdb.OperationalError): # Get read replica master DB alias (raise error if not a read replica) master = self.settings_dict.get('FAILOVER_MASTER', None) if not master: raise
34. except (django.db.OperationalError, MySQLdb.OperationalError): # Get read replica master DB alias (raise error if not a read replica) master = self.settings_dict.get('FAILOVER_MASTER', None) if not master: raise # settings.py DATABASES = { 'default': { }, 'read-replica-1': { 'NAME': 'mydb', 'USER': 'web', 'PASSWORD': 'scale', 'HOST': 'rr-1.mysite', 'ENGINE': 'mysql_failover_backend', 'OPTIONS': { 'connect_timeout': 5, }, 'FAILOVER_MASTER': 'default', }, 'read-replica-2': { }, }
35. except (django.db.OperationalError, MySQLdb.OperationalError): # Get read replica master DB alias (raise error if not a read replica) master = self.settings_dict.get('FAILOVER_MASTER', None) if not master: raise
36. except (django.db.OperationalError, MySQLdb.OperationalError): # Get read replica master DB alias (raise error if not a read replica) master = self.settings_dict.get('FAILOVER_MASTER', None) if not master: raise # List of previously failed DBs + current failure alias = kwargs.get('failover', self.alias) failed_dbs = kwargs.get('failed_dbs', []) + [alias]
37. except (django.db.OperationalError, MySQLdb.OperationalError): # Get read replica master DB alias (raise error if not a read replica) master = self.settings_dict.get('FAILOVER_MASTER', None) if not master: raise # List of previously failed DBs + current failure alias = kwargs.get('failover', self.alias) failed_dbs = kwargs.get('failed_dbs', []) + [alias] # Find first available read replica for db in settings.DATABASES.keys(): if db not in failed_dbs and \ settings.DATABASES[db].get('FAILOVER_MASTER', '') == master: # Found an available read replica new_db = db break else: # No other read replicas available? Use master new_db = master
38. except (django.db.OperationalError, MySQLdb.OperationalError): # Get read replica master DB alias (raise error if not a read replica) master = self.settings_dict.get('FAILOVER_MASTER', None) if not master: raise # List of previously failed DBs + current failure alias = kwargs.get('failover', self.alias) failed_dbs = kwargs.get('failed_dbs', []) + [alias] # Find first available read replica for db in settings.DATABASES.keys(): if db not in failed_dbs and \ settings.DATABASES[db].get('FAILOVER_MASTER', '') == master: # Found an available read replica new_db = db break else: # No other read replicas available? Use master new_db = master
39. except (django.db.OperationalError, MySQLdb.OperationalError): # Get read replica master DB alias (raise error if not a read replica) master = self.settings_dict.get('FAILOVER_MASTER', None) if not master: raise # List of previously failed DBs + current failure alias = kwargs.get('failover', self.alias) failed_dbs = kwargs.get('failed_dbs', []) + [alias] # Find first available read replica for db in settings.DATABASES.keys(): if db not in failed_dbs and \ settings.DATABASES[db].get('FAILOVER_MASTER', '') == master: # Found an available read replica new_db = db break else: # No other read replicas available? Use master new_db = master
40. except (django.db.OperationalError, MySQLdb.OperationalError): # Get read replica master DB alias (raise error if not a read replica) master = self.settings_dict.get('FAILOVER_MASTER', None) if not master: raise # List of previously failed DBs + current failure alias = kwargs.get('failover', self.alias) failed_dbs = kwargs.get('failed_dbs', []) + [alias] # Find first available read replica for db in settings.DATABASES.keys(): if db not in failed_dbs and \ settings.DATABASES[db].get('FAILOVER_MASTER', '') == master: # Found an available read replica new_db = db break else: # No other read replicas available? Use master new_db = master
41. except (django.db.OperationalError, MySQLdb.OperationalError): # Get read replica master DB alias (raise error if not a read replica) master = self.settings_dict.get('FAILOVER_MASTER', None) if not master: raise # List of previously failed DBs + current failure alias = kwargs.get('failover', self.alias) failed_dbs = kwargs.get('failed_dbs', []) + [alias] # Find first available read replica for db in settings.DATABASES.keys(): if db not in failed_dbs and \ settings.DATABASES[db].get('FAILOVER_MASTER', '') == master: # Found an available read replica new_db = db break else: # No other read replicas available? Use master new_db = master
42. except (django.db.OperationalError, MySQLdb.OperationalError): # Get read replica master DB alias (raise error if not a read replica) master = self.settings_dict.get('FAILOVER_MASTER', None) if not master: raise # List of previously failed DBs + current failure alias = kwargs.get('failover', self.alias) failed_dbs = kwargs.get('failed_dbs', []) + [alias] # Find first available read replica for db in settings.DATABASES.keys(): if db not in failed_dbs and \ settings.DATABASES[db].get('FAILOVER_MASTER', '') == master: # Found an available read replica new_db = db break else: # No other read replicas available? Use master new_db = master
43. except (django.db.OperationalError, MySQLdb.OperationalError): # Get read replica master DB alias (raise error if not a read replica) master = self.settings_dict.get('FAILOVER_MASTER', None) if not master: raise # List of previously failed DBs + current failure alias = kwargs.get('failover', self.alias) failed_dbs = kwargs.get('failed_dbs', []) + [alias] # Find first available read replica for db in settings.DATABASES.keys(): if db not in failed_dbs and \ settings.DATABASES[db].get('FAILOVER_MASTER', '') == master: # Found an available read replica new_db = db break else: # No other read replicas available? Use master new_db = master # Replace host in connection params and try new connection conn_params['host'] = settings.DATABASES[new_db]['HOST'] conn = get_new_connection(self, conn_params, failover=new_db, failed_dbs=failed_dbs)
44. except (django.db.OperationalError, MySQLdb.OperationalError): # Get read replica master DB alias (raise error if not a read replica) master = self.settings_dict.get('FAILOVER_MASTER', None) if not master: raise # List of previously failed DBs + current failure alias = kwargs.get('failover', self.alias) failed_dbs = kwargs.get('failed_dbs', []) + [alias] # Find first available read replica for db in settings.DATABASES.keys(): if db not in failed_dbs and \ settings.DATABASES[db].get('FAILOVER_MASTER', '') == master: # Found an available read replica new_db = db break else: # No other read replicas available? Use master new_db = master # Replace host in connection params and try new connection conn_params['host'] = settings.DATABASES[new_db]['HOST'] conn = get_new_connection(self, conn_params, failover=new_db, failed_dbs=failed_dbs)
45. 502 BAD GATEWAY :( POTENTIAL CUSTOMERS :)
46. POTENTIAL CUSTOMERS :) POTENTIAL CUSTOMERS :)
47. Data Caching Strategy Don’t Go for Broke
48. Why? Data Caching Strategy
49. Data Caching Strategy
50. Data Caching Strategy Database
51. Data Caching Strategy Shared Data Objects
 Shared View Models Data Objects View Models Database
52. Data Caching Strategy Shared Data Objects
 Shared View Models Data Objects View Models Database
53. Data Caching Strategy Shared Data Objects
 Shared View Models Data Objects View Models Views Database
54. Data Caching Strategy Shared Data Objects
 Shared View Models Data Objects View Models Views Database Cache
55. Data Caching Strategy Shared Data Objects
 Shared View Models Data Objects View Models Views Database Cache Scheduled Job: Refresh Cache for Frequently Used Objects
56. Two-pass Rendering with Class-based Views More Valuable than Space Cash
57. Caching Views Django Built-in Support
58. Caching Views • Example in documentation Django Built-in Support
59. Caching Views • Example in documentation • Set up cache server in CACHES setting Django Built-in Support
60. Caching Views • Example in documentation • Set up cache server in CACHES setting • Use cache_page Django Built-in Support
61. # views.py =========================================================== from django.views.decorators.cache import cache_page @cache_page(60 * 15) def my_view(request): # ... # urls.py =========================================================== from django.views.decorators.cache import cache_page urlpatterns = [ url(r'^foo/([0-9]{1,2})/$', cache_page(60 * 15)(my_view)), url(r'^foo/([0-9]{1,2})/$', cache_page(60 * 15)(MyView.as_view())), ]
66. Caching Views Django Built-in Support
67. Caching Views • Use Vary headers Django Built-in Support
68. Caching Views • Use Vary headers • Template fragment caching
 (cache template tag) Django Built-in Support
69. # views.py =========================================================== from django.views.decorators.cache import cache_page from django.views.decorators.vary import vary_on_cookie @cache_page(60 * 15) @vary_on_cookie def my_view(request): # ... # index.html ============================================================================== {% load cache %} {% cache 500 request.user.id %} <!-- sidebar content cached per user ID --> {% endcache %}
70. Django Built-in Support for Caching Views What’s the Problem?
72. Two-pass Rendering
73. Two-pass Rendering What is it?
74. Two-pass Rendering What is it? View Load data
75. Two-pass Rendering What is it? View Load data First Pass Render Render non-user specific content
76. Two-pass Rendering What is it? View Load data First Pass Render Render non-user specific content Cache Result
77. Two-pass Rendering What is it? View Load data First Pass Render Render non-user specific content Cache Result Second Pass Render Render user specific content
78. View Load data First Pass Render Render non-user specific content Load from Cache Second Pass Render Render user specific content Two-pass Rendering What is it?
79. View Load data First Pass Render Render non-user specific content Load from Cache Second Pass Render Render user specific content Two-pass Rendering What is it?
80. Two-pass Rendering with Class-based Views Implementation
81. class HomeView(CachedView): TEMPLATE = 'home.html' def get_first_pass_context_vars(self, request): # Non-user specific context vars return {'products': []} def get_second_pass_context_vars(self, request): # User specific context vars return {} def get(self, request): return self.render(request)
82. class HomeView(CachedView): TEMPLATE = 'home.html' def get_first_pass_context_vars(self, request): # Non-user specific context vars return {'products': []} def get_second_pass_context_vars(self, request): # User specific context vars return {} def get(self, request): return self.render(request)
83. class HomeView(CachedView): TEMPLATE = 'home.html' def get_first_pass_context_vars(self, request): # Non-user specific context vars return {'products': []} def get_second_pass_context_vars(self, request): # User specific context vars return {} def get(self, request): return self.render(request)
84. CSRF Token
85. class HomeView(CachedView): TEMPLATE = 'home.html' def get_first_pass_context_vars(self, request): # Non-user specific context vars return {'products': []} def get_second_pass_context_vars(self, request): # User specific context vars return {} def get(self, request): return self.render(request)
86. class CachedView(View): class HomeView(CachedView): TEMPLATE = 'home.html' def get(self, request): return self.render(request)
87. from django.core.cache import cache from django.http import HttpResponse from django.template import RequestContext, Template from django.template.loader import render_to_string from django.views.generic import View class CachedView(View): def render(self, request): # Render first pass template first_pass_render = self._render_first_pass(request) # Set up template, get second pass context vars & render context_vars = self.get_second_pass_context_vars(request) template = Template(first_pass_render) output = template.render(RequestContext(request, context_vars)) return HttpResponse(output) class HomeView(CachedView): TEMPLATE = 'home.html' def get(self, request): return self.render(request)
88. class CachedView(View): def render(self, request): # Render first pass template first_pass_render = self._render_first_pass(request) # Set up template, get second pass context vars & render context_vars = self.get_second_pass_context_vars(request) template = Template(first_pass_render) output = template.render(RequestContext(request, context_vars)) return HttpResponse(output) class HomeView(CachedView): TEMPLATE = 'home.html' def get(self, request): return self.render(request)
89. class CachedView(View): def render(self, request): # Render first pass template first_pass_render = self._render_first_pass(request) # Set up template, get second pass context vars & render context_vars = self.get_second_pass_context_vars(request) template = Template(first_pass_render) output = template.render(RequestContext(request, context_vars)) return HttpResponse(output) class HomeView(CachedView): TEMPLATE = 'home.html' def get(self, request): return self.render(request)
90. class CachedView(View): def render(self, request): # Render first pass template first_pass_render = self._render_first_pass(request) # Set up template, get second pass context vars & render context_vars = self.get_second_pass_context_vars(request) template = Template(first_pass_render) output = template.render(RequestContext(request, context_vars)) return HttpResponse(output) class HomeView(CachedView): TEMPLATE = 'home.html' def get(self, request): return self.render(request)
91. class CachedView(View): def render(self, request): # Render first pass template first_pass_render = self._render_first_pass(request) # Set up template, get second pass context vars & render context_vars = self.get_second_pass_context_vars(request) template = Template(first_pass_render) output = template.render(RequestContext(request, context_vars)) return HttpResponse(output)
92. class CachedView(View): def _render_first_pass(self, request): # Try to load rendered first pass from cache rendered_template = cache.get(self.CACHE_KEY, None) if rendered_template is None: # Get first pass context vars, render template and save to cache context_vars = self.get_first_pass_context_vars(request) rendered_template = render_to_string(self.TEMPLATE, context_vars) cache.set(self.CACHE_KEY, rendered_template) return rendered_template def render(self, request): # Render first pass template first_pass_render = self._render_first_pass(request) # Set up template, get second pass context vars & render context_vars = self.get_second_pass_context_vars(request) template = Template(first_pass_render) output = template.render(RequestContext(request, context_vars)) return HttpResponse(output)
93. class CachedView(View): def _render_first_pass(self, request): # Try to load rendered first pass from cache rendered_template = cache.get(self.CACHE_KEY, None) if rendered_template is None: # Get first pass context vars, render template and save to cache context_vars = self.get_first_pass_context_vars(request) rendered_template = render_to_string(self.TEMPLATE, context_vars) cache.set(self.CACHE_KEY, rendered_template) return rendered_template def render(self, request): # Render first pass template first_pass_render = self._render_first_pass(request) # Set up template, get second pass context vars & render context_vars = self.get_second_pass_context_vars(request) template = Template(first_pass_render) output = template.render(RequestContext(request, context_vars)) return HttpResponse(output)
94. class CachedView(View): def _render_first_pass(self, request): # Try to load rendered first pass from cache rendered_template = cache.get(self.CACHE_KEY, None) if rendered_template is None: # Get first pass context vars, render template and save to cache context_vars = self.get_first_pass_context_vars(request) rendered_template = render_to_string(self.TEMPLATE, context_vars) cache.set(self.CACHE_KEY, rendered_template) return rendered_template def render(self, request): # Render first pass template first_pass_render = self._render_first_pass(request) # Set up template, get second pass context vars & render context_vars = self.get_second_pass_context_vars(request) template = Template(first_pass_render) output = template.render(RequestContext(request, context_vars)) return HttpResponse(output)
95. class CachedView(View): def _render_first_pass(self, request): # Try to load rendered first pass from cache rendered_template = cache.get(self.CACHE_KEY, None) if rendered_template is None: # Get first pass context vars, render template and save to cache context_vars = self.get_first_pass_context_vars(request) rendered_template = render_to_string(self.TEMPLATE, context_vars) cache.set(self.CACHE_KEY, rendered_template) return rendered_template def render(self, request): # Render first pass template first_pass_render = self._render_first_pass(request) # Set up template, get second pass context vars & render context_vars = self.get_second_pass_context_vars(request) template = Template(first_pass_render) output = template.render(RequestContext(request, context_vars)) return HttpResponse(output)
96. class CachedView(View): def _render_first_pass(self, request): # Try to load rendered first pass from cache rendered_template = cache.get(self.CACHE_KEY, None) if rendered_template is None: # Get first pass context vars, render template and save to cache context_vars = self.get_first_pass_context_vars(request) rendered_template = render_to_string(self.TEMPLATE, context_vars) cache.set(self.CACHE_KEY, rendered_template) return rendered_template def render(self, request): # Render first pass template first_pass_render = self._render_first_pass(request) # Set up template, get second pass context vars & render context_vars = self.get_second_pass_context_vars(request) template = Template(first_pass_render) output = template.render(RequestContext(request, context_vars)) return HttpResponse(output)
97. class CachedView(View): def _render_first_pass(self, request): # Try to load rendered first pass from cache rendered_template = cache.get(self.CACHE_KEY, None) if rendered_template is None: # Get first pass context vars, render template and save to cache context_vars = self.get_first_pass_context_vars(request) rendered_template = render_to_string(self.TEMPLATE, context_vars) cache.set(self.CACHE_KEY, rendered_template) return rendered_template def render(self, request): # Render first pass template first_pass_render = self._render_first_pass(request) # Set up template, get second pass context vars & render context_vars = self.get_second_pass_context_vars(request) template = Template(first_pass_render) output = template.render(RequestContext(request, context_vars)) return HttpResponse(output)
98. class CachedView(View): def _render_first_pass(self, request): # Try to load rendered first pass from cache rendered_template = cache.get(self.CACHE_KEY, None) if rendered_template is None: # Get first pass context vars, render template and save to cache context_vars = self.get_first_pass_context_vars(request) rendered_template = render_to_string(self.TEMPLATE, context_vars) cache.set(self.CACHE_KEY, rendered_template) return rendered_template def render(self, request): # Render first pass template first_pass_render = self._render_first_pass(request) # Set up template, get second pass context vars & render context_vars = self.get_second_pass_context_vars(request) template = Template(first_pass_render) output = template.render(RequestContext(request, context_vars)) return HttpResponse(output)
99. class CachedView(View): def _render_first_pass(self, request): # Try to load rendered first pass from cache rendered_template = cache.get(self.CACHE_KEY, None) if rendered_template is None: # Get first pass context vars, render template and save to cache context_vars = self.get_first_pass_context_vars(request) rendered_template = render_to_string(self.TEMPLATE, context_vars) cache.set(self.CACHE_KEY, rendered_template) return rendered_template def render(self, request): # Render first pass template first_pass_render = self._render_first_pass(request) # Set up template, get second pass context vars & render context_vars = self.get_second_pass_context_vars(request) template = Template(first_pass_render) output = template.render(RequestContext(request, context_vars)) return HttpResponse(output)
100. class CachedView(View): def _render_first_pass(self, request): # Try to load rendered first pass from cache rendered_template = cache.get(self.CACHE_KEY, None) if rendered_template is None: # Get first pass context vars, render template and save to cache context_vars = self.get_first_pass_context_vars(request) rendered_template = render_to_string(self.TEMPLATE, context_vars) cache.set(self.CACHE_KEY, rendered_template) return rendered_template def render(self, request): # Render first pass template first_pass_render = self._render_first_pass(request) # Set up template, get second pass context vars & render context_vars = self.get_second_pass_context_vars(request) template = Template(first_pass_render) output = template.render(RequestContext(request, context_vars)) return HttpResponse(output)
101. class CachedView(View): def _render_first_pass(self, request): # Try to load rendered first pass from cache rendered_template = cache.get(self.CACHE_KEY, None) if rendered_template is None: # Get first pass context vars, render template and save to cache context_vars = self.get_first_pass_context_vars(request) rendered_template = render_to_string(self.TEMPLATE, context_vars) cache.set(self.CACHE_KEY, rendered_template) return rendered_template def render(self, request): # Render first pass template first_pass_render = self._render_first_pass(request) # Set up template, get second pass context vars & render context_vars = self.get_second_pass_context_vars(request) template = Template(first_pass_render) output = template.render(RequestContext(request, context_vars)) return HttpResponse(output)
102. class CachedView(View): def _render_first_pass(self, request): # Try to load rendered first pass from cache rendered_template = cache.get(self.CACHE_KEY, None) if rendered_template is None: # Get first pass context vars, render template and save to cache context_vars = self.get_first_pass_context_vars(request) rendered_template = render_to_string(self.TEMPLATE, context_vars) cache.set(self.CACHE_KEY, rendered_template) return rendered_template def render(self, request): # Render first pass template first_pass_render = self._render_first_pass(request) # Set up template, get second pass context vars & render context_vars = self.get_second_pass_context_vars(request) template = Template(first_pass_render) output = template.render(RequestContext(request, context_vars)) return HttpResponse(output)
103. class CachedView(View): def _render_first_pass(self, request): # Try to load rendered first pass from cache rendered_template = cache.get(self.CACHE_KEY, None) if rendered_template is None: # Get first pass context vars, render template and save to cache context_vars = self.get_first_pass_context_vars(request) rendered_template = render_to_string(self.TEMPLATE, context_vars) cache.set(self.CACHE_KEY, rendered_template) return rendered_template def render(self, request): # Render first pass template first_pass_render = self._render_first_pass(request) # Set up template, get second pass context vars & render context_vars = self.get_second_pass_context_vars(request) template = Template(first_pass_render) output = template.render(RequestContext(request, context_vars)) return HttpResponse(output)
104. class CachedView(View): def _render_first_pass(self, request): # Try to load rendered first pass from cache rendered_template = cache.get(self.CACHE_KEY, None) if rendered_template is None: # Get first pass context vars, render template and save to cache context_vars = self.get_first_pass_context_vars(request) rendered_template = render_to_string(self.TEMPLATE, context_vars) cache.set(self.CACHE_KEY, rendered_template) return rendered_template def render(self, request): # Render first pass template first_pass_render = self._render_first_pass(request) # Set up template, get second pass context vars & render context_vars = self.get_second_pass_context_vars(request) template = Template(first_pass_render) output = template.render(RequestContext(request, context_vars)) return HttpResponse(output)
105. CSRF Token
106. <html> <head> <title>Undercover Tourist</title> </head> <body> <h1>Undercover Tourist - Products</h1> {% if request.is_authenticated %} <p>Hi {{ request.user }}</p> {% endif %} <form action="{% url 'home' %}"> {% csrf_token %} {% for product in products %} <h4>{{ product.name }}</h4> <input type="text" name="{{ product.id }}"> {% endfor %} <button type="submit">Add to Cart</button> </form> </body> </html>
107. CSRF Token Warning UserWarning: A {% csrf_token %} was used in a template, but the context did not provide the value. This is usually caused by not using RequestContext.
108. class CachedView(View): def _render_first_pass(self, request): # Try to load rendered first pass from cache rendered_template = cache.get(self.CACHE_KEY, None) if rendered_template is None: # Get first pass context vars, render template and save to cache context_vars = self.get_first_pass_context_vars(request) rendered_template = render_to_string(self.TEMPLATE, context_vars) cache.set(self.CACHE_KEY, rendered_template) return rendered_template def render(self, request): # Render first pass template first_pass_render = self._render_first_pass(request) # Set up template, get second pass context vars & render context_vars = self.get_second_pass_context_vars(request) template = Template(first_pass_render) output = template.render(RequestContext(request, context_vars)) return HttpResponse(output)
109. <html> <head> <title>Undercover Tourist</title> </head> <body> <h1>Undercover Tourist - Products</h1> {% if request.is_authenticated %} <p>Hi {{ request.user }}</p> {% endif %} <form action="{% url 'home' %}"> {% csrf_token %} {% for product in products %} <h4>{{ product.name }}</h4> <input type="text" name="{{ product.id }}"> {% endfor %} <button type="submit">Add to Cart</button> </form> </body> </html>
110. <html> <head> <title>Undercover Tourist</title> </head> <body> <h1>Undercover Tourist - Products</h1> {% if request.is_authenticated %} <p>Hi {{ request.user }}</p> {% endif %} <form action="{% url 'home' %}"> {% csrf_token %} {% for product in products %} <h4>{{ product.name }}</h4> <input type="text" name="{{ product.id }}"> {% endfor %} <button type="submit">Add to Cart</button> </form> </body> </html>
111. <html> <head> <title>Undercover Tourist</title> </head> <body> <h1>Undercover Tourist - Products</h1> {% if request.is_authenticated %} <p>Hi {{ request.user }}</p> {% endif %} <form action="{% url 'home' %}"> {% csrf_token %} {% for product in products %} <h4>{{ product.name }}</h4> <input type="text" name="{{ product.id }}"> {% endfor %} <button type="submit">Add to Cart</button> </form> </body> </html>
112. <html> <head> <title>Undercover Tourist</title> </head> <body> <h1>Undercover Tourist - Products</h1> {% templatetag openblock %} if request.is_authenticated {% templatetag closeblock %} <p>Hi {% templatetag openvariable %} request.user {% templatetag closevariable %}</p> {% templatetag openblock %} endif {% templatetag closeblock %} <form action="{% url 'home' %}"> {% templatetag openblock %} csrf_token {% templatetag closeblock %} {% for product in products %} <h4>{{ product.name }}</h4> <input type="text" name="{{ product.id }}"> {% endfor %} <button type="submit">Add to Cart</button> </form> </body> </html>
113. <html> <head> <title>Undercover Tourist</title> </head> <body> <h1>Undercover Tourist - Products</h1> {% templatetag openblock %} if request.is_authenticated {% templatetag closeblock %} <p>Hi {% templatetag openvariable %} request.user {% templatetag closevariable %}</p> {% templatetag openblock %} endif {% templatetag closeblock %} <form action="{% url 'home' %}"> {% templatetag openblock %} csrf_token {% templatetag closeblock %} {% for product in products %} <h4>{{ product.name }}</h4> <input type="text" name="{{ product.id }}"> {% endfor %} <button type="submit">Add to Cart</button> </form> </body> </html>
114. <html> <head> <title>Undercover Tourist</title> </head> <body> <h1>Undercover Tourist - Products</h1> {% templatetag openblock %} if request.is_authenticated {% templatetag closeblock %} <p>Hi {% templatetag openvariable %} request.user {% templatetag closevariable %}</p> {% templatetag openblock %} endif {% templatetag closeblock %} <form action="{% url 'home' %}"> {% templatetag openblock %} csrf_token {% templatetag closeblock %} {% for product in products %} <h4>{{ product.name }}</h4> <input type="text" name="{{ product.id }}"> {% endfor %} <button type="submit">Add to Cart</button> </form> </body> </html>
115. <html> <head> <title>Undercover Tourist</title> </head> <body> <h1>Undercover Tourist - Products</h1> {% if request.is_authenticated %} <p>Hi {{ request.user }}</p> {% endif %} <form action="/home/"> {% csrf_token %} <h4>5-Day Magic Your Way</h4> <input type="text" name="1"> <h4>4-Day Water Park Fun &amp; More</h4> <input type="text" name="2"> <button type="submit">Add to Cart</button> </form> </body> </html>
116. <html> <head> <title>Undercover Tourist</title> </head> <body> <h1>Undercover Tourist - Products</h1> {% if request.is_authenticated %} <p>Hi {{ request.user }}</p> {% endif %} <form action="/home/"> {% csrf_token %} <h4>5-Day Magic Your Way</h4> <input type="text" name="1"> <h4>4-Day Water Park Fun & More</h4> <input type="text" name="2"> <button type="submit">Add to Cart</button> </form> </body> </html>
117. Two-pass Rendering with Class-based Views End Result
118. Two-pass Rendering with Class-based Views End Result CSRF Token
119. Speed! Two-pass Rendering with Class-based Views End Result
120. Migration Rules Avoid Production Deployment Issues
121. Migration Rules
122. Migration Rules • Don’t go from more precise to less precise
123. Migration Rules • Don’t go from more precise to less precise • Don’t rename columns or tables
124. Migration Rules • Don’t go from more precise to less precise • Don’t rename columns or tables • Don’t delete columns or tables