Django и HTTP-кэширование

Ринат Хабибиев (zvooq.com, ведущий разработчик) @ Moscow Python Conf 2017
"Много лет работая с Django, мне часто приходилось дорабатывать встроенные решения, либо заменять их на свои собственные реализации. На встрече речь пойдёт о механизме HTTP-кэширования Django, об эффективном его использовании и о том, как сделать этот инструмент ещё более эффективным, устранив в нем несколько проблем и добавив новых возможностей. Заодно, разберём лучшие практики HTTP-кэширования и попробуем решить главнейшую проблему в программировании - инвалидацию кэша".
Видео: https://conf.python.ru/django-and-http-cash/

Transcript

1. Django: HTTP-cache
   Developer’s guide
   Rinat Khabibiev, 2017
   

   View Slide

2. RINAT KHABIBIEV
   2
   https://github.com/renskiy https://www.facebook.com/rinat.khabibiev
   PostgreSQL
   

   View Slide

3. Музыкальный сервис
   для массовой
   аудитории
   

   View Slide

4. HTTP-CACHE
   • What is it
   • How it works
   • Cache management
   • Django HTTP tools
   

   View Slide

5. SERVER-SIDE CACHE
   HTML
   JSON
   XML
   …
   

   View Slide

6. CLIENT-SIDE CACHE
   

   View Slide

7. ALL CACHES (HTTP-CACHING)
   

   View Slide

8. CACHEABLE REQUESTS AND RESPONSES
   GET HEAD POST
   RFC 7231 HTTP 1.1:
   Semantics and Content
   Requests
   200 203 204
   300 301
   404 405 410 414
   501
   Responses
   

   View Slide

9. SERVER-SIDE CACHE MANAGEMENT
   

   View Slide

10. SERVER-SIDE CACHE MANAGEMENT
    Expires: Tue, 31 Jan 2012 15:02:53 GMT
    Last-Modified: Mon, 30 Jan 2012 15:02:53 GMT
    ETag: W/"56d-9989200-1132c580"
    Cache-Control: public, max-age=86400
    Age: 3600
    Vary: Cookie, Accept-Language
    

    View Slide

11. CLIENT-SIDE CACHE MANAGEMENT
    

    View Slide

12. CLIENT-SIDE CACHE MANAGEMENT
    If-Modified-Since: Tue, 31 Jan 2012 15:02:53 GMT
    If-Unmodified-Since: Tue, 31 Jan 2012 15:02:53 GMT
    Pragma: no-cache
    Cache-Control: max-age=30
    If-None-Match: W/"56d-9989200-1132c580"
    If-Match: "56d-9989200-1132c580"
    If-Range: "56d-9989200-1132c580"
    

    View Slide

13. DJANGO HTTP-CACHING: MIDDLEWARES
    # settings.py
    MIDDLEWARE = [
    ‘django.middleware.http.ConditionalGetMiddleware',
    'django.middleware.cache.UpdateCacheMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.cache.FetchFromCacheMiddleware',
    ]
    CACHES = {
    'default': {
    'BACKEND': 'django.core.cache.backends.locmem.LocMemCache',
    },
    }
    CACHE_MIDDLEWARE_ALIAS = 'default'
    CACHE_MIDDLEWARE_SECONDS = 600
    CACHE_MIDDLEWARE_KEY_PREFIX = ''
    

    View Slide

14. DJANGO HTTP-CACHING: DECORATORS
    from django.utils import timezone
    def get_last_modified(request: HttpRequest, *args, **kwargs):
    return timezone.now()
    def get_etag(request: HttpRequest, *args, **kwargs):
    return "56d-9989200-1132c580"
    @cache_page(600, cache='default', key_prefix='')
    @last_modified(get_last_modified)
    @etag(get_etag)
    def my_view(request: HttpRequest, *args, **kwargs):
    return HttpResponse()
    

    View Slide

15. DJANGO HTTP-CACHING
    # settings.py
    MIDDLEWARE = [
    'django.middleware.cache.UpdateCacheMiddleware',
    ...
    'django.middleware.cache.FetchFromCacheMiddleware',
    ]
    

    View Slide

16. HTTP/1.1 200 OK
    Date: Sun, 01 Jan 2017 00:00:00 GMT
    Expires: Sun, 01 Jan 2017 02:00:00 GMT
    Cache-Control: max-age=7200
    Client #1 makes request at 00:00 AM
    HTTP/1.1 200 OK
    Date: Sun, 01 Jan 2017 00:01:00 GMT
    Expires: Sun, 01 Jan 2017 02:00:00 GMT
    Cache-Control: max-age=7200
    Age: 3600
    Client #2 makes request 1 hour later and gets response from cache
    This makes cache expiration
    at 03:00, not at 02:00
    DJANGO HTTP-CACHE: MAX-AGE BUG
    

    View Slide

17. DJANGO HTTP-CACHING WITH CONDITIONAL REQUESTS
    

    View Slide

18. DJANGO HTTP-CACHING WITH CONDITIONAL REQUESTS
    

    View Slide

19. Sets header "Vary: Cookie"
    DJANGO HTTP-CACHE: VARY HEADER BUG
    

    View Slide

20. DJANGO HTTP-CACHE: VARY HEADER BUG
    HTTP/1.1 200 OK
    Date: Sun, 01 Jan 2017 00:00:00 GMT
    Expires: Sun, 01 Jan 2017 02:00:00 GMT
    Cache-Control: max-age=7200
    Vary: Cookie
    Authorised client makes request for private data
    HTTP/1.1 200 OK
    Date: Sun, 01 Jan 2017 00:01:00 GMT
    Expires: Sun, 01 Jan 2017 02:00:00 GMT
    Cache-Control: max-age=7200
    Another authorised client makes request later and gets response generated for another user!!
    Actual and very critical when using @cache_page
    (bug #15855)
    Set by SessionMiddleware after response
    was saved to cache
    

    View Slide

21. @cache_page(600)
    @vary_on_headers('Cookie', 'Accept-Language')
    def my_view(request: HttpRequest, *args, **kwargs):
    return HttpResponse()
    DJANGO HTTP-CACHE: VARY HEADER BUG FIX (FORCED)
    

    View Slide

22. DJANGO: CACHE INVALIDATION
    from django.utils.functional import lazy
    def get_key_prefix():
    return 'v1'
    @cache_page(600, key_prefix=lazy(get_key_prefix, str)())
    def my_view(request: HttpRequest, *args, **kwargs):
    return HttpResponse()
    threading.local() can be used to access current request
    

    View Slide

23. DJANGO HTTP-CACHE
    Not too hard, isn’t it?
    

    View Slide

24. DJANGOCACHE
    pip install django-cache
    from djangocache import cache_page
    def get_etag(request: HttpRequest, *args, **kwargs):
    return "56d-9989200-1132c580"
    @cache_page(cache_timeout=600, key_prefix=get_etag)
    @etag(get_etag)
    def my_view(request: HttpRequest, *args, **kwargs):
    return HttpResponse()
    • No middlewares needed
    
    • Works with Django >= 1.8
    
    • Fixed Django cache bugs
    
    • Callable ‘key_prefix’ for cache invalidation
    
    • Callable ‘cache_timeout’
    
    • Cache max-age can be limited by client
    https://github.com/renskiy/django-cache
    

    View Slide

25. LINKS & THANKS
    • RFC 7231 HTTP/1.1: Semantics and Content
    
    • RFC 7232 HTTP/1.1: Conditional Requests
    
    • RFC 7234 HTTP/1.1: Caching
    
    • https://www.djangoproject.com
    Thanks!
    

    View Slide