Building real time applications with Django

A primer on Django Channels

Since the introduction of Channels, real time web has become much easier to work with in Django: discover how you can easily create yours with Django!

Video: https://www.youtube.com/watch?v=lt5l3idQX60&list=PL4_MBPz5hOsLqrlGjX_emY4KMEfvqG0jG&index=5

This talk has been given at Swiss Python Summit 2018: http://www.python-summit.ch/

Transcript

1. A QUICK PRIMER ON CHANNELS A QUICK PRIMER ON CHANNELS BUILDING REAL TIME BUILDING REAL TIME APPLICATIONS WITH APPLICATIONS WITH DJANGO DJANGO SWISS PYTHON SUMMIT SWISS PYTHON SUMMIT RAPPERSWIL - 16TH FEBRUARY RAPPERSWIL - 16TH FEBRUARY
2. HELLO, I AM IACOPO HELLO, I AM IACOPO Founder and CTO @NephilaIt Djangonaut and django CMS core developer @yakkys @yakky@mastodon.social https://keybase.io/yakky https://github.com/yakky
3. REALTIME WEB REALTIME WEB No more pure request-response web Lot of tools (nice!) Lots of complexity (booo!) Why not doing it the Django-way?
4. SO, CHANNELS SO, CHANNELS Framework to use Django in non-HTTP world websockets any protocol (bots, IoT, ...)
5. SO, CHANNELS SO, CHANNELS CHANNELS VERSIONS CHANNELS VERSIONS Channels 2 is two weeks old But we will focus on Channels 1 I live-tested Channels 1 Channels 2 still a bit incomplete Little time to rewrite the talk
6. TALK OUTLINE TALK OUTLINE Channels concepts Demo! Code drill down Channels 2
7. CONCEPTS CONCEPTS
8. CONCEPTS CONCEPTS CHANNEL CHANNEL A FIFO at-most-once queue Messages written by producers (protocol server / consumers) and read by consumers
9. CONCEPTS CONCEPTS ASYNCRONOUS ASYNCRONOUS event-driven async code is more complex to understand Channels 1 "hides" the async part
10. CONCEPTS CONCEPTS ASGI ASGI Underlying protocol specs Version 1 for Channels 1 (retired) Version 2 for Channels 2
11. CONCEPTS CONCEPTS PROTOCOL SERVER PROTOCOL SERVER Implements the ASGI specs Bridge between the network and application (e.g.: Daphne for websockets) Defines the channels
12. CONCEPTS CONCEPTS ROUTING ROUTING Map channels to consumers Similar to Django URLconf
13. CONCEPTS CONCEPTS CONSUMERS CONSUMERS read / write messages on the channels similar to Django views
14. CONCEPTS CONCEPTS FRONTEND FRONTEND Websockets means Javascript Channels ships a lightweight js library
15. DEMO TIME! DEMO TIME! Enough theory
16. DEMO TIME DEMO TIME EXAMPLE APPLICATION EXAMPLE APPLICATION https://github.com/yakky/channeled-dashboard
17. DEMO TIME DEMO TIME FEATURES FEATURES Count active users Concurrency check Browser notifications
18. DEMO TIME DEMO TIME 0:00 / 1:05
19. DIVE IN DIVE IN Channel layers Routing Consumers
20. DIVE IN DIVE IN CHANNEL LAYERS CHANNEL LAYERS CHANNEL_LAYERS = { 'default': { 'BACKEND': 'asgi_redis.RedisChannelLayer', 'CONFIG': { 'hosts': [('localhost', 6379)], }, 'ROUTING': 'dashboard.routing.channel_routing', }, }
21. DIVE IN DIVE IN ROUTING ROUTING # equivalent to my_project.urls # tipically used to include application routing channel_routing = [ include(documents_routing, path=r'^/status'), ]
22. ROUTING ROUTING APPLICATION ROUTING APPLICATION ROUTING # consumers can be freely appended: path (and other filters) # used to match the right one # Channel are declared in the parent class (WebsocketConsumer) channel_routing = [ UserCounter.as_route(path=r'/users/'), DocListStatus.as_route(path=r'/documents/'), DocDetailStatus.as_route( path=r'/document/(?P<slug>[^/]+)/(?P<phase>[^/]+)/' ), route_class(Demultiplexer, path=r'/notifications/'), ] </phase></slug>
23. DIVE IN DIVE IN CONSUMERS CONSUMERS
24. DIVE IN DIVE IN CONSUMERS CONSUMERS class SomeConsumer(JsonWebsocketConsumer): """ Generic Websocket consumer """ channel_session_user = True http_user = True def connection_groups(self, **kwargs): ... def connect(self, message, **kwargs): ... def disconnect(self, message, **kwargs): ... def receive(self, message, **kwargs): ...
25. CONSUMERS CONSUMERS COUNTING USERS COUNTING USERS
26. CONSUMERS CONSUMERS COUNTING USERS COUNTING USERS class UserCounterConsumer(JsonWebsocketConsumer): def connection_groups(self, **kwargs): return 'users', def connect(self, message, **kwargs): """ Increment users on connect and notify users """ super().connect(message, **kwargs) if message.user.is_authenticated: increment_users(message.user) msg = {'users': count_users(),} self.group_send(self.users_group, msg) def disconnect(self, message, **kwargs): """ Decrement users on disconnect and notify users """ ...
27. COUNTING USERS COUNTING USERS FRONTEND FRONTEND // Library is instantiated and connected to the endpoint const users_count = new channels.WebSocketBridge(); users_count.connect(users_count_path); // whenever a message is received, the user counter is updated users_count.listen(function (data) { if (data.users) { document.getElementById( 'users-counter' ).textContent = data.users; } });
28. CONSUMERS CONSUMERS CONCURRENCY MONITOR CONCURRENCY MONITOR
29. CONCURRENCY MONITOR CONCURRENCY MONITOR SINGLE DOCUMENT SINGLE DOCUMENT class DocumentDetailStatusConsumer(DocumentListStatusConsumer): def connection_groups(self, **kwargs): return [self.kwargs.get('slug'), Document.Status.list] def connect(self, message, **kwargs): if message.user.is_authenticated: # kwargs contains slug and phase as mapped from path i key = Document.cache_key(**kwargs) update_status(key, message.user) super().connect(message, **kwargs) doc_status = { phase: doc_status(self.kwargs['slug'], phase) for phase in self.phases } # Notify users of the current document status doc_status['document'] = self.kwargs['slug'] self.group_send(self.kwargs.get('slug'), doc_status) # Notify users of the updated documents status statuses = doc_list_status()
30. CONCURRENCY MONITOR CONCURRENCY MONITOR DOCUMENTS LIST DOCUMENTS LIST class DocumentListStatusConsumer(JsonWebsocketConsumer): def connection_groups(self, **kwargs): return [Document.Status.list] def connect(self, message, **kwargs): super().connect(message, **kwargs) statuses = {} # Expanded view of doc_list_status of the previous slide # This is Django ORM code that retrieves all the docs slug for document in Document.objects.all().values_list( 'slug', flat=True): statuses[document] = { phase: doc_status(document, phase) for phase in self.phases } statuses[document]['document'] = document self.group_send(Document.Status.list, statuses)
31. CONCURRENCY MONITOR CONCURRENCY MONITOR FRONTEND FRONTEND // Library is instantiated and connected to the endpoint const document_status = new channels.WebSocketBridge(); document_status.connect(document_path); // whenever a message is received, the documents badges is updat document_status.listen(function (data) { if (data.document) { // we only have one document - detail view update_document_detail(data); } else { // all documents - list view update_document_list(data); } });
32. CONSUMERS CONSUMERS BROWSER NOTIFICATIONS BROWSER NOTIFICATIONS Let's tell users when a document changes
33. BROWSER NOTIFICATIONS BROWSER NOTIFICATIONS DATA BINDING DATA BINDING class DocumentBinding(WebsocketBinding): model = Document # The name of the stream must be matched in the js code stream = 'document' fields = ['title', 'image', 'abstract', ..., 'slug'] @classmethod def group_names(cls, instance): return ['notifications'] def has_permission(self, user, action, pk): """ Is current user authorized to the actions (create, update or delete)?""" return user.is_authenticated
34. BROWSER NOTIFICATIONS BROWSER NOTIFICATIONS DEMULTIPLEXER DEMULTIPLEXER class Demultiplexer(WebsocketDemultiplexer): """ Wrap multiple consumers in a single endpoint """ consumers = { 'document': DocumentBinding.consumer, } def connection_groups(self): return ['notifications']
35. BROWSER NOTIFICATIONS BROWSER NOTIFICATIONS DEMULTIPLEXER - ROUTING DEMULTIPLEXER - ROUTING channel_routing = [ ... Demultiplexer.as_route(path=r'/notifications/'), route_class(Demultiplexer, path=r'/notifications/'), ] In routing you can declare a demupltiplexer exactly like a consumer
36. BROWSER NOTIFICATIONS BROWSER NOTIFICATIONS FRONTEND FRONTEND // Library is instantiated and connected to the endpoint const notifications = new channels.WebSocketBridge(); notifications.connect(notifications_path); // Demultiplexing frontend-side notifications.demultiplex('document', function (action, stream) { let title = action; const data = { body: obj.title, icon: `${media_base_url}${obj.image}`, }; const note = new Notification(title, data); note.onclick = function () { document.location = `${document_base_url}${obj.slug}/`; }; }
37. THE FUTURE THE FUTURE CHANNELS 2 CHANNELS 2
38. CHANNELS 2 CHANNELS 2 HIGHTLIGHTS HIGHTLIGHTS Based on asyncio Python 3.5+ only Mix of sync / async code Single process In-process protocol termination Django runs in the same process No more separate worker required
39. Middlewares ASGI-all-the-way-down
40. CHANNELS 2 CHANNELS 2 UPGRADING UPGRADING Removed features function based consumers Group channel_session data binding Returning (soon?) Demultiplixer
41. CHANNELS 2 CHANNELS 2 UPGRADING UPGRADING Rewrite needed Routing Consumers (partial) Tests
42. FAQ FAQ Performance Deployments
43. WRAP IT UP WRAP IT UP Async made easier Access to Django API
44. GRAZIE! GRAZIE! Follow me on: https://github.com/yakky https://github.com/nephila @yakkys @yakky@mastodon.social https://keybase.io/yakky
45. QUESTIONS? QUESTIONS?