Building real time applications with Django and Channels 2 @ DjangoCon Europe

Since the introduction of Channels, real time web has become much easier to work with in Django. It’s now possible to build real time applications with much less effort in managing the idiosyncrasies of the async programming and a lot of batteries are included. Starting with a brief introduction to Channels (targeting version 2.0), we will see how to build a real time application, both on the Django and the frontend side and how easy it’s to start experimenting with it. The talk has a very hands-on approach, to allow the attendees to experiment with the proposed solution and approach, and starting immediately building their own real time applications with Django.

Video: https://www.youtube.com/watch?v=bhW7wDRtFEY

Transcript

1. A QUICK PRIMER ON CHANNELS A QUICK PRIMER ON CHANNELS BUILDING REAL TIME BUILDING REAL TIME APPLICATIONS WITH APPLICATIONS WITH DJANGO DJANGO DjangoCon Europe 2018
2. HELLO, I AM IACOPO HELLO, I AM IACOPO Founder and CTO @NephilaIt Djangonaut and django CMS core developer @yakkys @yakky@mastodon.social https://keybase.io/yakky https://github.com/yakky
3. REALTIME WEB REALTIME WEB No more pure request-response web Lot of tools (nice!) Lots of complexity (booo!) Why not doing it the Django-way?
4. SO, CHANNELS SO, CHANNELS Framework to use Django in non-HTTP world websockets any protocol (bots, IoT, ...)
5. SO, CHANNELS SO, CHANNELS CHANNELS VERSIONS CHANNELS VERSIONS Channels 2 is the shiny present and future Channels 1 served for almost three years Will focus on 2 (2.1, really)
6. TALK OUTLINE TALK OUTLINE Channels concepts Demo! Code drill down
7. CONCEPTS CONCEPTS
8. CONCEPTS CONCEPTS ASYNCHRONOUS ASYNCHRONOUS event-driven async code is more complex to understand Channels 2: best of both worlds
9. CONCEPTS CONCEPTS ASGI ASGI Protocol specs
10. CONCEPTS CONCEPTS ASGI-ALL-THE-WAY-DOWN ASGI-ALL-THE-WAY-DOWN
11. CONCEPTS CONCEPTS PROTOCOL SERVER PROTOCOL SERVER Implements the ASGI specs for a specific protocol Bridge between the network and the application (e.g.: Daphne for HTTP/websockets)
12. CONCEPTS CONCEPTS ROUTING ROUTING Map incoming messages to consumers nestable and composable
13. CONCEPTS CONCEPTS SCOPE SCOPE Connection data structure Connection ↔ Scope ↔ Application (consumer) instance
14. CONCEPTS CONCEPTS CHANNEL CHANNEL Channels IPC mechanism A FIFO at-most-once queue
15. CONCEPTS CONCEPTS CONSUMERS CONSUMERS Core abstraction to build ASGI app Handle events on a connection Stateful
16. CONCEPTS CONCEPTS GROUP GROUP Group consumer instances on channel layer
17. CONCEPTS CONCEPTS EVENTS EVENTS Triggered during a scope Consumed by consumers
18. CONCEPTS CONCEPTS FRONTEND FRONTEND Websockets means Javascript Channels ships a lightweight js library
19. CONCEPTS CONCEPTS HIGHLIGHTS HIGHLIGHTS Fully exposed async Based on asyncio Mix of sync / async code Single process In-process protocol termination No separate worker required Middlewares ASGI-all-the-way-down
20. DEMO TIME! DEMO TIME! Enough theory
21. DEMO TIME DEMO TIME EXAMPLE APPLICATION EXAMPLE APPLICATION https://github.com/yakky/channeled-dashboard
22. DEMO TIME DEMO TIME FEATURES FEATURES Count active users Concurrency check Browser notifications
23. DEMO TIME DEMO TIME 0:00 / 1:05
24. DIVE IN DIVE IN Channel layers Routing Consumers
25. DIVE IN DIVE IN CHANNEL LAYERS CHANNEL LAYERS ASGI_APPLICATION = 'dashboard.routing.application' CHANNEL_LAYERS = { 'default': { 'BACKEND': 'channels_redis.core.RedisChannelLayer', 'CONFIG': { 'hosts': [('localhost', 6379)], }, }, }
26. DIVE IN DIVE IN ROUTING ROUTING # equivalent to my_project.urls # tipically used to include application routing application = ProtocolTypeRouter({ 'websocket': AuthMiddlewareStack( URLRouter([ path('status/', documents_routing), ]) ), })
27. ROUTING ROUTING ROUTER TYPES ROUTER TYPES # routing is established on connection start application = ProtocolTypeRouter({ 'websocket': AuthMiddlewareStack( URLRouter([ path('status/', documents_routing), ]) ), }) Middlewares Routers A routing, is forever
28. ROUTING ROUTING APPLICATION ROUTING APPLICATION ROUTING channel_routing = URLRouter([ path('users/', UserCounterConsumer), path('documents/', DocumentListConsumer), path('document/<str:slug>/<str:phase>/', DocumentDetailConsum ])
29. DIVE IN DIVE IN CONSUMERS CONSUMERS
30. CONSUMERS CONSUMERS GENERIC WEBSOCKET GENERIC WEBSOCKET class SomeConsumer(WebsocketConsumer): """ Generic Websocket consumer """ groups = ['some_group', 'other'] def connect(self): ... def disconnect(self, code): ... def receive(self, text_data=None, bytes_data=None): ...
31. CONSUMERS CONSUMERS APPLICATION EVENTS APPLICATION EVENTS class UserCounterConsumer(JsonWebsocketConsumer): def connect(self): # this put a message in group users of the channel layer # type is the event to be generated async_to_sync(self.channel_layer.group_send)( 'users', {'type': 'users.count', 'value': 5} ) def users_count(self, event): # this catches the 'users.count' event / message # and send it to its connected client # it runs on all instances connected to the 'users' group self.send_json(content=event['value'])
32. CONSUMERS CONSUMERS COUNTING USERS COUNTING USERS
33. CONSUMERS CONSUMERS COUNTING USERS COUNTING USERS class UserCounterConsumer(JsonWebsocketConsumer): groups = 'users', def connect(self): """ Increment users on connect and notify other consumers super().connect() if self.scope['user'].is_authenticated: increment_users(message.user) msg = {'users': count_users(), 'type': 'users.count'} async_to_sync(self.channel_layer.group_send)('users', msg def users_count(self, event): """ Notify connected user """ self.send_json(content=event['message']) def disconnect(self, code): """ Decrement users on disconnect and notify users """ ...
34. COUNTING USERS COUNTING USERS FRONTEND FRONTEND if (users_count_path) { // Library is instantiated and connected to the endpoint const users_count = new channels.WebSocketBridge(); users_count.connect(users_count_path); users_count.listen((data) => { if (data.users) { document.getElementById( 'users-counter' ).textContent = data.users; } }); }
35. CONSUMERS CONSUMERS CONCURRENCY MONITOR CONCURRENCY MONITOR
36. CONCURRENCY MONITOR CONCURRENCY MONITOR SINGLE DOCUMENT SINGLE DOCUMENT class DocumentDetailConsumer(DocumentListConsumer): @property def groups(self): return [self.slug, Document.Status.list] def connect(self): self._update_document_count( self.scope['user'].get_full_name() ) super(DocumentDetailConsumer, self).connect() async_to_sync(self.channel_layer.group_send)( self.slug, { 'type': 'document.status', 'message': self.get_status() }) def document_status(self, event): self.send_json(content=event['message'])
37. CONSUMER CONSUMER DETAIL #1 DETAIL #1 def connect(): ... async_to_sync(self.channel_layer.group_send)( self.slug, { ... })
38. CONSUMER CONSUMER DETAIL #2 DETAIL #2 def connect(): ... async_to_sync(...)( self.slug, { 'type': 'document.status', 'message': self.get_status() }) def document_status(self, event): self.send_json(content=event['message'])
39. CONCURRENCY MONITOR CONCURRENCY MONITOR DOCUMENTS LIST DOCUMENTS LIST class DocumentListConsumer(JsonWebsocketConsumer): @property def groups(self): return Document.Status.list, def connect(self): super(DocumentListConsumer, self).connect() async_to_sync(self.channel_layer.group_send)( self.slug, { 'type': 'document.status', 'message': self.get_status_packet() }) def document_status(self, event): self.send_json(content=event['message'])
40. CONCURRENCY MONITOR CONCURRENCY MONITOR FRONTEND FRONTEND // Library is instantiated and connected to the endpoint const document_status = new channels.WebSocketBridge(); document_status.connect(document_path); // whenever a message is received, the documents badges is upda document_status.listen((data) => { if (data.document) { // we only have one document - detail view update_document_detail(data); } else { // all documents - list view update_document_list(data); } });
41. CHANNELS FROM OUTSIDE CHANNELS FROM OUTSIDE CHANNELS CHANNELS class Document(TimeStampedModel): ... def save(self, *args, **kwargs): channel_layer = get_channel_layer() async_to_sync(channel_layer.group_send)(self.slug, { 'type': 'document.saved', 'message': 'document saved' }) return super(Document, self).save(*args, **kwargs) class DocumentDetailConsumer(DocumentListConsumer): ... def document_saved(self, event): do_something(event)
42. DATA BINDING DATA BINDING No more in version 2
43. DATA BINDING DATA BINDING REJOYCE! REJOYCE! An opportunity for the community
44. DATA BINDING DATA BINDING AN EXPERIMENT AN EXPERIMENT resurrecting the dead django-knocker
45. DATA BINDING DATA BINDING KNOCKER KNOCKER based on django-meta hooks into signals sends messages to default consumer
46. KNOCKER KNOCKER OVERVIEW OVERVIEW class Document(KnockerModel, ModelMeta, TimeStampedModel): ... _knocker_data = { 'title': 'get_knocker_title', 'message': 'get_knocker_message', 'icon': 'get_knocker_icon', 'url': 'get_absolute_url', 'language': 'get_knocker_language', } ... def get_knocker_icon(self): ... def get_knocker_messageself): ... def get_knocker_language(self): ...
47. KNOCKER KNOCKER SIGNALS SIGNALS class KnockerModel(object): def __new__(cls, *args, **kwargs): ... new_cls._connect() return new_cls @classmethod def _connect(cls): pre_save.connect(notify_items_pre_save, sender=cls) ...
48. KNOCKER KNOCKER MESSAGES MESSAGES class KnockerModel(object): def send_knock(self, signal_type, created=False): if not self.should_knock(signal_type, created): return knock = self.as_knock(signal_type, created) if knock: channel_layer = get_channel_layer() group = 'knocker-%s' % knock['language'] async_to_sync(channel_layer.group_send)(group, { 'type': 'knocker.saved', 'message': json.dumps(knock) })
49. KNOCKER KNOCKER DEFAULT CONSUMER DEFAULT CONSUMER class KnockerConsumer(JsonWebsocketConsumer): @property def groups(self): lang = self.scope['url_route']['kwargs'].get('language') return 'knocker-%s' % lang, def knocker_saved(self, event): self.send_json(content=event['message'])
50. KNOCKER KNOCKER pretty rough
51. DATA BINDING DATA BINDING WHAT'S NEXT? WHAT'S NEXT? Next databinding library? Maybe talking about this during the sprints?
52. WRAP IT UP WRAP IT UP Async made easier Access to Django API
53. GRAZIE! GRAZIE! Follow me on: https://github.com/yakky https://github.com/nephila @yakkys @yakky@mastodon.social https://keybase.io/yakky
54. QUESTIONS? QUESTIONS?