A QUICK PRIMER ON CHANNELS A QUICK PRIMER ON CHANNELS BUILDING REAL TIME BUILDING REAL TIME APPLICATIONS WITH APPLICATIONS WITH DJANGO DJANGO PyCon Italia Firenze - 20 Aprile

HELLO, I AM IACOPO HELLO, I AM IACOPO Founder and CTO @NephilaIt Djangonaut and django CMS core developer @yakkys @[email protected] https://keybase.io/yakky https://github.com/yakky

REALTIME WEB REALTIME WEB No more pure request-response web Lot of tools (nice!) Lots of complexity (booo!) Why not doing it the Django-way?

SO, CHANNELS SO, CHANNELS Framework to use Django in non-HTTP world websockets any protocol (bots, IoT, ...)

SO, CHANNELS SO, CHANNELS CHANNELS VERSIONS CHANNELS VERSIONS Channels 2 is a few weeks old Channels 1 served for almost three years Will focus on 2 (2.1, really) (Check previous version of this talk)

TALK OUTLINE TALK OUTLINE Channels concepts Demo! Code drill down

CONCEPTS CONCEPTS

CONCEPTS CONCEPTS ASYNCRONOUS ASYNCRONOUS event-driven async code is more complex to understand Channels 2: best of both worlds

CONCEPTS CONCEPTS ASGI ASGI Protocol specs

CONCEPTS CONCEPTS ASGI-ALL-THE-WAY-DOWN ASGI-ALL-THE-WAY-DOWN

CONCEPTS CONCEPTS PROTOCOL SERVER PROTOCOL SERVER Implements the ASGI specs for a specific protocol Bridge between the network and the application (e.g.: Daphne for HTTP/websockets)

CONCEPTS CONCEPTS ROUTING ROUTING Map incoming messages to consumers nestable and composable

CONCEPTS CONCEPTS SCOPE SCOPE Connection data structure Connection ↔ Scope ↔ Application (consumer) instance

CONCEPTS CONCEPTS CHANNEL CHANNEL Channels IPC mechanism A FIFO at-most-once queue

CONCEPTS CONCEPTS CONSUMERS CONSUMERS Core abstraction to build ASGI app Handle events on a connection

CONCEPTS CONCEPTS EVENTS EVENTS Triggered during a scope Consumed by consumers

CONCEPTS CONCEPTS FRONTEND FRONTEND Websockets means Javascript Channels ships a lightweight js library

CONCEPTS CONCEPTS GROUP GROUP Group consumer instances on channel layer

TIME HAS PASSED TIME HAS PASSED CHANNELS 1 → 2 CHANNELS 1 → 2

CHANNELS 1 → 2 CHANNELS 1 → 2 HIGHLIGHTS HIGHLIGHTS Fully exposed async Based on asyncio Mix of sync / async code Single process In-process protocol termination No more separate worker required Middlewares ASGI-all-the-way-down

CHANNELS 1 → 2 CHANNELS 1 → 2 UPGRADING UPGRADING Major changes Channel-as-IPC only Stateful consumers Custom events architecture

CHANNELS 1 → 2 CHANNELS 1 → 2 UPGRADING UPGRADING Rewrite needed Routing Consumers (partial) Tests Check channels1 vs channels2 branches

DEMO TIME! DEMO TIME! Enough theory

DEMO TIME DEMO TIME EXAMPLE APPLICATION EXAMPLE APPLICATION https://github.com/yakky/channeled-dashboard

DEMO TIME DEMO TIME FEATURES FEATURES Count active users Concurrency check

DEMO TIME DEMO TIME 0:00 / 0:48

DIVE IN DIVE IN Channel layers Routing Consumers

DIVE IN DIVE IN CHANNEL LAYERS CHANNEL LAYERS CHANNEL_LAYERS = { 'default': { 'BACKEND': 'channels_redis.core.RedisChannelLayer', 'CONFIG': { 'hosts': [('localhost', 6379)], }, }, }

DIVE IN DIVE IN ROUTING ROUTING # equivalent to my_project.urls # tipically used to include application routing application = ProtocolTypeRouter({ 'websocket': AuthMiddlewareStack( URLRouter([ path('status/', documents_routing), ]) ), })

ROUTING ROUTING ROUTER TYPES ROUTER TYPES # routing is established on connection start application = ProtocolTypeRouter({ 'websocket': AuthMiddlewareStack( URLRouter([ path('status/', documents_routing), ]) ), }) Middlewares Routers A routing, is forever

ROUTING ROUTING APPLICATION ROUTING APPLICATION ROUTING channel_routing = URLRouter([ path('users/', UserCounterConsumer), path('documents/', DocumentListConsumer), path('document///', DocumentDetailConsum ])

DIVE IN DIVE IN CONSUMERS CONSUMERS

CONSUMERS CONSUMERS GENERIC WEBSOCKET GENERIC WEBSOCKET class SomeConsumer(WebsocketConsumer): """ Generic Websocket consumer """ groups = ['some_group', 'other'] def connect(self): ... def disconnect(self, code): ... def receive(self, text_data=None, bytes_data=None): ...

CONSUMERS CONSUMERS APPLICATION EVENTS APPLICATION EVENTS class UserCounterConsumer(JsonWebsocketConsumer): def connect(self): # this put a message in group users of the channel layer # type is the event to be generated async_to_sync(self.channel_layer.group_send)( 'users', {'type': 'users.count', 'value': 5} ) def users_count(self, event): # this catches the 'users.count' event / message # and send it to its connected client # it runs on all instances connected to the 'users' group self.send_json(content=event['value'])

CONSUMERS CONSUMERS COUNTING USERS COUNTING USERS

CONSUMERS CONSUMERS COUNTING USERS COUNTING USERS class UserCounterConsumer(JsonWebsocketConsumer): groups = 'users', def connect(self): """ Increment users on connect and notify other consumers super().connect() if self.scope['user'].is_authenticated: increment_users(message.user) msg = {'users': count_users(), 'type': 'users.count'} async_to_sync(self.channel_layer.group_send)('users', msg def users_count(self, event): """ Notify connected user """ self.send_json(content=event['message']) def disconnect(self, code): """ Decrement users on disconnect and notify users """ ...

COUNTING USERS COUNTING USERS FRONTEND FRONTEND if (users_count_path) { // Library is instantiated and connected to the endpoint const users_count = new channels.WebSocketBridge(); users_count.connect(users_count_path); users_count.listen((data) => { if (data.users) { document.getElementById( 'users-counter' ).textContent = data.users; } }); }

CONSUMERS CONSUMERS CONCURRENCY MONITOR CONCURRENCY MONITOR

CONCURRENCY MONITOR CONCURRENCY MONITOR SINGLE DOCUMENT SINGLE DOCUMENT class DocumentDetailConsumer(DocumentListConsumer): @property def groups(self): return [self.slug, Document.Status.list] def connect(self): self._update_document_count( self.scope['user'].get_full_name() ) super(DocumentDetailConsumer, self).connect() async_to_sync(self.channel_layer.group_send)( self.slug, { 'type': 'document.status', 'message': self.get_status() }) def document_status(self, event): self.send_json(content=event['message'])

CONSUMER CONSUMER DETAIL #1 DETAIL #1 def connect(): ... async_to_sync(self.channel_layer.group_send)( self.slug, { ... })

CONSUMER CONSUMER DETAIL #2 DETAIL #2 def connect(): ... async_to_sync(...)( self.slug, { 'type': 'document.status', 'message': self.get_status() }) def document_status(self, event): self.send_json(content=event['message'])

CONCURRENCY MONITOR CONCURRENCY MONITOR DOCUMENTS LIST DOCUMENTS LIST class DocumentListConsumer(JsonWebsocketConsumer): @property def groups(self): return Document.Status.list, def connect(self): super(DocumentListConsumer, self).connect() async_to_sync(self.channel_layer.group_send)( self.slug, { 'type': 'document.status', 'message': self.get_status_packet() }) def document_status(self, event): self.send_json(content=event['message'])

CONCURRENCY MONITOR CONCURRENCY MONITOR FRONTEND FRONTEND // Library is instantiated and connected to the endpoint const document_status = new channels.WebSocketBridge(); document_status.connect(document_path); // whenever a message is received, the documents badges is upda document_status.listen((data) => { if (data.document) { // we only have one document - detail view update_document_detail(data); } else { // all documents - list view update_document_list(data); } });

CHANNELS FROM OUTSIDE CHANNELS FROM OUTSIDE CHANNELS CHANNELS class Document(TimeStampedModel): ... def save(self, *args, **kwargs): channel_layer = get_channel_layer() async_to_sync(channel_layer.group_send)(self.slug, { 'type': 'document.saved', 'message': 'document saved' }) return super(Document, self).save(*args, **kwargs) class DocumentDetailConsumer(DocumentListConsumer): ... def document_saved(self, event): do_something(event)

FAQ FAQ Performance Deployments

WRAP IT UP WRAP IT UP Async made easier Access to Django API

GRAZIE! GRAZIE! Follow me on: https://github.com/yakky https://github.com/nephila @yakkys @[email protected] https://keybase.io/yakky

P.S. P.S. http://bit.ly/channels1

P.P.S. P.P.S. We are hiring https://nephila.digital/jobs

QUESTIONS? QUESTIONS?