Introduction to Django Channels - ConFoo Vancouver 2016

Slide 1

Slide 1 text

Eric Holscher ConFoo Vancouver December 7, 2016 Python & Django Go Async An introduction to Django Channels

Slide 2

Slide 2 text

Who am I • Co-Founder of Read the Docs • Co-Founder of Write the Docs • Longtime member of the Django community

Slide 3

Slide 3 text

Today • Understand the architecture of Channels • Understand the tradeoﬀs required • Know if it ﬁts your use case and organization • Know how to get started

Slide 4

Slide 4 text

A bit of history first

Slide 5

Slide 5 text

Django was created in 2005. Web was static pages rendered on the server side.

Slide 6

Slide 6 text

Channels takes us from 2005 to 2015

Slide 7

Slide 7 text

Channels adds ability for Django to handle H P/2, websockets, and other things outside a normal request/response cycle

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

Architecture

Slide 10

Slide 10 text

Channels

Slide 11

Slide 11 text

Current Status • 1.0 imminent • Lives outside of Django itself • An oﬃcial project of the Django team • Will perhaps merge into core for Django 2.0

Slide 12

Slide 12 text

A few different projects • ASGI • Daphne • Django Channels

Slide 13

Slide 13 text

Terms • Interface Server • Channels • Messages • Routes • Consumers • Groups

Slide 14

Slide 14 text

Interface Server • Holds connections open to the world • Speaks both H P and Websocket • Passes messages along the channels, doesn’t actually process anything

Slide 15

Slide 15 text

Channels • First In First Out • At most once delivery • No guaranteed ordering • Messages expire a er a set time • ‘websocket.connect’, ‘http.request’, ‘chat.message’

Slide 16

Slide 16 text

Messages • Send across the channels • Can have arbitrary content based on diﬀering protocols • Usually JSON

Slide 17

Slide 17 text

# Sent on a ‘http.request’ channel { 'reply_channel': 'http.response!g23vD2x5', 'method': 'GET', 'http_version': '2', 'path': '/chat/socket/', 'query_string': 'foo=bar', 'headers': [['cookie', 'abcdef...']], } Messages

Slide 18

Slide 18 text

Routing • Similar to Django URLConf • Allows for matching arbitrary things on the incoming messages • Can match against `path` on H P requests, or `address` on email sending

Slide 19

Slide 19 text

routing = [ route( "websocket.receive", consumers.chat_receive, path=r"^/chat/socket/$", ), route( "email.receive", comment_response, address=r".*@example.com$" ) include("stats.routing", path="^/stats/"), ] Routing

Slide 20

Slide 20 text

Consumer • Similar to a Django View • Takes a `message` and… does something

Slide 21

Slide 21 text

# Consumer def ws_connect(message): message.reply_channel.send(message['text']) Consumer

Slide 22

Slide 22 text

Groups • Contain sets of channels • Only allow add/remove/send — because distributed systems • Useful for keeping state and sending messages across sets of channels

Slide 23

Slide 23 text

Architecture

Slide 24

Slide 24 text

Basic Examples

Slide 25

Slide 25 text

# Routing channel_routing = { 'http.request': consumers.http_consumer } # Message { 'reply_channel': 'http.response!g23vD2x5', 'method': 'GET', 'http_version': '2', 'path': '/chat/socket/', 'query_string': 'foo=bar', 'headers': [['cookie', 'abcdef...']], } # Consumer def http_consumer(message): response = HttpResponse('Hey there') message.reply_channel.send(response.channel_encode()) HTTP Request

Slide 26

Slide 26 text

# Routing channel_routing = { 'websocket.connect': consumers.ws_connect, 'websocket.receive': consumers.ws_receive, 'websocket.disconnect': consumers.ws_disconnect, } # Message { 'text': 'Hello, world!', 'path': '/chat/socket/', 'reply_channel': 'websocket.send!9m12in2p', } # Consumer def ws_connect(message): message.reply_channel.send(message['text']) Websocket Request

Slide 27

Slide 27 text

Major Features

Slide 28

Slide 28 text

Authentication & Sessions • Slightly tricky • ‘websocket.connect’ has H P cookies & auth • There are two sessions, “http_sessions” & “channel_sessions”

Slide 29

Slide 29 text

@channel_session_user_from_http def ws_add(message): Group( "chat-%s" % message.user.username[0] ).add(message.reply_channel) @channel_session_user def ws_message(message): Group("chat-%s" % message.user.username[0]).send({ "text": message['text'], }) Authentication & Sessions

Slide 30

Slide 30 text

Generic Consumers • Similar to Generic Views • Abstracts common use cases for simpler logic • Ships with Websocket & JSON generic consumers currently

Slide 31

Slide 31 text

class MyConsumer(JsonWebsocketConsumer): def connection_groups(self, **kwargs): return ["test"] def receive(self, content, **kwargs): """ Called when a message is received with decoded JSON content """ # Simple echo self.send(content) Generic Consumers

Slide 32

Slide 32 text

MultiPlexing • Allows you to send multiple data types over a single connection • Uses ‘stream’ and ‘payload’ keys • Sends the payload to the mapped stream

Slide 33

Slide 33 text

class Demultiplexer(WebsocketDemultiplexer): mapping = { "intval": "binding.intval", "stats": "internal.stats", } MultiPlexing

Slide 34

Slide 34 text

Data Binding • Lets you keep client & server data models in sync • Update each side on change to either side in realtime • Uses Django serialization and multiplexing as standard wire format

Slide 35

Slide 35 text

# Routing channel_routing = [ route_class(Demultiplexer, path="^/binding/"), route("binding.intval", IntegerValueBinding.consumer), ] # Consumer class IntegerValueBinding(WebsocketBinding): model = IntegerValue stream = "intval" fields = ["name", "value"] def group_names(self, instance, action): return ["intval-updates"] def has_permission(self, user, action, pk): return True Data Binding

Slide 36

Slide 36 text

# Message (Client) { "payload": { "action": "update", "data": { "value": "56" }, "pk": "2" }, "stream": "intval" } # Message (Server) { "payload": { "action": "update", "data": { "name": "Cool", "value": 56 }, "model": "values.integervalue", "pk": 2 }, "stream": "intval" } Data Binding

Slide 37

Slide 37 text

Additional niceties • Testing helpers • Optional guaranteed ordering • Zero Downtime Deployments

Slide 38

Slide 38 text

Deploying

Slide 39

Slide 39 text

# settings.py CHANNEL_LAYERS = { "default": { "BACKEND": "asgi_redis.RedisChannelLayer", "CONFIG": { "hosts": [("localhost", 6379)], }, "ROUTING": "myproject.routing.channel_routing", }, } # asgi.py import os from channels.asgi import get_channel_layer os.environ.setdefault("DJANGO_SETTINGS_MODULE", "my_project.settings") channel_layer = get_channel_layer() Configuration

Slide 40

Slide 40 text

You need a Channel Layer, an Interface Server, and a pool of Workers

Slide 41

Slide 41 text

# Run a Channel Layer redis-server # Run an Interface Server daphne my_project.asgi:channel_layer # Run the Workers ./manage.py runworker All services

Slide 42

Slide 42 text

You should run more than 1 worker, as it’s single threaded

Slide 43

Slide 43 text

Can limit the channels workers listen on, to split up long running and short running processes

Slide 44

Slide 44 text

Websockets require a diﬀerent way of thinking about architecture. Channels embraces this

Slide 45

Slide 45 text

Slackbot

Slide 46

Slide 46 text

Usage of the same channel for Slack and IRC

Slide 47

Slide 47 text

{ 'stream': ‘chat.mesage’, 'payload': { 'channel': 'general', 'user': 'eric', 'text', '!post general echo', } } Message

Slide 48

Slide 48 text

Demo

Slide 49

Slide 49 text

Recap

Slide 50

Slide 50 text

Channels & ASGI is a swiss army knife of async

Slide 51

Slide 51 text

Not just for Django, but a lot of the neat bits on top are

Slide 52

Slide 52 text

Still relatively new, but deﬁnitely the future of realtime with Django

Slide 53

Slide 53 text

Next steps are to get some real deployments in the world

Slide 54

Slide 54 text

Resources • https://channels.readthedocs.io/ • https://github.com/andrewgodwin/ channels-examples/

Slide 55

Slide 55 text

Image Credits • https://blog.heroku.com/ in_deep_with_django_channels_the_fut ure_of_real_time_apps_in_django

Slide 56

Slide 56 text

Questions? • @ericholscher • eric@ericholscher.com • Come talk to me around the conference