Architecting with Channels

Slide 1

Slide 1 text

No content

Slide 2

Slide 2 text

Andrew Godwin Hi, I'm Django core developer Senior Software Engineer at Used to complain about migrations a lot

Slide 3

Slide 3 text

2010

Slide 4

Slide 4 text

WebSockets

Slide 5

Slide 5 text

WebSockets (and Eventlet)

Slide 6

Slide 6 text

2012

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

2014

Slide 9

Slide 9 text

"django-onair"

Slide 10

Slide 10 text

2015

Slide 11

Slide 11 text

Channels

Slide 12

Slide 12 text

Born from WebSockets

Slide 13

Slide 13 text

Born from WebSockets Expanded to be more

Slide 14

Slide 14 text

The "real hard problem"

Slide 15

Slide 15 text

Asynchronous coordination

Slide 16

Slide 16 text

Why?

Slide 17

Slide 17 text

WebSockets don't magically solve everything

Slide 18

Slide 18 text

Architecture is about tradeoffs

Slide 19

Slide 19 text

Bidirectional, low overhead Complex client-side, compatibility CONS PROS

Slide 20

Slide 20 text

Streaming updates Chat applications Collaborative editing Game backends

Slide 21

Slide 21 text

Broadcast

Slide 22

Slide 22 text

Server Client 1 Client 2 Client 3 Client 4

Slide 23

Slide 23 text

Server Client 1 Client 2 Client 3 Client 4 Server

Slide 24

Slide 24 text

Server Client 1 Client 2 Client 3 Client 4 Server The "hard problem"

Slide 25

Slide 25 text

Channels is a foundation for runnng 'async' at scale

Slide 26

Slide 26 text

WebSockets Task ofﬂoading Chat/Email integration IoT protocols

Slide 27

Slide 27 text

What makes it hard?

Slide 28

Slide 28 text

Stateful connections Internal network Bottlenecks

Slide 29

Slide 29 text

Server Server Django Script ASGI "Send to channel X" "Receive from channel X" "Send to group Y" "Add channel X to group Y" "Remove channel X from group Y"

Slide 30

Slide 30 text

Server Server Django Script ASGI ASGI ASGI ASGI Redis

Slide 31

Slide 31 text

Server Server Django Script ASGI ASGI ASGI ASGI Shared Memory

Slide 32

Slide 32 text

Server Server Django Script ASGI ASGI ASGI ASGI Shared Memory Redis Redis

Slide 33

Slide 33 text

bit.ly/asgi-spec

Slide 34

Slide 34 text

Channels wraps the low-level ASGI operations

Slide 35

Slide 35 text

Think of it as the "Django bit" of a larger whole.

Slide 36

Slide 36 text

Daphne HTTP/WebSocket Server Channels Django integration asgi-redis Redis backend asgi-ipc Local memory backend asgiref Shared code and libs

Slide 37

Slide 37 text

What does Channels provide? Routing Consumers Sessions Auth Helpers By channel, URL, etc. Standardised message handling Cross-network persistence on sockets Including HTTP cookies on WebSocket runserver, runworker, debugging info

Slide 38

Slide 38 text

Putting it to use Let's make a chat!

Slide 39

Slide 39 text

Consumers def on_connect(message): Group("chat").add(message.reply_channel) def on_receive(message): Group("chat").send({"text": message["text"]}) def on_disconnect(message): Group("chat").discard(message.reply_channel) websocket.connect websocket.receive websocket.disconnect

Slide 40

Slide 40 text

Routing from channels import route routing = [ route("websocket.connect", on_connect), route("websocket.receive", on_receive), route("websocket.disconnect", on_disconnect) ]

Slide 41

Slide 41 text

Class-based from channels import route_class routing = [ route_class(ChatConsumer), ] from channels.generic.websockets class ChatConsumer(WebsocketConsumer): def connection_groups(self): return ["chat"] def receive(self, text): self.group_send("chat", text=text) Routing

Slide 42

Slide 42 text

Full worked example github.com/andrewgodwin/channels-examples

Slide 43

Slide 43 text

Liveblog

Slide 44

Slide 44 text

Liveblog

Slide 45

Slide 45 text

Data Binding Multiplexing &

Slide 46

Slide 46 text

class IntegerValue(models.Model): name = models.CharField(max_length=100, unique=True) value = models.IntegerField(default=0) class IntegerValueBinding(WebsocketBinding): model = IntegerValue stream = "intval" def check_permission(self, user, action, instance): return True def group_names(self, instance, action): return ["binding.values"]

Slide 47

Slide 47 text

Demo Do I dare?

Slide 48

Slide 48 text

Slide 49

Slide 49 text

Custom Protocols

Slide 50

Slide 50 text

Decompose into messages Email email.send, email.receive Slack slack.receive, slack.send, slack.joined, ... MQTT mqtt.subscribe, mqtt.unsubscribe, mqtt.send, ...

Slide 51

Slide 51 text

Write a custom server Use asyncio, Twisted, Gevent, etc Server takes path to ASGI channel layer instance (CLI/conﬁg ﬁle) Roll layer.send, layer.receive_many into your event loop

Slide 52

Slide 52 text

Write Django consumers Same codebase as the main project Optional routing/per-worker channel limiting

Slide 53

Slide 53 text

IRC Worker ASGI Worker Worker Email MQTT Scheduler HTTP/WS Custom Daemon

Slide 54

Slide 54 text

Channels is a tool for you to use

Slide 55

Slide 55 text

There's more to be done (and some funding for it)

Slide 56

Slide 56 text

1995 You are a desktop app 2005 You are a website 2015 You are a rich web/mobile app 2025 ?

Slide 57

Slide 57 text

What are the goals of a framework? Do we adapt?

Slide 58

Slide 58 text

Let's keep Django and Python a great choice for complex backends

Slide 59

Slide 59 text

Thanks. Andrew Godwin @andrewgodwin channels.readthedocs.io github.com/andrewgodwin/channels-examples