Slide 1

Slide 1 text

Andrew Godwin @andrewgodwin Reinventing REAL-TIME WEB for the

Slide 2

Slide 2 text

Andrew Godwin Hi, I'm Django core developer Senior Software Engineer at Likes complaining about networking

Slide 3

Slide 3 text

Django, HTTP and WSGI

Slide 4

Slide 4 text

You take a request... ...and return a response.

Slide 5

Slide 5 text

Browser HTTP Webserver Django WSGI View Handler

Slide 6

Slide 6 text

HTTP 1 request response Browser Server request response request response request response

Slide 7

Slide 7 text

HTTP 2 request response Browser Server request response request 1 response 2 request 2 response 1

Slide 8

Slide 8 text

WebSockets receive send Browser Server send receive send send receive

Slide 9

Slide 9 text

???

Slide 10

Slide 10 text

Easy to use Secure by default Hard to break/deadlock Python 2 & 3 compatible Optional

Slide 11

Slide 11 text

Python Concurrency & Message-passing Django not being async

Slide 12

Slide 12 text

WSGI Server HTTP WSGI WSGI App WSGI Server HTTP WSGI WSGI App Process 1 Process 2

Slide 13

Slide 13 text

Worker Protocol Server WebSocket Process 1 Worker Worker Worker Protocol Server WebSocket Process 2 Worker

Slide 14

Slide 14 text

Protocol Server WebSocket Process 1 Worker Protocol Server WebSocket Process 2

Slide 15

Slide 15 text

Channel Layer Interface Server Worker Server Process 1 ASGI ASGI Asynchronous socket handling Synchronous Django project Interface Server Worker Server ASGI ASGI Worker Server ASGI Process 2 Process 3 Process 4

Slide 16

Slide 16 text

Channel Layer Worker Server Interface Server Channel Layer Worker Server Interface Server Channel Layer Worker Server Interface Server Channel Layer Worker 1 Worker 2 Worker 3 Redis

Slide 17

Slide 17 text

Groups Channels Named FIFO task queues Named sets of channels you broadcast to

Slide 18

Slide 18 text

send("channel_name", {"ponies": True}) receive_many(["channel_name", "channel_two"]) send_group("group_name", {"ponies": True}) group_add("group_name", "channel_name") group_discard("group_name", "channel_name")

Slide 19

Slide 19 text

You take a request... ...and return a response.

Slide 20

Slide 20 text

You receive a message... ...and send zero or more messages.

Slide 21

Slide 21 text

View Request Response Consumer Message Messages

Slide 22

Slide 22 text

Consumer websocket.connect Consumer websocket.receive websocket.send

Slide 23

Slide 23 text

Consumer websocket.connect Consumer websocket.receive websocket.send!jq3x41

Slide 24

Slide 24 text

Consumer http.request http.response!a34cxw

Slide 25

Slide 25 text

websocket.receive Worker Worker Worker websocket.send!a1b2c3 Socket

Slide 26

Slide 26 text

Receive events from channels, and send events to them/groups. WebSocket/HTTP messages come with a reply_channel

Slide 27

Slide 27 text

Using it

Slide 28

Slide 28 text

Installation pip install channels Add channels to INSTALLED_APPS

Slide 29

Slide 29 text

Liveblog We want people to get new blog posts as they are published, without refreshing

Slide 30

Slide 30 text

Liveblog People open a WebSocket when they open the page Their WebSocket is added to a group When the BlogPost model is saved, we send the post to that group

Slide 31

Slide 31 text

Their WebSocket is added to a group When the BlogPost model is saved, we send the post to that group def ws_connect(message): Group("liveblog").add(message.reply_channel) class BlogPost(models.Model): ... def save(self, *args, **kwargs): ... Group("liveblog").send({ "text": json.dumps({"id": self.id}), })

Slide 32

Slide 32 text

Chat People can send messages, and they get sent to everyone connected.

Slide 33

Slide 33 text

Chat When people connect they join a chat group When we receive a message we send it to the group

Slide 34

Slide 34 text

When people connect they join a chat group def ws_connect(message): Group("chat").add(message.reply_channel) When we receive a message we send it to the group def ws_receive(message): Group("chat").send({ "text": message["text"], })

Slide 35

Slide 35 text

...and we tell Django what consumers are joined to which actions. # in routing.py routing = [ route("websocket.connect", consumers.ws_connect), route("websocket.receive", consumers.ws_receive), ] ...and we tell Django what consumers are joined to which actions.

Slide 36

Slide 36 text

Important notes Runserver just works with WebSockets now Generic Consumers exist Fully worked versions of these two are at github.com/andrewgodwin/channels-examples Django sessions + auth work with WebSockets

Slide 37

Slide 37 text

FIFO queue with send and receive_many operations, named with a string. Channel Group Named set of channels with add/remove/send operations Cross-process Messages Representations for HTTP and WebSocket sessions

Slide 38

Slide 38 text

ASGI API specification for channel layer backends Message formats for HTTP and WebSocket &

Slide 39

Slide 39 text

Redis POSIX IPC In-memory Reference network layer For single-machine installs For testing or single-process installs

Slide 40

Slide 40 text

Channel Layer Interface Server Worker Server Process 1 ASGI ASGI Asynchronous socket handling Synchronous Django project Interface Server Worker Server ASGI ASGI Worker Server ASGI Process 2 Process 3 Process 4

Slide 41

Slide 41 text

Channel Layer Interface Server Worker Server Channel Layer Daphne HTTP + WS WSGI Adapter HTTP asgi_redis Cross-network, shards asgi_ipc Single-machine Django Consumer system WSGI Adapter Most WSGI apps

Slide 42

Slide 42 text

WSGI and/or ASGI An ASGI system can serve HTTP and WebSocket ...or your WSGI system can send onto channels

Slide 43

Slide 43 text

Channel Layer Worker Server Interface Server Channel Layer Worker Server Interface Server WSGI App WSGI Server Channel Layer Worker 1 Worker 2 Worker 3 Redis

Slide 44

Slide 44 text

Scaling?

Slide 45

Slide 45 text

Interface servers scale horizontally Worker servers scale horizontally Channel layer has to as well

Slide 46

Slide 46 text

Consistent hash sharding designed in Other approaches possible

Slide 47

Slide 47 text

Looking Ahead

Slide 48

Slide 48 text

Being Django Official external app Merge in 1.11 or 2.0

Slide 49

Slide 49 text

Maturity Load tests and tweaking Learning from production installs

Slide 50

Slide 50 text

Community Third-party tools and extensions Tutorials, write-ups and case studies

Slide 51

Slide 51 text

Co-existence Not everyone needs WebSockets! Make sure it's easy to add or remove.

Slide 52

Slide 52 text

Expansion Email, chat and other message formats Add more events to code against!

Slide 53

Slide 53 text

Specification ASGI is not Django-specific WSGI needs supplementing for new protocols

Slide 54

Slide 54 text

channels.readthedocs.io github.com/andrewgodwin/channels-examples

Slide 55

Slide 55 text

Thanks. Andrew Godwin @andrewgodwin