Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up for free
Django, Channels, and Distributed Systems
Andrew Godwin
August 21, 2016
Programming
1
390
Django, Channels, and Distributed Systems
A talk I gave at PyBay 2016
Andrew Godwin
August 21, 2016
Tweet
Share
More Decks by Andrew Godwin
See All by Andrew Godwin
Writing Maintainable Software At Scale
andrewgodwin
0
190
A Newcomer's Guide To Airflow's Architecture
andrewgodwin
0
150
Async, Python, and the Future
andrewgodwin
2
440
How To Break Django: With Async
andrewgodwin
1
420
Taking Django's ORM Async
andrewgodwin
0
400
The Long Road To Asynchrony
andrewgodwin
0
420
The Scientist & The Engineer
andrewgodwin
1
440
Pioneering Real-Time
andrewgodwin
0
190
Just Add Await: Retrofitting Async Into Django
andrewgodwin
2
1.2k
Other Decks in Programming
See All in Programming
TypeScript 4.9のas const satisfiesが便利
tonkotsuboy_com
9
2.2k
社会人 20 年目エンジニア、発信で技術学びなおしてる話
e99h2121
1
140
Form実装基本を学び直してみた
hyugatsukui
0
240
僕が考えた超最強のKMMアプリの作り方
spbaya0141
0
180
23年のJavaトレンドは?Quarkusで理解するコンテナネイティブJava
tatsuya1bm
1
120
Gradle build: The time is now
nonews
1
440
Cloudflare Workersと状態管理
chimame
3
480
AWSにおける標的型Bot対策
hacomono
0
410
ペパカレで入社した私が感じた2つのギャップと向き合い方
kosuke_ito
0
160
Excelの助けを借りて楽にシナリオを作ろう
rpa_niiyama
0
270
コンピュータビジョンセミナー2 / computer_vision_seminar_libSGM
fixstars
0
320
%q is for Quine
koic
0
400
Featured
See All Featured
VelocityConf: Rendering Performance Case Studies
addyosmani
317
22k
The Web Native Designer (August 2011)
paulrobertlloyd
76
2.2k
How New CSS Is Changing Everything About Graphic Design on the Web
jensimmons
214
12k
A Philosophy of Restraint
colly
193
15k
Side Projects
sachag
451
37k
What's new in Ruby 2.0
geeforr
336
30k
The Illustrated Children's Guide to Kubernetes
chrisshort
22
42k
Clear Off the Table
cherdarchuk
79
290k
Adopting Sorbet at Scale
ufuk
65
7.8k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
239
19k
Rails Girls Zürich Keynote
gr2m
87
12k
GraphQLとの向き合い方2022年版
quramy
20
9.8k
Transcript
None
Andrew Godwin Hi, I'm Django core developer Senior Software Engineer
at Used to complain about migrations a lot
Channels
Born from WebSockets
Born from WebSockets Expanded to be more
The "real hard problem"
Asynchronous coordination
You take a request... ...and return a response.
HTTP 1 request response Browser Server request response request response
request response
HTTP 2 request response Browser Server request response request 1
response 2 request 2 response 1
WebSockets receive send Browser Server send receive send send receive
???
Server Client 1 Client 2 Client 3 Client 4
Server Client 1 Client 2 Client 3 Client 4 Server
Server Client 1 Client 2 Client 3 Client 4 Server
The "hard problem"
Broadcast
We need to coordinate between servers
Channels is a foundation for runnng 'async' at scale
Architecture is about tradeoffs
At most once / At least once Ordered / Unordered
FIFO / FILO Expiry / Persistence
WebSockets Service-oriented Architecture Chat/Email integration IoT protocols
What makes it hard?
DISTRIBUTED SYSTEMS
Stateful connections Internal network Bottlenecks Machines dying
Server Server Django Script ASGI "Send to channel X" "Receive
from channel X" "Send to group Y" "Add channel X to group Y" "Remove channel X from group Y"
Server Server Django Script ASGI ASGI ASGI ASGI Redis
Server Server Django Script ASGI ASGI ASGI ASGI Shared Memory
Server Server Django Script ASGI ASGI ASGI ASGI Shared Memory
Redis Redis
bit.ly/asgi-spec
Channels wraps the low-level ASGI operations
Think of it as the "Django bit" of a larger
whole.
Daphne HTTP/WebSocket Server Channels Django integration asgi-redis Redis backend asgi-ipc
Local memory backend asgiref Shared code and libs
What does Channels provide? Routing Consumers Sessions Auth Helpers By
channel, URL, etc. Standardised message handling Cross-network persistence on sockets Including HTTP cookies on WebSocket runserver, runworker, debugging info
Putting it to use Let's make a chat!
Consumers def on_connect(message): Group("chat").add(message.reply_channel) def on_receive(message): Group("chat").send({"text": message["text"]}) def on_disconnect(message):
Group("chat").discard(message.reply_channel) websocket.connect websocket.receive websocket.disconnect
Routing from channels import route routing = [ route("websocket.connect", on_connect),
route("websocket.receive", on_receive), route("websocket.disconnect", on_disconnect) ]
Class-based from channels import route_class routing = [ route_class(ChatConsumer), ]
from channels.generic.websockets class ChatConsumer(WebsocketConsumer): def connection_groups(self): return ["chat"] def receive(self, text): self.group_send("chat", text=text) Routing
Full worked example github.com/andrewgodwin/channels-examples
Ignoring Django
1. Take a channel layer daphne myproject.asgi:channel_layer
2. Tie it into an event loop Twisted, asyncio, or
while-True
3. Call send/receive It's a communication channel!
Example: SOA Services receive()-block waiting for tasks Clients use send()
with a reply-channel to call an endpoint Servers process and send() the reply
IRC Worker ASGI Worker Worker Email MQTT Scheduler HTTP/WS Custom
Daemon
Channels is a tool for you to use
There's more to be done (and some funding for it)
The tradeoffs may not be for you! (Especially as you
specialise)
1995 You are a desktop app 2005 You are a
website 2015 You are a rich web/mobile app 2025 ?
What are the goals of a framework? Do we adapt?
Thanks. Andrew Godwin @andrewgodwin channels.readthedocs.io github.com/andrewgodwin/channels-examples