Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Django, Channels, and Distributed Systems
Search
Andrew Godwin
August 21, 2016
Programming
1
530
Django, Channels, and Distributed Systems
A talk I gave at PyBay 2016
Andrew Godwin
August 21, 2016
Tweet
Share
More Decks by Andrew Godwin
See All by Andrew Godwin
Reconciling Everything
andrewgodwin
1
210
Django Through The Years
andrewgodwin
0
88
Writing Maintainable Software At Scale
andrewgodwin
0
330
A Newcomer's Guide To Airflow's Architecture
andrewgodwin
0
240
Async, Python, and the Future
andrewgodwin
2
540
How To Break Django: With Async
andrewgodwin
1
580
Taking Django's ORM Async
andrewgodwin
0
590
The Long Road To Asynchrony
andrewgodwin
0
510
The Scientist & The Engineer
andrewgodwin
1
580
Other Decks in Programming
See All in Programming
Milestoner
bkuhlmann
1
410
Documentation for users with AsciiDoc and Antora
ahus1
0
370
業務ツールとして使うPostman
msys75
0
100
Fragment Composition of GraphQL
quramy
13
1.4k
#phpcon_odawara オープン・クローズドなテストフィクスチャを求めて / open closed test fixtures
77web
3
240
Go製Webアプリケーションのエラーとの向き合い方大全、あるいはやっぱりスタックトレース欲しいやん / Kyoto.go #50
utgwkk
6
1.8k
Code Reviews
bkuhlmann
4
900
サイコロで理解する統計的仮説検定の考え方
tatamiya
4
1k
Let's learn code review
riofujimon
2
570
Azure OpenAI Serviceのプロンプトエンジニアリング入門
tomokusaba
3
870
MetricKitで予期せぬ終了を検知する話 / Detect unexpected termination with MetricKit
nekowen
1
200
CREってこういうこと? 体験入社 - 提案資料 - / what-is-cre-trial-employment
shinden
1
510
Featured
See All Featured
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
34
8.9k
The Power of CSS Pseudo Elements
geoffreycrofte
62
5k
Mobile First: as difficult as doing things right
swwweet
217
8.6k
Clear Off the Table
cherdarchuk
85
310k
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
188
16k
Robots, Beer and Maslow
schacon
PRO
155
7.9k
jQuery: Nuts, Bolts and Bling
dougneiner
59
7.2k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
245
20k
Designing for Performance
lara
601
67k
Agile that works and the tools we love
rasmusluckow
325
20k
WebSockets: Embracing the real-time Web
robhawkes
59
7k
The Brand Is Dead. Long Live the Brand.
mthomps
49
29k
Transcript
None
Andrew Godwin Hi, I'm Django core developer Senior Software Engineer
at Used to complain about migrations a lot
Channels
Born from WebSockets
Born from WebSockets Expanded to be more
The "real hard problem"
Asynchronous coordination
You take a request... ...and return a response.
HTTP 1 request response Browser Server request response request response
request response
HTTP 2 request response Browser Server request response request 1
response 2 request 2 response 1
WebSockets receive send Browser Server send receive send send receive
???
Server Client 1 Client 2 Client 3 Client 4
Server Client 1 Client 2 Client 3 Client 4 Server
Server Client 1 Client 2 Client 3 Client 4 Server
The "hard problem"
Broadcast
We need to coordinate between servers
Channels is a foundation for runnng 'async' at scale
Architecture is about tradeoffs
At most once / At least once Ordered / Unordered
FIFO / FILO Expiry / Persistence
WebSockets Service-oriented Architecture Chat/Email integration IoT protocols
What makes it hard?
DISTRIBUTED SYSTEMS
Stateful connections Internal network Bottlenecks Machines dying
Server Server Django Script ASGI "Send to channel X" "Receive
from channel X" "Send to group Y" "Add channel X to group Y" "Remove channel X from group Y"
Server Server Django Script ASGI ASGI ASGI ASGI Redis
Server Server Django Script ASGI ASGI ASGI ASGI Shared Memory
Server Server Django Script ASGI ASGI ASGI ASGI Shared Memory
Redis Redis
bit.ly/asgi-spec
Channels wraps the low-level ASGI operations
Think of it as the "Django bit" of a larger
whole.
Daphne HTTP/WebSocket Server Channels Django integration asgi-redis Redis backend asgi-ipc
Local memory backend asgiref Shared code and libs
What does Channels provide? Routing Consumers Sessions Auth Helpers By
channel, URL, etc. Standardised message handling Cross-network persistence on sockets Including HTTP cookies on WebSocket runserver, runworker, debugging info
Putting it to use Let's make a chat!
Consumers def on_connect(message): Group("chat").add(message.reply_channel) def on_receive(message): Group("chat").send({"text": message["text"]}) def on_disconnect(message):
Group("chat").discard(message.reply_channel) websocket.connect websocket.receive websocket.disconnect
Routing from channels import route routing = [ route("websocket.connect", on_connect),
route("websocket.receive", on_receive), route("websocket.disconnect", on_disconnect) ]
Class-based from channels import route_class routing = [ route_class(ChatConsumer), ]
from channels.generic.websockets class ChatConsumer(WebsocketConsumer): def connection_groups(self): return ["chat"] def receive(self, text): self.group_send("chat", text=text) Routing
Full worked example github.com/andrewgodwin/channels-examples
Ignoring Django
1. Take a channel layer daphne myproject.asgi:channel_layer
2. Tie it into an event loop Twisted, asyncio, or
while-True
3. Call send/receive It's a communication channel!
Example: SOA Services receive()-block waiting for tasks Clients use send()
with a reply-channel to call an endpoint Servers process and send() the reply
IRC Worker ASGI Worker Worker Email MQTT Scheduler HTTP/WS Custom
Daemon
Channels is a tool for you to use
There's more to be done (and some funding for it)
The tradeoffs may not be for you! (Especially as you
specialise)
1995 You are a desktop app 2005 You are a
website 2015 You are a rich web/mobile app 2025 ?
What are the goals of a framework? Do we adapt?
Thanks. Andrew Godwin @andrewgodwin channels.readthedocs.io github.com/andrewgodwin/channels-examples