Django, Channels, and Distributed Systems

Transcript

1. None

2. Andrew Godwin Hi, I'm Django core developer Senior Software Engineer

   at Used to complain about migrations a lot

3. Channels

4. Born from WebSockets

5. Born from WebSockets Expanded to be more

6. The "real hard problem"

7. Asynchronous coordination

8. You take a request... ...and return a response.

9. HTTP 1 request response Browser Server request response request response

   request response

10. HTTP 2 request response Browser Server request response request 1

    response 2 request 2 response 1

11. WebSockets receive send Browser Server send receive send send receive

12. ???

13. Server Client 1 Client 2 Client 3 Client 4

14. Server Client 1 Client 2 Client 3 Client 4 Server

15. Server Client 1 Client 2 Client 3 Client 4 Server

    The "hard problem"

16. Broadcast

17. We need to coordinate between servers

18. Channels is a foundation for runnng 'async' at scale

19. Architecture is about tradeoffs

20. At most once / At least once Ordered / Unordered

    FIFO / FILO Expiry / Persistence

21. WebSockets Service-oriented Architecture Chat/Email integration IoT protocols

22. What makes it hard?

23. DISTRIBUTED SYSTEMS

24. Stateful connections Internal network Bottlenecks Machines dying

25. Server Server Django Script ASGI "Send to channel X" "Receive

    from channel X" "Send to group Y" "Add channel X to group Y" "Remove channel X from group Y"

26. Server Server Django Script ASGI ASGI ASGI ASGI Redis

27. Server Server Django Script ASGI ASGI ASGI ASGI Shared Memory

28. Server Server Django Script ASGI ASGI ASGI ASGI Shared Memory

    Redis Redis

29. bit.ly/asgi-spec

30. Channels wraps the low-level ASGI operations

31. Think of it as the "Django bit" of a larger

    whole.

32. Daphne HTTP/WebSocket Server Channels Django integration asgi-redis Redis backend asgi-ipc

    Local memory backend asgiref Shared code and libs

33. What does Channels provide? Routing Consumers Sessions Auth Helpers By

    channel, URL, etc. Standardised message handling Cross-network persistence on sockets Including HTTP cookies on WebSocket runserver, runworker, debugging info

34. Putting it to use Let's make a chat!

35. Consumers def on_connect(message): Group("chat").add(message.reply_channel) def on_receive(message): Group("chat").send({"text": message["text"]}) def on_disconnect(message):

    Group("chat").discard(message.reply_channel) websocket.connect websocket.receive websocket.disconnect

36. Routing from channels import route routing = [ route("websocket.connect", on_connect),

    route("websocket.receive", on_receive), route("websocket.disconnect", on_disconnect) ]

37. Class-based from channels import route_class routing = [ route_class(ChatConsumer), ]

    from channels.generic.websockets class ChatConsumer(WebsocketConsumer): def connection_groups(self): return ["chat"] def receive(self, text): self.group_send("chat", text=text) Routing

38. Full worked example github.com/andrewgodwin/channels-examples

39. Ignoring Django

40. 1. Take a channel layer daphne myproject.asgi:channel_layer

41. 2. Tie it into an event loop Twisted, asyncio, or

    while-True

42. 3. Call send/receive It's a communication channel!

43. Example: SOA Services receive()-block waiting for tasks Clients use send()

    with a reply-channel to call an endpoint Servers process and send() the reply

44. IRC Worker ASGI Worker Worker Email MQTT Scheduler HTTP/WS Custom

    Daemon

45. Channels is a tool for you to use

46. There's more to be done (and some funding for it)

47. The tradeoffs may not be for you! (Especially as you

    specialise)

48. 1995 You are a desktop app 2005 You are a

    website 2015 You are a rich web/mobile app 2025 ?

49. What are the goals of a framework? Do we adapt?

50. Thanks. Andrew Godwin @andrewgodwin channels.readthedocs.io github.com/andrewgodwin/channels-examples