Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Architecting with Channels

Architecting with Channels

My keynote from DjangoCon US 2016.

Andrew Godwin

July 19, 2016

More Decks by Andrew Godwin

Other Decks in Programming


  1. None
  2. Andrew Godwin Hi, I'm Django core developer Senior Software Engineer

    at Used to complain about migrations a lot
  3. 2010

  4. WebSockets

  5. WebSockets (and Eventlet)

  6. 2012

  7. None
  8. 2014

  9. "django-onair"

  10. 2015

  11. Channels

  12. Born from WebSockets

  13. Born from WebSockets Expanded to be more

  14. The "real hard problem"

  15. Asynchronous coordination

  16. Why?

  17. WebSockets don't magically solve everything

  18. Architecture is about tradeoffs

  19. Bidirectional, low overhead Complex client-side, compatibility CONS PROS

  20. Streaming updates Chat applications Collaborative editing Game backends

  21. Broadcast

  22. Server Client 1 Client 2 Client 3 Client 4

  23. Server Client 1 Client 2 Client 3 Client 4 Server

  24. Server Client 1 Client 2 Client 3 Client 4 Server

    The "hard problem"
  25. Channels is a foundation for runnng 'async' at scale

  26. WebSockets Task offloading Chat/Email integration IoT protocols

  27. What makes it hard?

  28. Stateful connections Internal network Bottlenecks

  29. Server Server Django Script ASGI "Send to channel X" "Receive

    from channel X" "Send to group Y" "Add channel X to group Y" "Remove channel X from group Y"
  30. Server Server Django Script ASGI ASGI ASGI ASGI Redis

  31. Server Server Django Script ASGI ASGI ASGI ASGI Shared Memory

  32. Server Server Django Script ASGI ASGI ASGI ASGI Shared Memory

    Redis Redis
  33. bit.ly/asgi-spec

  34. Channels wraps the low-level ASGI operations

  35. Think of it as the "Django bit" of a larger

  36. Daphne HTTP/WebSocket Server Channels Django integration asgi-redis Redis backend asgi-ipc

    Local memory backend asgiref Shared code and libs
  37. What does Channels provide? Routing Consumers Sessions Auth Helpers By

    channel, URL, etc. Standardised message handling Cross-network persistence on sockets Including HTTP cookies on WebSocket runserver, runworker, debugging info
  38. Putting it to use Let's make a chat!

  39. Consumers def on_connect(message): Group("chat").add(message.reply_channel) def on_receive(message): Group("chat").send({"text": message["text"]}) def on_disconnect(message):

    Group("chat").discard(message.reply_channel) websocket.connect websocket.receive websocket.disconnect
  40. Routing from channels import route routing = [ route("websocket.connect", on_connect),

    route("websocket.receive", on_receive), route("websocket.disconnect", on_disconnect) ]
  41. Class-based from channels import route_class routing = [ route_class(ChatConsumer), ]

    from channels.generic.websockets class ChatConsumer(WebsocketConsumer): def connection_groups(self): return ["chat"] def receive(self, text): self.group_send("chat", text=text) Routing
  42. Full worked example github.com/andrewgodwin/channels-examples

  43. Liveblog

  44. Liveblog

  45. Data Binding Multiplexing &

  46. class IntegerValue(models.Model): name = models.CharField(max_length=100, unique=True) value = models.IntegerField(default=0) class

    IntegerValueBinding(WebsocketBinding): model = IntegerValue stream = "intval" def check_permission(self, user, action, instance): return True def group_names(self, instance, action): return ["binding.values"]
  47. Demo Do I dare?

  48. More in channels-examples!

  49. Custom Protocols

  50. Decompose into messages Email email.send, email.receive Slack slack.receive, slack.send, slack.joined,

    ... MQTT mqtt.subscribe, mqtt.unsubscribe, mqtt.send, ...
  51. Write a custom server Use asyncio, Twisted, Gevent, etc Server

    takes path to ASGI channel layer instance (CLI/config file) Roll layer.send, layer.receive_many into your event loop
  52. Write Django consumers Same codebase as the main project Optional

    routing/per-worker channel limiting
  53. IRC Worker ASGI Worker Worker Email MQTT Scheduler HTTP/WS Custom

  54. Channels is a tool for you to use

  55. There's more to be done (and some funding for it)

  56. 1995 You are a desktop app 2005 You are a

    website 2015 You are a rich web/mobile app 2025 ?
  57. What are the goals of a framework? Do we adapt?

  58. Let's keep Django and Python a great choice for complex

  59. Thanks. Andrew Godwin @andrewgodwin channels.readthedocs.io github.com/andrewgodwin/channels-examples