Channels (Under The Hood)

077e9a0cb34fa3eba2699240c9509717?s=47 Andrew Godwin
November 03, 2016

Channels (Under The Hood)

A talk I gave at Django Under The Hood 2016.

077e9a0cb34fa3eba2699240c9509717?s=128

Andrew Godwin

November 03, 2016
Tweet

Transcript

  1. None
  2. Andrew Godwin Hi, I'm Django core developer Senior Software Engineer

    at Used to complain about migrations a lot
  3. It's magic.

  4. It's magic.

  5. The Problem 1

  6. The Web is changing.

  7. WebSockets

  8. WebSockets WebRTC Long-polling MQTT Server-Sent Events

  9. Python is synchronous. Django is synchronous.

  10. Synchronous code is easier to write. Single-process async is not

    enough.
  11. Proven design pattern Not too hard to reason about

  12. What could fit these constraints?

  13. Loose Coupling 2

  14. Not too tied to WebSockets Not too tied to Django

  15. Well-defined, minimal interfaces

  16. Easy to swap out or rewrite

  17. The Message Bus HTTP Server Message Bus WSock Server Django

    Project
  18. What do you send? How do you send it?

  19. ASGI

  20. nonblocking send blocking receive add to group discard from group

    send to group
  21. JSON-compatible, dictionary-based messages onto named channels

  22. Concrete Ideas 3

  23. Develop using concrete examples

  24. WebSocket connect receive disconnect accept/reject send

  25. WebSocket websocket.connect websocket.send!abc1234 websocket.receive websocket.disconnect

  26. At-most-once First In First Out Backpressure via capacity Not sticky

    No guaranteed ordering No serial processing
  27. HTTP & WS Channel Layer Django Worker HTTP & WS

    Django Worker Django Worker Django Worker
  28. { "text": "Hello, world!", "path": "/chat/socket/", "reply_channel": "websocket.send!9m12in2p", }

  29. Developed and spec'd HTTP WebSocket Rough drafts IRC Email Slack

    Please, no. Minecraft Mainframe Terminal
  30. { "reply_channel": "http.response!g23vD2x5", "method": "GET", "http_version": "2", "path": "/chat/socket/", "query_string":

    "foo=bar", "headers": [["cookie", "abcdef..."]], }
  31. At-most-once First In First Out Backpressure via capacity Not sticky

    No guaranteed ordering No serial processing
  32. At-most-once First In First Out Backpressure via capacity Not sticky

    No guaranteed ordering No serial processing
  33. "order" key on receive messages Connection acceptance

  34. Daphne HTTP/WebSocket Server Channels Django integration asgi-redis Redis backend asgi-ipc

    Local memory backend asgiref Shared code and libs
  35. Django-ish 4

  36. It can take several tries to get a nice API.

  37. Consumers based on Views Callable that takes an object Decorators

    for functionality Class-based generics
  38. @channel_session def chat_receive(message): name = message.channel_session["name"] message.reply_channel.send({"text": "OK"}) Group("chat").send({ "text":

    "%s: %s" % (name, message["text"]), }) Message.objects.create( name=name, content=message["text"], )
  39. Routing based on URLs List of regex-based matches Includes with

    prefix stripping on paths More standardised interface
  40. routing = [ route( "websocket.receive", consumers.chat_receive, path=r"^/chat/socket/$", ), include("stats.routing", path="^/stats/"),

    route_class(ConsumerClass, path="^/v1/"), ]
  41. Sessions are the only state Sessions hang off reply channels

    not cookies Uses same sessions backends Available on the consumer's argument Can also access long-term cookie sessions
  42. @enforce_ordering def receive_consumer(message): Log.objects.create(...)

  43. session = session_for_reply_channel( message.reply_channel.name ) if not session.exists(session.session_key): try: session.save(must_create=True)

    except CreateError: # Session wasn't unique raise ConsumeLater() message.channel_session = session
  44. No Middleware New-style middleware half works No ability to capture

    sends Decorators replace most cases
  45. View/HTTP Django still there Can intermingle or just use one

    type View system is just a consumer now
  46. def view_consumer(message): replies = AsgiHandler()(message) for reply in replies: while

    True: try: message.reply_channel.send(reply) except ChannelFull: time.sleep(0.05) else: break
  47. Signals and commands runserver works as expected Signals for handling

    lifecycle staticfiles configured for development
  48. Beyond 5

  49. Generalised async communication

  50. Service messaging Security/CPU separation Sync & Async / Py2 &

    Py3
  51. Diversity of implementations More web servers More channel layers

  52. More optimisation More efficient bulk sends Less network traffic on

    receive
  53. More maintainers More viewpoints, more time

  54. 1.0 coming soon Stable APIs for everything except binding

  55. Thanks. Andrew Godwin @andrewgodwin channels.readthedocs.io github.com/andrewgodwin/channels-examples