Architecting with Channels

My keynote from DjangoCon US 2016.

Transcript

1. View Slide

2. Andrew Godwin
   Hi, I'm
   Django core developer
   Senior Software Engineer at
   Used to complain about migrations a lot
   

   View Slide

3. View Slide

4. WebSockets
   

   View Slide

5. WebSockets
   (and Eventlet)
   

   View Slide

6. View Slide

7. View Slide

8. View Slide

9. "django-onair"
   

   View Slide

10. View Slide

11. Channels
    

    View Slide

12. Born from WebSockets
    

    View Slide

13. Born from WebSockets
    Expanded to be more
    

    View Slide

14. The "real hard problem"
    

    View Slide

15. Asynchronous coordination
    

    View Slide

16. Why?
    

    View Slide

17. WebSockets don't
    magically solve everything
    

    View Slide

18. Architecture is
    about tradeoffs
    

    View Slide

19. Bidirectional, low overhead
    Complex client-side, compatibility
    CONS
    PROS
    

    View Slide

20. Streaming updates
    Chat applications
    Collaborative editing
    Game backends
    

    View Slide

21. Broadcast
    

    View Slide

22. Server
    Client 1
    Client 2
    Client 3
    Client 4
    

    View Slide

23. Server
    Client 1
    Client 2
    Client 3
    Client 4
    Server
    

    View Slide

24. Server
    Client 1
    Client 2
    Client 3
    Client 4
    Server
    The "hard problem"
    

    View Slide

25. Channels is a foundation
    for runnng 'async' at scale
    

    View Slide

26. WebSockets
    Task offloading
    Chat/Email integration
    IoT protocols
    

    View Slide

27. What makes it hard?
    

    View Slide

28. Stateful connections
    Internal network
    Bottlenecks
    

    View Slide

29. Server Server Django Script
    ASGI
    "Send to channel X"
    "Receive from channel X"
    "Send to group Y"
    "Add channel X to group Y"
    "Remove channel X from group Y"
    

    View Slide

30. Server Server Django Script
    ASGI ASGI ASGI ASGI
    Redis
    

    View Slide

31. Server Server Django Script
    ASGI ASGI ASGI ASGI
    Shared
    Memory
    

    View Slide

32. Server Server Django Script
    ASGI ASGI ASGI ASGI
    Shared
    Memory
    Redis Redis
    

    View Slide

33. bit.ly/asgi-spec
    

    View Slide

34. Channels wraps the low-level
    ASGI operations
    

    View Slide

35. Think of it as the "Django bit"
    of a larger whole.
    

    View Slide

36. Daphne
    HTTP/WebSocket Server
    Channels
    Django integration
    asgi-redis
    Redis backend
    asgi-ipc
    Local memory backend
    asgiref
    Shared code and libs
    

    View Slide

37. What does Channels provide?
    Routing
    Consumers
    Sessions
    Auth
    Helpers
    By channel, URL, etc.
    Standardised message handling
    Cross-network persistence on sockets
    Including HTTP cookies on WebSocket
    runserver, runworker, debugging info
    

    View Slide

38. Putting it to use
    Let's make a chat!
    

    View Slide

39. Consumers
    def on_connect(message):
    Group("chat").add(message.reply_channel)
    def on_receive(message):
    Group("chat").send({"text": message["text"]})
    def on_disconnect(message):
    Group("chat").discard(message.reply_channel)
    websocket.connect
    websocket.receive
    websocket.disconnect
    

    View Slide

40. Routing
    from channels import route
    routing = [
    route("websocket.connect", on_connect),
    route("websocket.receive", on_receive),
    route("websocket.disconnect", on_disconnect)
    ]
    

    View Slide

41. Class-based
    from channels import route_class
    routing = [
    route_class(ChatConsumer),
    ]
    from channels.generic.websockets
    class ChatConsumer(WebsocketConsumer):
    def connection_groups(self):
    return ["chat"]
    def receive(self, text):
    self.group_send("chat", text=text)
    Routing
    

    View Slide

42. Full worked example
    github.com/andrewgodwin/channels-examples
    

    View Slide

43. Liveblog
    

    View Slide

44. Liveblog
    

    View Slide

45. Data Binding
    Multiplexing &
    

    View Slide

46. class IntegerValue(models.Model):
    name = models.CharField(max_length=100, unique=True)
    value = models.IntegerField(default=0)
    class IntegerValueBinding(WebsocketBinding):
    model = IntegerValue
    stream = "intval"
    def check_permission(self, user, action, instance):
    return True
    def group_names(self, instance, action):
    return ["binding.values"]
    

    View Slide

47. Demo
    Do I dare?
    

    View Slide

48. More in channels-examples!
    

    View Slide

49. Custom Protocols
    

    View Slide

50. Decompose into messages
    Email
    email.send, email.receive
    Slack
    slack.receive, slack.send, slack.joined, ...
    MQTT
    mqtt.subscribe, mqtt.unsubscribe, mqtt.send, ...
    

    View Slide

51. Write a custom server
    Use asyncio, Twisted, Gevent, etc
    Server takes path to ASGI channel
    layer instance (CLI/config file)
    Roll layer.send, layer.receive_many
    into your event loop
    

    View Slide

52. Write Django consumers
    Same codebase as the main project
    Optional routing/per-worker channel
    limiting
    

    View Slide

53. IRC
    Worker
    ASGI Worker
    Worker
    Email
    MQTT
    Scheduler
    HTTP/WS
    Custom
    Daemon
    

    View Slide

54. Channels is a tool
    for you to use
    

    View Slide

55. There's more to be done
    (and some funding for it)
    

    View Slide

56. 1995 You are a desktop app
    2005 You are a website
    2015 You are a rich web/mobile app
    2025 ?
    

    View Slide

57. What are the goals of a framework?
    Do we adapt?
    

    View Slide

58. Let's keep Django and Python a
    great choice for complex backends
    

    View Slide

59. Thanks.
    Andrew Godwin
    @andrewgodwin
    channels.readthedocs.io
    github.com/andrewgodwin/channels-examples
    

    View Slide