Concurrency to Channels

Concurrency to Channels

My keynote from Python Brasil 13 (2017).

077e9a0cb34fa3eba2699240c9509717?s=128

Andrew Godwin

October 08, 2017
Tweet

Transcript

  1. Andrew Godwin @andrewgodwin

  2. Hi, I’m Andrew Godwin • Django core developer • Senior

    Software Engineer at • South, Django migrations, Channels
  3. What is Channels?

  4. What is Channels?

  5. Why Channels?

  6. 2000 2008 2015

  7. Python 1.5 Threading 2000

  8. Twisted 1.0 2002

  9. WSGI (PEP 333) 2003

  10. Python 2.6 Multiprocessing 2008

  11. Django 1.0 2008

  12. Eventlet / gevent 2009

  13. Python 3.4 asyncio 2014

  14. Python 3.5 async / await 2015

  15. Web request handling is “solved” We have good patterns and

    standards
  16. Web request handling is easy It’s stateless, and has a

    fixed response pattern
  17. Client Server Request Response Process

  18. Other protocols are not so easy Like WebSockets and MQTT

  19. This is why they are more efficient No overheads of

    cookies, headers and TCP handshake
  20. Client Server Connect Data ??? Data Data Data

  21. Writing async code is harder There’s a reason we don’t

    do it all the time
  22. Deadlocks Livelocks Race conditions Resource starvation

  23. Simple Race Condition

  24. Thread 1 Thread 2 if self.funds[..] < .. if self.funds[..]

    < .. self.funds[..] += .. self.funds[..] -= .. self.funds[..] += .. self.funds[..] -= .. {"a": 20, "b": 0} {"a": 0, "b": 20} {"a": -20, "b": 40}
  25. “Fixing” it, but actually adding a deadlock

  26. Thread 1 Thread 2 get lock a get lock b

    release lock b release lock a get lock b get lock a release lock a release lock b
  27. Thread 1 Thread 2 get lock a get lock b

    get lock b get lock a
  28. More threads = more problems And hard to find in

    development when you have no users
  29. CSP “Communicating Sequential Processes” Algebra for talking about concurrency &

    problems
  30. You don’t need everything async Only the important bits

  31. Channels: sync + async tooling Traditional synchronous Django with async

    handling
  32. “ASGI”: WSGI + async An interface that works for more

    than request-response.
  33. Goal: You mostly write sync code Channels/Django handles the annoying

    part
  34. Sync code for async behaviour

  35. Why not just use asyncio? Do we need channels at

    all?
  36. Channels 1: Can’t write async code too Have to write

    it separate outside of Channels framework
  37. Channels 2: asyncio + sync support You can write code

    as complex as you need to
  38. Channels does not solve everything Neither does Django, the goal

    is a good place to start
  39. Working on more than just WebSocket Ways to write Django/Python

    against anything with “events”
  40. WebSockets? Emails? Chat? SMS?

  41. What is Django in future? Is it HTTP requests forever?

  42. Making async code easier = good Give people a way

    to learn safely
  43. Make Python the best place for these Even if it

    doesn’t end up being Django.
  44. Thanks. Andrew Godwin @andrewgodwin aeracode.org