Async, Python, and the Future

Transcript

1. ASYNC, PYTHON, AND ANDREW GODWIN // @andrewgodwin THE FUTURE

2. Hi, I’m Andrew Godwin • Django core developer • Worked

   on Migrations, Channels & Async • Dabbled with Python async since 2009
3. None

4. The Past What is all this async business anyway? The

   Present The long road and where we've got to The Future Is there a perfect solution to all this?

5. The path was forged by other languages And continues to

   be - we're all one community in the end

6. 1998 threading module, Stackless Python 2002 Twisted 2006 Greenlets (later

   gevent, eventlet) 2008 multiprocessing module 2012 Tulip, PEP 3156 2014 asyncio module 2005 Coroutine-friendly generators (PEP 342)

7. Threading & Multiprocessing They're concurrency, but not really "async" in

   the way we use it now

8. Twisted The original, and ahead of its time!

9. Greenlets & Gevent An almost drop-in solution… but it's never

   that easy

10. Generators & Tulip The foundation of our current, cooperative async

11. 1998 threading module, Stackless Python 2002 Twisted 2006 Greenlets (later

    gevent, eventlet) 2008 multiprocessing module 2012 Tulip, PEP 3156 2014 asyncio module 2005 Coroutine-friendly generators (PEP 342)

12. What did we learn? A lot, but not everything.

13. Let's talk about the present And, of course, asyncio

14. Asyncio is here, and it's gaining traction Library support! Framework

    support!

15. # Ready when a timer finishes await asyncio.sleep(1) # Ready

    when network packets return await client.get("http://example.com") # Ready when the coroutine exits await my_function("hello", 64.2)

16. Network/timer updates An event loop's flow Select a ready task

    Run task Add new tasks to queue await

17. Coroutines Time →

18. It is, however, not yet perfect. Turns out, it's a

    really hard problem to solve

19. Everything must cooperate! One bit of synchronous code will ruin

    the whole thing.

20. Can't tell if a function returns a coroutine! There are

    standard hints, but no actual guaranteed way

21. async def calculate(x): result = await coroutine(x) return result #

    These both return a coroutine def calculate(x): result = coroutine(x) return result

22. Can't have one function service both How we got here

    makes sense, but it's still annoying sometimes.

23. You have to namespace async functions I really, really wish

    we didn't have to

24. instance = MyModel.objects.get(id=3) instance = await MyModel.objects.a.get(id=3)

25. WSGIHandler __call__ WSGI Server WSGIRequest URLs Middleware View __call__ ASGIHandler

    __call__ ASGI Server ASGIRequest Asynchronous request path BaseHandler get_response_async BaseHandler get_response URLs Middleware Async View __call__ Django's dual request flows

26. But, in many ways, the future is here You can

    just write full async Python now, and it works pretty well.

27. So what does the future hold? Apart from, in my

    case, a very delicious meal.

28. Obviously, more library support Databases & common services are still

    thin on the ground

29. Safety, Safety, Safety Async code is HARD. Really hard.

30. How do we design out silent failure? Deadlocks, livelocks, race

    conditions...

31. How do we prioritise? It's not like you have all

    day to add new things.

32. Horizontal scalability is more important It's the difference between life

    and death for a web service.

33. Long-polling and sockets need async Or your server bill will,

    instead, be the death of you

34. I think we need both Sync and async code both

    have their place.

35. Some things don't need async They're better off a little

    slower and safer

36. Asyncio only benefits IO-bound code Code that thrashes the CPU

    doesn't benefit at all

37. What does this mean for the Web? Our roles are

    changing along with our technology

38. Parallel Queries After all, we're the experts in fetching data

    Notifications & Events Polling will absolutely wreck your servers Microservices / API aggregation It's a lot quicker to call those 10 things in parallel

39. What does this mean for you? You're probably not in

    quite as deep as I am

40. Mixed-mode is coming Django has it, Flask is really close.

41. Think about your architecture Group things that all do I/O

    & requests together, for future parallelisation

42. Experiment! Take some async code for a spin.

43. Be the change you want to see. There's a lot

    of work to be done, and never enough of us to do it.

44. Thanks. Andrew Godwin @andrewgodwin // aeracode.org