Slide 1

Slide 1 text

ASYNC, PYTHON, AND ANDREW GODWIN // @andrewgodwin THE FUTURE

Slide 2

Slide 2 text

Hi, I’m Andrew Godwin • Django core developer • Worked on Migrations, Channels & Async • Dabbled with Python async since 2009

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

The Past What is all this async business anyway? The Present The long road and where we've got to The Future Is there a perfect solution to all this?

Slide 5

Slide 5 text

The path was forged by other languages And continues to be - we're all one community in the end

Slide 6

Slide 6 text

1998 threading module, Stackless Python 2002 Twisted 2006 Greenlets (later gevent, eventlet) 2008 multiprocessing module 2012 Tulip, PEP 3156 2014 asyncio module 2005 Coroutine-friendly generators (PEP 342)

Slide 7

Slide 7 text

Threading & Multiprocessing They're concurrency, but not really "async" in the way we use it now

Slide 8

Slide 8 text

Twisted The original, and ahead of its time!

Slide 9

Slide 9 text

Greenlets & Gevent An almost drop-in solution… but it's never that easy

Slide 10

Slide 10 text

Generators & Tulip The foundation of our current, cooperative async

Slide 11

Slide 11 text

1998 threading module, Stackless Python 2002 Twisted 2006 Greenlets (later gevent, eventlet) 2008 multiprocessing module 2012 Tulip, PEP 3156 2014 asyncio module 2005 Coroutine-friendly generators (PEP 342)

Slide 12

Slide 12 text

What did we learn? A lot, but not everything.

Slide 13

Slide 13 text

Let's talk about the present And, of course, asyncio

Slide 14

Slide 14 text

Asyncio is here, and it's gaining traction Library support! Framework support!

Slide 15

Slide 15 text

# Ready when a timer finishes await asyncio.sleep(1) # Ready when network packets return await client.get("http://example.com") # Ready when the coroutine exits await my_function("hello", 64.2)

Slide 16

Slide 16 text

Network/timer updates An event loop's flow Select a ready task Run task Add new tasks to queue await

Slide 17

Slide 17 text

Coroutines Time →

Slide 18

Slide 18 text

It is, however, not yet perfect. Turns out, it's a really hard problem to solve

Slide 19

Slide 19 text

Everything must cooperate! One bit of synchronous code will ruin the whole thing.

Slide 20

Slide 20 text

Can't tell if a function returns a coroutine! There are standard hints, but no actual guaranteed way

Slide 21

Slide 21 text

async def calculate(x): result = await coroutine(x) return result # These both return a coroutine def calculate(x): result = coroutine(x) return result

Slide 22

Slide 22 text

Can't have one function service both How we got here makes sense, but it's still annoying sometimes.

Slide 23

Slide 23 text

You have to namespace async functions I really, really wish we didn't have to

Slide 24

Slide 24 text

instance = MyModel.objects.get(id=3) instance = await MyModel.objects.a.get(id=3)

Slide 25

Slide 25 text

WSGIHandler __call__ WSGI Server WSGIRequest URLs Middleware View __call__ ASGIHandler __call__ ASGI Server ASGIRequest Asynchronous request path BaseHandler get_response_async BaseHandler get_response URLs Middleware Async View __call__ Django's dual request flows

Slide 26

Slide 26 text

But, in many ways, the future is here You can just write full async Python now, and it works pretty well.

Slide 27

Slide 27 text

So what does the future hold? Apart from, in my case, a very delicious meal.

Slide 28

Slide 28 text

Obviously, more library support Databases & common services are still thin on the ground

Slide 29

Slide 29 text

Safety, Safety, Safety Async code is HARD. Really hard.

Slide 30

Slide 30 text

How do we design out silent failure? Deadlocks, livelocks, race conditions...

Slide 31

Slide 31 text

How do we prioritise? It's not like you have all day to add new things.

Slide 32

Slide 32 text

Horizontal scalability is more important It's the difference between life and death for a web service.

Slide 33

Slide 33 text

Long-polling and sockets need async Or your server bill will, instead, be the death of you

Slide 34

Slide 34 text

I think we need both Sync and async code both have their place.

Slide 35

Slide 35 text

Some things don't need async They're better off a little slower and safer

Slide 36

Slide 36 text

Asyncio only benefits IO-bound code Code that thrashes the CPU doesn't benefit at all

Slide 37

Slide 37 text

What does this mean for the Web? Our roles are changing along with our technology

Slide 38

Slide 38 text

Parallel Queries After all, we're the experts in fetching data Notifications & Events Polling will absolutely wreck your servers Microservices / API aggregation It's a lot quicker to call those 10 things in parallel

Slide 39

Slide 39 text

What does this mean for you? You're probably not in quite as deep as I am

Slide 40

Slide 40 text

Mixed-mode is coming Django has it, Flask is really close.

Slide 41

Slide 41 text

Think about your architecture Group things that all do I/O & requests together, for future parallelisation

Slide 42

Slide 42 text

Experiment! Take some async code for a spin.

Slide 43

Slide 43 text

Be the change you want to see. There's a lot of work to be done, and never enough of us to do it.

Slide 44

Slide 44 text

Thanks. Andrew Godwin @andrewgodwin // aeracode.org