Just Add Await: Retrofitting Async Into Django

Slide 1

Slide 1 text

JUST ADD AWAIT ANDREW GODWIN // @andrewgodwin RETROFITTING ASYNC INTO DJANGO

Slide 2

Slide 2 text

Hi, I’m Andrew Godwin • Django contributor (Migrations/Channels) • Principal Engineer at • I see Python threads in my sleep now

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

async def view(request): return TemplateResponse( request, "template.html", {"article": await api_call(pk=5)} )

Slide 5

Slide 5 text

Django In Depth 1. 2. 3. Threading, cooperation, and intrigue Spanning two worlds with one vision Handlers, Requests, Middleware & Views Big Framework Problems Async In Brief

Slide 6

Slide 6 text

ASYNC IN BRIEF 1.

Slide 7

Slide 7 text

You don't know when it'll switch! Threads are preemptive

Slide 8

Slide 8 text

You don't know when it'll switch! Threads are preemptive

Slide 9

Slide 9 text

You don't know when it'll switch! Threads are preemptive Coroutines are cooperative They only yield at an await.

Slide 10

Slide 10 text

Coroutines are cooperative They only yield at an await.

Slide 11

Slide 11 text

You don't know when it'll switch! Threads are preemptive Coroutines are cooperative They only yield at an await.

Slide 12

Slide 12 text

Coroutines need an event loop It's where the program idles between tasks

Slide 13

Slide 13 text

An event loop runs in a single thread Yes, you can have threads and coroutines!

Slide 14

Slide 14 text

Sync Thread Sync Thread Async Thread

Slide 15

Slide 15 text

Threads are slow! The more you add, the worse it gets.

Slide 16

Slide 16 text

Async is fast... As long as you are I/O bound!

Slide 17

Slide 17 text

Async functions are different to sync They are not cross-compatible!

Slide 18

Slide 18 text

# Call sync from sync result = function() # Call async from async result = await function()

Slide 19

Slide 19 text

# Call async from sync in Python 3.7 result = asyncio.run(function()) # Python 3.6 loop = asyncio.new_event_loop() asyncio.set_event_loop(loop) result = loop.run_until_complete(function())

Slide 20

Slide 20 text

# Call sync from async executor = ThreadPoolExecutor(max_workers=3) loop = asyncio.get_event_loop() result = await loop.run_in_executor( executor, function, *args, )

Slide 21

Slide 21 text

Async calling Sync is dangerous It has to be in a separate thread or it'll block the event loop

Slide 22

Slide 22 text

Sync Thread Async Thread

Slide 23

Slide 23 text

Sync Thread Async Thread

Slide 24

Slide 24 text

It's complicated This is why I encourage writing sync code at first!

Slide 25

Slide 25 text

BIG FRAMEWORK PROBLEMS 2.

Slide 26

Slide 26 text

Backwards compatibility is crucial Throw it away, and nobody will adopt your new thing

Slide 27

Slide 27 text

A function cannot be both sync and async You have to pick one. I've tried.

Slide 28

Slide 28 text

# You can have this... result = cache.get("my-key") # Or this. Not both. result = await cache.get("my-key")

Slide 29

Slide 29 text

Totally different libraries And there's not even standards like DBAPI2

Slide 30

Slide 30 text

# Even sleep is different! time.sleep(0.01) await asyncio.sleep(0.1)

Slide 31

Slide 31 text

Lack of standards It's just too early on for things to coalesce.

Slide 32

Slide 32 text

async def application(scope, receive, send): await receive() ... send({"type": "http.response", ...})

Slide 33

Slide 33 text

Different language features No more attribute access

Slide 34

Slide 34 text

# While this can have an async version... instance = await Model.objects.get() # There's no async way of doing this print(instance.foreign_key.name)

Slide 35

Slide 35 text

Threads matter! Sync code all wants to run in the same thread still.

Slide 36

Slide 36 text

Async has to add, not replace Sync Django is still important Things need to look familiar We don't want a wildly different API feel Things need to be safe Deadlocking or blocking is easier than ever

Slide 37

Slide 37 text

DJANGO IN DEPTH 3.

Slide 38

Slide 38 text

Handler ASGI / WSGI Server Middleware View ORM Template URL Router Forms

Slide 39

Slide 39 text

Outside-in approach Async outside, sync inside

Slide 40

Slide 40 text

Handler ASGI / WSGI Server Middleware View ORM Template URL Router Forms Phase One Phase Two Phase Three

Slide 41

Slide 41 text

Phase One: ASGI Support Allowing Django to be async at all Phase Two: Async Views Unlocking async use in normal apps Phase Three: The ORM High-level async use for the most common case

Slide 42

Slide 42 text

Phase One: Django 3.0 It's already committed! Phase Two: Django 3.1 Unless things really do turn out nicely... Phase Three: Django 3.2/4.0 There's a lot of work here.

Slide 43

Slide 43 text

Each phase brings concrete benefits Even if we stop!

Slide 44

Slide 44 text

Phase One: ASGI

Slide 45

Slide 45 text

Django predates WSGI Which turns out to actually help, in the end

Slide 46

Slide 46 text

“ James Bennett, "Django and NIH", 2006 Just so you know, Django is a smug, arrogant framework that doesn’t play nice with others.

Slide 47

Slide 47 text

“ James Bennett, "Django and NIH", 2006 Just so you know, Django is a smug, arrogant framework that doesn’t play nice with others. [...] Or at least, that’s the impression you’d get from reading the rants...

Slide 48

Slide 48 text

Custom request/response objects Most other frameworks did this too Custom "handler" classes Abstracts away WSGI Custom middleware Wow, was this contentious at the time!

Slide 49

Slide 49 text

WSGIHandler __call__ WSGI Server WSGIRequest BaseHandler get_response URLs Middleware View __call__ HTTP protocol Socket handling Transfer encodings Headers-to-META Upload file wrapping GET/POST parsing Exception catching Atomic view wrapper

Slide 50

Slide 50 text

WSGIHandler __call__ WSGI Server WSGIRequest BaseHandler get_response URLs Middleware View __call__ ASGIHandler __call__ ASGI Server ASGIRequest

Slide 51

Slide 51 text

WSGIHandler __call__ WSGI Server WSGIRequest BaseHandler get_response URLs Middleware View __call__ ASGIHandler __call__ ASGI Server ASGIRequest Asynchronous

Slide 52

Slide 52 text

ASGI is mostly WSGI-compatible With better definitions of bytes versus unicode

Slide 53

Slide 53 text

if self.scope.get('client'): self.META['REMOTE_ADDR'] = self.scope['client'][0] self.META['REMOTE_HOST'] = self.META['REMOTE_ADDR'] self.META['REMOTE_PORT'] = self.scope['client'][1]

Slide 54

Slide 54 text

body_file = tempfile.SpooledTemporaryFile(max_size=..., mode='w+b') while True: message = await receive() if message['type'] == 'http.disconnect': # Early client disconnect. raise RequestAborted() # Add a body chunk from the message, if provided. if 'body' in message: body_file.write(message['body']) # Quit out if that's the end. if not message.get('more_body', False): break body_file.seek(0) return body_file

Slide 55

Slide 55 text

if response.streaming: # Access `__iter__` and not `streaming_content` directly in case # it has been overridden in a subclass. for part in response: for chunk, _ in self.chunk_bytes(part): await send({ 'type': 'http.response.body', 'body': chunk, # Ignore "more" as there may be more parts; instead, # use an empty final closing message with False. 'more_body': True, }) # Final closing message. await send({'type': 'http.response.body'})

Slide 56

Slide 56 text

WSGIHandler __call__ WSGI Server WSGIRequest BaseHandler get_response URLs Middleware View __call__ ASGIHandler __call__ ASGI Server ASGIRequest Asynchronous

Slide 57

Slide 57 text

“ Me, earlier in this talk Async calling sync is dangerous!

Slide 58

Slide 58 text

from asgiref.sync import sync_to_async result = await sync_to_async(callable)(arg1, name=arg2)

Slide 59

Slide 59 text

Propagates exceptions nicely Really helps with debugging! Proxies threadlocals down correctly Because people really love threadlocals. Stickies sync code into one thread We'll get back to this. It's nasty.

Slide 60

Slide 60 text

Result: Django 3.0 can speak ASGI But it can't do much else async... yet.

Slide 61

Slide 61 text

Phase Two: Views

Slide 62

Slide 62 text

WSGIHandler __call__ WSGI Server WSGIRequest BaseHandler get_response URLs Middleware View __call__ ASGIHandler __call__ ASGI Server ASGIRequest Asynchronous

Slide 63

Slide 63 text

WSGIHandler __call__ WSGI Server WSGIRequest BaseHandler get_response URLs Middleware Async View __call__ ASGIHandler __call__ ASGI Server ASGIRequest Asynchronous Sync View __call__

Slide 64

Slide 64 text

WSGIHandler __call__ BaseHandler get_response URLs Middleware Async View __call__ ASGIHandler __call__ Sync View __call__ TestClient get/post

Slide 65

Slide 65 text

BaseHandler get_response Sync View __call__ TestClient get/post Main Thread Event Loop Sub Thread asyncio.run ThreadPool

Slide 66

Slide 66 text

SQLite hates this Try This One Weird Trick To Help Thread-Sensitive Libraries

Slide 67

Slide 67 text

result = async_to_sync(awaitable)(arg1, name=arg2) result = await sync_to_async(callable)(arg1, name=arg2)

Slide 68

Slide 68 text

BaseHandler get_response Sync View __call__ TestClient get/post Main Thread Event Loop Main Thread async_to_sync sync_to_async

Slide 69

Slide 69 text

There's a whole talk in how this works! Also, it's not pretty or nice and it really shouldn't be necessary.

Slide 70

Slide 70 text

M I D D L E W A R E

Slide 71

Slide 71 text

get_response Middleware 1 Middleware 2 View

Slide 72

Slide 72 text

get_response Sync Middleware Async Middleware Async View async_to_sync sync_to_async

Slide 73

Slide 73 text

Transactions Views are auto-wrapped in them with ATOMIC_REQUESTS Templates Direct calls from error handlers Tracebacks They're really long with all the switch functions

Slide 74

Slide 74 text

Goal: Django 3.1 has async def views They already work on the branch right now!

Slide 75

Slide 75 text

Phase Three: ORM

Slide 76

Slide 76 text

API Design is crucial It must be familiar, yet safe.

Slide 77

Slide 77 text

# Iteration is the one transparent thing for result in Model.objects.filter(name="Andrew"): >>> QuerySet.__iter__ # This can work in the same codebase! async for result in Model.objects.filter(name="Andrew"): >>> QuerySet.__aiter__

Slide 78

Slide 78 text

# But some things will never work - # we'll need to force select_related result = instance.foreign_key.name

Slide 79

Slide 79 text

QuerySet Query Compiler Connection

Slide 80

Slide 80 text

QuerySet Query Compiler Connection

Slide 81

Slide 81 text

QuerySet Query Compiler Connection

Slide 82

Slide 82 text

In the meantime, async-safety You just try calling the ORM from async code in 3.0!

Slide 83

Slide 83 text

async def random_code(): result = Model.objects.get(pk=5) >>> SynchronousOnlyOperation("You cannot call this from an async context - use a thread or sync_to_async.")

Slide 84

Slide 84 text

This needs a lot more research It's also not going to happen straight away.

Slide 85

Slide 85 text

LOOKING AHEAD 4.

Slide 86

Slide 86 text

Cache? Templates? Forms? Some will benefit from async, some will not

Slide 87

Slide 87 text

Some things don't need to be async URL routing is just fine as it is.

Slide 88

Slide 88 text

Async views are the cornerstone Once we get those working, all other paths open up

Slide 89

Slide 89 text

Being careful about performance Things could easily slow down for synchronous applications

Slide 90

Slide 90 text

Being careful about people We need to bring on new faces, and not burn out others

Slide 91

Slide 91 text

Documentation Async needs to be clear, safe, and clearly optional

Slide 92

Slide 92 text

Funding Async expertise is rare. We need to pay people for their knowledge.

Slide 93

Slide 93 text

Organisation One of the largest changes in Django's history.

Slide 94

Slide 94 text

aeracode.org/2018/02/19/python-async-simplified/ A deeper dive into async vs. sync functions github.com/django/deps/blob/master/accepted/0009-async.rst DEP 0009, the proposal for async in Django code.djangoproject.com/wiki/AsyncProject Where to go to help

Slide 95

Slide 95 text

Thanks. Andrew Godwin @andrewgodwin // aeracode.org