How To Break Django: With Async

A talk I gave at DjangoCon Europe 2020.

Transcript

1. HOW TO BREAK DJANGO: ANDREW GODWIN // @andrewgodwin WITH ASYNC
2. Andrew Godwin / @andrewgodwin Hi, I’m Andrew Godwin • Django core developer • Worked on Migrations, Channels & Async • Not currently in Europe
3. Andrew Godwin / @andrewgodwin
4. Andrew Godwin / @andrewgodwin Async views are released! Find them in a Django 3.1 near you.
5. Andrew Godwin / @andrewgodwin WSGIHandler __call__ WSGI Server WSGIRequest URLs Middleware View __call__ ASGIHandler __call__ ASGI Server ASGIRequest Asynchronous request path BaseHandler get_response_async BaseHandler get_response URLs Middleware Async View __call__ Implemented async request flow
6. Andrew Godwin / @andrewgodwin Synchronous programming is quite safe Sometimes that's worth the performance cost
7. Andrew Godwin / @andrewgodwin Async programming is hard But it can definitely be worth it!
8. Andrew Godwin / @andrewgodwin I love learning by example … so please don't do any of these things.
9. Andrew Godwin / @andrewgodwin async def my_view(request, book_id): book = Book.objects.get(pk=book_id) return render( request, "book.html", {"book": book} )
10. Andrew Godwin / @andrewgodwin async def my_view(request, book_id): book = await Book.objects.get_async( pk=book_id ) return render( request, "book.html", {"book": book} )
11. Andrew Godwin / @andrewgodwin SynchronousOnlyOperation: You cannot call this from an async context - use a thread or sync_to_async
12. Andrew Godwin / @andrewgodwin async def my_view(request, book_id): book = Book.objects.get_async( pk=book_id ) return render( request, "book.html", {"book": book} )
13. Andrew Godwin / @andrewgodwin Django's got your back ...this time, anyway.
14. Andrew Godwin / @andrewgodwin async def my_view(request): ... asyncio.wait({ create_user_account(), send_created_email(), }) ...
15. Andrew Godwin / @andrewgodwin There's no ordering guarantee! Or even a guarantee it will execute.
16. Andrew Godwin / @andrewgodwin async def my_view(request): ... await create_user_account() await send_created_email() ...
17. Andrew Godwin / @andrewgodwin Race conditions! Names are power - now you can research it!
18. Andrew Godwin / @andrewgodwin # This is how you need to do DB access in 3.1 @sync_to_async def create_user_account(): User.objects.create(...)
19. Andrew Godwin / @andrewgodwin async def my_view(request): with transaction.atomic(): await verify_valid_user(...) await create_user_account(...)
20. Andrew Godwin / @andrewgodwin async def my_view(request): with transaction.atomic(): await verify_valid_user(...) await create_user_account(...) Runs in async thread In different, sync thread
21. Andrew Godwin / @andrewgodwin You don't actually have a transaction. We need to address this when we make the ORM async-aware!
22. Andrew Godwin / @andrewgodwin stats = {"alive": 0, "dead": 0} async def fetch_site(site): alive = stats["alive"] response = await client.get(site) if response.status_code < 400: stats["alive"] = alive + 1 ... asyncio.wait([ fetch_site(site) for site in sites ])
23. Andrew Godwin / @andrewgodwin stats = {"alive": 0, "dead": 0} async def fetch_site(site): response = await client.get(site) if response.status_code < 400: alive = stats["alive"] stats["alive"] = alive + 1 ... asyncio.wait([ fetch_site(site) for site in sites ])
24. Andrew Godwin / @andrewgodwin Bad with threads. Good with async! The await model brings us atomic sections for free
25. Andrew Godwin / @andrewgodwin ready = False async def run(): await client.get("some-url") ready = True async def notify(): while not ready: time.sleep(0.01) send_email() asyncio.wait({run, notify})
26. Andrew Godwin / @andrewgodwin ready = False async def run(): await client.get("some-url") ready = True async def notify(): while not ready: await time.sleep(0.01) send_email() asyncio.wait({run, notify})
27. Andrew Godwin / @andrewgodwin ready = False async def run(): await client.get("some-url") ready = True async def notify(): while not ready: await asyncio.sleep(0.01) send_email() asyncio.wait({run, notify})
28. Andrew Godwin / @andrewgodwin This maybe is a Python thing We can't have functions be both sync and async.
29. Andrew Godwin / @andrewgodwin @sync_to_async def setup_conn(): connection.execute("SET SESSION CHARACTERISTICS…") @sync_to_async def deactivate_users(): with transaction.atomic(): for user in User.objects.filter(old=True): ... async def my_view(request): await setup_conn() await deactivate_users(users)
30. Andrew Godwin / @andrewgodwin @sync_to_async def setup_conn(): connection.execute("SET SESSION CHARACTERISTICS…") @sync_to_async def deactivate_users(): with transaction.atomic(): for user in User.objects.filter(old=True): ... async def my_view(request): await setup_conn() await deactivate_users(users)
31. Andrew Godwin / @andrewgodwin This one's on me. Fix is already in the works!
32. Andrew Godwin / @andrewgodwin How do we defend against these? Seems awfully easy to slip up, doesn't it.
33. Andrew Godwin / @andrewgodwin It's an env variable. Set it to 1! PYTHONASYNCIODEBUG
34. Andrew Godwin / @andrewgodwin Detects slow coroutines Like ones that are calling synchronous code! Detects unawaited coroutines Because we all forget await sometimes Slow I/O & thread-safety too Though these are not usually your fault
35. Andrew Godwin / @andrewgodwin SynchronousOnlyOperation Django's here to save you from the worst errors, where we can
36. Andrew Godwin / @andrewgodwin Again, async programming is hard This is not just Python!
37. Andrew Godwin / @andrewgodwin Write things sync first, with tests As they say, "make it work, then make it fast"
38. Andrew Godwin / @andrewgodwin Django will do our part - where we can We want to provide a place where you can quickly & safely add async.
39. Andrew Godwin / @andrewgodwin We'll get better patterns! Until then, be safe out there
40. Thanks. Andrew Godwin @andrewgodwin // aeracode.org