Upgrade to Pro — share decks privately, control downloads, hide ads and more …

How To Break Django: With Async

Andrew Godwin
September 19, 2020

How To Break Django: With Async

A talk I gave at DjangoCon Europe 2020.

Andrew Godwin

September 19, 2020
Tweet

More Decks by Andrew Godwin

Other Decks in Programming

Transcript

  1. Andrew Godwin / @andrewgodwin Hi, I’m Andrew Godwin • Django

    core developer • Worked on Migrations, Channels & Async • Not currently in Europe
  2. Andrew Godwin / @andrewgodwin WSGIHandler __call__ WSGI Server WSGIRequest URLs

    Middleware View __call__ ASGIHandler __call__ ASGI Server ASGIRequest Asynchronous request path BaseHandler get_response_async BaseHandler get_response URLs Middleware Async View __call__ Implemented async request flow
  3. Andrew Godwin / @andrewgodwin I love learning by example …

    so please don't do any of these things.
  4. Andrew Godwin / @andrewgodwin async def my_view(request, book_id): book =

    Book.objects.get(pk=book_id) return render( request, "book.html", {"book": book} )
  5. Andrew Godwin / @andrewgodwin async def my_view(request, book_id): book =

    await Book.objects.get_async( pk=book_id ) return render( request, "book.html", {"book": book} )
  6. Andrew Godwin / @andrewgodwin async def my_view(request, book_id): book =

    Book.objects.get_async( pk=book_id ) return render( request, "book.html", {"book": book} )
  7. Andrew Godwin / @andrewgodwin # This is how you need

    to do DB access in 3.1 @sync_to_async def create_user_account(): User.objects.create(...)
  8. Andrew Godwin / @andrewgodwin async def my_view(request): with transaction.atomic(): await

    verify_valid_user(...) await create_user_account(...) Runs in async thread In different, sync thread
  9. Andrew Godwin / @andrewgodwin You don't actually have a transaction.

    We need to address this when we make the ORM async-aware!
  10. Andrew Godwin / @andrewgodwin stats = {"alive": 0, "dead": 0}

    async def fetch_site(site): alive = stats["alive"] response = await client.get(site) if response.status_code < 400: stats["alive"] = alive + 1 ... asyncio.wait([ fetch_site(site) for site in sites ])
  11. Andrew Godwin / @andrewgodwin stats = {"alive": 0, "dead": 0}

    async def fetch_site(site): response = await client.get(site) if response.status_code < 400: alive = stats["alive"] stats["alive"] = alive + 1 ... asyncio.wait([ fetch_site(site) for site in sites ])
  12. Andrew Godwin / @andrewgodwin Bad with threads. Good with async!

    The await model brings us atomic sections for free
  13. Andrew Godwin / @andrewgodwin ready = False async def run():

    await client.get("some-url") ready = True async def notify(): while not ready: time.sleep(0.01) send_email() asyncio.wait({run, notify})
  14. Andrew Godwin / @andrewgodwin ready = False async def run():

    await client.get("some-url") ready = True async def notify(): while not ready: await time.sleep(0.01) send_email() asyncio.wait({run, notify})
  15. Andrew Godwin / @andrewgodwin ready = False async def run():

    await client.get("some-url") ready = True async def notify(): while not ready: await asyncio.sleep(0.01) send_email() asyncio.wait({run, notify})
  16. Andrew Godwin / @andrewgodwin This maybe is a Python thing

    We can't have functions be both sync and async.
  17. Andrew Godwin / @andrewgodwin @sync_to_async def setup_conn(): connection.execute("SET SESSION CHARACTERISTICS…")

    @sync_to_async def deactivate_users(): with transaction.atomic(): for user in User.objects.filter(old=True): ... async def my_view(request): await setup_conn() await deactivate_users(users)
  18. Andrew Godwin / @andrewgodwin @sync_to_async def setup_conn(): connection.execute("SET SESSION CHARACTERISTICS…")

    @sync_to_async def deactivate_users(): with transaction.atomic(): for user in User.objects.filter(old=True): ... async def my_view(request): await setup_conn() await deactivate_users(users)
  19. Andrew Godwin / @andrewgodwin How do we defend against these?

    Seems awfully easy to slip up, doesn't it.
  20. Andrew Godwin / @andrewgodwin Detects slow coroutines Like ones that

    are calling synchronous code! Detects unawaited coroutines Because we all forget await sometimes Slow I/O & thread-safety too Though these are not usually your fault
  21. Andrew Godwin / @andrewgodwin Write things sync first, with tests

    As they say, "make it work, then make it fast"
  22. Andrew Godwin / @andrewgodwin Django will do our part -

    where we can We want to provide a place where you can quickly & safely add async.