Slide 1

Slide 1 text

HOW TO BREAK DJANGO: ANDREW GODWIN // @andrewgodwin WITH ASYNC

Slide 2

Slide 2 text

Andrew Godwin / @andrewgodwin Hi, I’m Andrew Godwin • Django core developer • Worked on Migrations, Channels & Async • Not currently in Europe

Slide 3

Slide 3 text

Andrew Godwin / @andrewgodwin

Slide 4

Slide 4 text

Andrew Godwin / @andrewgodwin Async views are released! Find them in a Django 3.1 near you.

Slide 5

Slide 5 text

Andrew Godwin / @andrewgodwin WSGIHandler __call__ WSGI Server WSGIRequest URLs Middleware View __call__ ASGIHandler __call__ ASGI Server ASGIRequest Asynchronous request path BaseHandler get_response_async BaseHandler get_response URLs Middleware Async View __call__ Implemented async request flow

Slide 6

Slide 6 text

Andrew Godwin / @andrewgodwin Synchronous programming is quite safe Sometimes that's worth the performance cost

Slide 7

Slide 7 text

Andrew Godwin / @andrewgodwin Async programming is hard But it can definitely be worth it!

Slide 8

Slide 8 text

Andrew Godwin / @andrewgodwin I love learning by example … so please don't do any of these things.

Slide 9

Slide 9 text

Andrew Godwin / @andrewgodwin async def my_view(request, book_id): book = Book.objects.get(pk=book_id) return render( request, "book.html", {"book": book} )

Slide 10

Slide 10 text

Andrew Godwin / @andrewgodwin async def my_view(request, book_id): book = await Book.objects.get_async( pk=book_id ) return render( request, "book.html", {"book": book} )

Slide 11

Slide 11 text

Andrew Godwin / @andrewgodwin SynchronousOnlyOperation: You cannot call this from an async context - use a thread or sync_to_async

Slide 12

Slide 12 text

Andrew Godwin / @andrewgodwin async def my_view(request, book_id): book = Book.objects.get_async( pk=book_id ) return render( request, "book.html", {"book": book} )

Slide 13

Slide 13 text

Andrew Godwin / @andrewgodwin Django's got your back ...this time, anyway.

Slide 14

Slide 14 text

Andrew Godwin / @andrewgodwin async def my_view(request): ... asyncio.wait({ create_user_account(), send_created_email(), }) ...

Slide 15

Slide 15 text

Andrew Godwin / @andrewgodwin There's no ordering guarantee! Or even a guarantee it will execute.

Slide 16

Slide 16 text

Andrew Godwin / @andrewgodwin async def my_view(request): ... await create_user_account() await send_created_email() ...

Slide 17

Slide 17 text

Andrew Godwin / @andrewgodwin Race conditions! Names are power - now you can research it!

Slide 18

Slide 18 text

Andrew Godwin / @andrewgodwin # This is how you need to do DB access in 3.1 @sync_to_async def create_user_account(): User.objects.create(...)

Slide 19

Slide 19 text

Andrew Godwin / @andrewgodwin async def my_view(request): with transaction.atomic(): await verify_valid_user(...) await create_user_account(...)

Slide 20

Slide 20 text

Andrew Godwin / @andrewgodwin async def my_view(request): with transaction.atomic(): await verify_valid_user(...) await create_user_account(...) Runs in async thread In different, sync thread

Slide 21

Slide 21 text

Andrew Godwin / @andrewgodwin You don't actually have a transaction. We need to address this when we make the ORM async-aware!

Slide 22

Slide 22 text

Andrew Godwin / @andrewgodwin stats = {"alive": 0, "dead": 0} async def fetch_site(site): alive = stats["alive"] response = await client.get(site) if response.status_code < 400: stats["alive"] = alive + 1 ... asyncio.wait([ fetch_site(site) for site in sites ])

Slide 23

Slide 23 text

Andrew Godwin / @andrewgodwin stats = {"alive": 0, "dead": 0} async def fetch_site(site): response = await client.get(site) if response.status_code < 400: alive = stats["alive"] stats["alive"] = alive + 1 ... asyncio.wait([ fetch_site(site) for site in sites ])

Slide 24

Slide 24 text

Andrew Godwin / @andrewgodwin Bad with threads. Good with async! The await model brings us atomic sections for free

Slide 25

Slide 25 text

Andrew Godwin / @andrewgodwin ready = False async def run(): await client.get("some-url") ready = True async def notify(): while not ready: time.sleep(0.01) send_email() asyncio.wait({run, notify})

Slide 26

Slide 26 text

Andrew Godwin / @andrewgodwin ready = False async def run(): await client.get("some-url") ready = True async def notify(): while not ready: await time.sleep(0.01) send_email() asyncio.wait({run, notify})

Slide 27

Slide 27 text

Andrew Godwin / @andrewgodwin ready = False async def run(): await client.get("some-url") ready = True async def notify(): while not ready: await asyncio.sleep(0.01) send_email() asyncio.wait({run, notify})

Slide 28

Slide 28 text

Andrew Godwin / @andrewgodwin This maybe is a Python thing We can't have functions be both sync and async.

Slide 29

Slide 29 text

Andrew Godwin / @andrewgodwin @sync_to_async def setup_conn(): connection.execute("SET SESSION CHARACTERISTICS…") @sync_to_async def deactivate_users(): with transaction.atomic(): for user in User.objects.filter(old=True): ... async def my_view(request): await setup_conn() await deactivate_users(users)

Slide 30

Slide 30 text

Andrew Godwin / @andrewgodwin @sync_to_async def setup_conn(): connection.execute("SET SESSION CHARACTERISTICS…") @sync_to_async def deactivate_users(): with transaction.atomic(): for user in User.objects.filter(old=True): ... async def my_view(request): await setup_conn() await deactivate_users(users)

Slide 31

Slide 31 text

Andrew Godwin / @andrewgodwin This one's on me. Fix is already in the works!

Slide 32

Slide 32 text

Andrew Godwin / @andrewgodwin How do we defend against these? Seems awfully easy to slip up, doesn't it.

Slide 33

Slide 33 text

Andrew Godwin / @andrewgodwin It's an env variable. Set it to 1! PYTHONASYNCIODEBUG

Slide 34

Slide 34 text

Andrew Godwin / @andrewgodwin Detects slow coroutines Like ones that are calling synchronous code! Detects unawaited coroutines Because we all forget await sometimes Slow I/O & thread-safety too Though these are not usually your fault

Slide 35

Slide 35 text

Andrew Godwin / @andrewgodwin SynchronousOnlyOperation Django's here to save you from the worst errors, where we can

Slide 36

Slide 36 text

Andrew Godwin / @andrewgodwin Again, async programming is hard This is not just Python!

Slide 37

Slide 37 text

Andrew Godwin / @andrewgodwin Write things sync first, with tests As they say, "make it work, then make it fast"

Slide 38

Slide 38 text

Andrew Godwin / @andrewgodwin Django will do our part - where we can We want to provide a place where you can quickly & safely add async.

Slide 39

Slide 39 text

Andrew Godwin / @andrewgodwin We'll get better patterns! Until then, be safe out there

Slide 40

Slide 40 text

Thanks. Andrew Godwin @andrewgodwin // aeracode.org