Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
The Long Road To Asynchrony
Search
Andrew Godwin
February 22, 2020
Programming
0
660
The Long Road To Asynchrony
My keynote from PyCon Belarus 2020.
Andrew Godwin
February 22, 2020
Tweet
Share
More Decks by Andrew Godwin
See All by Andrew Godwin
Reconciling Everything
andrewgodwin
1
310
Django Through The Years
andrewgodwin
0
200
Writing Maintainable Software At Scale
andrewgodwin
0
440
A Newcomer's Guide To Airflow's Architecture
andrewgodwin
0
350
Async, Python, and the Future
andrewgodwin
2
660
How To Break Django: With Async
andrewgodwin
1
730
Taking Django's ORM Async
andrewgodwin
0
720
The Scientist & The Engineer
andrewgodwin
1
760
Pioneering Real-Time
andrewgodwin
0
420
Other Decks in Programming
See All in Programming
“いい感じ“な定量評価を求めて - Four Keysとアウトカムの間の探求 -
nealle
2
12k
20250708_JAWS_opscdk
takuyay0ne
2
130
iOS 26にアップデートすると実機でのHot Reloadができない?
umigishiaoi
0
140
ふつうの技術スタックでアート作品を作ってみる
akira888
1
1.3k
Google Agent Development Kit でLINE Botを作ってみた
ymd65536
2
260
「テストは愚直&&網羅的に書くほどよい」という誤解 / Test Smarter, Not Harder
munetoshi
0
200
PipeCDのプラグイン化で目指すところ
warashi
1
300
型で語るカタ
irof
0
700
AI時代のソフトウェア開発を考える(2025/07版) / Agentic Software Engineering Findy 2025-07 Edition
twada
PRO
99
37k
AWS Summit Japan 2024と2025の比較/はじめてのKiro、今あなたは岐路に立つ
satoshi256kbyte
0
120
マッチングアプリにおけるフリックUIで苦労したこと
yuheiito
0
190
The Modern View Layer Rails Deserves: A Vision For 2025 And Beyond @ RailsConf 2025, Philadelphia, PA
marcoroth
2
730
Featured
See All Featured
Why You Should Never Use an ORM
jnunemaker
PRO
58
9.5k
Principles of Awesome APIs and How to Build Them.
keavy
126
17k
BBQ
matthewcrist
89
9.7k
10 Git Anti Patterns You Should be Aware of
lemiorhan
PRO
656
60k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
229
22k
Bash Introduction
62gerente
613
210k
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
irinanazarova
8
830
Making the Leap to Tech Lead
cromwellryan
134
9.4k
Build your cross-platform service in a week with App Engine
jlugia
231
18k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
667
120k
How GitHub (no longer) Works
holman
314
140k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
29
9.6k
Transcript
LONG ROAD ANDREW GODWIN // @andrewgodwin ASYNCHRONY TO THE
Andrew Godwin / @andrewgodwin Hi, I’m Andrew Godwin • Django
core developer • Worked on Migrations, Channels & Async • Once a Londoner, now from Denver, USA
Andrew Godwin / @andrewgodwin
Andrew Godwin / @andrewgodwin "Asynchronous Programming" What is it, really?
Andrew Godwin / @andrewgodwin Concurrent Programming The more general term
Andrew Godwin / @andrewgodwin Input Process Output Sequential execution
Andrew Godwin / @andrewgodwin Input Process Output Process Concurrent execution
Archive
Andrew Godwin / @andrewgodwin Shared use of a single resource
In this case, CPUs
Andrew Godwin / @andrewgodwin "Communicating Sequential Processes", C. A. R
Hoare
Andrew Godwin / @andrewgodwin func main() { messages := make(chan
string) go func() { messages <- "ping" }() msg := <-messages fmt.Println(msg) }
Andrew Godwin / @andrewgodwin Multiple processes multiprocessing Threads threading Event
loops asyncio / twisted
Andrew Godwin / @andrewgodwin Multiple processes scale best It's also
difficult and costs the most!
Andrew Godwin / @andrewgodwin Threads are unpredictable Also, the GIL
is our ever-present friend
Andrew Godwin / @andrewgodwin Event loops are a good compromise
They do require shared memory, though.
Andrew Godwin / @andrewgodwin Asynchronous ≈ Event loops Most of
the time!
Andrew Godwin / @andrewgodwin # Ready when a timer finishes
await asyncio.sleep(1) # Ready when network packets return await client.get("http://example.com") # Ready when the coroutine exits await my_function("hello", 64.2)
Andrew Godwin / @andrewgodwin Network/timer updates An event loop's flow
Select a ready task Run task Add new tasks to queue await
Andrew Godwin / @andrewgodwin Coroutines Time →
Andrew Godwin / @andrewgodwin How did we get here?
Andrew Godwin / @andrewgodwin 1998 threading module, Stackless Python 2002
Twisted 2006 Greenlets (later gevent, eventlet) 2008 multiprocessing module 2012 Tulip, PEP 3156 2014 asyncio module 2005 Coroutine-friendly generators (PEP 342)
Andrew Godwin / @andrewgodwin 2017 Django Channels 1.0 2018 Django
Channels 2.0 2019 DEP 9 (Async support) 2020 Async views land in Django
Andrew Godwin / @andrewgodwin No solution is perfect Everyone chooses
different tradeoffs
Andrew Godwin / @andrewgodwin Asyncio is based on yield from
Because it was prototyped in Python 2
Andrew Godwin / @andrewgodwin Can't tell if a function returns
a coroutine! There are standard hints, but no actual guaranteed way
Andrew Godwin / @andrewgodwin async def calculate(x): result = await
coroutine(x) return result # These both return a coroutine def calculate(x): result = coroutine(x) return result
Andrew Godwin / @andrewgodwin Can't have one function service both
How we got here makes sense, but it's still annoying sometimes.
Andrew Godwin / @andrewgodwin # Calls get.__call__ instance = MyModel.objects.get(id=3)
# Calls get.__call__ # and then awaits its result instance = await MyModel.objects.get(id=3)
Andrew Godwin / @andrewgodwin You have to namespace async functions
I really, really wish we didn't have to
Andrew Godwin / @andrewgodwin instance = MyModel.objects.get(id=3) instance = await
MyModel.objects.async.get(id=3)
Andrew Godwin / @andrewgodwin Completely different libraries! Even sleep() is
different.
Andrew Godwin / @andrewgodwin time.sleep ➞ asyncio.sleep requests ➞ httpx
psycopg2 ➞ aiopg WSGI ➞ ASGI Django ➞ Django?
Andrew Godwin / @andrewgodwin Django & Async
Andrew Godwin / @andrewgodwin Asyncio only benefits IO-bound code Code
that thrashes the CPU doesn't benefit at all
Andrew Godwin / @andrewgodwin We're adding async to some parts
The bits where it makes sense!
Andrew Godwin / @andrewgodwin But, you can't mix sync and
async So we have to have two parallel request paths
Andrew Godwin / @andrewgodwin WSGIHandler __call__ WSGI Server WSGIRequest BaseHandler
get_response URLs Middleware View __call__ HTTP protocol Socket handling Transfer encodings Headers-to-META Upload file wrapping GET/POST parsing Exception catching Atomic view wrapper Django 3.0 Request Flow
Andrew Godwin / @andrewgodwin WSGIHandler __call__ WSGI Server WSGIRequest BaseHandler
get_response URLs Middleware Async View __call__ ASGIHandler __call__ ASGI Server ASGIRequest Sync View __call__ Asynchronous request path Proposed async request flow
Andrew Godwin / @andrewgodwin WSGIHandler __call__ WSGI Server WSGIRequest URLs
Middleware View __call__ ASGIHandler __call__ ASGI Server ASGIRequest Asynchronous request path BaseHandler get_response_async BaseHandler get_response URLs Middleware Async View __call__ Implemented async request flow
Andrew Godwin / @andrewgodwin We have to work with what
we have I'd rather let people ship code than argue about perfection.
Andrew Godwin / @andrewgodwin Django's main job is safety It
matters more than anything else
Andrew Godwin / @andrewgodwin Deadlocks Livelocks Starvation Race conditions
Andrew Godwin / @andrewgodwin Coroutines & the GIL actually help!
You're saved from all the awful memory corruption bugs
Andrew Godwin / @andrewgodwin async def transfer_money(p1, p2): await get_lock()
# Code between awaits is atomic! subtract_money(p1) add_money(p2) await release_lock()
Andrew Godwin / @andrewgodwin You can still screw up a
lot Trust me, I have lived it while developing async
Andrew Godwin / @andrewgodwin async def log_message(m): await client.post("log-server", m)
result = calculate_result() log_message(m)
Andrew Godwin / @andrewgodwin async def log_message(m): await client.post("log-server", m)
result = calculate_result() await log_message(m)
Andrew Godwin / @andrewgodwin Async usability has a long way
to go But it is undoubtedly the future!
Andrew Godwin / @andrewgodwin Python is the language of pragmatism
If anyone can get it right, we can
Andrew Godwin / @andrewgodwin What does the future hold? Hopefully,
no GIL!
Andrew Godwin / @andrewgodwin Let's make async understandable Almost every
project could benefit, if we made it worth their time.
Thanks. Andrew Godwin @andrewgodwin // aeracode.org