Python async/await unraveled

Python coroutines in detail -- understand the magical await keyword and how it manages to transform callback hell into coroutine bliss.

If you have ever wondered what await actually does, explained in terms of ordinary code, this talk is for you.

I’ll introduce coroutines in async programming and briefly compare them to callbacks and promises, showing examples of their use in asyncio alongside equivalent blocking code. Then I’ll use these examples to delve further into the seemingly magical await keyword and the programming model it brings about. After the talk you should be able to understand what await X expands to, how a coroutine could be desugared into an ordinary function, and why you’re happy you don’t have to do it yourself.

Transcript

1. python async/await unraveled Hrvoje Nikšić WebCamp Zagreb 2019 AVL-AST Croatia 1
2. Synchronous IO • Synchronous IO: API calls block. – return after the operation completes or fails – traditional IO: Python file objects, C stdio • Parallelism only possible with threads or processes. • No cancellation. • Easy to use, scales poorly. 2
3. Asynchronous IO • Asynchronous IO: API calls don’t block. – return immediately, complete or fail with EAGAIN – explicit poll tells when to retry • Parallelism within a single thread. – event loop for central polling and dispatch – application logic in callbacks • Harder to use, scales well. 3
4. Callbacks • JavaScript: all-async, event loop invisible. • Async calls accept continuation callbacks. 4
5. Callbacks • JavaScript: all-async, event loop invisible. • Async calls accept continuation callbacks. sync function greet() { console.log("hello"); sleep(1000); // XXX console.log("world"); } 4
6. Callbacks • JavaScript: all-async, event loop invisible. • Async calls accept continuation callbacks. sync function greet() { console.log("hello"); sleep(1000); // XXX console.log("world"); } async function greet() { console.log("hello"); setTimeout(() => { console.log("world"); }, 1000); } 4
7. Callbacks • JavaScript: all-async, event loop invisible. • Async calls accept continuation callbacks. sync function greet() { console.log("hello"); sleep(1000); // XXX console.log("wide"); sleep(1000); // XXX console.log("world"); } async function greet() { console.log("hello"); setTimeout(() => { console.log("world"); }, 1000); } 4
8. Callbacks • JavaScript: all-async, event loop invisible. • Async calls accept continuation callbacks. sync function greet() { console.log("hello"); sleep(1000); // XXX console.log("wide"); sleep(1000); // XXX console.log("world"); } async function greet() { console.log("hello"); setTimeout(() => { console.log("wide"); setTimeout(() => { console.log("world"); }, 1000); }, 1000); } 4
9. Callback Chaining Sync function verifyUser(username, password) { let userInfo = database.checkUser(username, password); let rolesInfo = database.getRoles(userInfo); database.logAccess(rolesInfo); return userInfo; } 5
10. Callback Chaining Sync function verifyUser(username, password) { let userInfo = database.checkUser(username, password); let rolesInfo = database.getRoles(userInfo); database.logAccess(rolesInfo); return userInfo; } Async — callback hell function verifyUser(username, password, callback) { database.checkUser(username, password, (error, userInfo) => { if (error) { callback(error); } else { database.getRoles(username, (error, roles) => { if (error) { callback(error); } else { database.logAccess(username, (error) => { if (error) { callback(error); } else { callback(null, userInfo); } }); } }); } }); } 5
11. Promises Async — promises function verifyUser(username, password) { let retUserInfo; return database.checkUser(username, password) .then(userInfo => { retUserInfo = userInfo; return database.getRoles(userInfo); }) .then(rolesInfo => database.logAccess(rolesInfo)) .then(_ignore => retUserInfo) } 6
12. Promises Async — promises function verifyUser(username, password) { let retUserInfo; return database.checkUser(username, password) .then(userInfo => { retUserInfo = userInfo; return database.getRoles(userInfo); }) .then(rolesInfo => database.logAccess(rolesInfo)) .then(_ignore => retUserInfo) } • Async functions return promises. – no callback parameter • Better than callback hell, but. . . – still based on callbacks – reinvents control flow, feels like different language 6
13. JavaScript async/await Async — async/await async function verifyUser(username, password) { const userInfo = await database.checkUser(username, password); const rolesInfo = await database.getRoles(userInfo); await database.logAccess(userInfo); return userInfo; } 7
14. JavaScript async/await Async — async/await async function verifyUser(username, password) { const userInfo = await database.checkUser(username, password); const rolesInfo = await database.getRoles(userInfo); await database.logAccess(userInfo); return userInfo; } • Automatically returns a promise. • Synchronous look and feel, async execution. • Native control flow. • How? 7
15. Python Async • asyncore — stdlib, deprecated since Python 3.6. • Twisted, Tornado, gevent — separate and incompatible event loops. • asyncio — new async for the standard library. – pluggable event loop – futures, coroutines • uvloop — drop-in replacement for asyncio. • curio, trio — coroutine-first. 8
16. Coroutine Intro Async function def verify_user(username, password): user_info = [-] database.check_user(username, password) roles_info = [-] database.get_roles(user_info) [-] database.log_access(roles_info) return user_info 9
17. Coroutine Intro Async function def verify_user(username, password): user_info = [-] database.check_user(username, password) roles_info = [-] database.get_roles(user_info) [-] database.log_access(roles_info) return user_info • Goal: use normal syntax to write async functions. • Idea: suspend execution at [-], resume when ready. • If we only had a way to magically suspend a function. . . 9
18. Coroutine Intro Async function def verify_user(username, password): user_info = yield database.check_user(username, password) roles_info = yield database.get_roles(user_info) yield database.log_access(roles_info) return user_info • Goal: use normal syntax to write async functions. • Idea: suspend execution at [-], resume when ready. • If we only had a way to magically suspend a function. . . • Let’s just use generators! 9
19. Coroutine Intro Async function def verify_user(username, password): user_info = yield database.check_user(username, password) roles_info = yield database.get_roles(user_info) yield database.log_access(roles_info) return user_info • Goal: use normal syntax to write async functions. • Idea: suspend execution at [-], resume when ready. • If we only had a way to magically suspend a function. . . • Let’s just use generators! • Problem: inner async call may need to suspend more than once — or not at all. • Problem: classic generators don’t return values. 9
20. Generators Rebooted yield from ITERABLE for _x in ITERABLE: yield _x 10
21. Generators Rebooted yield from ITERABLE for _x in ITERABLE: yield _x – splitting up plain generators – multiple suspensions in coroutines 10
22. Generators Rebooted yield from ITERABLE for _x in ITERABLE: yield _x – splitting up plain generators – multiple suspensions in coroutines VAR = yield from ITERABLE _it = iter(ITERABLE) while True: try: _x = next(_it) except StopIteration as e: VAR = e.value break else: yield _x 10
23. Generators Rebooted yield from ITERABLE for _x in ITERABLE: yield _x – splitting up plain generators – multiple suspensions in coroutines VAR = yield from ITERABLE _it = iter(ITERABLE) while True: try: _x = next(_it) except StopIteration as e: VAR = e.value break else: yield _x 10
24. Generators Rebooted yield from ITERABLE for _x in ITERABLE: yield _x – splitting up plain generators – multiple suspensions in coroutines VAR = yield from ITERABLE _it = iter(ITERABLE) while True: try: _x = next(_it) except StopIteration as e: VAR = e.value break else: yield _x • return v in a generator stops iteration and stores v in the StopIteration exception. • yield from gen retrieves the value returned by gen • Actual desugaring more complex to support send, throw, and close. 10
25. Generator Based Coroutines High-level def verify_user(username, password): user_info = yield from database.check_user(username, password) roles_info = yield from database.get_roles(user_info) yield from database.log_access(roles_info) return user_info 11
26. Generator Based Coroutines High-level def verify_user(username, password): user_info = yield from database.check_user(username, password) roles_info = yield from database.get_roles(user_info) yield from database.log_access(roles_info) return user_info • Coroutines yield from other coroutines. • When the inner coroutine suspends, so does the calling one, transparently. • yield from can be used anywhere inside the coroutine. 11
27. Generator Based Coroutines Low-level class Database: def get_roles(self, user_info): request = self._make_get_roles_request(user_info) yield from write(self._sock, request) resp = yield from readexactly(self._sock, _RESP_SIZE) roles_info = self._parse_get_roles_response(resp) return roles_info 12
28. Generator Based Coroutines Low-level class Database: def get_roles(self, user_info): request = self._make_get_roles_request(user_info) yield from write(self._sock, request) resp = yield from readexactly(self._sock, _RESP_SIZE) roles_info = self._parse_get_roles_response(resp) return roles_info OS-level def write(fd, data): nwritten = 0 while nwritten < len(data): try: nwritten += os.write(fd, data[nwritten:]) except BlockingIOError: yield WantWrite(fd) 12
29. Generator Based Coroutines Low-level class Database: def get_roles(self, user_info): request = self._make_get_roles_request(user_info) yield from write(self._sock, request) resp = yield from readexactly(self._sock, _RESP_SIZE) roles_info = self._parse_get_roles_response(resp) return roles_info OS-level def write(fd, data): nwritten = 0 while nwritten < len(data): try: nwritten += os.write(fd, data[nwritten:]) except BlockingIOError: yield WantWrite(fd) • OS-level coroutines suspend with yield. • Yielded object visible only to the event loop. • Entire call chain must be async. • Async coroutines do not use extended generators. 12
30. async/await Coroutine function async def verify_user(username, password): user_info = await database.check_user(username, password) roles_info = await database.get_roles(user_info) await database.log_access(roles_info) return user_info 13
31. async/await Coroutine function async def verify_user(username, password): user_info = await database.check_user(username, password) roles_info = await database.get_roles(user_info) await database.log_access(roles_info) return user_info • async def defines a coroutine function. • await synchronizes with another async operation. – only allowed in coroutines • await requires an awaitable, not a generator. – uses yield from under the hood, but generators now invisible – @types.coroutine makes a generator awaitable 13
32. Example: Parallel Download Threads import concurrent.futures import requests URLS = [ http://www.cnn.com/ , http://www.huffpost.com/ , http://europe.wsj.com/ , http://www.bbc.co.uk/ , http://failfailfail.com/ ] def load_url(url): try: with requests.get(url) as resp: content = resp.content print(f {url!r} is {len(content)} bytes ) except IOError: print(f failed to load {url} ) def main(): with concurrent.futures.ThreadPoolExecutor() as executor: futures = [executor.submit(load_url, url) for url in URLS] concurrent.futures.wait(futures) if __name__ == __main__ : main() 14
33. Example: Parallel Download Threads import concurrent.futures import requests URLS = [ http://www.cnn.com/ , http://www.huffpost.com/ , http://europe.wsj.com/ , http://www.bbc.co.uk/ , http://failfailfail.com/ ] def load_url(url): try: with requests.get(url) as resp: content = resp.content print(f {url!r} is {len(content)} bytes ) except IOError: print(f failed to load {url} ) def main(): with concurrent.futures.ThreadPoolExecutor() as executor: futures = [executor.submit(load_url, url) for url in URLS] concurrent.futures.wait(futures) if __name__ == __main__ : main() Async import asyncio import aiohttp URLS = [ http://www.cnn.com/ , http://www.huffpost.com/ , http://europe.wsj.com/ , http://www.bbc.co.uk/ , http://failfailfail.com/ ] async def load_url(url, session): try: async with session.get(url) as resp: content = await resp.read() print(f {url!r} is {len(content)} bytes ) except IOError: print(f failed to load {url} ) async def main(): async with aiohttp.ClientSession() as session: tasks = [load_url(url, session) for url in URLS] await asyncio.wait(tasks) if __name__ == __main__ : asyncio.run(main()) 14
34. Caveats • Calling a coroutine function just constructs an awaitable coroutine object. – await or nothing happens! • await doesn’t introduce parallelism. • No blocking or long-running code allowed. – but off-thread is ok • All coroutines run in a single thread. – caution with classic “async” APIs 15
35. Awaitable From the Ground Up Coroutine awaitable async def print_len(url, resp): content = await resp.read() print(f {url!r} is {len(content)} bytes ) 16
36. Awaitable From the Ground Up Coroutine awaitable async def print_len(url, resp): # CHALLENGE content = await resp.read() # write print_len without using print(f {url!r} is {len(content)} bytes ) # await or yield from 16
37. Awaitable From the Ground Up Coroutine awaitable async def print_len(url, resp): # CHALLENGE content = await resp.read() # write print_len without using print(f {url!r} is {len(content)} bytes ) # await or yield from • Learning exercise: write an awaitable! • print_len(...) must return object that defines __await__. • __await__ must return a running generator. 16
38. Awaitable With Generator Generator using yield from class print_len: def __init__(self, url, resp): self._url, self._resp = url, resp ... 17
39. Awaitable With Generator Generator using yield from class print_len: def __init__(self, url, resp): self._url, self._resp = url, resp def __await__(self): content = yield from self._resp.read().__await__() print(f {self._url!r} is {len(content)} bytes ) 17
40. Awaitable With Generator Generator using yield from class print_len: def __init__(self, url, resp): self._url, self._resp = url, resp def __await__(self): content = yield from self._resp.read().__await__() print(f {self._url!r} is {len(content)} bytes ) • Constructor only stores data. • awaitable.__await__() requests generator. 17
41. Awaitable With Plain Generator Plain generator class print_len: def __init__(self, url, resp): self._url, self._resp = url, resp def __await__(self): # content = yield from self._resp.read().__await__() ... 18
42. Awaitable With Plain Generator Plain generator class print_len: def __init__(self, url, resp): self._url, self._resp = url, resp def __await__(self): # content = yield from self._resp.read().__await__() it = iter(self._resp.read().__await__()) while True: try: _token = next(it) except StopIteration as e: content = e.value break else: yield _token print(f {self._url!r} is {len(content)} bytes ) 18
43. Awaitable With Plain Generator Plain generator class print_len: def __init__(self, url, resp): self._url, self._resp = url, resp def __await__(self): # content = yield from self._resp.read().__await__() it = iter(self._resp.read().__await__()) while True: try: _token = next(it) except StopIteration as e: content = e.value break else: yield _token print(f {self._url!r} is {len(content)} bytes ) • for not allowed — would swallow StopIteration. 18
44. Awaitable With Iterator Awaitable class class print_len: def __init__(self, url, resp): self._url, self._resp = url, resp def __await__(self): return _print_len_iter(self._url, self._resp) 19
45. Awaitable With Iterator Awaitable class class print_len: def __init__(self, url, resp): self._url, self._resp = url, resp def __await__(self): return _print_len_iter(self._url, self._resp) Iterator class class _print_len_iter: def __init__(self, url, resp): self._url, self._resp = url, resp self._state = 0 def __iter__(self): return self 19
46. Awaitable With Iterator Awaitable class class print_len: def __init__(self, url, resp): self._url, self._resp = url, resp def __await__(self): return _print_len_iter(self._url, self._resp) Iterator class class _print_len_iter: def __init__(self, url, resp): self._url, self._resp = url, resp self._state = 0 def __iter__(self): return self Iterator class, cont. def __next__(self): if self._state == 0: self._it = iter(self._resp.read().__await__()) self._state = 1 ... 19
47. Awaitable With Iterator Awaitable class class print_len: def __init__(self, url, resp): self._url, self._resp = url, resp def __await__(self): return _print_len_iter(self._url, self._resp) Iterator class class _print_len_iter: def __init__(self, url, resp): self._url, self._resp = url, resp self._state = 0 def __iter__(self): return self Iterator class, cont. def __next__(self): if self._state == 0: self._it = iter(self._resp.read().__await__()) self._state = 1 if self._state == 1: try: _token = next(self._it) except StopIteration as e: content = e.value else: return _token ... 19
48. Awaitable With Iterator Awaitable class class print_len: def __init__(self, url, resp): self._url, self._resp = url, resp def __await__(self): return _print_len_iter(self._url, self._resp) Iterator class class _print_len_iter: def __init__(self, url, resp): self._url, self._resp = url, resp self._state = 0 def __iter__(self): return self Iterator class, cont. def __next__(self): if self._state == 0: self._it = iter(self._resp.read().__await__()) self._state = 1 if self._state == 1: try: _token = next(self._it) except StopIteration as e: content = e.value else: return _token print(f {self._url!r} is {len(content)} bytes ) self._state = 2 if self._state == 2: raise StopIteration 19
49. Awaitable With Iterator Awaitable class class print_len: def __init__(self, url, resp): self._url, self._resp = url, resp def __await__(self): return _print_len_iter(self._url, self._resp) Iterator class class _print_len_iter: def __init__(self, url, resp): self._url, self._resp = url, resp self._state = 0 def __iter__(self): return self Iterator class, cont. def __next__(self): if self._state == 0: self._it = iter(self._resp.read().__await__()) self._state = 1 if self._state == 1: try: _token = next(self._it) except StopIteration as e: content = e.value else: return _token print(f {self._url!r} is {len(content)} bytes ) self._state = 2 if self._state == 2: raise StopIteration 19
50. Futher reading • Python Concurrency From the Ground Up https://www.youtube.com/watch?v=MCs5OvhV9S4 • Notes on structured concurrency, or: Go statement considered harmful https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/ 20