www.ska.ac.za PYTHON ASYNCIO A SHORT INTRODUCTION PyCon 8 Bruce Merry www.ska.ac.za 

www.ska.ac.za Outline • Concurrent programming • First look at asyncio • Futures and Tasks • Cancellation • Blocking code • Testing • Debugging 

www.ska.ac.za Outline • Concurrent programming • First look at asyncio • Futures and Tasks • Cancellation • Blocking code • Testing • Debugging 

www.ska.ac.za The trouble with threads What does this code output? import concurrent.futures a = 0 def incr(): global a 6 for i in range(1000000): a += 1 8 executor = concurrent.futures.ThreadPoolExecutor() futures = [executor.submit(incr) for i in range(4)] concurrent.futures.wait(futures) print(a) 

www.ska.ac.za The trouble with threads What does this code output? import concurrent.futures a = 0 def incr(): global a 6 for i in range(1000000): a += 1 8 executor = concurrent.futures.ThreadPoolExecutor() futures = [executor.submit(incr) for i in range(4)] concurrent.futures.wait(futures) print(a) 688 

www.ska.ac.za More trouble with threads While connection is open: . Read a request . Process request . Send reply 

www.ska.ac.za More trouble with threads While connection is open: . Read a request . Process request . Send reply Stop the world, I want to get off! 

www.ska.ac.za Concurrent programming Concurrency is about dealing with lots of things at once. Parallelism is about doing lots of things at once. — Rob Pike 

www.ska.ac.za Concurrent programming Concurrency is about dealing with lots of things at once. Parallelism is about doing lots of things at once. — Rob Pike • Multiple tasks active 

www.ska.ac.za Concurrent programming Concurrency is about dealing with lots of things at once. Parallelism is about doing lots of things at once. — Rob Pike • Multiple tasks active • Only progress one at a time 

www.ska.ac.za Concurrent programming Concurrency is about dealing with lots of things at once. Parallelism is about doing lots of things at once. — Rob Pike • Multiple tasks active • Only progress one at a time • Only switch tasks at known points (typically I/O) 

www.ska.ac.za Concurrent programming Concurrency is about dealing with lots of things at once. Parallelism is about doing lots of things at once. — Rob Pike • Multiple tasks active • Only progress one at a time • Only switch tasks at known points (typically I/O) • Better scalability 

www.ska.ac.za Concurrent programming Concurrency is about dealing with lots of things at once. Parallelism is about doing lots of things at once. — Rob Pike • Multiple tasks active • Only progress one at a time • Only switch tasks at known points (typically I/O) • Better scalability • Easier to reason about 

www.ska.ac.za Outline • Concurrent programming • First look at asyncio • Futures and Tasks • Cancellation • Blocking code • Testing • Debugging 

www.ska.ac.za Example time import asyncio import aiohttp async def grab(session, url): resp = await session.get(url) 6 text = await resp.text() return text.splitlines()[0] 8 async def grab_urls(urls): session = aiohttp.ClientSession() coros = [grab(session, url) for url in urls] result = await asyncio.gather(*coros) await session.close() return result 6 loop = asyncio.get_event_loop() work = grab_urls([’https://www.python.org/’, ’http://www.ska.ac.za’]) 8 print(loop.run_until_complete(work)) 

www.ska.ac.za Support in Python versions Active development — use the newest Python you can . Provisional support, based on generators, no async / await . Adds async and await .6 API stable, numerous small additions (with backports to . ) . More small improvements, async / await are now true keywords 

www.ska.ac.za About this talk × Every API call × All the syntax 

www.ska.ac.za About this talk × Every API call × All the syntax Concepts Practical tips 

www.ska.ac.za The asyncio onion Event loop Future The rest 

www.ska.ac.za Event loop Minimal low-level API • call_soon, call_soon_threadsafe • call_later, call_at • add_reader, add_writer • add_signal_handler • run_forever, run_until_complete • Assorted networking/socket functions 

www.ska.ac.za Event loop Minimal low-level API • call_soon, call_soon_threadsafe • call_later, call_at • add_reader, add_writer • add_signal_handler • run_forever, run_until_complete • Assorted networking/socket functions Can be replaced by an alternative implementation 

www.ska.ac.za Continuation style Painful way to code def step1(loop): do_stuff() loop.call_later(1, step2) def step2(loop): 6 do_more_stuff() 

www.ska.ac.za Continuation style Painful way to code def step1(loop): do_stuff() loop.call_later(1, step2) def step2(loop): 6 do_more_stuff() versus async def run(): do_stuff() await asyncio.sleep(1) do_more_stuff() 

www.ska.ac.za Outline • Concurrent programming • First look at asyncio • Futures and Tasks • Cancellation • Blocking code • Testing • Debugging 

www.ska.ac.za Futures The future is already here — it’s just not very evenly distributed. — William Gibson When an expression is given to the evaluator by the user, a future for that expression is returned which is a promise to deliver the value of that expression at some later time... — Baker and Hewitt, The Incremental Garbage Collection of Processes 

www.ska.ac.za Futures The future is already here — it’s just not very evenly distributed. — William Gibson When an expression is given to the evaluator by the user, a future for that expression is returned which is a promise to deliver the value of that expression at some later time... — Baker and Hewitt, The Incremental Garbage Collection of Processes A future • Provides storage for a result (or exception) • Allows the result to be waited for with await 

www.ska.ac.za Future example A simple implementation of asyncio.sleep async def my_sleep(delay): loop = asyncio.get_event_loop() future = loop.create_future() loop.call_later(delay, future.set_result, None) await future 6 # Usage: 8 await my_sleep(1) 

www.ska.ac.za Tasks Tasks are coroutines executing concurrently async def my_coroutine(): ... async def run_concurrent(): task = loop.create_task(my_coroutine()) 6 ... result = await task 

www.ska.ac.za What the What is a What? Coroutine func Awaitable Coroutine Future Task () create_task ensure_future ensure_future subclass create 

www.ska.ac.za Outline • Concurrent programming • First look at asyncio • Futures and Tasks • Cancellation • Blocking code • Testing • Debugging 

www.ska.ac.za Simple server example async def do_connection(reader, writer): try: while True: line = await reader.readline() if not line: 6 break # Connection was closed writer.write(await process(line)) 8 finally: writer.close() 

www.ska.ac.za Simple server example async def do_connection(reader, writer): try: while True: line = await reader.readline() if not line: 6 break # Connection was closed writer.write(await process(line)) 8 finally: writer.close() loop = asyncio.get_event_loop() task = loop.create_task(do_connection(reader, writer)) ... await task 

www.ska.ac.za Server shutdown async def do_connection(reader, writer): try: while True: line = await reader.readline() if not line: 6 break # Connection was closed writer.write(await process(line)) 8 finally: writer.close() task.cancel() 

www.ska.ac.za Server shutdown async def do_connection(reader, writer): try: while True: line = await reader.readline() if not line: 6 break # Connection was closed writer.write(await process(line)) 8 finally: writer.close() task.cancel() CancelledError 

www.ska.ac.za Catching cancellation async def do_connection(reader, writer): try: while True: line = await reader.readline() if not line: 6 break # Connection was closed writer.write(await process(line)) 8 except asyncio.CancelledError: writer.write(b'Server shutting down\n') raise finally: writer.close() 

www.ska.ac.za Anti-pattern: Futures as Events async def eat_sandwich(): await lunchtime_future mouth.insert("sandwich") async def drink_coffee(): 6 await lunchtime_future mouth.insert("coffee") 8 def finish_talk(): lunchtime_future.set_result(None) 

www.ska.ac.za Anti-pattern: Futures as Events async def eat_sandwich(): await lunchtime_future mouth.insert("sandwich") async def drink_coffee(): 6 await lunchtime_future mouth.insert("coffee") 8 def finish_talk(): lunchtime_future.set_result(None) What if eat_sandwich is cancelled? 

www.ska.ac.za Anti-pattern: Futures as Events async def eat_sandwich(): await lunchtime_future mouth.insert("sandwich") async def drink_coffee(): 6 await lunchtime_future mouth.insert("coffee") 8 def finish_talk(): lunchtime_future.set_result(None) What if eat_sandwich is cancelled? Use a asyncio.Event instead. 

www.ska.ac.za Timeouts Don’t add timeout arguments to your async APIs. The caller can cancel. try: await asyncio.wait_for(fetch(), timeout=10) except asyncio.TimeoutError: ... 

www.ska.ac.za Timeouts Don’t add timeout arguments to your async APIs. The caller can cancel. try: await asyncio.wait_for(fetch(), timeout=10) except asyncio.TimeoutError: ... Or, with async_timeout library, try: with async_timeout.timeout(10): await fetch('thing1') await fetch('thing2') except asyncio.TimeoutError: 6 ... 

www.ska.ac.za Outline • Concurrent programming • First look at asyncio • Futures and Tasks • Cancellation • Blocking code • Testing • Debugging 

www.ska.ac.za Blocking calls Main disadvantage of asyncio versus threads • Blocking I/O blocks the entire event loop... 

www.ska.ac.za Blocking calls Main disadvantage of asyncio versus threads • Blocking I/O blocks the entire event loop... • ...even if it drops the GIL 

www.ska.ac.za Solutions • Just accept it 

www.ska.ac.za Solutions • Just accept it • Run blocking code on another thread 

www.ska.ac.za Solutions • Just accept it • Run blocking code on another thread • Migrate to an async library 

www.ska.ac.za Dispatching to a thread asyncio makes it easy: await loop.run_in_executor(executor, func, arg) 

www.ska.ac.za Dispatching to a thread asyncio makes it easy: await loop.run_in_executor(executor, func, arg) But there are caveats • Cancellation won’t interrupt execution • func can’t directly use the event loop • All the usual thread safety pitfalls 

www.ska.ac.za Async libraries aioamqp aiobotocore aiodns aiodocker aioes / aioelasticsearch aioetcd aiofiles aiohttp aiokafka aiomcache / aiomemcache aiomysql aioodbc aiopg aioprocessing aiopyramid aioredis / asyncio-redis aiosqlite aiozmq aiozipkin asyncio-mongo 

www.ska.ac.za Outline • Concurrent programming • First look at asyncio • Futures and Tasks • Cancellation • Blocking code • Testing • Debugging 

www.ska.ac.za unittest/nosetests Without asyncio import unittest class MyTest(unittest.TestCase): def setUp(self): ... 6 def tearDown(self): 8 ... def testThing(self): ... 

www.ska.ac.za unittest/nosetests With asyncio plus asynctest import asynctest class MyTest(asynctest.TestCase): async def setUp(self): ... 6 async def tearDown(self): 8 ... async def testThing(self): ... 

www.ska.ac.za asynctest.ClockedTestCase Test time-dependent code e.g. timeouts import asyncio, async_timeout, asynctest async def times_out(): with async_timeout.timeout(10): await asyncio.sleep(20) 6 class TestTimeout(asynctest.ClockedTestCase): 8 async def test_times_out(self): task = self.loop.create_task(times_out()) await self.advance(11) self.assertTrue(task.done()) # Avoid hanging forever with self.assertRaises(asyncio.TimeoutError): await task 

www.ska.ac.za pytest Without asyncio import subprocess def test_it(): proc = subprocess.Popen(['/bin/false']) 6 returncode = proc.wait() assert returncode == 1 

www.ska.ac.za pytest With asyncio and pytest-asyncio import asyncio, pytest @pytest.mark.asyncio async def test_it(): proc = await asyncio.create_subprocess_exec('/bin/false') 6 returncode = await proc.wait() assert returncode == 1 

www.ska.ac.za Outline • Concurrent programming • First look at asyncio • Futures and Tasks • Cancellation • Blocking code • Testing • Debugging 

www.ska.ac.za Debugging To catch common mistakes: • Set PYTHONASYNCIODEBUG=1 in environment • Enable debug logging 

www.ska.ac.za An easy mistake async def double(x): print('double called with', x) return 2 * x async def main(): 6 y = double(2) # Oops! print('Double 2 is', y) 

www.ska.ac.za An easy mistake async def double(x): print('double called with', x) return 2 * x async def main(): 6 y = double(2) # Oops! print('Double 2 is', y) Double 2 is 

www.ska.ac.za An easy mistake Now with PYTHONASYNCIODEBUG=1: Double 2 is was never yielded from Coroutine object created at (most recent call last): File "code/debugmode.py", line 10, in asyncio.get_event_loop().run_until_complete(main()) ... File "/usr/lib/python3.5/asyncio/tasks.py", line 239, in _step result = coro.send(None) File "code/debugmode.py", line 6, in main y = double(2) # Oops! File "/usr/lib/python3.5/asyncio/coroutines.py", line 80, in debug_wrapper return CoroWrapper(gen, None) 

www.ska.ac.za Blocking the event loop import time async def sleepy(): print(time.strftime("%H:%M:%S")) time.sleep(2) 6 print(time.strftime("%H:%M:%S")) 19:50:45 19:50:47 

www.ska.ac.za Blocking the event loop With PYTHONASYNCIODEBUG=1: 19:50:45 19:50:47 Executing result=None created at /usr/lib/python3.5/asyncio/base_events.py:367> took 2.002 seconds 

www.ska.ac.za Fire-and-forget tasks import signal, asyncio async def run_server(): raise NotImplementedError # oops 6 def shutdown(): task.cancel() 8 loop.stop() loop = asyncio.get_event_loop() task = loop.create_task(run_server()) loop.add_signal_handler(signal.SIGINT, shutdown) loop.run_forever() loop.close() 

www.ska.ac.za Uncaught exception Exception seen during shutdown (even without debug mode) ^CTask exception was never retrieved future: exception=NotImplementedError()> Traceback (most recent call last): File "/usr/lib/python3.5/asyncio/tasks.py", line 239, in _step result = coro.send(None) File "code/debugmode-nocatch.py", line 4, in run_server raise NotImplementedError # oops NotImplementedError 

www.ska.ac.za Catch and log exceptions import signal, asyncio, logging async def run_server(): try: raise NotImplementedError # oops 6 except Exception: logging.exception('Server failed') 8 def shutdown(): task.cancel() loop.stop() loop = asyncio.get_event_loop() task = loop.create_task(run_server()) loop.add_signal_handler(signal.SIGINT, shutdown) 6 loop.run_forever() loop.close() 

www.ska.ac.za aiomonitor TCP server that lets you inspect running tasks monitor >>> help Commands: ps : Show task table where taskid : Show stack frames for a task cancel taskid : Cancel an indicated task signal signame : Send a Unix signal stacktrace : Print a stack trace from the event loop thread console : Switch to async Python REPL quit : Leave the monitor 

www.ska.ac.za aiomonitor TCP server that lets you inspect running tasks monitor >>> ps +-----------------+----------+----------------------------------------------------------------------------------+ | Task ID | State | Task | +-----------------+----------+----------------------------------------------------------------------------------+ | 140592037503664 | PENDING | | | | | wait_for= cb=[Connection._done_callback()]> | | 140592037504504 | PENDING | | | | | wait_for= cb=[_run_until_complete_cb() at | | | | /usr/lib/python3.5/asyncio/base_events.py:164]> | | 140592037504784 | FINISHED | | | | | result=]>> | | 140592037591416 | PENDING | | | | | wait_for=> | | 140592037759128 | FINISHED | | | | | result=None> | | 140592038199872 | PENDING | wait_for=> | | 140592038202784 | PENDING | .cleanup() running | | | | at /home/kat/ve3/lib/python3.5/site-packages/aiokatcp/server.py:494> | | | | wait_for=> | +-----------------+----------+----------------------------------------------------------------------------------+ 

www.ska.ac.za aiomonitor TCP server that lets you inspect running tasks monitor >>> where 140592038199872 Stack for

www.ska.ac.za aioconsole TCP server running an async REPL: Python 3.5.2 (default, Nov 23 2017, 16:37:01) [GCC 5.4.0 20160609] on linux Type "help", "copyright", "credits" or "license" for more information. --- This console is running in an asyncio event loop. It allows you to wait for coroutines using the 'await' syntax. Try: await asyncio.sleep(1, result=3) --- >>> 

www.ska.ac.za SARAO, a business unit of the National Research Foundation. The South African Radio Astronomy Observatory (SARAO) spearheads South Africa’s activities in the Square Kilometre Array Radio Telescope, commonly known as the SKA, in engineering, science and construction. SARAO is a National Facility managed by the National Research Foundation and incorporates radio astronomy instruments and programmes such as the MeerKAT and KAT- telescopes in the Karoo, the Hartebeesthoek Radio Astronomy Observatory (HartRAO) in Gauteng, the African Very Long Baseline Interferometry (AVN) programme in nine African countries as well as the associated human capital development and commercialisation endeavours. Contact information Bruce Merry Senior Science Processing Developer Email: [email protected]