Asynchronous Programming in Python

A lite introduction to asynchronous programming in Python using asyncio standard library.

https://www.meetup.com/HelPy-meetups/events/263798711/

Transcript

1. Lite Intro to Asynchronous Programming in Python @AsherShekhamis
3. Synchronous vs. Asynchronous • Blocking (busy, but not doing anything useful). • One task must finish execution before the other can start. • Tasks are dependent. • Used for pretty much anything; OG. • Non-blocking. • One task can start execution before the other is finished. • Tasks are independent. • Suitable for I/O, networking...etc.
4. Asynchronous != Multithreaded Synchronous/Asynchronous has nothing to do with threads. A machine with single CPU and a single thread can still execute tasks asynchronously (seemingly in parallel). However, it’s unusual to find async tasks running on the same thread.
5. Single threaded vs. Multi-threaded Synchronous Asynchronous
6. Why not using threads? • Don’t scale well; context switches. • Number of sockets. • Synchronization is hard to achieve. • Race conditions. • Unpredictable scheduling. • Deadlock. • Resource starvation. “Threads are the `goto` of our generation” - Someone
7. Python’s Coroutines • Twisted. • Stackless Python and greenlets. • Python 2.5: coroutines with `yield` • Python 3.3: coroutines with `yield from` • Python 3.5: `async/await` History
8. Using `yield` - Twisted @inlineCallbacks def some_fun(): returned_deferred = yield makeRequestReturningDeferred() if returned_deferred == 'This is a Deferred': returnValue('Yep!') else: raise Exception('Nooooooooo!')
9. Using `yield from` - asyncio @asyncio.coroutine def async_sum(x, y): yield from asyncio.sleep(1.0) return x + y
10. Problems when refactoring `yield from` def async_sum(x, y): yield from asyncio.sleep(1.0) return x + y def async_sum(x, y): return x + y
11. Using `async/await` async def sum(x, y): await asyncio.sleep(1.0) return x + y async def sum(x, y): return x + y Cleaner and more readable. Decorator free.
12. Using `async/await` • Readability; concise and dedicated syntax • Framework agnostic design; generic • Built-in coroutines; no additives • `async for` and `async with` • Better performance than `yield` statement; 10 - 100 times faster
13. What is asyncio? Module; part of the standard library. Pluggable event-loop. Collection of tools for other frameworks to use; no http implementation, no database drivers...etc. Interfaces, factories, futures, tasks, subprocess. Simple and easy to learn. Really tho!
14. Everyday asyncio • asyncio.get_event_loop() • asyncio.create_task(), asyncio.current_task(), and asyncio.all_tasks() • loop.run_forever() and loop.run_until_complete() • asyncio.wait() and asyncio.gather() • loop.run_in_executor() • loop.close() • asyncio.set_debug() Unless you’re building a framework
15. asyncio and async/await out there import asyncio import asyncpg async def run(): conn = await asyncpg.connect(user='user', password='password',database='database', host='127.0.0.1') values = await conn.fetch('''SELECT * FROM mytable''') await conn.close() loop = asyncio.get_event_loop() loop.run_until_complete(run()) DB interface library `asyncpg`
16. Sync to Async using asyncio and async/await syntax
17. Asyncio event loop alternatives • A fast, drop-in replacement of the built-in asyncio event loop. • Cython • libuv under the hood. • Faster compared to asyncio; wins performance-wise. uvloop
18. Thanks! Questions?