High performance networking in asyncio

The talk focuses on writing efficient protocols for asyncio and uvloop.

Transcript

1. High-performance
 networking in Python Yury Selivanov EuroPython 2016
2. Yury Selivanov – @1st1 About me • Co-founder of MagicStack (http://magic.io); check our company out! • Avid Python user since 2008. • CPython core developer since 2013.
3. Yury Selivanov – @1st1 What I do for Python • Co-authored and implemented PEP 362:
 inspect.signature API. • Authored and implemented PEP 492:
 async/await syntax. • Co-maintainer of asyncio with Guido and Victor Stinner. • Created uvloop.
4. Yury Selivanov – @1st1 Structure of this talk • Overview of async/await. • asyncio and uvloop. • Sockets, Protocols or Streams? • Learn by example: high-performance PostgreSQL driver. • Recap.
5. Performance is hard.
6. Part I async/await
7. Yury Selivanov – @1st1 One obvious way to do it • Callbacks and Deferred. • Stackless Python & greenlets. • Coroutines using generators and yield. • Coroutines using generators and yield from. • And now we have async/await.
8. Yury Selivanov – @1st1 • Dedicated syntax; concise and readable. • New builtin type for coroutines. • New concepts: async for and async with. • Generic, framework agnostic design. • Fast: only ~2x slower than a function call. Why async/await is the answer
9. Yury Selivanov – @1st1 async/await • Subtype of generators; shares a lot of code. • await —> yield from —> YIELD_FROM opcode. • @types.coroutine. • __await__. • __aenter__, __aexit__, __aiter__, __anext__.
10. Part II asyncio, libuv, cython, uvloop
11. Yury Selivanov – @1st1 asyncio • Developed and actively maintained by the BDFL. • A toolbox for frameworks and protocols. • Part of Standard Library.
12. Yury Selivanov – @1st1 asyncio: what’s inside? • Standardized pluggable event loop. • Interfaces for Protocols and Transports. • Factories for servers and connections; streams. • Futures and Tasks: callbacks + coroutines, timeouts, cancellation, etc. • Subprocess, queues, synchronisation primitives.
13. Yury Selivanov – @1st1 Event Loop • Event loop is the foundation of asyncio. • Factories for Tasks and Futures. • IO multiplexor. • Low level APIs for timed events, scheduling callbacks, subprocesses and signals. • And… you can replace it.
14. Yury Selivanov – @1st1 github.com/magicstack/uvloop • 99.(9)% compatible asyncio event loop. • Written in Cython. • Uses libuv under the hood. • Fast Tasks and Futures = faster async/await. • Super fast IO.
15. Yury Selivanov – @1st1 How fast is uvloop? 2-4x faster than asyncio.
16. Yury Selivanov – @1st1 OK, it’s faster than asyncio…
17. Just use uvloop Thank you for your time. Questions? It’s not that easy, unfortunately…
18. Part III Sockets, Streams and Protocols
19. Yury Selivanov – @1st1 One obvious way to do it: episode 2 • Low level sock_sendall, sock_recv, sock_connect. • High level StreamReader and StreamWriter. • Low level Protocols and Transports.
20. Yury Selivanov – @1st1 loop.sock_* methods
21. Yury Selivanov – @1st1 Streams
22. Yury Selivanov – @1st1 Protocols and Transports
23. Yury Selivanov – @1st1 Downsides • loop.sock_*: loop cannot buffer data for you. 
 No flow control. • Streams: easy to use, but too generic. • Protocols & Transports: say hello to callbacks.
24. Flow Control.
25. Yury Selivanov – @1st1 So which API should I use? • Use loop.sock_* for easy porting synchronous code to asyncio. Better consider using Streams. • Use Streams for implementing protocols with async/await. • Use Protocols and Transports for performance.
26. Yury Selivanov – @1st1 Let’s focus on Protocols. • loop pushes data to Protocols. • Protocols send data back using Transports. • Protocols can implement specialized read and write buffers. • Protocols can do flow-control. • Full control over how IO is performed.
27. Yury Selivanov – @1st1 Protocols & async/await • Strategy #1: develop custom streaming abstraction tailored for the concrete protocol and use async/await.
 
 A good example is aiohttp.
28. Yury Selivanov – @1st1 Protocols & async/await • Strategy #2: Implement protocol parsing and logic using callbacks. 
 
 Hide that under an easy to use async/await facade.
29. Part IV asyncpg
30. Yury Selivanov – @1st1 github.com/magicstack/asyncpg • The fastest PostgreSQL driver for asyncio. 
 And for Python in general. • Open-sourced 2 hours ago! • Written from scratch in Cython and Python. 
 Does not use libpq.
31. Yury Selivanov – @1st1 github.com/magicstack/asyncpg • Uses Postgres binary data format. • That means more efficient encoding of data -> less network traffic -> faster IO. • Binary means much faster parsing. Always prefer binary. • Not all Postgres types can be unambiguously parsed 
 when transferred as text.
32. Yury Selivanov – @1st1 • Ditches DB API. • The idea: let’s build an efficient driver for Postgres. Use its features to the max. • Supports all built-in Postgres types; composite and custom types. github.com/magicstack/asyncpg
33. Yury Selivanov – @1st1 • Server loves prepared statements. • So we use them extensively. • Each connection has an LRU cache of prepared statements. • We dynamically build and cache pipelines of data encoders and decoders for each statement. github.com/magicstack/asyncpg
34. Yury Selivanov – @1st1 Did I say asyncpg is fast?
35. Yury Selivanov – @1st1 asyncpg architecture • Protocol is implemented in two classes:
 CoreProtocol and Protocol(CoreProtocol). • CoreProtocol class knows almost nothing about asyncio. • Protocol class is the bridge between the world of callbacks and async/await.
36. Yury Selivanov – @1st1 Parsing PostgreSQL protocol • Naïve approach: use Python bytes type and/or memoryviews. • Problem: the driver will spend a lot of time allocating and deallocating memory. The performance will suffer.
37. Yury Selivanov – @1st1 Parsing PostgreSQL protocol • Solution: use Cython to work with ‘char*’; 
 create high-level Pythonic buffers for that. • asyncpg has three types of buffers!
38. Yury Selivanov – @1st1 ReadBuffer
39. Yury Selivanov – @1st1 asyncpg & async/await • High level logic and API is built in pure Python 
 with async/await.
40. Part V Recap
41. Yury Selivanov – @1st1 Don’t be afraid of Protocols • Use asyncio Protocols and Transports to create high performance protocols. • Use Cython for low level code. It’s amazing. • async/await should always be the preferred public API. • Once you have fast drivers, your application will work fast. Use async/await.
42. Yury Selivanov – @1st1 loop.create_future() • Use `loop.create_future()` instead of `asyncio.Future(loop=loop)`. • Added in Python 3.5.2. • Allows uvloop to inject high performance Future implementation in your code.
43. Yury Selivanov – @1st1 Binary Protocols • Always use binary protocols. • Less traffic between server and clients, 
 faster parsing.
44. Yury Selivanov – @1st1 Profiling and Benchmarking • Always profile your code. Always. • Cython code can be profiled with valgrind;
 results visualized with KCachegrind. • Use `cython -a` option to generate HTML output for your Cython code.
45. Yury Selivanov – @1st1 Zero-copy • Implement custom buffer classes. • Prefer C data types; working with Python 
 bytes is expensive. • Have efficient buffer to build requests and one buffer with ‘transport.write’. Or use multiple buffers and a single ‘transport.writelines’.
46. Yury Selivanov – @1st1 TCP options • When you have no control over how transport.write is called, use TCP_CORK option to buffer writes and send as few TCP packets as possible. • Use TCP_NODELAY to send data as soon as you call ‘transport.write’.
47. Yury Selivanov – @1st1 Timeouts • Implement timeouts as part of your APIs. • ‘asyncio.wait_for’ has a lot of overhead; use ‘loop.call_later’ and build custom cancellation logic at the Protocol level.
48. That’s it. Thank you. Questions?