Cory Benfield - Building Protocol Libraries The Right Way

One of the great strengths of the Python ecosystem is the enormous collection of powerful, flexible libraries. However, these libraries tend to suffer from one extremely common design flaw that mean that the work done is not easily re-usable or transferable. In this talk, we talk about how to build libraries that can be used as widely as possible, through the lens of the Python Hyper HTTP project.

https://us.pycon.org/2016/schedule/presentation/1743/

Transcript

1. Building Protocol Libraries The Right Way
2. Hi!
3. @lukasaoz @lukasa
5. Python HTTP • Hyper • h2, hpack, priority, etc. • Requests + urllib3 • PyCA (PyOpenSSL)
6. Why All Your Libraries Are Garbage, And All But One Of Mine Are Too
7. Python HTTP Rocks
8. • Requests/urllib3/HTTPie • gunicorn • aiohttp • Twisted • Tornado
9. ~0% Shared Code
10. Code Re-Use
11. I/O
12. • httplib: state machine and parser mixed with I/O • aiohttp/Tornado/Twisted: state machine and parser use concurrency primitives
13. Choice of I/O limits choice of libraries
14. So What?
15. Wasted Effort
16. Reduces Experimentation
17. Duplicate Bugs
18. Limits Optimisation
19. Avoiding This Problem
20. Don’t Do I/O
21. Alternative API events = handle_data(in_bytes) out_bytes = perform_action()
22. Not All-Purpose
23. Build Wrappers
24. Why Bother?
25. ===================== test session starts ====================== ------- coverage: platform darwin, python 2.7.11-final-0 ------- Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------------- h2/__init__.py 2 0 0 0 100% h2/connection.py 505 0 120 0 100% h2/errors.py 17 0 0 0 100% h2/events.py 107 0 4 0 100% h2/exceptions.py 41 0 0 0 100% h2/frame_buffer.py 55 0 18 0 100% h2/settings.py 79 0 26 0 100% h2/stream.py 356 0 76 0 100% h2/utilities.py 76 0 44 0 100% ---------------------------------------------------------------- TOTAL 1238 0 288 0 100% ================== 894 passed in 6.87 seconds ==================
26. I/O Is Application- Specific
27. Build A Toolbox
28. Examples • hyper-h2, a HTTP/2 protocol stack. • http://python-hyper.org/h2 • h11, for HTTP/1.1 • https://github.com/njsmith/h11
29. Help Us