Cory Benfield - Building Protocol Libraries The Right Way

Slide 1

Slide 1 text

Building Protocol Libraries The Right Way

Slide 2

Slide 2 text

Hi!

Slide 3

Slide 3 text

@lukasaoz @lukasa

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

Python HTTP • Hyper • h2, hpack, priority, etc. • Requests + urllib3 • PyCA (PyOpenSSL)

Slide 6

Slide 6 text

Why All Your Libraries Are Garbage, And All But One Of Mine Are Too

Slide 7

Slide 7 text

Python HTTP Rocks

Slide 8

Slide 8 text

• Requests/urllib3/HTTPie • gunicorn • aiohttp • Twisted • Tornado

Slide 9

Slide 9 text

~0% Shared Code

Slide 10

Slide 10 text

Code Re-Use

Slide 11

Slide 11 text

I/O

Slide 12

Slide 12 text

• httplib: state machine and parser mixed with I/O • aiohttp/Tornado/Twisted: state machine and parser use concurrency primitives

Slide 13

Slide 13 text

Choice of I/O limits choice of libraries

Slide 14

Slide 14 text

So What?

Slide 15

Slide 15 text

Wasted Effort

Slide 16

Slide 16 text

Reduces Experimentation

Slide 17

Slide 17 text

Duplicate Bugs

Slide 18

Slide 18 text

Limits Optimisation

Slide 19

Slide 19 text

Avoiding This Problem

Slide 20

Slide 20 text

Don’t Do I/O

Slide 21

Slide 21 text

Alternative API events = handle_data(in_bytes) out_bytes = perform_action()

Slide 22

Slide 22 text

Not All-Purpose

Slide 23

Slide 23 text

Build Wrappers

Slide 24

Slide 24 text

Why Bother?

Slide 25

Slide 25 text

===================== test session starts ====================== ------- coverage: platform darwin, python 2.7.11-final-0 ------- Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------------- h2/__init__.py 2 0 0 0 100% h2/connection.py 505 0 120 0 100% h2/errors.py 17 0 0 0 100% h2/events.py 107 0 4 0 100% h2/exceptions.py 41 0 0 0 100% h2/frame_buffer.py 55 0 18 0 100% h2/settings.py 79 0 26 0 100% h2/stream.py 356 0 76 0 100% h2/utilities.py 76 0 44 0 100% ---------------------------------------------------------------- TOTAL 1238 0 288 0 100% ================== 894 passed in 6.87 seconds ==================

Slide 26

Slide 26 text

I/O Is Application- Specific

Slide 27

Slide 27 text

Build A Toolbox

Slide 28

Slide 28 text

Examples • hyper-h2, a HTTP/2 protocol stack. • http://python-hyper.org/h2 • h11, for HTTP/1.1 • https://github.com/njsmith/h11