Why Twisted Is The Best (WOOTConf @ LCA 2017)

Twisted's pretty good.

Transcript

1. Why Twisted Is The Best (and how it will make your application awesome) KatieConf/WOOTConf @ linux.conf.au
2. Hello, I’m Amber Brown (HawkOwl)
3. @hawkieowl atleastfornow.net
4. I live in Melbourne, Victoria
5. Twisted
7. @hawkieowl "Why Twisted Is The Best" Release Manager for: 13.2, 14.0, 15.0, 15.1, 15.2, 15.3, 15.4, 15.5, 16.0, 16.1, 16.2, 16.3, 16.4, 16.5, 16.6, 16.7
8. @hawkieowl "Why Twisted Is The Best" Framework (written in) Python 2+3 (for) concurrent networking
9. @hawkieowl "Why Twisted Is The Best" "Standard" Web App Client Client Client Client Client Client
10. @hawkieowl "Why Twisted Is The Best" WSGI TCP "Standard" Web App Client Client Client Client Client Client
11. @hawkieowl "Why Twisted Is The Best" TCP Twisted Application Client Client Client Client Client Client
12. @hawkieowl "Why Twisted Is The Best" How?
13. @hawkieowl "Why Twisted Is The Best" ✨ Operating System ✨ ✨ I/O Primitives! ✨
14. @hawkieowl "Why Twisted Is The Best" 1. Twisted keeps a list of connections
 your app is interested in Connection 1 Connection 2 Connection 3 Your App
15. @hawkieowl "Why Twisted Is The Best" 2. Twisted asks the OS which connections
 have data ready, or can take more data select([1, 2, 3]) Connection 1 Connection 2 Connection 3 Your App
16. @hawkieowl "Why Twisted Is The Best" 3. The OS responds with the ones that are. { "readable": [1], "writable: [2, 3] } Connection 1 Connection 2 Connection 3 Your App
17. @hawkieowl "Why Twisted Is The Best" 4. Twisted recieves data and notifies
 your app, which processes the data. dataRecieved("how are you?") Connection 1 Connection 2 Connection 3 Your App
18. @hawkieowl "Why Twisted Is The Best" 5. Your application can send data back on the connection, which Twisted will give the OS. transport.send("good!") socket.send(1, "good!") Connection 1 Connection 2 Connection 3 Your App
19. @hawkieowl "Why Twisted Is The Best" Connection 1 Connection 2 Connection 3 6. GOTO 1 Your App
20. @hawkieowl "Why Twisted Is The Best" Your Application (is implemented as) Protocols (connected by) Transports
21. @hawkieowl "Why Twisted Is The Best" All coordinated by a Reactor (or "event loop")
22. @hawkieowl "Why Twisted Is The Best" Transport Operating System Network Protocol Business Logic
23. @hawkieowl "Why Twisted Is The Best" Transport Operating System Network Protocol Business Logic
24. Protocols
25. @hawkieowl "Why Twisted Is The Best" Protocols implement and parse a network protocol
26. @hawkieowl "Why Twisted Is The Best" Created by a Factory
27. @hawkieowl "Why Twisted Is The Best" Factory Listening Socket Protocol Instance Transport Connected Socket Protocol Instance Transport Connected Socket Protocol Instance Transport Connected Socket
28. @hawkieowl "Why Twisted Is The Best" Receives and sends data Decodes the data into useful events Handles the connection lifecycle
29. @hawkieowl "Why Twisted Is The Best" Data is what is sent over the "wire"
30. @hawkieowl "Why Twisted Is The Best" Data is sent to a Transport A Transport may get it there in different ways
31. @hawkieowl "Why Twisted Is The Best" Base Transports: UDP, TCP Transport Protocols: TLS/SSL, SSH, Tor
32. @hawkieowl "Why Twisted Is The Best" Events are what your application acts on
33. @hawkieowl "Why Twisted Is The Best" Protocol consumes a full HTTP request, tells your application about it
34. @hawkieowl "Why Twisted Is The Best" Creation Connection Operation Disconnection Destruction
35. Out-of-the-box Protocols
36. @hawkieowl "Why Twisted Is The Best" HTTP (1.0, 1.1, 2.0) SMTP, ESMTP, IMAP, POP3 DNS, SSH, Telnet, TLS ...and a bunch more
37. @hawkieowl "Why Twisted Is The Best" Web Server DNS Server Email Server SSH Server
38. Building an Application
39. @hawkieowl "Why Twisted Is The Best" Let's build a website!
40. @hawkieowl "Why Twisted Is The Best" from klein import Klein app = Klein() @app.route("/") def main(self, request): return b"Hello!!!!" app.run('0.0.0.0', 8080)
41. @hawkieowl "Why Twisted Is The Best" Transport Operating System Network Protocol Business Logic
42. @hawkieowl "Why Twisted Is The Best" Transport Operating System Network Protocol Business Logic
43. @hawkieowl "Why Twisted Is The Best" Now lets make it... be a Google proxy!
44. @hawkieowl "Why Twisted Is The Best" We send a request, which Twisted sends immediately... ...but how do we know when we've got all the data back?
45. @hawkieowl "Why Twisted Is The Best" ✨ Deferreds! ✨
46. @hawkieowl "Why Twisted Is The Best" d = Deferred() d.addCallback(do_thing)
47. @hawkieowl "Why Twisted Is The Best" var p = new Promise() p.then(do_thing)
48. @hawkieowl "Why Twisted Is The Best" Twisted Deferred (~2003, Python) Dojo Deferred (2008, JavaScript) Promises/A (2009, JavaScript) jQuery Deferred (2011, JavaScript) Promises/A+ (2013, JavaScript)
49. @hawkieowl "Why Twisted Is The Best" @app.route("/<ip>") async def main(self, request, ip): p = await treq.get( "https://google.com") return await p.content() app.run('0.0.0.0', 8080)
50. @hawkieowl "Why Twisted Is The Best" Transport Operating System Network Protocol Business Logic
51. @hawkieowl "Why Twisted Is The Best" Transport Operating System Network Protocol Business Logic
52. @hawkieowl "Why Twisted Is The Best" Twisted makes harder things easier.
53. @hawkieowl "Why Twisted Is The Best" Transport Operating System Network Protocol Business Logic Parsing protocols Abstractions on top TLS (handshakes, certificates) Mostly same interface on Windows, Mac, Linux, BSDs Out-of-the-box logic for some tasks Utilities for coping with network failure/reconnects
54. @hawkieowl "Why Twisted Is The Best" Opt-in to the separate layers
55. @hawkieowl "Why Twisted Is The Best" Transport Operating System Network Protocol Business Logic Example: txtorcon A Tor implementation that implements the Protocol and Transport layers Exposes itself to app devs as an Endpoint (which creates Transports)
56. @hawkieowl "Why Twisted Is The Best" Transport Operating System Network Protocol Business Logic Example: qt5reactor A reactor/event loop that interoperates with QT5. Installable and usable by app developers.
57. Not just web, not just one web
58. @hawkieowl "Why Twisted Is The Best" Twisted is a general purpose networking framework
59. @hawkieowl "Why Twisted Is The Best" Python Django/Flask/Pyramid
60. @hawkieowl "Why Twisted Is The Best" Python Twisted Reactor Web Client Client Client
61. @hawkieowl "Why Twisted Is The Best" Python Twisted Reactor Web Client DNS Client Client Client
62. @hawkieowl "Why Twisted Is The Best" Twisted allows you to listen/connect to as many things as you want*
63. @hawkieowl "Why Twisted Is The Best" * as long as you don't block the reactor with heavy CPU usage
64. @hawkieowl "Why Twisted Is The Best" Twisted has GUI interop reactors too!
65. @hawkieowl "Why Twisted Is The Best" Toga (http://pybee.org/project/projects/libraries/toga/) + Twisted, on Python 3.5, macOS + Linux GTK
66. Testing
67. @hawkieowl "Why Twisted Is The Best" Twisted's explicit layering allows for writing simple, deterministic tests.
68. @hawkieowl "Why Twisted Is The Best" Protocol Factory Transport Endpoint Reactor
69. @hawkieowl "Why Twisted Is The Best" Protocol Fake Transport Fake Reactor
70. @hawkieowl "Why Twisted Is The Best" Unit tests require zero real network connections
71. @hawkieowl "Why Twisted Is The Best" Twisted encourages writing code that can have network data replayed
72. @hawkieowl "Why Twisted Is The Best" Twisted encourages writing code that is free of mocks in business logic
73. Performance
74. @hawkieowl "Why Twisted Is The Best" Usually you are presented with two paths: - the easy way - the performant way
75. @hawkieowl "Why Twisted Is The Best" Django runserver nginx + uWSGI
76. @hawkieowl "Why Twisted Is The Best" All ways of starting a Twisted app are equally performant
77. @hawkieowl "Why Twisted Is The Best" twist, meant for deployment task.react, meant for scripts calling reactor.run()
78. @hawkieowl "Why Twisted Is The Best" CPython too slow? Move to PyPy!
79. @hawkieowl "Why Twisted Is The Best"
80. @hawkieowl "Why Twisted Is The Best" Drop-in compatible with Python 2.7 & 3.5 (soon) Twisted works pretty good!
81. What this all means
82. @hawkieowl "Why Twisted Is The Best" Using Python, you can create networking servers that handle thousands of connections at once
83. @hawkieowl "Why Twisted Is The Best" Speak any protocol you want, talk to basically anything that exists
84. @hawkieowl "Why Twisted Is The Best" You can write hybrid networking applications super easily!
85. @hawkieowl "Why Twisted Is The Best" High-level enough to get things done, low-level enough that you can make it do exactly as you want
86. @hawkieowl "Why Twisted Is The Best" It's all Python! Write Python, use Twisted whenever you need to talk to the network!
87. @hawkieowl "Why Twisted Is The Best" No configuration Opinionated in security Unopinionated in how you build your application
88. @hawkieowl "Why Twisted Is The Best" twistedmatrix.com atleastfornow.net/blog
89. Come talk to me after the session!