Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Why Twisted Is The Best (WOOTConf @ LCA 2017)

Why Twisted Is The Best (WOOTConf @ LCA 2017)

Twisted's pretty good.

3d37232726396a1d3c7412dd915095ea?s=128

Amber Brown (HawkOwl)

January 16, 2017
Tweet

More Decks by Amber Brown (HawkOwl)

Other Decks in Technology

Transcript

  1. Why Twisted Is The Best (and how it will make

    your application awesome) KatieConf/WOOTConf @ linux.conf.au
  2. Hello, I’m Amber Brown (HawkOwl)

  3. @hawkieowl atleastfornow.net

  4. I live in Melbourne, Victoria

  5. Twisted

  6. None
  7. @hawkieowl "Why Twisted Is The Best" Release Manager for: 13.2,

    14.0, 15.0, 15.1, 15.2, 15.3, 15.4, 15.5, 16.0, 16.1, 16.2, 16.3, 16.4, 16.5, 16.6, 16.7
  8. @hawkieowl "Why Twisted Is The Best" Framework (written in) Python

    2+3 (for) concurrent networking
  9. @hawkieowl "Why Twisted Is The Best" "Standard" Web App Client

    Client Client Client Client Client
  10. @hawkieowl "Why Twisted Is The Best" WSGI TCP "Standard" Web

    App Client Client Client Client Client Client
  11. @hawkieowl "Why Twisted Is The Best" TCP Twisted Application Client

    Client Client Client Client Client
  12. @hawkieowl "Why Twisted Is The Best" How?

  13. @hawkieowl "Why Twisted Is The Best" ✨ Operating System ✨

    ✨ I/O Primitives! ✨
  14. @hawkieowl "Why Twisted Is The Best" 1. Twisted keeps a

    list of connections
 your app is interested in Connection 1 Connection 2 Connection 3 Your App
  15. @hawkieowl "Why Twisted Is The Best" 2. Twisted asks the

    OS which connections
 have data ready, or can take more data select([1, 2, 3]) Connection 1 Connection 2 Connection 3 Your App
  16. @hawkieowl "Why Twisted Is The Best" 3. The OS responds

    with the ones that are. { "readable": [1], "writable: [2, 3] } Connection 1 Connection 2 Connection 3 Your App
  17. @hawkieowl "Why Twisted Is The Best" 4. Twisted recieves data

    and notifies
 your app, which processes the data. dataRecieved("how are you?") Connection 1 Connection 2 Connection 3 Your App
  18. @hawkieowl "Why Twisted Is The Best" 5. Your application can

    send data back on the connection, which Twisted will give the OS. transport.send("good!") socket.send(1, "good!") Connection 1 Connection 2 Connection 3 Your App
  19. @hawkieowl "Why Twisted Is The Best" Connection 1 Connection 2

    Connection 3 6. GOTO 1 Your App
  20. @hawkieowl "Why Twisted Is The Best" Your Application (is implemented

    as) Protocols (connected by) Transports
  21. @hawkieowl "Why Twisted Is The Best" All coordinated by a

    Reactor (or "event loop")
  22. @hawkieowl "Why Twisted Is The Best" Transport Operating System Network

    Protocol Business Logic
  23. @hawkieowl "Why Twisted Is The Best" Transport Operating System Network

    Protocol Business Logic
  24. Protocols

  25. @hawkieowl "Why Twisted Is The Best" Protocols implement and parse

    a network protocol
  26. @hawkieowl "Why Twisted Is The Best" Created by a Factory

  27. @hawkieowl "Why Twisted Is The Best" Factory Listening Socket Protocol

    Instance Transport Connected Socket Protocol Instance Transport Connected Socket Protocol Instance Transport Connected Socket
  28. @hawkieowl "Why Twisted Is The Best" Receives and sends data

    Decodes the data into useful events Handles the connection lifecycle
  29. @hawkieowl "Why Twisted Is The Best" Data is what is

    sent over the "wire"
  30. @hawkieowl "Why Twisted Is The Best" Data is sent to

    a Transport A Transport may get it there in different ways
  31. @hawkieowl "Why Twisted Is The Best" Base Transports: UDP, TCP

    Transport Protocols: TLS/SSL, SSH, Tor
  32. @hawkieowl "Why Twisted Is The Best" Events are what your

    application acts on
  33. @hawkieowl "Why Twisted Is The Best" Protocol consumes a full

    HTTP request, tells your application about it
  34. @hawkieowl "Why Twisted Is The Best" Creation Connection Operation Disconnection

    Destruction
  35. Out-of-the-box Protocols

  36. @hawkieowl "Why Twisted Is The Best" HTTP (1.0, 1.1, 2.0)

    SMTP, ESMTP, IMAP, POP3 DNS, SSH, Telnet, TLS ...and a bunch more
  37. @hawkieowl "Why Twisted Is The Best" Web Server DNS Server

    Email Server SSH Server
  38. Building an Application

  39. @hawkieowl "Why Twisted Is The Best" Let's build a website!

  40. @hawkieowl "Why Twisted Is The Best" from klein import Klein

    app = Klein() @app.route("/") def main(self, request): return b"Hello!!!!" app.run('0.0.0.0', 8080)
  41. @hawkieowl "Why Twisted Is The Best" Transport Operating System Network

    Protocol Business Logic
  42. @hawkieowl "Why Twisted Is The Best" Transport Operating System Network

    Protocol Business Logic
  43. @hawkieowl "Why Twisted Is The Best" Now lets make it...

    be a Google proxy!
  44. @hawkieowl "Why Twisted Is The Best" We send a request,

    which Twisted sends immediately... ...but how do we know when we've got all the data back?
  45. @hawkieowl "Why Twisted Is The Best" ✨ Deferreds! ✨

  46. @hawkieowl "Why Twisted Is The Best" d = Deferred() d.addCallback(do_thing)

  47. @hawkieowl "Why Twisted Is The Best" var p = new

    Promise() p.then(do_thing)
  48. @hawkieowl "Why Twisted Is The Best" Twisted Deferred (~2003, Python)

    Dojo Deferred (2008, JavaScript) Promises/A (2009, JavaScript) jQuery Deferred (2011, JavaScript) Promises/A+ (2013, JavaScript)
  49. @hawkieowl "Why Twisted Is The Best" @app.route("/<ip>") async def main(self,

    request, ip): p = await treq.get( "https://google.com") return await p.content() app.run('0.0.0.0', 8080)
  50. @hawkieowl "Why Twisted Is The Best" Transport Operating System Network

    Protocol Business Logic
  51. @hawkieowl "Why Twisted Is The Best" Transport Operating System Network

    Protocol Business Logic
  52. @hawkieowl "Why Twisted Is The Best" Twisted makes harder things

    easier.
  53. @hawkieowl "Why Twisted Is The Best" Transport Operating System Network

    Protocol Business Logic Parsing protocols Abstractions on top TLS (handshakes, certificates) Mostly same interface on Windows, Mac, Linux, BSDs Out-of-the-box logic for some tasks Utilities for coping with network failure/reconnects
  54. @hawkieowl "Why Twisted Is The Best" Opt-in to the separate

    layers
  55. @hawkieowl "Why Twisted Is The Best" Transport Operating System Network

    Protocol Business Logic Example: txtorcon A Tor implementation that implements the Protocol and Transport layers Exposes itself to app devs as an Endpoint (which creates Transports)
  56. @hawkieowl "Why Twisted Is The Best" Transport Operating System Network

    Protocol Business Logic Example: qt5reactor A reactor/event loop that interoperates with QT5. Installable and usable by app developers.
  57. Not just web, not just one web

  58. @hawkieowl "Why Twisted Is The Best" Twisted is a general

    purpose networking framework
  59. @hawkieowl "Why Twisted Is The Best" Python Django/Flask/Pyramid

  60. @hawkieowl "Why Twisted Is The Best" Python Twisted Reactor Web

    Client Client Client
  61. @hawkieowl "Why Twisted Is The Best" Python Twisted Reactor Web

    Client DNS Client Client Client
  62. @hawkieowl "Why Twisted Is The Best" Twisted allows you to

    listen/connect to as many things as you want*
  63. @hawkieowl "Why Twisted Is The Best" * as long as

    you don't block the reactor with heavy CPU usage
  64. @hawkieowl "Why Twisted Is The Best" Twisted has GUI interop

    reactors too!
  65. @hawkieowl "Why Twisted Is The Best" Toga (http://pybee.org/project/projects/libraries/toga/) + Twisted,

    on Python 3.5, macOS + Linux GTK
  66. Testing

  67. @hawkieowl "Why Twisted Is The Best" Twisted's explicit layering allows

    for writing simple, deterministic tests.
  68. @hawkieowl "Why Twisted Is The Best" Protocol Factory Transport Endpoint

    Reactor
  69. @hawkieowl "Why Twisted Is The Best" Protocol Fake Transport Fake

    Reactor
  70. @hawkieowl "Why Twisted Is The Best" Unit tests require zero

    real network connections
  71. @hawkieowl "Why Twisted Is The Best" Twisted encourages writing code

    that can have network data replayed
  72. @hawkieowl "Why Twisted Is The Best" Twisted encourages writing code

    that is free of mocks in business logic
  73. Performance

  74. @hawkieowl "Why Twisted Is The Best" Usually you are presented

    with two paths: - the easy way - the performant way
  75. @hawkieowl "Why Twisted Is The Best" Django runserver nginx +

    uWSGI
  76. @hawkieowl "Why Twisted Is The Best" All ways of starting

    a Twisted app are equally performant
  77. @hawkieowl "Why Twisted Is The Best" twist, meant for deployment

    task.react, meant for scripts calling reactor.run()
  78. @hawkieowl "Why Twisted Is The Best" CPython too slow? Move

    to PyPy!
  79. @hawkieowl "Why Twisted Is The Best"

  80. @hawkieowl "Why Twisted Is The Best" Drop-in compatible with Python

    2.7 & 3.5 (soon) Twisted works pretty good!
  81. What this all means

  82. @hawkieowl "Why Twisted Is The Best" Using Python, you can

    create networking servers that handle thousands of connections at once
  83. @hawkieowl "Why Twisted Is The Best" Speak any protocol you

    want, talk to basically anything that exists
  84. @hawkieowl "Why Twisted Is The Best" You can write hybrid

    networking applications super easily!
  85. @hawkieowl "Why Twisted Is The Best" High-level enough to get

    things done, low-level enough that you can make it do exactly as you want
  86. @hawkieowl "Why Twisted Is The Best" It's all Python! Write

    Python, use Twisted whenever you need to talk to the network!
  87. @hawkieowl "Why Twisted Is The Best" No configuration Opinionated in

    security Unopinionated in how you build your application
  88. @hawkieowl "Why Twisted Is The Best" twistedmatrix.com atleastfornow.net/blog

  89. Come talk to me after the session!