Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Why Twisted Is The Best (WOOTConf @ LCA 2017)

Why Twisted Is The Best (WOOTConf @ LCA 2017)

Twisted's pretty good.

3d37232726396a1d3c7412dd915095ea?s=128

Amber Brown (HawkOwl)

January 16, 2017
Tweet

Transcript

  1. Why Twisted Is The Best (and how it will make

    your application awesome) KatieConf/WOOTConf @ linux.conf.au
  2. Hello, I’m Amber Brown (HawkOwl)

  3. @hawkieowl atleastfornow.net

  4. I live in Melbourne, Victoria

  5. Twisted

  6. None
  7. @hawkieowl "Why Twisted Is The Best" Release Manager for: 13.2,

    14.0, 15.0, 15.1, 15.2, 15.3, 15.4, 15.5, 16.0, 16.1, 16.2, 16.3, 16.4, 16.5, 16.6, 16.7
  8. @hawkieowl "Why Twisted Is The Best" Framework (written in) Python

    2+3 (for) concurrent networking
  9. @hawkieowl "Why Twisted Is The Best" "Standard" Web App Client

    Client Client Client Client Client
  10. @hawkieowl "Why Twisted Is The Best" WSGI TCP "Standard" Web

    App Client Client Client Client Client Client
  11. @hawkieowl "Why Twisted Is The Best" TCP Twisted Application Client

    Client Client Client Client Client
  12. @hawkieowl "Why Twisted Is The Best" How?

  13. @hawkieowl "Why Twisted Is The Best" ✨ Operating System ✨

    ✨ I/O Primitives! ✨
  14. @hawkieowl "Why Twisted Is The Best" 1. Twisted keeps a

    list of connections
 your app is interested in Connection 1 Connection 2 Connection 3 Your App
  15. @hawkieowl "Why Twisted Is The Best" 2. Twisted asks the

    OS which connections
 have data ready, or can take more data select([1, 2, 3]) Connection 1 Connection 2 Connection 3 Your App
  16. @hawkieowl "Why Twisted Is The Best" 3. The OS responds

    with the ones that are. { "readable": [1], "writable: [2, 3] } Connection 1 Connection 2 Connection 3 Your App
  17. @hawkieowl "Why Twisted Is The Best" 4. Twisted recieves data

    and notifies
 your app, which processes the data. dataRecieved("how are you?") Connection 1 Connection 2 Connection 3 Your App
  18. @hawkieowl "Why Twisted Is The Best" 5. Your application can

    send data back on the connection, which Twisted will give the OS. transport.send("good!") socket.send(1, "good!") Connection 1 Connection 2 Connection 3 Your App
  19. @hawkieowl "Why Twisted Is The Best" Connection 1 Connection 2

    Connection 3 6. GOTO 1 Your App
  20. @hawkieowl "Why Twisted Is The Best" Your Application (is implemented

    as) Protocols (connected by) Transports
  21. @hawkieowl "Why Twisted Is The Best" All coordinated by a

    Reactor (or "event loop")
  22. @hawkieowl "Why Twisted Is The Best" Transport Operating System Network

    Protocol Business Logic
  23. @hawkieowl "Why Twisted Is The Best" Transport Operating System Network

    Protocol Business Logic
  24. Protocols

  25. @hawkieowl "Why Twisted Is The Best" Protocols implement and parse

    a network protocol
  26. @hawkieowl "Why Twisted Is The Best" Created by a Factory

  27. @hawkieowl "Why Twisted Is The Best" Factory Listening Socket Protocol

    Instance Transport Connected Socket Protocol Instance Transport Connected Socket Protocol Instance Transport Connected Socket
  28. @hawkieowl "Why Twisted Is The Best" Receives and sends data

    Decodes the data into useful events Handles the connection lifecycle
  29. @hawkieowl "Why Twisted Is The Best" Data is what is

    sent over the "wire"
  30. @hawkieowl "Why Twisted Is The Best" Data is sent to

    a Transport A Transport may get it there in different ways
  31. @hawkieowl "Why Twisted Is The Best" Base Transports: UDP, TCP

    Transport Protocols: TLS/SSL, SSH, Tor
  32. @hawkieowl "Why Twisted Is The Best" Events are what your

    application acts on
  33. @hawkieowl "Why Twisted Is The Best" Protocol consumes a full

    HTTP request, tells your application about it
  34. @hawkieowl "Why Twisted Is The Best" Creation Connection Operation Disconnection

    Destruction
  35. Out-of-the-box Protocols

  36. @hawkieowl "Why Twisted Is The Best" HTTP (1.0, 1.1, 2.0)

    SMTP, ESMTP, IMAP, POP3 DNS, SSH, Telnet, TLS ...and a bunch more
  37. @hawkieowl "Why Twisted Is The Best" Web Server DNS Server

    Email Server SSH Server
  38. Building an Application

  39. @hawkieowl "Why Twisted Is The Best" Let's build a website!

  40. @hawkieowl "Why Twisted Is The Best" from klein import Klein

    app = Klein() @app.route("/") def main(self, request): return b"Hello!!!!" app.run('0.0.0.0', 8080)
  41. @hawkieowl "Why Twisted Is The Best" Transport Operating System Network

    Protocol Business Logic
  42. @hawkieowl "Why Twisted Is The Best" Transport Operating System Network

    Protocol Business Logic
  43. @hawkieowl "Why Twisted Is The Best" Now lets make it...

    be a Google proxy!
  44. @hawkieowl "Why Twisted Is The Best" We send a request,

    which Twisted sends immediately... ...but how do we know when we've got all the data back?
  45. @hawkieowl "Why Twisted Is The Best" ✨ Deferreds! ✨

  46. @hawkieowl "Why Twisted Is The Best" d = Deferred() d.addCallback(do_thing)

  47. @hawkieowl "Why Twisted Is The Best" var p = new

    Promise() p.then(do_thing)
  48. @hawkieowl "Why Twisted Is The Best" Twisted Deferred (~2003, Python)

    Dojo Deferred (2008, JavaScript) Promises/A (2009, JavaScript) jQuery Deferred (2011, JavaScript) Promises/A+ (2013, JavaScript)
  49. @hawkieowl "Why Twisted Is The Best" @app.route("/<ip>") async def main(self,

    request, ip): p = await treq.get( "https://google.com") return await p.content() app.run('0.0.0.0', 8080)
  50. @hawkieowl "Why Twisted Is The Best" Transport Operating System Network

    Protocol Business Logic
  51. @hawkieowl "Why Twisted Is The Best" Transport Operating System Network

    Protocol Business Logic
  52. @hawkieowl "Why Twisted Is The Best" Twisted makes harder things

    easier.
  53. @hawkieowl "Why Twisted Is The Best" Transport Operating System Network

    Protocol Business Logic Parsing protocols Abstractions on top TLS (handshakes, certificates) Mostly same interface on Windows, Mac, Linux, BSDs Out-of-the-box logic for some tasks Utilities for coping with network failure/reconnects
  54. @hawkieowl "Why Twisted Is The Best" Opt-in to the separate

    layers
  55. @hawkieowl "Why Twisted Is The Best" Transport Operating System Network

    Protocol Business Logic Example: txtorcon A Tor implementation that implements the Protocol and Transport layers Exposes itself to app devs as an Endpoint (which creates Transports)
  56. @hawkieowl "Why Twisted Is The Best" Transport Operating System Network

    Protocol Business Logic Example: qt5reactor A reactor/event loop that interoperates with QT5. Installable and usable by app developers.
  57. Not just web, not just one web

  58. @hawkieowl "Why Twisted Is The Best" Twisted is a general

    purpose networking framework
  59. @hawkieowl "Why Twisted Is The Best" Python Django/Flask/Pyramid

  60. @hawkieowl "Why Twisted Is The Best" Python Twisted Reactor Web

    Client Client Client
  61. @hawkieowl "Why Twisted Is The Best" Python Twisted Reactor Web

    Client DNS Client Client Client
  62. @hawkieowl "Why Twisted Is The Best" Twisted allows you to

    listen/connect to as many things as you want*
  63. @hawkieowl "Why Twisted Is The Best" * as long as

    you don't block the reactor with heavy CPU usage
  64. @hawkieowl "Why Twisted Is The Best" Twisted has GUI interop

    reactors too!
  65. @hawkieowl "Why Twisted Is The Best" Toga (http://pybee.org/project/projects/libraries/toga/) + Twisted,

    on Python 3.5, macOS + Linux GTK
  66. Testing

  67. @hawkieowl "Why Twisted Is The Best" Twisted's explicit layering allows

    for writing simple, deterministic tests.
  68. @hawkieowl "Why Twisted Is The Best" Protocol Factory Transport Endpoint

    Reactor
  69. @hawkieowl "Why Twisted Is The Best" Protocol Fake Transport Fake

    Reactor
  70. @hawkieowl "Why Twisted Is The Best" Unit tests require zero

    real network connections
  71. @hawkieowl "Why Twisted Is The Best" Twisted encourages writing code

    that can have network data replayed
  72. @hawkieowl "Why Twisted Is The Best" Twisted encourages writing code

    that is free of mocks in business logic
  73. Performance

  74. @hawkieowl "Why Twisted Is The Best" Usually you are presented

    with two paths: - the easy way - the performant way
  75. @hawkieowl "Why Twisted Is The Best" Django runserver nginx +

    uWSGI
  76. @hawkieowl "Why Twisted Is The Best" All ways of starting

    a Twisted app are equally performant
  77. @hawkieowl "Why Twisted Is The Best" twist, meant for deployment

    task.react, meant for scripts calling reactor.run()
  78. @hawkieowl "Why Twisted Is The Best" CPython too slow? Move

    to PyPy!
  79. @hawkieowl "Why Twisted Is The Best"

  80. @hawkieowl "Why Twisted Is The Best" Drop-in compatible with Python

    2.7 & 3.5 (soon) Twisted works pretty good!
  81. What this all means

  82. @hawkieowl "Why Twisted Is The Best" Using Python, you can

    create networking servers that handle thousands of connections at once
  83. @hawkieowl "Why Twisted Is The Best" Speak any protocol you

    want, talk to basically anything that exists
  84. @hawkieowl "Why Twisted Is The Best" You can write hybrid

    networking applications super easily!
  85. @hawkieowl "Why Twisted Is The Best" High-level enough to get

    things done, low-level enough that you can make it do exactly as you want
  86. @hawkieowl "Why Twisted Is The Best" It's all Python! Write

    Python, use Twisted whenever you need to talk to the network!
  87. @hawkieowl "Why Twisted Is The Best" No configuration Opinionated in

    security Unopinionated in how you build your application
  88. @hawkieowl "Why Twisted Is The Best" twistedmatrix.com atleastfornow.net/blog

  89. Come talk to me after the session!