Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Amber Brown - The Report Of Twisted’s Death or: Why Twisted and Tornado Are Relevant In The Asyncio Age

Amber Brown - The Report Of Twisted’s Death or: Why Twisted and Tornado Are Relevant In The Asyncio Age

With asyncio on the scene, the question has been asked: is there any point in having Twisted or Tornado around?


PyCon 2016

May 29, 2016

More Decks by PyCon 2016

Other Decks in Programming


  1. Binary release management across 3 distros Ported Autobahn|Python (Tx) and

    Crossbar.io to Python 3 Web API/REST integration in CB
  2. Selector functions take a list of file descriptors (e.g. sockets,

    open files) and tell you what is ready for reading or writing
  3. Nothing blocks! It just waits until the network is ready

    for more data to be sent, or more data has arrived
  4. They provide an object which is a standin for some

    future result, and a way of being notified of that result
  5. Deferreds run callbacks as soon as they are able Futures

    schedule a callback for the next reactor loop
  6. import asyncio import datetime async def display_date(loop): end_time = loop.time()

    + 5.0 while True: print(datetime.datetime.now()) if (loop.time() + 1.0) >= end_time: break await asyncio.sleep(1) loop = asyncio.get_event_loop() # Blocking call which returns when the display_date() # coroutine is done loop.run_until_complete(display_date(loop)) loop.close()
  7. “It should be easy for (Python 3.3 ports of) frameworks

    like Twisted, Tornado, or even gevent to either adapt the default event loop implementation to their needs using a lightweight adapter or proxy, or to replace the default event loop implementation with an adaptation of their own event loop implementation.” “Interoperability - asyncio”
  8. “For this interoperability to be effective, the preferred direction of

    adaptation in third party frameworks is to keep the default event loop and adapt it to the framework's API. Ideally all third party frameworks would give up their own event loop implementation in favor of the standard implementation.” “Interoperability - asyncio”
  9. “Twisted is an async I/O thing, asyncio is an async

    I/O thing. Therefore they are the same kind of thing. I only need one kind of thing in each category of thing. Therefore I only need one of them, and the “standard” one is probably the better one to depend on. So I guess nobody will need Twisted any more!” https://glyph.twistedmatrix.com/2014/05/the-report-of-our-death.html
  10. 0 75000 150000 225000 300000 Twisted 16.1 asyncio 3,352 107,612

    21,902 176,927 Code (lines) Comments (lines) Lines of Code (Python & C) With Tests
  11. 0 35000 70000 105000 140000 Twisted 16.1 asyncio 2,355 54,242

    8,452 74,250 Code (lines) Comments (lines) Lines of Code (Python & C) Without Tests
  12. Twisted did a lot of things because none of these

    things were otherwise available
  13. 0 35000 70000 105000 140000 Twisted 16.1 Django 1.9 25,625

    54,242 74,033 74,250 Code (lines) Comments (lines) Lines of Code (Python & C) Without Tests
  14. 0 7500 15000 22500 30000 Twisted 16.1
 (asyncio parity) asyncio

    (+ concurrent.futures) 2,712 8,434 9,143 13,722 Code (lines) Comments (lines) Lines of Code (Python & C) Without Tests
  15. Similar to yield from, delegates to subgenerator Asynchronous code executed

    in a synchronous style Yielding for other things while it waits for a new result
  16. import treq from twisted.internet.defer import deferredCoroutine from twisted.internet.task import react

    @deferredCoroutine async def main(reactor): pages = [ "http://google.com/", "http://twistedmatrix.com", ] results = {} for page in pages: results[page] = await treq.content(await treq.get(page)) print(results) react(main)
  17. import asyncio import uvloop asyncio.set_event_loop_policy(uvloop.EventLoopPolicy()) from twisted.internet.asyncioreactor import install install()

    import treq from aiohttp import web from twisted.internet import reactor, defer reactor.startRunning(True) user_agent = 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.1' async def handle(request): url = "http://www.google.com" + request.path_qs headers = dict(request.headers) try: headers.pop('HOST') headers.pop('CONNECTION') except: pass get = await defer.deferredToFuture(treq.get(url, headers=headers)) text = await defer.deferredToFuture(treq.content(get)) return web.Response( body=text, content_type=get.headers.getRawHeaders("content-type")[0]) app = web.Application() app.router.add_route('GET', '/{tail:.*}', handle) web.run_app(app)
  18. from twisted.internet import reactor from twisted.internet.endpoints import serverFromString from twisted.internet.protocol

    import Factory from twisted.protocols.basic import LineReceiver class MyProtocol(LineReceiver): def lineReceived(self, line): print("I got this line!") print(line) factory = Factory.forProtocol(MyProtocol) endpoint = serverFromString(reactor, "tcp:7000") endpoint.listen(factory) reactor.run()
  19. Example: hendrix WSGI runner on top of Twisted Websockets, TLS,

    run Twisted code github.com/hendrix/hendrix
  20. Example: Autobahn|Python Websockets for Twisted and asyncio WS + WAMP,

    a RPC & PubSub framework Super fast under PyPy! I work on it :D github.com/crossbario/autobahnpython
  21. Python 2.7 (all platforms) Python 3.4 (Linux, FreeBSD) Python 3.5

    (Linux, FreeBSD) Earlier versions support 2.6 and 3.3
  22. PyPy is close, only a handful of tests remain, nearly

    all are due to CPython assumptions