The Report of Twisted's Death; or Twisted & Tornado in the Asyncio Age (PyCon US 2016)

The Report of Twisted's Death; or Twisted & Tornado in the Asyncio Age (PyCon US 2016)

3d37232726396a1d3c7412dd915095ea?s=128

Amber Brown (HawkOwl)

May 31, 2016
Tweet

Transcript

  1. 4.
  2. 7.
  3. 8.

    Binary release management across 3 distros Ported Autobahn|Python (Tx) and

    Crossbar.io to Python 3 Web API/REST integration in CB
  4. 12.
  5. 16.
  6. 17.
  7. 30.
  8. 33.

    Selector functions take a list of file descriptors (e.g. sockets,

    open files) and tell you what is ready for reading or writing
  9. 35.
  10. 38.

    Nothing blocks! It just waits until the network is ready

    for more data to be sent, or more data has arrived
  11. 44.

    They provide an object which is a standin for some

    future result, and a way of being notified of that result
  12. 48.

    Deferreds run callbacks as soon as they are able Futures

    schedule a callback for the next reactor loop
  13. 58.

    import asyncio import datetime async def display_date(loop): end_time = loop.time()

    + 5.0 while True: print(datetime.datetime.now()) if (loop.time() + 1.0) >= end_time: break await asyncio.sleep(1) loop = asyncio.get_event_loop() # Blocking call which returns when the display_date() # coroutine is done loop.run_until_complete(display_date(loop)) loop.close()
  14. 61.

    “It should be easy for (Python 3.3 ports of) frameworks

    like Twisted, Tornado, or even gevent to either adapt the default event loop implementation to their needs using a lightweight adapter or proxy, or to replace the default event loop implementation with an adaptation of their own event loop implementation.” “Interoperability - asyncio”
 https://www.python.org/dev/peps/pep-3156/
  15. 63.

    “For this interoperability to be effective, the preferred direction of

    adaptation in third party frameworks is to keep the default event loop and adapt it to the framework's API. Ideally all third party frameworks would give up their own event loop implementation in favor of the standard implementation.” “Interoperability - asyncio”
 https://www.python.org/dev/peps/pep-3156/
  16. 70.

    “Twisted is an async I/O thing, asyncio is an async

    I/O thing. Therefore they are the same kind of thing. I only need one kind of thing in each category of thing. Therefore I only need one of them, and the “standard” one is probably the better one to depend on. So I guess nobody will need Twisted any more!” https://glyph.twistedmatrix.com/2014/05/the-report-of-our-death.html
  17. 72.

    0 75000 150000 225000 300000 Twisted 16.1 asyncio 3,352 107,612

    21,902 176,927 Code (lines) Comments (lines) Lines of Code (Python & C) With Tests
  18. 73.

    0 35000 70000 105000 140000 Twisted 16.1 asyncio 2,355 54,242

    8,452 74,250 Code (lines) Comments (lines) Lines of Code (Python & C) Without Tests
  19. 74.

    Twisted did a lot of things because none of these

    things were otherwise available
  20. 76.

    0 35000 70000 105000 140000 Twisted 16.1 Django 1.9 25,625

    54,242 74,033 74,250 Code (lines) Comments (lines) Lines of Code (Python & C) Without Tests
  21. 78.

    0 7500 15000 22500 30000 Twisted 16.1
 (asyncio parity) asyncio


    (+ concurrent.futures) 2,712 8,434 9,143 13,722 Code (lines) Comments (lines) Lines of Code (Python & C) Without Tests
  22. 81.
  23. 96.

    Similar to yield from, delegates to subgenerator Asynchronous code executed

    in a synchronous style Yielding for other things while it waits for a new result
  24. 97.
  25. 100.

    import treq from twisted.internet.defer import deferredCoroutine from twisted.internet.task import react

    @deferredCoroutine async def main(reactor): pages = [ "http://google.com/", "http://twistedmatrix.com", ] results = {} for page in pages: results[page] = await treq.content(await treq.get(page)) print(results) react(main)
  26. 104.

    import asyncio import uvloop asyncio.set_event_loop_policy(uvloop.EventLoopPolicy()) from twisted.internet.asyncioreactor import install install()

    import treq from aiohttp import web from twisted.internet import reactor, defer reactor.startRunning(True) user_agent = 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.1' async def handle(request): url = "http://www.google.com" + request.path_qs headers = dict(request.headers) try: headers.pop('HOST') headers.pop('CONNECTION') except: pass get = await defer.deferredToFuture(treq.get(url, headers=headers)) text = await defer.deferredToFuture(treq.content(get)) return web.Response( body=text, content_type=get.headers.getRawHeaders("content-type")[0]) app = web.Application() app.router.add_route('GET', '/{tail:.*}', handle) web.run_app(app)
  27. 115.

    from twisted.internet import reactor from twisted.internet.endpoints import serverFromString from twisted.internet.protocol

    import Factory from twisted.protocols.basic import LineReceiver class MyProtocol(LineReceiver): def lineReceived(self, line): print("I got this line!") print(line) factory = Factory.forProtocol(MyProtocol) endpoint = serverFromString(reactor, "tcp:7000") endpoint.listen(factory) reactor.run()
  28. 119.

    Example: hendrix WSGI runner on top of Twisted Websockets, TLS,

    run Twisted code github.com/hendrix/hendrix
  29. 120.

    Example: Autobahn|Python Websockets for Twisted and asyncio WS + WAMP,

    a RPC & PubSub framework Super fast under PyPy! I work on it :D github.com/crossbario/autobahnpython
  30. 127.
  31. 131.

    Python 2.7 (all platforms) Python 3.4 (Linux, FreeBSD) Python 3.5

    (Linux, FreeBSD) Earlier versions support 2.6 and 3.3
  32. 132.

    PyPy is close, only a handful of tests remain, nearly

    all are due to CPython assumptions
  33. 139.