WTF Is Twisted? (DjangoCon AU 2014 Lightning Talk)

A lightning talk going over the very basics of Twisted.

Transcript

1. WTF is Twisted? (or: Owl Amongst The Ponies)
 Lightning Talk Edition
2. Who Am I? • Hi, I’m HawkOwl! • hawkowl@atleastfornow.net • @hawkieowl on Twitter • Twisted Core Dev & Release Manager (13.2 & 14.0)
3. Twisted in One Slide • Twisted is a framework for writing network applications in Python • Best known for “Deferreds”, Twisted’s async abstraction • Contains protocol implementations for HTTP, SMTP, IMAP, DNS, etc.
4. Why would you want to use Twisted? • Higher performance in I/O • Stable, mature codebase • High test coverage • Great community consisting of excellent individuals who do excellent work
5. The Tenets of Twisted (the time-tested, battle-hardened principals of the Twisted project, as interpreted by HawkOwl)
6. Subclassing is bad. Composition is the new inheritance!
7. Multiple inheritance is really bad. Handy hint: If you’ve written the word “super”, it’s already too late.
8. Time-based releases are king. For large, evolving frameworks, it’s better to gradually change than have a massive app migration effort.
9. Early APIs > No APIS You won’t know how your API works in the real world until other people use it. You can always deprecate it later.
10. All code needs tests. No exceptions. It doesn’t matter if you’re fixing an urgent release regression - an eye on coverage saves you a lot of time later.
11. All code needs reviewing. No exceptions. Code review is invaluable, even if it does descend into bikeshedding if they can’t find anything wrong.
12. Only bad APIs can’t be unit tested. Can’t unit test your API? Too bad, time to refactor it.
13. Build slaves are amazing. Test your code on everything. Virtual machines are cheap, compatibility problems down the line are not.
14. How Twisted Works (a short, mostly-correct overview)
15. The Event Loop (Spin me right round, baby, right round)
16. The Reactor • Twisted, being an event-driven framework, provides an event loop, or “reactor”. • This is an infinite loop, wrapped around a blocking call that waits until some I/O is ready • Once the I/O is ready, this fires off an event to run the code that was waiting for it • Runs other tasks while I/O blocked tasks are waiting
17. Using the Reactor • To utilise the reactor, you use Deferreds, which are an IOU for a result • A Deferred contains a callback chain, a series of functions run in sequence after it gets a result • Deferreds get their results from events occurring, such as I/O
18. Event Loop Integrations • select • epoll • kqueue • GTK • QT
19. The Callback Chain (Hey, I just met you, and this is crazy,
 but here’s a function, so call it maybe?)
20. The Callback Chain • The easiest way to understand the callback chain is to visualise it as a production line. • A production line is made up of discrete blocks - each doing a separate thing.
21. Deferreds • Each function in the chain is called with the result of the previous one as its first argument • Functions in the callback chain can return either a result or a Deferred that fires with a result sometime in the future
22. Error Handling • Error handling is done by errbacks. • Errbacks are like callbacks, but are run when an exception is raised. • An errback will catch all errors that occur on the chain before it.
23. Deferred Benefits • Forces you to break up your logic into sections, broken up over external interface accesses • Encourages reuse, as these sections quite commonly do one thing to an input and return it • Can be cancelled (eg. if a client disconnected and you don’t care about the result anymore)
24. Deferred Benefits • Discourages global mutable state by having different execution threads share the same global state • No global mutable state means you can scale easier • If your global state is databases, just spin up more Twisted instances to use all your CPUs
25. What can Twisted do? (Enough of the boring stuff!)
26. Web • Klein, a Flask-like API in Twisted • Saratoga, a HTTP API framework • Autobahn, a Websockets implementation • Static file serving: 
 twistd -n web --port tcp:8080 --path /tmp
27. Klein (Flask in Twisted) from klein import Klein! app = Klein()! ! @app.route('/')! def home(request):! return 'Hello, world!'! ! app.run("localhost", 8080)
28. Saratoga (HTTP API) {! "metadata": {! "name": “APIExample", "versions": [1]! },! "endpoints": [{! ! ! "endpoint": “example", "getProcessors": [! {"versions": [1]}! ]}! ]}! from saratoga.api import SaratogaAPI! import json! ! class APIExample(object):! class v1(object):! def example_GET(self, request, params):! return {"hello": "world"}! ! api = SaratogaAPI(APIExample, json.load(open("apidef.json")))! api.run()
29. Email • Complete SMTP and ESMTP server and clients • POP3 and IMAP4 servers and clients • Support for TLS-secured email out of the box
 
 For more info, see:
 http://twistedmatrix.com/documents/ current/mail/index.html
30. DNS • A caching, recursive DNS server out of the box
 twistd -n dns -c —port 10053 --recursive • DNS client APIs • DNS server toolkit
 
 For more info, see:
 http://twistedmatrix.com/documents/ current/names/howto/custom-server.html
31. Chat • IRC Server
 twistd -n words --irc-port tcp:8400 --group roomname --auth memory:usr:pass • IRC client (for making IRC bots) • XMPP client
 
 For more info, see: 
 http://twistedmatrix.com/documents/ current/words/examples/
32. Want to know more? • Twisted Docs
 http://twistedmatrix.com/documents/current/ • Iffy’s Twisted FTW
 http://iffycan.com/twistedftw/ • HawkOwl’s Technicals
 http://technicals.atleastfornow.net/
33. Thank you! BTW, I’m talking before lunch on Sunday!