Tornado in 1 Hour (or Less)

Transcript

1. 1.

   TORNADO IN 1 HOUR (OR LESS) Kārlis Lauva @skazhy python.lv

   meetup 2013.12.04
2. 2.

   Tornado in 3 bullet points • High performance™ web framework

   • Async networking library • Gets out of your way (most of the time)
3. 3.

   Timeline • 2007.10 - FriendFeed is launched • 2009.08 -

   Facebook acquires FriendFeed • 2009.09 - Tornado is open sourced • 2013.09 - Last stable release (3.1.1)
4. 4.

   Who’s using it? …..and your company.

5. 5.

   Installing pip install tornado pip install git+https://github.com/facebook/tornado.git

6. 6.

   “Hello, World!” import tornado.web import tornado.ioloop class Handler(tornado.web.RequestHandler): def get(self):

   self.finish("Hello World!") handlers = [tornado.web.URLSpec(r"/$", Handler)] app = tornado.web.Application(handlers) app.listen(1337) tornado.ioloop.IOLoop.instance().start()
7. 7.

   MAIN INGREDIENTS

8. 8.

   tornado.ioloop • I/O loop for non-blocking sockets • One per

   application (or thread) • Uses epoll or kqueue
9. 9.

   Example - TCP server def connection_ready(sock, fd, events): while True:

   try: connection, address = sock.accept() except socket.error: return connection.setblocking(0) connection.send("Hello") sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0) sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) sock.setblocking(0) sock.bind(("", 1337)) sock.listen(128) io_loop = ioloop.IOLoop.instance() callback = functools.partial(connection_ready, sock) io_loop.add_handler(sock.fileno(), callback, io_loop.READ) io_loop.start()
10. 10.

    tornado.httpserver • Async web server • Utilizes a singleton IOLoop

    • Subclass of TCP server (duh)
11. 11.

    Example - HTTP server import tornado.httpserver import tornado.ioloop def handle_request(request):

    message = "Hello world\n" request.write("HTTP/1.1 200 OK\r\nContent-Length: %d\r\n\r\n%s" % \ (len(message), message)) request.finish() http_server = tornado.httpserver.HTTPServer(handle_request) http_server.listen(8080) tornado.ioloop.IOLoop.instance().start()
12. 12.

    tornado.web.Application • A collection of resource - handler mappings •

    Setting container • Manages UI modules & rendering • Wraps tornado.httpserver • Supports virtual hosts
13. 13.

    tornado.web.RequestHandler • Basic building block • If method exists -

    call it, else raise HTTP 405 • Not limited to “standard” verbs • Unified request preparation / finalizing methods
14. 14.

    A (blocking) request handler # in handler listing URLSpec(r"/users/(\w+)$", SomeHandler)

    class Somehandler(tornado.web.RequestHandler): def post(self, user_id): if not self.get_argument(message): raise HTTPError(400, "No message") payload = { "user_id": user_id, "message": self.get_argument(message), } self.application.db.messages.insert(payload) response = HTTPClient().fetch("http://foo.bar.baz") self.render("template.html", {"data": response})
15. 15.

    What else is there? • HTTP client • Templating system

    • Option management / collection system • Generator interface for async calls • Auth mixins for OAuth • ...and more
16. 16.

    ADDING ASYNC-SAUCE

17. 17.

    Blocking request redux def post(self, user_id): if not self.get_argument(message): raise

    HTTPError(400, "No message") payload = { "user_id": user_id, "message": self.get_argument(message), } self.application.db.messages.insert(payload) response = HTTPClient().fetch("http://foo.bar.baz") self.render("template.html", {"data": response})
18. 18.

    Non-blocking request with callback @web.asynchronous def post(self, user_id): def callback(response):

    self.render("template.html", {"data": response}) if not self.get_argument(message): raise HTTPError(400, "No message") payload = { "user_id": user_id, "message": self.get_argument(message), } self.application.db.messages.insert(payload) AsyncHTTPClient().fetch("http://foo.bar.baz", callback)
19. 19.

    A non-blocking request with @gen @web.asynchronous @gen.engine def post(self, user_id):

    def callback(response): if not self.get_argument(message): raise HTTPError(400, "No message") payload = { "user_id": user_id, "message": self.get_argument(message), } self.application.db.messages.insert(payload) resp = yield gen.Task(AsyncHTTPClient().fetch, "http://foo.bar.baz") self.render("template.html", {"data": resp})
20. 20.

    Non-blocking calls for pymongo @gen.engine @web.asynchronous def post(self, user_id): def

    callback(response): if not self.get_argument(message): raise HTTPError(400, "No message") payload = { "user_id": user_id, "message": self.get_argument(message), } motor.Op(self.application.db.messages.insert, payload) resp = yield gen.Task(AsyncHTTPClient().fetch, "http://foo.bar.baz") self.render("template.html", {"data": resp})
21. 21.

    Caveats • Always raise in main thread • Don’t forget

    to add decorators • @gen.Tasks are toxic
22. 22.

    tornado.websocket class EchoHandler(websocket.WebSocketHandler): def open(self): # the same as handler.get()

    logging.debug("WebSocket opened") def on_message(self, message): self.write_message(message) def on_close(self): logging.debug("WebSocket closed") // Client side var ws = new WebSocket("ws://localhost:1337/echo"); ws.onopen = function() { ws.send("Hello, world"); }; ws.onmessage = function (evt) { alert(evt.data); };
23. 23.

    TORNADO TEMPLATING

24. 24.

    • Django & Jinja2-esque • Easily extendible • Built-in helpers

    for reverse urls, etc tornado.template
25. 25.

    Example context = { "bugs": {"123": "Fix color scheme"} "uppercase":

    lambda x: x.upper() } <ul> {% for bug_id, description in bugs.items() %} <li>{{ bug_id }} - {{ description }}</li> <p>{{ shout(“whoa”) }}</p> {% end %} </li>
26. 26.

    TESTING + DEBUGGING

27. 27.

    tornado.testing • Wrappers for unittest.TestCase • Helpers for testing async

    handlers
28. 28.

    PDB • Works as you'd expect app.listen(options.port) pdb.set_trace() tornado.ioloop.IOLoop.instance().start()

29. 29.

    REAL WORLD TORNADO

30. 30.

    Managing instances with supervisord [program:app-8000] directory=/srv/app/ command=python server.py --port=8101 --mongodb_host=localhost:27017

    user=nginx stdout_logfile=/srv/app/logs/app.8000.log [program:app-8001] directory=/srv/app/ command=python server.py --port=8001 --mongodb_host=localhost:27017 user=nginx stdout_logfile=/srv/app/logs/app.8001.log [group:webapp] programs=app-8000,app-8001
31. 31.

    Load balancing with nginx upstream webapp { server localhost:8000; server

    localhost:8001; } server { server_name foobar; location / { proxy_pass http://webapp; } }
32. 32.

    Utilize ngnix to it's full power • Do not serve

    static files from Tornado • Whitelist routing (don't bug Tornado with 403s)
33. 33.

    Load balancing while you load balance with Amazon ELB

34. 34.

    Deployment / production settings python server.py --port=8001 --mongodb_host=localhost:27017 # In

    application file from tornado.options import define, options define("port", default=1337, type=int) define("mongodb_host", default="localhost:27017", type=str) define("version", default="1", type=str) class Application(tornado.web.Application): def __init__(self): self.db = MongoClient(options.mongodb_host) handlers = [....] super(Application, self).__init__(handlers, version=options.version) def main(): options.parse_command_line() app = Application() application.listen(options.port)
35. 35.

    Tornado and HTTPS • As easy as adding an option

    to the application instance ssl_options={ "certfile": path_to_certfile, "keyfile": path_to_keyfile, } app = Application(handlers, ssl_options=ssl_options)
36. 36.

    Error collection • Override RequestHandler.send_error() • Use error collection mixins

37. 37.

    Sentry integration with Raven • Using mixins to capture exceptions

    class Hadler(SentryMixin, tornado.web.RequestHandler): @tornado.web.asynchronous @tornado.gen.engine def get(self): try: raise ValueError() except Exception as e: response = yield tornado.gen.Task( self.captureException, exc_info=True ) self.finish()
38. 38.

    Where to go next? • Read through the source •

    Start building rad hacks • ??? • PROFIT!!
39. 39.

    QUESTION TIME!