Tornado in 1 Hour (or Less)

Tornado in 1 Hour (or Less)

Tornado crash-course

085d620aca53f8b5a7630021747d9c9d?s=128

Kārlis Lauva

December 04, 2013
Tweet

Transcript

  1. 2.

    Tornado in 3 bullet points • High performance™ web framework

    • Async networking library • Gets out of your way (most of the time)
  2. 3.

    Timeline • 2007.10 - FriendFeed is launched • 2009.08 -

    Facebook acquires FriendFeed • 2009.09 - Tornado is open sourced • 2013.09 - Last stable release (3.1.1)
  3. 6.

    “Hello, World!” import tornado.web import tornado.ioloop class Handler(tornado.web.RequestHandler): def get(self):

    self.finish("Hello World!") handlers = [tornado.web.URLSpec(r"/$", Handler)] app = tornado.web.Application(handlers) app.listen(1337) tornado.ioloop.IOLoop.instance().start()
  4. 8.

    tornado.ioloop • I/O loop for non-blocking sockets • One per

    application (or thread) • Uses epoll or kqueue
  5. 9.

    Example - TCP server def connection_ready(sock, fd, events): while True:

    try: connection, address = sock.accept() except socket.error: return connection.setblocking(0) connection.send("Hello") sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0) sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) sock.setblocking(0) sock.bind(("", 1337)) sock.listen(128) io_loop = ioloop.IOLoop.instance() callback = functools.partial(connection_ready, sock) io_loop.add_handler(sock.fileno(), callback, io_loop.READ) io_loop.start()
  6. 11.

    Example - HTTP server import tornado.httpserver import tornado.ioloop def handle_request(request):

    message = "Hello world\n" request.write("HTTP/1.1 200 OK\r\nContent-Length: %d\r\n\r\n%s" % \ (len(message), message)) request.finish() http_server = tornado.httpserver.HTTPServer(handle_request) http_server.listen(8080) tornado.ioloop.IOLoop.instance().start()
  7. 12.

    tornado.web.Application • A collection of resource - handler mappings •

    Setting container • Manages UI modules & rendering • Wraps tornado.httpserver • Supports virtual hosts
  8. 13.

    tornado.web.RequestHandler • Basic building block • If method exists -

    call it, else raise HTTP 405 • Not limited to “standard” verbs • Unified request preparation / finalizing methods
  9. 14.

    A (blocking) request handler # in handler listing URLSpec(r"/users/(\w+)$", SomeHandler)

    class Somehandler(tornado.web.RequestHandler): def post(self, user_id): if not self.get_argument(message): raise HTTPError(400, "No message") payload = { "user_id": user_id, "message": self.get_argument(message), } self.application.db.messages.insert(payload) response = HTTPClient().fetch("http://foo.bar.baz") self.render("template.html", {"data": response})
  10. 15.

    What else is there? • HTTP client • Templating system

    • Option management / collection system • Generator interface for async calls • Auth mixins for OAuth • ...and more
  11. 17.

    Blocking request redux def post(self, user_id): if not self.get_argument(message): raise

    HTTPError(400, "No message") payload = { "user_id": user_id, "message": self.get_argument(message), } self.application.db.messages.insert(payload) response = HTTPClient().fetch("http://foo.bar.baz") self.render("template.html", {"data": response})
  12. 18.

    Non-blocking request with callback @web.asynchronous def post(self, user_id): def callback(response):

    self.render("template.html", {"data": response}) if not self.get_argument(message): raise HTTPError(400, "No message") payload = { "user_id": user_id, "message": self.get_argument(message), } self.application.db.messages.insert(payload) AsyncHTTPClient().fetch("http://foo.bar.baz", callback)
  13. 19.

    A non-blocking request with @gen @web.asynchronous @gen.engine def post(self, user_id):

    def callback(response): if not self.get_argument(message): raise HTTPError(400, "No message") payload = { "user_id": user_id, "message": self.get_argument(message), } self.application.db.messages.insert(payload) resp = yield gen.Task(AsyncHTTPClient().fetch, "http://foo.bar.baz") self.render("template.html", {"data": resp})
  14. 20.

    Non-blocking calls for pymongo @gen.engine @web.asynchronous def post(self, user_id): def

    callback(response): if not self.get_argument(message): raise HTTPError(400, "No message") payload = { "user_id": user_id, "message": self.get_argument(message), } motor.Op(self.application.db.messages.insert, payload) resp = yield gen.Task(AsyncHTTPClient().fetch, "http://foo.bar.baz") self.render("template.html", {"data": resp})
  15. 21.

    Caveats • Always raise in main thread • Don’t forget

    to add decorators • @gen.Tasks are toxic
  16. 22.

    tornado.websocket class EchoHandler(websocket.WebSocketHandler): def open(self): # the same as handler.get()

    logging.debug("WebSocket opened") def on_message(self, message): self.write_message(message) def on_close(self): logging.debug("WebSocket closed") // Client side var ws = new WebSocket("ws://localhost:1337/echo"); ws.onopen = function() { ws.send("Hello, world"); }; ws.onmessage = function (evt) { alert(evt.data); };
  17. 25.

    Example context = { "bugs": {"123": "Fix color scheme"} "uppercase":

    lambda x: x.upper() } <ul> {% for bug_id, description in bugs.items() %} <li>{{ bug_id }} - {{ description }}</li> <p>{{ shout(“whoa”) }}</p> {% end %} </li>
  18. 30.

    Managing instances with supervisord [program:app-8000] directory=/srv/app/ command=python server.py --port=8101 --mongodb_host=localhost:27017

    user=nginx stdout_logfile=/srv/app/logs/app.8000.log [program:app-8001] directory=/srv/app/ command=python server.py --port=8001 --mongodb_host=localhost:27017 user=nginx stdout_logfile=/srv/app/logs/app.8001.log [group:webapp] programs=app-8000,app-8001
  19. 31.

    Load balancing with nginx upstream webapp { server localhost:8000; server

    localhost:8001; } server { server_name foobar; location / { proxy_pass http://webapp; } }
  20. 32.

    Utilize ngnix to it's full power • Do not serve

    static files from Tornado • Whitelist routing (don't bug Tornado with 403s)
  21. 34.

    Deployment / production settings python server.py --port=8001 --mongodb_host=localhost:27017 # In

    application file from tornado.options import define, options define("port", default=1337, type=int) define("mongodb_host", default="localhost:27017", type=str) define("version", default="1", type=str) class Application(tornado.web.Application): def __init__(self): self.db = MongoClient(options.mongodb_host) handlers = [....] super(Application, self).__init__(handlers, version=options.version) def main(): options.parse_command_line() app = Application() application.listen(options.port)
  22. 35.

    Tornado and HTTPS • As easy as adding an option

    to the application instance ssl_options={ "certfile": path_to_certfile, "keyfile": path_to_keyfile, } app = Application(handlers, ssl_options=ssl_options)
  23. 37.

    Sentry integration with Raven • Using mixins to capture exceptions

    class Hadler(SentryMixin, tornado.web.RequestHandler): @tornado.web.asynchronous @tornado.gen.engine def get(self): try: raise ValueError() except Exception as e: response = yield tornado.gen.Task( self.captureException, exc_info=True ) self.finish()
  24. 38.

    Where to go next? • Read through the source •

    Start building rad hacks • ??? • PROFIT!!