Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Tornado in 1 Hour (or Less)

Tornado in 1 Hour (or Less)

Tornado crash-course

Kārlis Lauva

December 04, 2013
Tweet

More Decks by Kārlis Lauva

Other Decks in Programming

Transcript

  1. TORNADO IN 1 HOUR
    (OR LESS)
    Kārlis Lauva
    @skazhy
    python.lv meetup
    2013.12.04

    View Slide

  2. Tornado in 3 bullet points
    ● High performance™ web framework
    ● Async networking library
    ● Gets out of your way (most of the time)

    View Slide

  3. Timeline
    ● 2007.10 - FriendFeed is launched
    ● 2009.08 - Facebook acquires FriendFeed
    ● 2009.09 - Tornado is open sourced
    ● 2013.09 - Last stable release (3.1.1)

    View Slide

  4. Who’s using it?
    …..and your company.

    View Slide

  5. Installing
    pip install tornado
    pip install git+https://github.com/facebook/tornado.git

    View Slide

  6. “Hello, World!”
    import tornado.web
    import tornado.ioloop
    class Handler(tornado.web.RequestHandler):
    def get(self):
    self.finish("Hello World!")
    handlers = [tornado.web.URLSpec(r"/$", Handler)]
    app = tornado.web.Application(handlers)
    app.listen(1337)
    tornado.ioloop.IOLoop.instance().start()

    View Slide

  7. MAIN INGREDIENTS

    View Slide

  8. tornado.ioloop
    ● I/O loop for non-blocking sockets
    ● One per application (or thread)
    ● Uses epoll or kqueue

    View Slide

  9. Example - TCP server
    def connection_ready(sock, fd, events):
    while True:
    try:
    connection, address = sock.accept()
    except socket.error:
    return
    connection.setblocking(0)
    connection.send("Hello")
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0)
    sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    sock.setblocking(0)
    sock.bind(("", 1337))
    sock.listen(128)
    io_loop = ioloop.IOLoop.instance()
    callback = functools.partial(connection_ready, sock)
    io_loop.add_handler(sock.fileno(), callback, io_loop.READ)
    io_loop.start()

    View Slide

  10. tornado.httpserver
    ● Async web server
    ● Utilizes a singleton IOLoop
    ● Subclass of TCP server (duh)

    View Slide

  11. Example - HTTP server
    import tornado.httpserver
    import tornado.ioloop
    def handle_request(request):
    message = "Hello world\n"
    request.write("HTTP/1.1 200 OK\r\nContent-Length: %d\r\n\r\n%s" % \
    (len(message), message))
    request.finish()
    http_server = tornado.httpserver.HTTPServer(handle_request)
    http_server.listen(8080)
    tornado.ioloop.IOLoop.instance().start()

    View Slide

  12. tornado.web.Application
    ● A collection of resource - handler mappings
    ● Setting container
    ● Manages UI modules & rendering
    ● Wraps tornado.httpserver
    ● Supports virtual hosts

    View Slide

  13. tornado.web.RequestHandler
    ● Basic building block
    ● If method exists - call it, else raise
    HTTP 405
    ● Not limited to “standard” verbs
    ● Unified request preparation / finalizing
    methods

    View Slide

  14. A (blocking) request handler
    # in handler listing
    URLSpec(r"/users/(\w+)$", SomeHandler)
    class Somehandler(tornado.web.RequestHandler):
    def post(self, user_id):
    if not self.get_argument(message):
    raise HTTPError(400, "No message")
    payload = {
    "user_id": user_id,
    "message": self.get_argument(message),
    }
    self.application.db.messages.insert(payload)
    response = HTTPClient().fetch("http://foo.bar.baz")
    self.render("template.html", {"data": response})

    View Slide

  15. What else is there?
    ● HTTP client
    ● Templating system
    ● Option management / collection system
    ● Generator interface for async calls
    ● Auth mixins for OAuth
    ● ...and more

    View Slide

  16. ADDING ASYNC-SAUCE

    View Slide

  17. Blocking request redux
    def post(self, user_id):
    if not self.get_argument(message):
    raise HTTPError(400, "No message")
    payload = {
    "user_id": user_id,
    "message": self.get_argument(message),
    }
    self.application.db.messages.insert(payload)
    response = HTTPClient().fetch("http://foo.bar.baz")
    self.render("template.html", {"data": response})

    View Slide

  18. Non-blocking request with callback
    @web.asynchronous
    def post(self, user_id):
    def callback(response):
    self.render("template.html", {"data": response})
    if not self.get_argument(message):
    raise HTTPError(400, "No message")
    payload = {
    "user_id": user_id,
    "message": self.get_argument(message),
    }
    self.application.db.messages.insert(payload)
    AsyncHTTPClient().fetch("http://foo.bar.baz", callback)

    View Slide

  19. A non-blocking request with @gen
    @web.asynchronous
    @gen.engine
    def post(self, user_id):
    def callback(response):
    if not self.get_argument(message):
    raise HTTPError(400, "No message")
    payload = {
    "user_id": user_id,
    "message": self.get_argument(message),
    }
    self.application.db.messages.insert(payload)
    resp = yield gen.Task(AsyncHTTPClient().fetch, "http://foo.bar.baz")
    self.render("template.html", {"data": resp})

    View Slide

  20. Non-blocking calls for pymongo
    @gen.engine
    @web.asynchronous
    def post(self, user_id):
    def callback(response):
    if not self.get_argument(message):
    raise HTTPError(400, "No message")
    payload = {
    "user_id": user_id,
    "message": self.get_argument(message),
    }
    motor.Op(self.application.db.messages.insert, payload)
    resp = yield gen.Task(AsyncHTTPClient().fetch, "http://foo.bar.baz")
    self.render("template.html", {"data": resp})

    View Slide

  21. Caveats
    ● Always raise in main thread
    ● Don’t forget to add decorators
    ● @gen.Tasks are toxic

    View Slide

  22. tornado.websocket
    class EchoHandler(websocket.WebSocketHandler):
    def open(self):
    # the same as handler.get()
    logging.debug("WebSocket opened")
    def on_message(self, message):
    self.write_message(message)
    def on_close(self):
    logging.debug("WebSocket closed")
    // Client side
    var ws = new WebSocket("ws://localhost:1337/echo");
    ws.onopen = function() { ws.send("Hello, world"); };
    ws.onmessage = function (evt) { alert(evt.data); };

    View Slide

  23. TORNADO TEMPLATING

    View Slide

  24. ● Django & Jinja2-esque
    ● Easily extendible
    ● Built-in helpers for reverse urls, etc
    tornado.template

    View Slide

  25. Example
    context = {
    "bugs": {"123": "Fix color scheme"}
    "uppercase": lambda x: x.upper()
    }

    {% for bug_id, description in bugs.items() %}
    {{ bug_id }} - {{ description }}
    {{ shout(“whoa”) }}
    {% end %}

    View Slide

  26. TESTING + DEBUGGING

    View Slide

  27. tornado.testing
    ● Wrappers for unittest.TestCase
    ● Helpers for testing async handlers

    View Slide

  28. PDB
    ● Works as you'd expect
    app.listen(options.port)
    pdb.set_trace()
    tornado.ioloop.IOLoop.instance().start()

    View Slide

  29. REAL WORLD TORNADO

    View Slide

  30. Managing instances with
    supervisord
    [program:app-8000]
    directory=/srv/app/
    command=python server.py --port=8101 --mongodb_host=localhost:27017
    user=nginx
    stdout_logfile=/srv/app/logs/app.8000.log
    [program:app-8001]
    directory=/srv/app/
    command=python server.py --port=8001 --mongodb_host=localhost:27017
    user=nginx
    stdout_logfile=/srv/app/logs/app.8001.log
    [group:webapp]
    programs=app-8000,app-8001

    View Slide

  31. Load balancing with nginx
    upstream webapp {
    server localhost:8000;
    server localhost:8001;
    }
    server {
    server_name foobar;
    location / {
    proxy_pass http://webapp;
    }
    }

    View Slide

  32. Utilize ngnix to it's full power
    ● Do not serve static files from Tornado
    ● Whitelist routing (don't bug Tornado with
    403s)

    View Slide

  33. Load balancing while you load
    balance with Amazon ELB

    View Slide

  34. Deployment / production settings
    python server.py --port=8001 --mongodb_host=localhost:27017
    # In application file
    from tornado.options import define, options
    define("port", default=1337, type=int)
    define("mongodb_host", default="localhost:27017", type=str)
    define("version", default="1", type=str)
    class Application(tornado.web.Application):
    def __init__(self):
    self.db = MongoClient(options.mongodb_host)
    handlers = [....]
    super(Application, self).__init__(handlers, version=options.version)
    def main():
    options.parse_command_line()
    app = Application()
    application.listen(options.port)

    View Slide

  35. Tornado and HTTPS
    ● As easy as adding an option to the
    application instance
    ssl_options={
    "certfile": path_to_certfile,
    "keyfile": path_to_keyfile,
    }
    app = Application(handlers, ssl_options=ssl_options)

    View Slide

  36. Error collection
    ● Override RequestHandler.send_error()
    ● Use error collection mixins

    View Slide

  37. Sentry integration with Raven
    ● Using mixins to capture exceptions
    class Hadler(SentryMixin, tornado.web.RequestHandler):
    @tornado.web.asynchronous
    @tornado.gen.engine
    def get(self):
    try:
    raise ValueError()
    except Exception as e:
    response = yield tornado.gen.Task(
    self.captureException, exc_info=True
    )
    self.finish()

    View Slide

  38. Where to go next?
    ● Read through the source
    ● Start building rad hacks
    ● ???
    ● PROFIT!!

    View Slide

  39. QUESTION TIME!

    View Slide