Tornado in 1 Hour (or Less)

Transcript

1. TORNADO IN 1 HOUR (OR LESS) Kārlis Lauva @skazhy python.lv

   meetup 2013.12.04

2. Tornado in 3 bullet points • High performance™ web framework

   • Async networking library • Gets out of your way (most of the time)

3. Timeline • 2007.10 - FriendFeed is launched • 2009.08 -

   Facebook acquires FriendFeed • 2009.09 - Tornado is open sourced • 2013.09 - Last stable release (3.1.1)

4. Who’s using it? …..and your company.

5. Installing pip install tornado pip install git+https://github.com/facebook/tornado.git

6. “Hello, World!” import tornado.web import tornado.ioloop class Handler(tornado.web.RequestHandler): def get(self):

   self.finish("Hello World!") handlers = [tornado.web.URLSpec(r"/$", Handler)] app = tornado.web.Application(handlers) app.listen(1337) tornado.ioloop.IOLoop.instance().start()

7. MAIN INGREDIENTS

8. tornado.ioloop • I/O loop for non-blocking sockets • One per

   application (or thread) • Uses epoll or kqueue

9. Example - TCP server def connection_ready(sock, fd, events): while True:

   try: connection, address = sock.accept() except socket.error: return connection.setblocking(0) connection.send("Hello") sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0) sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) sock.setblocking(0) sock.bind(("", 1337)) sock.listen(128) io_loop = ioloop.IOLoop.instance() callback = functools.partial(connection_ready, sock) io_loop.add_handler(sock.fileno(), callback, io_loop.READ) io_loop.start()

10. tornado.httpserver • Async web server • Utilizes a singleton IOLoop

    • Subclass of TCP server (duh)

11. Example - HTTP server import tornado.httpserver import tornado.ioloop def handle_request(request):

    message = "Hello world\n" request.write("HTTP/1.1 200 OK\r\nContent-Length: %d\r\n\r\n%s" % \ (len(message), message)) request.finish() http_server = tornado.httpserver.HTTPServer(handle_request) http_server.listen(8080) tornado.ioloop.IOLoop.instance().start()

12. tornado.web.Application • A collection of resource - handler mappings •

    Setting container • Manages UI modules & rendering • Wraps tornado.httpserver • Supports virtual hosts

13. tornado.web.RequestHandler • Basic building block • If method exists -

    call it, else raise HTTP 405 • Not limited to “standard” verbs • Unified request preparation / finalizing methods

14. A (blocking) request handler # in handler listing URLSpec(r"/users/(\w+)$", SomeHandler)

    class Somehandler(tornado.web.RequestHandler): def post(self, user_id): if not self.get_argument(message): raise HTTPError(400, "No message") payload = { "user_id": user_id, "message": self.get_argument(message), } self.application.db.messages.insert(payload) response = HTTPClient().fetch("http://foo.bar.baz") self.render("template.html", {"data": response})

15. What else is there? • HTTP client • Templating system

    • Option management / collection system • Generator interface for async calls • Auth mixins for OAuth • ...and more

16. ADDING ASYNC-SAUCE

17. Blocking request redux def post(self, user_id): if not self.get_argument(message): raise

    HTTPError(400, "No message") payload = { "user_id": user_id, "message": self.get_argument(message), } self.application.db.messages.insert(payload) response = HTTPClient().fetch("http://foo.bar.baz") self.render("template.html", {"data": response})

18. Non-blocking request with callback @web.asynchronous def post(self, user_id): def callback(response):

    self.render("template.html", {"data": response}) if not self.get_argument(message): raise HTTPError(400, "No message") payload = { "user_id": user_id, "message": self.get_argument(message), } self.application.db.messages.insert(payload) AsyncHTTPClient().fetch("http://foo.bar.baz", callback)

19. A non-blocking request with @gen @web.asynchronous @gen.engine def post(self, user_id):

    def callback(response): if not self.get_argument(message): raise HTTPError(400, "No message") payload = { "user_id": user_id, "message": self.get_argument(message), } self.application.db.messages.insert(payload) resp = yield gen.Task(AsyncHTTPClient().fetch, "http://foo.bar.baz") self.render("template.html", {"data": resp})

20. Non-blocking calls for pymongo @gen.engine @web.asynchronous def post(self, user_id): def

    callback(response): if not self.get_argument(message): raise HTTPError(400, "No message") payload = { "user_id": user_id, "message": self.get_argument(message), } motor.Op(self.application.db.messages.insert, payload) resp = yield gen.Task(AsyncHTTPClient().fetch, "http://foo.bar.baz") self.render("template.html", {"data": resp})

21. Caveats • Always raise in main thread • Don’t forget

    to add decorators • @gen.Tasks are toxic

22. tornado.websocket class EchoHandler(websocket.WebSocketHandler): def open(self): # the same as handler.get()

    logging.debug("WebSocket opened") def on_message(self, message): self.write_message(message) def on_close(self): logging.debug("WebSocket closed") // Client side var ws = new WebSocket("ws://localhost:1337/echo"); ws.onopen = function() { ws.send("Hello, world"); }; ws.onmessage = function (evt) { alert(evt.data); };

23. TORNADO TEMPLATING

24. • Django & Jinja2-esque • Easily extendible • Built-in helpers

    for reverse urls, etc tornado.template

25. Example context = { "bugs": {"123": "Fix color scheme"} "uppercase":

    lambda x: x.upper() } <ul> {% for bug_id, description in bugs.items() %} <li>{{ bug_id }} - {{ description }}</li> <p>{{ shout(“whoa”) }}</p> {% end %} </li>

26. TESTING + DEBUGGING

27. tornado.testing • Wrappers for unittest.TestCase • Helpers for testing async

    handlers

28. PDB • Works as you'd expect app.listen(options.port) pdb.set_trace() tornado.ioloop.IOLoop.instance().start()

29. REAL WORLD TORNADO

30. Managing instances with supervisord [program:app-8000] directory=/srv/app/ command=python server.py --port=8101 --mongodb_host=localhost:27017

    user=nginx stdout_logfile=/srv/app/logs/app.8000.log [program:app-8001] directory=/srv/app/ command=python server.py --port=8001 --mongodb_host=localhost:27017 user=nginx stdout_logfile=/srv/app/logs/app.8001.log [group:webapp] programs=app-8000,app-8001

31. Load balancing with nginx upstream webapp { server localhost:8000; server

    localhost:8001; } server { server_name foobar; location / { proxy_pass http://webapp; } }

32. Utilize ngnix to it's full power • Do not serve

    static files from Tornado • Whitelist routing (don't bug Tornado with 403s)

33. Load balancing while you load balance with Amazon ELB

34. Deployment / production settings python server.py --port=8001 --mongodb_host=localhost:27017 # In

    application file from tornado.options import define, options define("port", default=1337, type=int) define("mongodb_host", default="localhost:27017", type=str) define("version", default="1", type=str) class Application(tornado.web.Application): def __init__(self): self.db = MongoClient(options.mongodb_host) handlers = [....] super(Application, self).__init__(handlers, version=options.version) def main(): options.parse_command_line() app = Application() application.listen(options.port)

35. Tornado and HTTPS • As easy as adding an option

    to the application instance ssl_options={ "certfile": path_to_certfile, "keyfile": path_to_keyfile, } app = Application(handlers, ssl_options=ssl_options)

36. Error collection • Override RequestHandler.send_error() • Use error collection mixins

37. Sentry integration with Raven • Using mixins to capture exceptions

    class Hadler(SentryMixin, tornado.web.RequestHandler): @tornado.web.asynchronous @tornado.gen.engine def get(self): try: raise ValueError() except Exception as e: response = yield tornado.gen.Task( self.captureException, exc_info=True ) self.finish()

38. Where to go next? • Read through the source •

    Start building rad hacks • ??? • PROFIT!!

39. QUESTION TIME!