Tornado in 1 Hour (or Less)

Tornado crash-course

Transcript

1. TORNADO IN 1 HOUR (OR LESS) Kārlis Lauva @skazhy python.lv meetup 2013.12.04
2. Tornado in 3 bullet points • High performance™ web framework • Async networking library • Gets out of your way (most of the time)
3. Timeline • 2007.10 - FriendFeed is launched • 2009.08 - Facebook acquires FriendFeed • 2009.09 - Tornado is open sourced • 2013.09 - Last stable release (3.1.1)
4. Who’s using it? …..and your company.
5. Installing pip install tornado pip install git+https://github.com/facebook/tornado.git
6. “Hello, World!” import tornado.web import tornado.ioloop class Handler(tornado.web.RequestHandler): def get(self): self.finish("Hello World!") handlers = [tornado.web.URLSpec(r"/$", Handler)] app = tornado.web.Application(handlers) app.listen(1337) tornado.ioloop.IOLoop.instance().start()
7. MAIN INGREDIENTS
8. tornado.ioloop • I/O loop for non-blocking sockets • One per application (or thread) • Uses epoll or kqueue
9. Example - TCP server def connection_ready(sock, fd, events): while True: try: connection, address = sock.accept() except socket.error: return connection.setblocking(0) connection.send("Hello") sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0) sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) sock.setblocking(0) sock.bind(("", 1337)) sock.listen(128) io_loop = ioloop.IOLoop.instance() callback = functools.partial(connection_ready, sock) io_loop.add_handler(sock.fileno(), callback, io_loop.READ) io_loop.start()
10. tornado.httpserver • Async web server • Utilizes a singleton IOLoop • Subclass of TCP server (duh)
11. Example - HTTP server import tornado.httpserver import tornado.ioloop def handle_request(request): message = "Hello world\n" request.write("HTTP/1.1 200 OK\r\nContent-Length: %d\r\n\r\n%s" % \ (len(message), message)) request.finish() http_server = tornado.httpserver.HTTPServer(handle_request) http_server.listen(8080) tornado.ioloop.IOLoop.instance().start()
12. tornado.web.Application • A collection of resource - handler mappings • Setting container • Manages UI modules & rendering • Wraps tornado.httpserver • Supports virtual hosts
13. tornado.web.RequestHandler • Basic building block • If method exists - call it, else raise HTTP 405 • Not limited to “standard” verbs • Unified request preparation / finalizing methods
14. A (blocking) request handler # in handler listing URLSpec(r"/users/(\w+)$", SomeHandler) class Somehandler(tornado.web.RequestHandler): def post(self, user_id): if not self.get_argument(message): raise HTTPError(400, "No message") payload = { "user_id": user_id, "message": self.get_argument(message), } self.application.db.messages.insert(payload) response = HTTPClient().fetch("http://foo.bar.baz") self.render("template.html", {"data": response})
15. What else is there? • HTTP client • Templating system • Option management / collection system • Generator interface for async calls • Auth mixins for OAuth • ...and more
16. ADDING ASYNC-SAUCE
17. Blocking request redux def post(self, user_id): if not self.get_argument(message): raise HTTPError(400, "No message") payload = { "user_id": user_id, "message": self.get_argument(message), } self.application.db.messages.insert(payload) response = HTTPClient().fetch("http://foo.bar.baz") self.render("template.html", {"data": response})
18. Non-blocking request with callback @web.asynchronous def post(self, user_id): def callback(response): self.render("template.html", {"data": response}) if not self.get_argument(message): raise HTTPError(400, "No message") payload = { "user_id": user_id, "message": self.get_argument(message), } self.application.db.messages.insert(payload) AsyncHTTPClient().fetch("http://foo.bar.baz", callback)
19. A non-blocking request with @gen @web.asynchronous @gen.engine def post(self, user_id): def callback(response): if not self.get_argument(message): raise HTTPError(400, "No message") payload = { "user_id": user_id, "message": self.get_argument(message), } self.application.db.messages.insert(payload) resp = yield gen.Task(AsyncHTTPClient().fetch, "http://foo.bar.baz") self.render("template.html", {"data": resp})
20. Non-blocking calls for pymongo @gen.engine @web.asynchronous def post(self, user_id): def callback(response): if not self.get_argument(message): raise HTTPError(400, "No message") payload = { "user_id": user_id, "message": self.get_argument(message), } motor.Op(self.application.db.messages.insert, payload) resp = yield gen.Task(AsyncHTTPClient().fetch, "http://foo.bar.baz") self.render("template.html", {"data": resp})
21. Caveats • Always raise in main thread • Don’t forget to add decorators • @gen.Tasks are toxic
22. tornado.websocket class EchoHandler(websocket.WebSocketHandler): def open(self): # the same as handler.get() logging.debug("WebSocket opened") def on_message(self, message): self.write_message(message) def on_close(self): logging.debug("WebSocket closed") // Client side var ws = new WebSocket("ws://localhost:1337/echo"); ws.onopen = function() { ws.send("Hello, world"); }; ws.onmessage = function (evt) { alert(evt.data); };
23. TORNADO TEMPLATING
24. • Django & Jinja2-esque • Easily extendible • Built-in helpers for reverse urls, etc tornado.template
25. Example context = { "bugs": {"123": "Fix color scheme"} "uppercase": lambda x: x.upper() } <ul> {% for bug_id, description in bugs.items() %} <li>{{ bug_id }} - {{ description }}</li> <p>{{ shout(“whoa”) }}</p> {% end %} </li>
26. TESTING + DEBUGGING
27. tornado.testing • Wrappers for unittest.TestCase • Helpers for testing async handlers
28. PDB • Works as you'd expect app.listen(options.port) pdb.set_trace() tornado.ioloop.IOLoop.instance().start()
29. REAL WORLD TORNADO
30. Managing instances with supervisord [program:app-8000] directory=/srv/app/ command=python server.py --port=8101 --mongodb_host=localhost:27017 user=nginx stdout_logfile=/srv/app/logs/app.8000.log [program:app-8001] directory=/srv/app/ command=python server.py --port=8001 --mongodb_host=localhost:27017 user=nginx stdout_logfile=/srv/app/logs/app.8001.log [group:webapp] programs=app-8000,app-8001
31. Load balancing with nginx upstream webapp { server localhost:8000; server localhost:8001; } server { server_name foobar; location / { proxy_pass http://webapp; } }
32. Utilize ngnix to it's full power • Do not serve static files from Tornado • Whitelist routing (don't bug Tornado with 403s)
33. Load balancing while you load balance with Amazon ELB
34. Deployment / production settings python server.py --port=8001 --mongodb_host=localhost:27017 # In application file from tornado.options import define, options define("port", default=1337, type=int) define("mongodb_host", default="localhost:27017", type=str) define("version", default="1", type=str) class Application(tornado.web.Application): def __init__(self): self.db = MongoClient(options.mongodb_host) handlers = [....] super(Application, self).__init__(handlers, version=options.version) def main(): options.parse_command_line() app = Application() application.listen(options.port)
35. Tornado and HTTPS • As easy as adding an option to the application instance ssl_options={ "certfile": path_to_certfile, "keyfile": path_to_keyfile, } app = Application(handlers, ssl_options=ssl_options)
36. Error collection • Override RequestHandler.send_error() • Use error collection mixins
37. Sentry integration with Raven • Using mixins to capture exceptions class Hadler(SentryMixin, tornado.web.RequestHandler): @tornado.web.asynchronous @tornado.gen.engine def get(self): try: raise ValueError() except Exception as e: response = yield tornado.gen.Task( self.captureException, exc_info=True ) self.finish()
38. Where to go next? • Read through the source • Start building rad hacks • ??? • PROFIT!!
39. QUESTION TIME!