API-centric Web Development with Tornado or The Great Refactoring Story

API-centric Web Development with Tornado or The Great Refactoring Story

Choose proper stack to build and distribute your apps. You know about monolith style, but what about horizontal scalability, what about your web and mobile clients? How to build API properly and why it's the most important task? You can use several good solutions, balancing between time, money and quality.


Roman Zaiev

June 20, 2015


  1. API-centric Web Development with Tornado or The Great Refactoring Story

    Roman Zaiev, special for
  2. Tornado? Why not Django / Flask / etc

  3. None
  4. is

  5. is prototyping tool

  6. None
  7. djangoproject/ manage.py project/ __init__.py urls.py wsgi.py settings/ __init__.py base.py dev.py

    prod.py blog/ __init__.py models.py managers.py views.py urls.py templates/ blog/ base.html list.html detail.html static/ … tests/ __init__.py test_models.py test_managers.py test_views.py static/ css/ … js/ … templates/ base.html index.html requirements/ base.txt dev.txt test.txt prod.txt Django Project Structure
  8. Modern Django Admin

  9. Django ORM Hell

  10. Django NoSQL? No. SQL!

  11. RedirectView TemplateView DetailView UpdateView CreateView FormView DeleteView DateDetailView ListView DayArchiveView

    ArchiveIndexView TodayArchiveView MonthArchiveView YearArchiveView WeekArchiveView BaseUpdateView BaseCreateView BaseFormView SingleObjectTemplateResponseMixin BaseListView BaseArchiveIndexView MultipleObjectTemplateResponseMixin BaseTodayArchiveView ModelFormMixin ProcessFormMixin TemplateResponseMixin BaseDeleteView BaseDateDetailView BaseDayArchiveView BaseMonthArchiveView BaseYearArchiveView BaseWeekArchiveView FormMixin DeletionMixin BaseDetailView SingleObjectMixin MultipleObjectMixin ContextMixin DateMixin BaseDateListView WeekMixin | connect the dots CBV
  12. Kill me WTF?!

  13. Django is bad

  14. Monolith is bad

  15. first step refactoring

  16. auth landings basket checkout product second step API emails

  17. API to rule them all

  18. ADD. API Driven Development

  19. POST /api/v1/authorize GET /api/v1/user GET /api/v1/product POST /users/ajax_user_auth GET /products/ajax_current_user

    GET /products/ajax_prod_info API Business logic Templates rendering SEO Urls Mob Web UI AJAX UI API Calls
  20. Front Mob API Templates rendering API Calls SEO Urls Web

    UI AJAX POST /api/v1/authorize GET /api/v1/user GET /api/v1/product UI API Calls
  21. API Monolith Services

  22. A A A A A A A A A A

    A A A A A A A A A A A A A A A A auth product landings A A A A basket A P T API
  23. Stack Choice framework? transport? proxy? P T A

  24. A framework Flask? Yes. But no.

  25. Easy Light Fast Async Python 2.7 Django Flask Pyramid Tornado

    A framework
  26. tornadoweb.org A framework

  27. import tornado.ioloop import tornado.web class MainHandler(tornado.web.RequestHandler): def get(self): self.write("Hello, world")

    application = tornado.web.Application([ (r"/", MainHandler), ]) if __name__ == "__main__": application.listen(8888) tornado.ioloop.IOLoop.current().start() A Tornado | Start
  28. A Tornado | Async handler class GenAsyncHandler(tornado.web.RequestHandler): @gen.coroutine def get(self):

    http_client = AsyncHTTPClient() response = yield http_client.fetch("http://async.ua") do_something_with_response(response) self.render("template.html")
  29. P T Tornado transport? proxy?

  30. transport ZeroMQ + Protocol Buffers T

  31. Protobuf T message BannerItem { required string link = 1;

    required string image = 2; optional string title = 3; required string alt = 4; required string selector = 5; required string last_edited = 6; required string last_editor = 7; optional string date_start = 8; optional string date_end = 9; optional int32 order = 10; } ‘\n\x01/\x12\x07img.png\x1a\x04Frau "\x06Muller*\x07.snoopy2\tyesterday:\x02me' new_banner = banner_pb2.BannerItem() new_banner.link = ‘/' new_banner.image = ‘img.png' new_banner.title = ‘Frau' new_banner.alt = ‘Muller' new_banner.selector = ‘.snoopy' new_banner.last_edited = ‘yesterday' new_banner.last_editor = ‘me' new_banner.SerializeToString()
  32. P Tornado ZeroMQ proxy?

  33. proxy P Nginx

  34. Tornado ZeroMQ Nginx

  35. auth product landings basket API

  36. API Monolith Services

  37. API Front

  38. A Tornado | API HTTP class ProductAPIHandler(BaseAPIHandler): def get(self): user_attrs

    = self.request.arguments pbf_response = get_product(self, **user_attrs) response = protobuf_to_dict( pbf_response, ignore_list=['body'] ) if pbf_response.success: product = product_pb2.ProductSample() product.ParseFromString(pbf_response.body) response['body'] = protobuf_to_dict(product) self.write(response)
  39. A Tornado | API ZeroMQ class ProductZMQHandler(BaseZMQHandler): def get(self): user_attrs

    = self.request.arguments pbf_response = get_product(self, **user_attrs) return pbf_response
  40. A Tornado | API Call class ProductHandler(BaseHandler): TEMPLATE = 'pdp.html'

    @tornado.web.asynchronous def get(self, slug): zmq_stream = get_zmq_client(self) requests = [{ 'url': '/api/v1/product', 'callback': self.gen_product, 'data': { 'url': slug }, }] self.pbc_client( zmq_stream=zmq_stream, requests=requests, callback=self.on_fetch, ) ...
  41. A Tornado | API Call Processing class ProductHandler(BaseHandler): ... def

    gen_product(self, response): if not response.success: raise HTTPError(404) product_pbc = product_pb2.ProductSample() product_pbc.ParseFromString(response.body) product_dict = protobuf_to_dict(product_pbc) self.context['product'] = product_dict def on_fetch(self): self.render(self.TEMPLATE, **self.context)
  42. Find your balance

  43. Use proper technologies

  44. github.com/semirook ua.linkedin.com/in/semirook Thank you! And good luck!