Upgrade to Pro — share decks privately, control downloads, hide ads and more …

API-centric Web Development with Tornado or The Great Refactoring Story

API-centric Web Development with Tornado or The Great Refactoring Story

Choose proper stack to build and distribute your apps. You know about monolith style, but what about horizontal scalability, what about your web and mobile clients? How to build API properly and why it's the most important task? You can use several good solutions, balancing between time, money and quality.

Roman Zaiev

June 20, 2015
Tweet

More Decks by Roman Zaiev

Other Decks in Programming

Transcript

  1. API-centric Web Development with Tornado
    or The Great Refactoring Story
    Roman Zaiev, special for

    View full-size slide

  2. Tornado?
    Why not Django / Flask / etc

    View full-size slide

  3. is prototyping tool

    View full-size slide

  4. djangoproject/
    manage.py
    project/
    __init__.py
    urls.py
    wsgi.py
    settings/
    __init__.py
    base.py
    dev.py
    prod.py
    blog/
    __init__.py
    models.py
    managers.py
    views.py
    urls.py
    templates/
    blog/
    base.html
    list.html
    detail.html
    static/

    tests/
    __init__.py
    test_models.py
    test_managers.py
    test_views.py
    static/
    css/

    js/

    templates/
    base.html
    index.html
    requirements/
    base.txt
    dev.txt
    test.txt
    prod.txt
    Django Project Structure

    View full-size slide

  5. Modern
    Django
    Admin

    View full-size slide

  6. Django ORM Hell

    View full-size slide

  7. Django NoSQL? No. SQL!

    View full-size slide

  8. RedirectView
    TemplateView
    DetailView
    UpdateView
    CreateView
    FormView
    DeleteView
    DateDetailView
    ListView
    DayArchiveView
    ArchiveIndexView
    TodayArchiveView
    MonthArchiveView
    YearArchiveView
    WeekArchiveView
    BaseUpdateView
    BaseCreateView
    BaseFormView
    SingleObjectTemplateResponseMixin
    BaseListView
    BaseArchiveIndexView
    MultipleObjectTemplateResponseMixin
    BaseTodayArchiveView
    ModelFormMixin
    ProcessFormMixin
    TemplateResponseMixin
    BaseDeleteView
    BaseDateDetailView
    BaseDayArchiveView
    BaseMonthArchiveView
    BaseYearArchiveView
    BaseWeekArchiveView
    FormMixin
    DeletionMixin
    BaseDetailView
    SingleObjectMixin
    MultipleObjectMixin
    ContextMixin
    DateMixin
    BaseDateListView
    WeekMixin
    | connect the dots
    CBV

    View full-size slide

  9. Kill me
    WTF?!

    View full-size slide

  10. Django is bad

    View full-size slide

  11. Monolith is bad

    View full-size slide

  12. first step
    refactoring

    View full-size slide

  13. auth
    landings
    basket
    checkout
    product
    second step
    API
    emails

    View full-size slide

  14. API to rule them all

    View full-size slide

  15. ADD. API Driven Development

    View full-size slide

  16. POST /api/v1/authorize
    GET /api/v1/user
    GET /api/v1/product
    POST /users/ajax_user_auth
    GET /products/ajax_current_user
    GET /products/ajax_prod_info
    API
    Business logic
    Templates rendering
    SEO Urls
    Mob
    Web
    UI
    AJAX
    UI
    API Calls

    View full-size slide

  17. Front
    Mob
    API
    Templates rendering
    API Calls
    SEO Urls
    Web
    UI
    AJAX
    POST /api/v1/authorize
    GET /api/v1/user
    GET /api/v1/product
    UI
    API Calls

    View full-size slide

  18. API
    Monolith Services

    View full-size slide

  19. A A
    A A
    A A
    A A
    A A
    A
    A
    A A
    A A
    A A
    A
    A
    A
    A
    A A A A
    auth
    product landings
    A A
    A A
    basket
    A
    P
    T
    API

    View full-size slide

  20. Stack Choice
    framework? transport? proxy?
    P
    T
    A

    View full-size slide

  21. A framework
    Flask? Yes. But no.

    View full-size slide

  22. Easy Light Fast Async Python 2.7
    Django
    Flask
    Pyramid
    Tornado
    A framework

    View full-size slide

  23. tornadoweb.org
    A framework

    View full-size slide

  24. import tornado.ioloop
    import tornado.web
    class MainHandler(tornado.web.RequestHandler):
    def get(self):
    self.write("Hello, world")
    application = tornado.web.Application([
    (r"/", MainHandler),
    ])
    if __name__ == "__main__":
    application.listen(8888)
    tornado.ioloop.IOLoop.current().start()
    A Tornado | Start

    View full-size slide

  25. A Tornado | Async handler
    class GenAsyncHandler(tornado.web.RequestHandler):
    @gen.coroutine
    def get(self):
    http_client = AsyncHTTPClient()
    response = yield http_client.fetch("http://async.ua")
    do_something_with_response(response)
    self.render("template.html")

    View full-size slide

  26. P
    T
    Tornado transport? proxy?

    View full-size slide

  27. transport
    ZeroMQ + Protocol Buffers
    T

    View full-size slide

  28. Protobuf
    T
    message BannerItem {
    required string link = 1;
    required string image = 2;
    optional string title = 3;
    required string alt = 4;
    required string selector = 5;
    required string last_edited = 6;
    required string last_editor = 7;
    optional string date_start = 8;
    optional string date_end = 9;
    optional int32 order = 10;
    }
    ‘\n\x01/\x12\x07img.png\x1a\x04Frau
    "\x06Muller*\x07.snoopy2\tyesterday:\x02me'
    new_banner = banner_pb2.BannerItem()
    new_banner.link = ‘/'
    new_banner.image = ‘img.png'
    new_banner.title = ‘Frau'
    new_banner.alt = ‘Muller'
    new_banner.selector = ‘.snoopy'
    new_banner.last_edited = ‘yesterday'
    new_banner.last_editor = ‘me'
    new_banner.SerializeToString()

    View full-size slide

  29. P
    Tornado ZeroMQ proxy?

    View full-size slide

  30. Tornado ZeroMQ Nginx

    View full-size slide

  31. auth
    product landings
    basket
    API

    View full-size slide

  32. API
    Monolith Services

    View full-size slide

  33. A Tornado | API HTTP
    class ProductAPIHandler(BaseAPIHandler):
    def get(self):
    user_attrs = self.request.arguments
    pbf_response = get_product(self, **user_attrs)
    response = protobuf_to_dict(
    pbf_response,
    ignore_list=['body']
    )
    if pbf_response.success:
    product = product_pb2.ProductSample()
    product.ParseFromString(pbf_response.body)
    response['body'] = protobuf_to_dict(product)
    self.write(response)

    View full-size slide

  34. A Tornado | API ZeroMQ
    class ProductZMQHandler(BaseZMQHandler):
    def get(self):
    user_attrs = self.request.arguments
    pbf_response = get_product(self, **user_attrs)
    return pbf_response

    View full-size slide

  35. A Tornado | API Call
    class ProductHandler(BaseHandler):
    TEMPLATE = 'pdp.html'
    @tornado.web.asynchronous
    def get(self, slug):
    zmq_stream = get_zmq_client(self)
    requests = [{
    'url': '/api/v1/product',
    'callback': self.gen_product,
    'data': {
    'url': slug
    },
    }]
    self.pbc_client(
    zmq_stream=zmq_stream,
    requests=requests,
    callback=self.on_fetch,
    )
    ...

    View full-size slide

  36. A Tornado | API Call Processing
    class ProductHandler(BaseHandler):
    ...
    def gen_product(self, response):
    if not response.success:
    raise HTTPError(404)
    product_pbc = product_pb2.ProductSample()
    product_pbc.ParseFromString(response.body)
    product_dict = protobuf_to_dict(product_pbc)
    self.context['product'] = product_dict
    def on_fetch(self):
    self.render(self.TEMPLATE, **self.context)

    View full-size slide

  37. Find your balance

    View full-size slide

  38. Use proper technologies

    View full-size slide

  39. github.com/semirook
    ua.linkedin.com/in/semirook
    Thank you! And good luck!

    View full-size slide