API-centric Web Development with Tornado or The Great Refactoring Story

Slide 1

Slide 1 text

API-centric Web Development with Tornado or The Great Refactoring Story Roman Zaiev, special for

Slide 2

Slide 2 text

Tornado? Why not Django / Flask / etc

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

Slide 5

Slide 5 text

is prototyping tool

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

djangoproject/ manage.py project/ __init__.py urls.py wsgi.py settings/ __init__.py base.py dev.py prod.py blog/ __init__.py models.py managers.py views.py urls.py templates/ blog/ base.html list.html detail.html static/ … tests/ __init__.py test_models.py test_managers.py test_views.py static/ css/ … js/ … templates/ base.html index.html requirements/ base.txt dev.txt test.txt prod.txt Django Project Structure

Slide 8

Slide 8 text

Modern Django Admin

Slide 9

Slide 9 text

Django ORM Hell

Slide 10

Slide 10 text

Django NoSQL? No. SQL!

Slide 11

Slide 11 text

RedirectView TemplateView DetailView UpdateView CreateView FormView DeleteView DateDetailView ListView DayArchiveView ArchiveIndexView TodayArchiveView MonthArchiveView YearArchiveView WeekArchiveView BaseUpdateView BaseCreateView BaseFormView SingleObjectTemplateResponseMixin BaseListView BaseArchiveIndexView MultipleObjectTemplateResponseMixin BaseTodayArchiveView ModelFormMixin ProcessFormMixin TemplateResponseMixin BaseDeleteView BaseDateDetailView BaseDayArchiveView BaseMonthArchiveView BaseYearArchiveView BaseWeekArchiveView FormMixin DeletionMixin BaseDetailView SingleObjectMixin MultipleObjectMixin ContextMixin DateMixin BaseDateListView WeekMixin | connect the dots CBV

Slide 12

Slide 12 text

Kill me WTF?!

Slide 13

Slide 13 text

Django is bad

Slide 14

Slide 14 text

Monolith is bad

Slide 15

Slide 15 text

ﬁrst step refactoring

Slide 16

Slide 16 text

auth landings basket checkout product second step API emails

Slide 17

Slide 17 text

API to rule them all

Slide 18

Slide 18 text

ADD. API Driven Development

Slide 19

Slide 19 text

POST /api/v1/authorize GET /api/v1/user GET /api/v1/product POST /users/ajax_user_auth GET /products/ajax_current_user GET /products/ajax_prod_info API Business logic Templates rendering SEO Urls Mob Web UI AJAX UI API Calls

Slide 20

Slide 20 text

Front Mob API Templates rendering API Calls SEO Urls Web UI AJAX POST /api/v1/authorize GET /api/v1/user GET /api/v1/product UI API Calls

Slide 21

Slide 21 text

API Monolith Services

Slide 22

Slide 22 text

A A A A A A A A A A A A A A A A A A A A A A A A A A auth product landings A A A A basket A P T API

Slide 23

Slide 23 text

Stack Choice framework? transport? proxy? P T A

Slide 24

Slide 24 text

A framework Flask? Yes. But no.

Slide 25

Slide 25 text

Easy Light Fast Async Python 2.7 Django Flask Pyramid Tornado A framework

Slide 26

Slide 26 text

tornadoweb.org A framework

Slide 27

Slide 27 text

import tornado.ioloop import tornado.web class MainHandler(tornado.web.RequestHandler): def get(self): self.write("Hello, world") application = tornado.web.Application([ (r"/", MainHandler), ]) if __name__ == "__main__": application.listen(8888) tornado.ioloop.IOLoop.current().start() A Tornado | Start

Slide 28

Slide 28 text

A Tornado | Async handler class GenAsyncHandler(tornado.web.RequestHandler): @gen.coroutine def get(self): http_client = AsyncHTTPClient() response = yield http_client.fetch("http://async.ua") do_something_with_response(response) self.render("template.html")

Slide 29

Slide 29 text

P T Tornado transport? proxy?

Slide 30

Slide 30 text

transport ZeroMQ + Protocol Buffers T

Slide 31

Slide 31 text

Protobuf T message BannerItem { required string link = 1; required string image = 2; optional string title = 3; required string alt = 4; required string selector = 5; required string last_edited = 6; required string last_editor = 7; optional string date_start = 8; optional string date_end = 9; optional int32 order = 10; } ‘\n\x01/\x12\x07img.png\x1a\x04Frau "\x06Muller*\x07.snoopy2\tyesterday:\x02me' new_banner = banner_pb2.BannerItem() new_banner.link = ‘/' new_banner.image = ‘img.png' new_banner.title = ‘Frau' new_banner.alt = ‘Muller' new_banner.selector = ‘.snoopy' new_banner.last_edited = ‘yesterday' new_banner.last_editor = ‘me' new_banner.SerializeToString()

Slide 32

Slide 32 text

P Tornado ZeroMQ proxy?

Slide 33

Slide 33 text

proxy P Nginx

Slide 34

Slide 34 text

Tornado ZeroMQ Nginx

Slide 35

Slide 35 text

auth product landings basket API

Slide 36

Slide 36 text

API Monolith Services

Slide 37

Slide 37 text

API Front

Slide 38

Slide 38 text

A Tornado | API HTTP class ProductAPIHandler(BaseAPIHandler): def get(self): user_attrs = self.request.arguments pbf_response = get_product(self, **user_attrs) response = protobuf_to_dict( pbf_response, ignore_list=['body'] ) if pbf_response.success: product = product_pb2.ProductSample() product.ParseFromString(pbf_response.body) response['body'] = protobuf_to_dict(product) self.write(response)

Slide 39

Slide 39 text

A Tornado | API ZeroMQ class ProductZMQHandler(BaseZMQHandler): def get(self): user_attrs = self.request.arguments pbf_response = get_product(self, **user_attrs) return pbf_response

Slide 40

Slide 40 text

A Tornado | API Call class ProductHandler(BaseHandler): TEMPLATE = 'pdp.html' @tornado.web.asynchronous def get(self, slug): zmq_stream = get_zmq_client(self) requests = [{ 'url': '/api/v1/product', 'callback': self.gen_product, 'data': { 'url': slug }, }] self.pbc_client( zmq_stream=zmq_stream, requests=requests, callback=self.on_fetch, ) ...

Slide 41

Slide 41 text

A Tornado | API Call Processing class ProductHandler(BaseHandler): ... def gen_product(self, response): if not response.success: raise HTTPError(404) product_pbc = product_pb2.ProductSample() product_pbc.ParseFromString(response.body) product_dict = protobuf_to_dict(product_pbc) self.context['product'] = product_dict def on_fetch(self): self.render(self.TEMPLATE, **self.context)