Slide 1
Slide 1 text
Designing Fast &
Scalable Python
MicroServices with
Django
Dinesh A. Joshi
@dineshjoshi
Feb 2018
Slide 2
Slide 2 text
About
● Senior Software Engineer
● Distributed Systems @ Georgia Tech, Atlanta
● Python Enthusiast
Slide 3
Slide 3 text
Outline
● Reference App
● Request & Response Pipeline
● Middleware
● ORM
● Measuring Performance
● Caching
● Async Processing
● Questions
Slide 4
Slide 4 text
● Python 2 or 3
● Packages
○ django (1.9+)
○ django-restframework
○ django-redis
○ psycopg2
○ celery
● Postgres
● Redis
Reference App
Database
(Postgres)
Cache
(Redis)
Celery
Tasks
Queue
Django
App
django
App
Slide 5
Slide 5 text
Request &
Response
Pipeline
Turn a Request into a Response...
Slide 6
Slide 6 text
Request & Response Pipeline
1
WSGI Handler
Middleware
(Request)
2
View
3
Request duration
Middleware
(Response)
4
Don’t Block!
Slide 7
Slide 7 text
Middleware
The code that runs in the middle...
Slide 8
Slide 8 text
● Audit & Remove unnecessary middleware
● Use
○ Conditional GET middleware
○ GzipMiddleware
○ Cached Sessions
Middleware: Accelerating Responses
Slide 9
Slide 9 text
Middleware
Slide 10
Slide 10 text
ORM
Mapping Objects to Database
Table, etc...
Slide 11
Slide 11 text
● Add indexes
● Enable persistent connections to database
● Use Connection Pooling
ORM (Settings)
Slide 12
Slide 12 text
● prefetch_related()
○ many-to-many, many-to-one relationships
● select_related()
○ Single valued relationships
● Reuse QuerySets
ORM (Code)
Slide 13
Slide 13 text
len(cars)
cars.count()
ORM (Code)
Slide 14
Slide 14 text
ORM (Code)
len(cars)
cars.count()
Slide 15
Slide 15 text
Measurement
Something about premature
optimization being root of all evil...
Slide 16
Slide 16 text
django-debug-toolbar
Slide 17
Slide 17 text
django-debug-toolbar
Slide 18
Slide 18 text
django-debug-toolbar
Slide 19
Slide 19 text
Caching
When everything else fails...
Slide 20
Slide 20 text
● Read heavy workloads
● Boosts performance many-folds for cached responses
● View Level or custom
● Invalidation is complex
Caching
Slide 21
Slide 21 text
Caching
Slide 22
Slide 22 text
Async
Processing
Why do it now when you can do it
later…?
Slide 23
Slide 23 text
Request & Response Pipeline
1
WSGI Handler
Middleware
(Request)
2
View
3
Request duration
Middleware
(Response)
4
Don’t Block!
Slide 24
Slide 24 text
Async Processing
Database
(Postgres)
Cache
(Redis)
Celery
Tasks
Queue
Django
App
django
App
1 2
3
Service
X
4
Slide 25
Slide 25 text
Questions?