Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Taking Django Distributed
Search
Andrew Godwin
August 16, 2017
Programming
1
990
Taking Django Distributed
A talk I gave at DjangoCon US 2017.
Andrew Godwin
August 16, 2017
Tweet
Share
More Decks by Andrew Godwin
See All by Andrew Godwin
Reconciling Everything
andrewgodwin
1
330
Django Through The Years
andrewgodwin
0
220
Writing Maintainable Software At Scale
andrewgodwin
0
460
A Newcomer's Guide To Airflow's Architecture
andrewgodwin
0
370
Async, Python, and the Future
andrewgodwin
2
680
How To Break Django: With Async
andrewgodwin
1
740
Taking Django's ORM Async
andrewgodwin
0
740
The Long Road To Asynchrony
andrewgodwin
0
680
The Scientist & The Engineer
andrewgodwin
1
790
Other Decks in Programming
See All in Programming
エンジニアインターン「Treasure」とHonoの2年、そして未来へ / Our Journey with Hono Two Years at Treasure and Beyond
carta_engineering
0
440
Leading Effective Engineering Teams in the AI Era
addyosmani
7
640
The Past, Present, and Future of Enterprise Java
ivargrimstad
0
300
Server Side Kotlin Meetup vol.16: 内部動作を理解して ハイパフォーマンスなサーバサイド Kotlin アプリケーションを書こう
ternbusty
3
260
SwiftDataを使って10万件のデータを読み書きする
akidon0000
0
240
Domain-centric? Why Hexagonal, Onion, and Clean Architecture Are Answers to the Wrong Question
olivergierke
3
980
pnpm に provenance のダウングレード を検出する PR を出してみた
ryo_manba
1
160
TransformerからMCPまで(現代AIを理解するための羅針盤)
mickey_kubo
7
5.5k
Webサーバーサイド言語としてのRustについて
kouyuume
1
4.9k
理論と実務のギャップを超える
eycjur
0
180
AI駆動で0→1をやって見えた光と伸びしろ
passion0102
1
860
Pythonに漸進的に型をつける
nealle
1
130
Featured
See All Featured
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
30
2.9k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
132
19k
Automating Front-end Workflow
addyosmani
1371
200k
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
marcduiker
34
2.3k
Rails Girls Zürich Keynote
gr2m
95
14k
Building a Scalable Design System with Sketch
lauravandoore
463
33k
Practical Orchestrator
shlominoach
190
11k
RailsConf 2023
tenderlove
30
1.3k
Improving Core Web Vitals using Speculation Rules API
sergeychernyshev
21
1.2k
GraphQLの誤解/rethinking-graphql
sonatard
73
11k
Optimizing for Happiness
mojombo
379
70k
Principles of Awesome APIs and How to Build Them.
keavy
127
17k
Transcript
Taking Django Distributed Andrew Godwin @andrewgodwin Taking Django Distributed
Hi, I’m Andrew Godwin • Django core developer • Senior
Software Engineer at • Needs to stop running towards code on fire
Computers hate you.
This makes distributed hard.
2001: A Space Odyssey Copyright Warner Brothers
It’s time to split things up a bit. But how?
And why?
Code Databases Team
There is no one solution
Read-heavy? Write-heavy? Spiky? Predictable? Chatty?
Code
Use apps! They’re a good start! Ignore the way I
wrote code for the first 5 years of Django.
Formalise interfaces between apps Preferably in an RPC style
Split along those interfaces Into separate processes, or machines
Inventory Payments Presentation
How do you communicate? HTTP? Channels? Smoke signals?
None
None
None
None
Databases
Users Vertically Partitioned Database Images Comments
Main DB Replica Replica Replica Single main database with replication
Partition Tolerant Available Consistent
Non-consistency is everywhere It’s sneaky like that
National Museum of American History
Load Balancing
Equally balanced servers Consistent load times Similar users
Split logic Different processor loads Wildly varying users
Reqs Time
Reqs Time
W E B S O C K E T S
W E B S O C K E T S
• They can last for hours • There’s not many tools that handle them • They have 4 different kinds of failure
Design for failure, and then use it! Kill off sockets
early and often.
Team
Developers are people too! They need time and interesting things
Technical debt can be poisonous But you need a little
bit to compete
Single repo? Multiple repos? Each has distinct advantages.
Teams per service? Split responsibility? Do you split ops/QA across
teams too?
Ownership gaps They’re very hard to see.
Strategies
Don’t go too micro on those services It’s easier in
the short term, but will confuse you in the long term.
Communicate over a service bus Preferably Channels, but you get
to choose.
Work out where to allow old data Build in deliberate
caching or read only modes
Design for future sharding Route everything through one model or
set of functions
Expect long-polls/sockets to die Design for load every time, and
treat as a happy optimisation
Independent, full-stack teams From ops to frontend, per major service
Architect as a part-time position You need some, but not
in an ivory tower
2001: A Space Odyssey Copyright Warner Brothers
Maybe, just maybe, keep that monolith A well maintained and
separated one beats bad distributed
Thanks. Andrew Godwin @andrewgodwin aeracode.org