Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Taking Django Distributed
Search
Andrew Godwin
August 16, 2017
Programming
1
830
Taking Django Distributed
A talk I gave at DjangoCon US 2017.
Andrew Godwin
August 16, 2017
Tweet
Share
More Decks by Andrew Godwin
See All by Andrew Godwin
Reconciling Everything
andrewgodwin
1
230
Django Through The Years
andrewgodwin
0
110
Writing Maintainable Software At Scale
andrewgodwin
0
350
A Newcomer's Guide To Airflow's Architecture
andrewgodwin
0
270
Async, Python, and the Future
andrewgodwin
2
560
How To Break Django: With Async
andrewgodwin
1
610
Taking Django's ORM Async
andrewgodwin
0
620
The Long Road To Asynchrony
andrewgodwin
0
540
The Scientist & The Engineer
andrewgodwin
1
630
Other Decks in Programming
See All in Programming
Composing an API the *right* way (Droidcon Berlin 2024)
zsmb
1
450
DMMプラットフォームにおけるTiDBの導入から運用まで
pospome
7
3k
英語
s_shimotori
1
220
開発部に不満を持っていたCSがエンジニアにジョブチェンしてわかった「勝手に諦めない」ことの大切さ
sakuraikotone
28
16k
CSC307 Lecture 11
javiergs
PRO
0
240
みんなのオブザーバビリティプラットフォームを作ってるんだがパフォーマンスがやばい #mackerelio #srenext
ne_sachirou
0
370
feature環境をGitHub ActionsとCloudFormationでいい感じに管理する
nealle
2
310
DDDを志して3年経ったら「DDDの皮を被ったクリーンアーキテクチャ」になった話【デブサミ2024夏】
texmeijin
1
620
Terraformテスト入門
msato
0
540
AWS CDKにおける「再利用性」を考える / aws-cdk-reusability
gotok365
6
1.3k
小さな開発会社を作った理由
polidog
0
1.9k
Ruby メモリ管理 プログラミング
megmogmog1965
0
130
Featured
See All Featured
From Idea to $5000 a Month in 5 Months
shpigford
377
46k
KATA
mclloyd
20
13k
Intergalactic Javascript Robots from Outer Space
tanoku
266
26k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
18
1.2k
Teambox: Starting and Learning
jrom
130
8.6k
Git: the NoSQL Database
bkeepers
PRO
423
64k
Infographics Made Easy
chrislema
238
18k
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
13
430
Designing for Performance
lara
604
67k
Music & Morning Musume
bryan
43
5.9k
Building Flexible Design Systems
yeseniaperezcruz
323
37k
Why You Should Never Use an ORM
jnunemaker
PRO
51
8.9k
Transcript
Taking Django Distributed Andrew Godwin @andrewgodwin Taking Django Distributed
Hi, I’m Andrew Godwin • Django core developer • Senior
Software Engineer at • Needs to stop running towards code on fire
Computers hate you.
This makes distributed hard.
2001: A Space Odyssey Copyright Warner Brothers
It’s time to split things up a bit. But how?
And why?
Code Databases Team
There is no one solution
Read-heavy? Write-heavy? Spiky? Predictable? Chatty?
Code
Use apps! They’re a good start! Ignore the way I
wrote code for the first 5 years of Django.
Formalise interfaces between apps Preferably in an RPC style
Split along those interfaces Into separate processes, or machines
Inventory Payments Presentation
How do you communicate? HTTP? Channels? Smoke signals?
None
None
None
None
Databases
Users Vertically Partitioned Database Images Comments
Main DB Replica Replica Replica Single main database with replication
Partition Tolerant Available Consistent
Non-consistency is everywhere It’s sneaky like that
National Museum of American History
Load Balancing
Equally balanced servers Consistent load times Similar users
Split logic Different processor loads Wildly varying users
Reqs Time
Reqs Time
W E B S O C K E T S
W E B S O C K E T S
• They can last for hours • There’s not many tools that handle them • They have 4 different kinds of failure
Design for failure, and then use it! Kill off sockets
early and often.
Team
Developers are people too! They need time and interesting things
Technical debt can be poisonous But you need a little
bit to compete
Single repo? Multiple repos? Each has distinct advantages.
Teams per service? Split responsibility? Do you split ops/QA across
teams too?
Ownership gaps They’re very hard to see.
Strategies
Don’t go too micro on those services It’s easier in
the short term, but will confuse you in the long term.
Communicate over a service bus Preferably Channels, but you get
to choose.
Work out where to allow old data Build in deliberate
caching or read only modes
Design for future sharding Route everything through one model or
set of functions
Expect long-polls/sockets to die Design for load every time, and
treat as a happy optimisation
Independent, full-stack teams From ops to frontend, per major service
Architect as a part-time position You need some, but not
in an ivory tower
2001: A Space Odyssey Copyright Warner Brothers
Maybe, just maybe, keep that monolith A well maintained and
separated one beats bad distributed
Thanks. Andrew Godwin @andrewgodwin aeracode.org