Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Taking Django Distributed
Search
Andrew Godwin
August 16, 2017
Programming
1
960
Taking Django Distributed
A talk I gave at DjangoCon US 2017.
Andrew Godwin
August 16, 2017
Tweet
Share
More Decks by Andrew Godwin
See All by Andrew Godwin
Reconciling Everything
andrewgodwin
1
310
Django Through The Years
andrewgodwin
0
200
Writing Maintainable Software At Scale
andrewgodwin
0
430
A Newcomer's Guide To Airflow's Architecture
andrewgodwin
0
350
Async, Python, and the Future
andrewgodwin
2
650
How To Break Django: With Async
andrewgodwin
1
720
Taking Django's ORM Async
andrewgodwin
0
720
The Long Road To Asynchrony
andrewgodwin
0
650
The Scientist & The Engineer
andrewgodwin
1
750
Other Decks in Programming
See All in Programming
2度もゼロから書き直して、やっとブラウザでぬるぬる動くAIに辿り着いた話
tomoino
0
150
Bytecode Manipulation 으로 생산성 높이기
bigstark
1
190
XSLTで作るBrainfuck処理系
makki_d
0
180
MLOps Japan 勉強会 #52 - 特徴量を言語を越えて一貫して管理する, 『特徴量ドリブン』な MLOps の実現への試み
taniiicom
2
650
ReadMoreTextView
fornewid
0
250
社内での開発コミュニティ活動とモジュラーモノリス標準化事例のご紹介/xPalette and Introduction of Modular monolith standardization
m4maruyama
0
110
Datadog RUM 本番導入までの道
shinter61
1
220
Zennの運営完全に理解した #完全に理解したTalk
wadayusuke
1
180
Blueskyのプラグインを作ってみた
hakkadaikon
1
470
TypeScript を活かしてデザインシステム MCP を作る / #tskaigi_after_night
izumin5210
5
520
実はすごいスピードで進化しているCSS
hayato_yokoyama
0
110
つよそうにふるまい、つよい成果を出すのなら、つよいのかもしれない
irof
1
270
Featured
See All Featured
The Art of Programming - Codeland 2020
erikaheidi
54
13k
The Myth of the Modular Monolith - Day 2 Keynote - Rails World 2024
eileencodes
26
2.8k
Imperfection Machines: The Place of Print at Facebook
scottboms
267
13k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
34
3k
XXLCSS - How to scale CSS and keep your sanity
sugarenia
248
1.3M
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
52
2.8k
Writing Fast Ruby
sferik
628
61k
Bootstrapping a Software Product
garrettdimon
PRO
307
110k
Fantastic passwords and where to find them - at NoRuKo
philnash
51
3.3k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
29
9.5k
It's Worth the Effort
3n
184
28k
Fireside Chat
paigeccino
37
3.5k
Transcript
Taking Django Distributed Andrew Godwin @andrewgodwin Taking Django Distributed
Hi, I’m Andrew Godwin • Django core developer • Senior
Software Engineer at • Needs to stop running towards code on fire
Computers hate you.
This makes distributed hard.
2001: A Space Odyssey Copyright Warner Brothers
It’s time to split things up a bit. But how?
And why?
Code Databases Team
There is no one solution
Read-heavy? Write-heavy? Spiky? Predictable? Chatty?
Code
Use apps! They’re a good start! Ignore the way I
wrote code for the first 5 years of Django.
Formalise interfaces between apps Preferably in an RPC style
Split along those interfaces Into separate processes, or machines
Inventory Payments Presentation
How do you communicate? HTTP? Channels? Smoke signals?
None
None
None
None
Databases
Users Vertically Partitioned Database Images Comments
Main DB Replica Replica Replica Single main database with replication
Partition Tolerant Available Consistent
Non-consistency is everywhere It’s sneaky like that
National Museum of American History
Load Balancing
Equally balanced servers Consistent load times Similar users
Split logic Different processor loads Wildly varying users
Reqs Time
Reqs Time
W E B S O C K E T S
W E B S O C K E T S
• They can last for hours • There’s not many tools that handle them • They have 4 different kinds of failure
Design for failure, and then use it! Kill off sockets
early and often.
Team
Developers are people too! They need time and interesting things
Technical debt can be poisonous But you need a little
bit to compete
Single repo? Multiple repos? Each has distinct advantages.
Teams per service? Split responsibility? Do you split ops/QA across
teams too?
Ownership gaps They’re very hard to see.
Strategies
Don’t go too micro on those services It’s easier in
the short term, but will confuse you in the long term.
Communicate over a service bus Preferably Channels, but you get
to choose.
Work out where to allow old data Build in deliberate
caching or read only modes
Design for future sharding Route everything through one model or
set of functions
Expect long-polls/sockets to die Design for load every time, and
treat as a happy optimisation
Independent, full-stack teams From ops to frontend, per major service
Architect as a part-time position You need some, but not
in an ivory tower
2001: A Space Odyssey Copyright Warner Brothers
Maybe, just maybe, keep that monolith A well maintained and
separated one beats bad distributed
Thanks. Andrew Godwin @andrewgodwin aeracode.org