Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
Migrations Under The Hood
Search
Andrew Godwin
November 14, 2014
Programming
13k
6
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
Migrations Under The Hood
A talk I gave at Django Under The Hood 2014.
Andrew Godwin
November 14, 2014
Transcript
52 slides
Migrations UNDER THE HOOD Andrew Godwin Migrations
Andrew Godwin Author of South migrations library Hi, I'm Author
of 1.7 Django migrations Senior Software Engineer at Eventbrite Apparently at every Django conference
Migrations
Migrations
Other talks are available. This is a technical dive!
The Initial Plan Django Schema backend ORM Hooks South 2
Migration handling User interface
TheBRevisedBPlan Django SchemaBbackend ORMBHooks SouthB2 MigrationBhandling UserBinterface BackportBforB1.4B-B1.6
The Revised Revised Plan Django Schema backend ORM Hooks Migration
handling User interface
Logically Separate SchemaEditor Schema Migrations field.deconstruct() ModelOptions.apps Operations Loader /
Graph Executor Autodetector Optimiser State
Logically Separate SchemaEditor Schema Migrations field.deconstruct() ModelOptions.apps Operations Loader /
Graph Executor Autodetector Optimiser State
myapp/0001 myapp/0002 otherapp/0001 contenttypes/0001 appthree/0001 myapp/0003 otherapp/0002
(insert wavy memory effect!) Why do we need dependencies?
Makes all tables syncdb Sets all constraints Adds all ForeignKeys
Makes appA tables migrations Makes appB tables Adds FKs from
appB to appA Adds FKs from appA to appB
Makes appA tables migrations Makes appB tables Adds FKs from
appB to appA Adds FKs from appA to appB
The obvious stuff. Basic Dependencies
Or your database gets really mopey. FK target add before
FK added
In the general case; some DBs less fussy FK target
delete after FK deleted
Good luck doing it otherwise. Model create before field create
Also after field create Model delete after field alter
Stuff you occasionally forget Django has. Trickier Dependencies
And, of course, the reverse for deletion. M2M through model
before M2M
I'll let you figure out deletion reversals from now on.
MTI parent before MTI child
Index together, too Unique together after fields
Well done if you've actually used this feature. Order with
respect to after target
This happens when you turn proxies into real models. Same
delete before same create
Impossible dependencies
OH PLEASE NO TAKE IT AWAY Swappable models
Your migration dependencies myapp/0001 myapp/0002 otherapp/0001 auth/0001 contenttypes/0001
myapp/0001 myapp/0002 otherapp/0001 auth/0001 contenttypes/0001 thirdapp/0001 Your migration dependencies on
swappable models
what? Your migration dependencies on swappable models myapp/0001 myapp/0002 otherapp/0001
auth/0001 contenttypes/0001 thirdapp/0001 ???
what? Your migration dependencies on swappable models myapp/0001 myapp/0002 otherapp/0001
auth/0001 contenttypes/0001 thirdapp/0001 ??? argh
End of presentation. Click to exit.
They can't change at runtime, for swappable models or anything
else. Dependencies are static
(for a while, the answer was "careful function ordering") How
is it implemented?
no issues depend match shift operations restart
Using dependencies to define boundaries Chop operations into migrations
Optimisation
There's really quite a lot. Even more rules
create model + delete model = nothing create model +
create field = create model create field + alter field = create field alter field + delete field = delete field alter field + alter field = alter field
This time, with collapses rather than swaps Even more shuffling
can't reduce reduce possible check conflicts reduced, reset
Lets us be more verbose and clear with operations Applied
after autodetection
Makes your history a bit shorter Applied after squash
Loading & Graph
Builds graph of all basic migrations in memory Loads from
disk first
Marks applied state on each node Loads from db second
Only if all replaced nodes have same applied state Applies
squashes third
In-order list of all a node's ancestors in the tree
Graph returns "plans"
• Autodetector is slow • Optimizer is not great •
Graph state building is inefficient Room for improvement!
django/db/migrations/autodetector.py Autodetector django/db/migrations/optimizer.py Optimiser reduce()GfunctionGisGentryGforGlogic StartGatG_detect_changes()GandGfollowGcalls django/db/migrations/graph.py Graph django/db/migrations/loader.py
Thank you. Andrew Godwin @andrewgodwin
More Decks by Andrew Godwin
See All by Andrew Godwin
Working With Risk: Lessons From Mountain Rescue
andrewgodwin
1
37
Reconciling Everything
andrewgodwin
1
430
Django Through The Years
andrewgodwin
0
350
Writing Maintainable Software At Scale
andrewgodwin
0
560
A Newcomer's Guide To Airflow's Architecture
andrewgodwin
0
450
Async, Python, and the Future
andrewgodwin
2
780
How To Break Django: With Async
andrewgodwin
1
860
Taking Django's ORM Async
andrewgodwin
0
890
The Long Road To Asynchrony
andrewgodwin
0
790
Other Decks in Programming
See All in Programming
Proxmox x Talos Linux x Terraform で実現するなんかいい感じの Kubernetes クラスタ / A Sleek Kubernetes Cluster Powered by Proxmox, Talos Linux, and Terraform
nnaka2992
0
260
SODA - FACT BOOK(JP)
sodainc
1
40k
선언형 햅틱 라이브러리 진동을 소개합니다
l2hyunwoo
1
320
なぜ私たちは「現時点では」KMPの導入を見送ったのか
chigichan24
1
310
Kotlin JVM用に作られたライブラリのKMPを前提とした再設計
jsoizo
0
210
Crash Auto Pipelines
veronikapj
1
430
私なりのハーネスエンジニアリング ~独自の言語処理系の開発を通して~
seosoft
0
220
[Kotlin Fest 2026] eBPF Tracing for Kotlin
takehaya
1
380
DroidKnights 2026 Android A2UI Jetpack Compose 기반의 안전한 AI UI 렌더링 - 박제창
itsmedreamwalker
1
250
[DroidKnights 2026] Workflow AX 적용기 — 자동 정책 정리부터 E2E까지
dayeon
0
110
Spec-Driven Development - Why Vaadin Is the Perfect Fit
simas
PRO
0
260
AIが実装を担う時代の「技術設計」と「チームマネジメント」
kamina_zzz
2
280
Featured
See All Featured
The B2B funnel & how to create a winning content strategy
katarinadahlin
PRO
1
620
Marketing to machines
jonoalderson
1
6k
How to Think Like a Performance Engineer
csswizardry
28
2.9k
Un-Boring Meetings
codingconduct
0
500
AI Search: Where Are We & What Can We Do About It?
aleyda
0
8.2k
Taking LLMs out of the black box: A practical guide to human-in-the-loop distillation
inesmontani
PRO
3
2.5k
Raft: Consensus for Rubyists
vanstee
143
7.9k
ラッコキーワード サービス紹介資料
rakko
5
5.9M
Building a Modern Day E-commerce SEO Strategy
aleyda
45
9.3k
Music & Morning Musume
bryan
48
7.5k
Building the Perfect Custom Keyboard
takai
2
960
Reflections from 52 weeks, 52 projects
jeffersonlam
356
22k