Migrations Under The Hood

Migrations UNDER THE HOOD Andrew Godwin Migrations

Andrew Godwin Author of South migrations library Hi, I'm Author
of 1.7 Django migrations Senior Software Engineer at Eventbrite Apparently at every Django conference

Migrations

Other talks are available. This is a technical dive!

The Initial Plan Django Schema backend ORM Hooks South 2
Migration handling User interface

TheBRevisedBPlan Django SchemaBbackend ORMBHooks SouthB2 MigrationBhandling UserBinterface BackportBforB1.4B-B1.6

The Revised Revised Plan Django Schema backend ORM Hooks Migration
handling User interface

Logically Separate SchemaEditor Schema Migrations field.deconstruct() ModelOptions.apps Operations Loader /
Graph Executor Autodetector Optimiser State

myapp/0001 myapp/0002 otherapp/0001 contenttypes/0001 appthree/0001 myapp/0003 otherapp/0002

(insert wavy memory effect!) Why do we need dependencies?

Makes all tables syncdb Sets all constraints Adds all ForeignKeys

Makes appA tables migrations Makes appB tables Adds FKs from
appB to appA Adds FKs from appA to appB

The obvious stuff. Basic Dependencies

Or your database gets really mopey. FK target add before
FK added

In the general case; some DBs less fussy FK target
delete after FK deleted

Good luck doing it otherwise. Model create before field create

Also after field create Model delete after field alter

Stuff you occasionally forget Django has. Trickier Dependencies

And, of course, the reverse for deletion. M2M through model
before M2M

I'll let you figure out deletion reversals from now on.
MTI parent before MTI child

Index together, too Unique together after fields

Well done if you've actually used this feature. Order with
respect to after target

This happens when you turn proxies into real models. Same
delete before same create

Impossible dependencies

OH PLEASE NO TAKE IT AWAY Swappable models

Your migration dependencies myapp/0001 myapp/0002 otherapp/0001 auth/0001 contenttypes/0001

myapp/0001 myapp/0002 otherapp/0001 auth/0001 contenttypes/0001 thirdapp/0001 Your migration dependencies on
swappable models

what? Your migration dependencies on swappable models myapp/0001 myapp/0002 otherapp/0001
auth/0001 contenttypes/0001 thirdapp/0001 ???

what? Your migration dependencies on swappable models myapp/0001 myapp/0002 otherapp/0001
auth/0001 contenttypes/0001 thirdapp/0001 ??? argh

End of presentation. Click to exit.

They can't change at runtime, for swappable models or anything
else. Dependencies are static

(for a while, the answer was "careful function ordering") How
is it implemented?

no issues depend match shift operations restart

Using dependencies to define boundaries Chop operations into migrations

Optimisation

There's really quite a lot. Even more rules

create model + delete model = nothing create model +
create field = create model create field + alter field = create field alter field + delete field = delete field alter field + alter field = alter field

This time, with collapses rather than swaps Even more shuffling

can't reduce reduce possible check conflicts reduced, reset

Lets us be more verbose and clear with operations Applied
after autodetection

Makes your history a bit shorter Applied after squash

Loading & Graph

Builds graph of all basic migrations in memory Loads from
disk first

Marks applied state on each node Loads from db second

Only if all replaced nodes have same applied state Applies
squashes third

In-order list of all a node's ancestors in the tree
Graph returns "plans"

• Autodetector is slow • Optimizer is not great •
Graph state building is inefficient Room for improvement!

django/db/migrations/autodetector.py Autodetector django/db/migrations/optimizer.py Optimiser reduce()GfunctionGisGentryGforGlogic StartGatG_detect_changes()GandGfollowGcalls django/db/migrations/graph.py Graph django/db/migrations/loader.py

Thank you. Andrew Godwin @andrewgodwin

Migrations Under The Hood

Migrations Under The Hood

More Decks by Andrew Godwin

Other Decks in Programming

Featured

Transcript