Digging Into Django's Migrations

Slide 1

Slide 1 text

's migrations digging into

Slide 2

Slide 2 text

Andrew Godwin Author of South migrations library Hi, I'm Author of 1.7 Django migrations Senior Software Engineer at

Slide 3

Slide 3 text

south migrations for django south.aeracode.org DjangoCon 2008 https://speakerdeck.com/andrewgodwin/south-migrations-for-django

Slide 4

Slide 4 text

Migrations: They're pretty good.

Slide 5

Slide 5 text

South 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 1.0 Aug 2008 Aug 2008 Sep 2008 Jan 2009 Apr 2009 Jun 2009 Mar 2010 May 2013 Jul 2014

Slide 6

Slide 6 text

The Early Years 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 1.0 Aug 2008 Aug 2008 Sep 2008 Jan 2009 Apr 2009 Jun 2009 Mar 2010 May 2013 Jul 2014

Slide 7

Slide 7 text

Approaching Stability 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 1.0 Aug 2008 Aug 2008 Sep 2008 Jan 2009 Apr 2009 Jun 2009 Mar 2010 May 2013 Jul 2014 Well, in the short term, South 1.0 will be released. “ “ June 2010

Slide 8

Slide 8 text

The Long Gap & Kickstarter 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 1.0 Aug 2008 Aug 2008 Sep 2008 Jan 2009 Apr 2009 Jun 2009 Mar 2010 May 2013 Jul 2014

Slide 9

Slide 9 text

Today 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 1.0 Aug 2008 Aug 2008 Sep 2008 Jan 2009 Apr 2009 Jun 2009 Mar 2010 May 2013 Jul 2014

Slide 10

Slide 10 text

For at least a year now, people have been suggesting to me that South should be in Django core. “ “ June 2010

Slide 11

Slide 11 text

The Initial Plan Django Schema backend ORM Hooks South 2 Migration handling User interface

Slide 12

Slide 12 text

The Revised Plan Django Schema backend ORM Hooks South 2 Migration handling User interface Backport for 1.4 - 1.6

Slide 13

Slide 13 text

The Revised Revised Plan Django Schema backend ORM Hooks Migration handling User interface

Slide 14

Slide 14 text

Moving South into Django instead, "Adding migrations to Django"

Slide 15

Slide 15 text

Logically Separate SchemaEditor Schema Migrations field.deconstruct() ModelOptions.apps Operations Loader / Graph Executor Autodetector Optimiser State

Slide 16

Slide 16 text

Operations & State

Slide 17

Slide 17 text

Operations are a declarative representation of model changes State is an in-memory representation of entire project state

Slide 18

Slide 18 text

State State Operation AddModel No Models 1 Model

Slide 19

Slide 19 text

State State Operation State Operation State Operation State Operation Migration 1 Migration 2

Slide 20

Slide 20 text

AddModel DeleteModel RenameModel AddField DeleteField AlterModelOptions RenameField AlterModelTable AlterField AlterUniqueTogether AlterIndexTogether RunSQL RunPython

Slide 21

Slide 21 text

SchemaEditor

Slide 22

Slide 22 text

connection.schema_editor() Abstraction over database DDL Takes Django models and fields

Slide 23

Slide 23 text

with connection.schema_editor() as ed: ed.create_model(Author) ed.add_field( Book, "author", models.ForeignKey(Author), )

Slide 24

Slide 24 text

Field Deconstruction

Slide 25

Slide 25 text

Every field has deconstruct() Returns arguments passed to __init__ to duplicate itself

Slide 26

Slide 26 text

>>> field = CharField(max_length=255) >>> field.deconstruct() ( None, 'django.db.models.CharField', [], {'max_length': 255} )

Slide 27

Slide 27 text

The Graph

Slide 28

Slide 28 text

Represents migrations and dependencies as directed graph Loop detection, leaf/root selection

Slide 29

Slide 29 text

myapp/0001 myapp/0002 otherapp/0001 contenttypes/0001 appthree/0001 myapp/0003 otherapp/0002

Slide 30

Slide 30 text

The Autodetector

Slide 31

Slide 31 text

Compares two States and outputs a set of operations to perform Operations have own dependencies and resolver

Slide 32

Slide 32 text

Model A Model A w/FK, Model B diff AddModel(B) AddField(A, "b", FK(B)) dependency sort AddModel(B) AddField(A, "b", FK(B)) BEFORE AFTER

Slide 33

Slide 33 text

The Optimiser

Slide 34

Slide 34 text

Takes one set of Operations and outputs another with the same effect Steps over pairs and checks if they combine and what they pass over

Slide 35

Slide 35 text

can't reduce reduce possible check conflicts reduced, reset

Slide 36

Slide 36 text

A New Format More concise Declarative Introspectable

Slide 37

Slide 37 text

Migration actions Frozen ORM

Slide 38

Slide 38 text

No content

Slide 39

Slide 39 text

makemigrations field.deconstruct() Loader / Graph Autodetector Optimiser State Writer 1 2 State 3 5 4

Slide 40

Slide 40 text

migrate SchemaEditor ModelOptions.apps Operations Loader / Graph Executor State 1 2 Recorder 3

Slide 41

Slide 41 text

In-memory running Creates models from migration sets Autodetector diffs created from on-disk Used to feed SchemaEditor / ORM

Slide 42

Slide 42 text

State State Operation State Operation State Operation State Operation Migration 1 Migration 2 Models Models Render Render

Slide 43

Slide 43 text

But what went wrong?

Slide 44

Slide 44 text

Swappable Models

Slide 45

Slide 45 text

Your migration dependencies myapp/0001 myapp/0002 otherapp/0001 auth/0001 contenttypes/0001

Slide 46

Slide 46 text

myapp/0001 myapp/0002 otherapp/0001 auth/0001 contenttypes/0001 thirdapp/0001 Your migration dependencies on swappable models

Slide 47

Slide 47 text

what? Your migration dependencies on swappable models myapp/0001 myapp/0002 otherapp/0001 auth/0001 contenttypes/0001 thirdapp/0001 ???

Slide 48

Slide 48 text

what? Your migration dependencies on swappable models myapp/0001 myapp/0002 otherapp/0001 auth/0001 contenttypes/0001 thirdapp/0001 ???