Digging Into Django's Migrations

My keynote from DjangoCon US 2014

1. 's migrations digging into
2. Andrew Godwin Author of South migrations library Hi, I'm Author of 1.7 Django migrations Senior Software Engineer at
3. south migrations for django south.aeracode.org DjangoCon 2008 https://speakerdeck.com/andrewgodwin/south-migrations-for-django
4. Migrations: They're pretty good.
5. South 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 1.0 Aug 2008 Aug 2008 Sep 2008 Jan 2009 Apr 2009 Jun 2009 Mar 2010 May 2013 Jul 2014
6. The Early Years 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 1.0 Aug 2008 Aug 2008 Sep 2008 Jan 2009 Apr 2009 Jun 2009 Mar 2010 May 2013 Jul 2014
7. Approaching Stability 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 1.0 Aug 2008 Aug 2008 Sep 2008 Jan 2009 Apr 2009 Jun 2009 Mar 2010 May 2013 Jul 2014 Well, in the short term, South 1.0 will be released. “ “ June 2010
8. The Long Gap & Kickstarter 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 1.0 Aug 2008 Aug 2008 Sep 2008 Jan 2009 Apr 2009 Jun 2009 Mar 2010 May 2013 Jul 2014
9. Today 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 1.0 Aug 2008 Aug 2008 Sep 2008 Jan 2009 Apr 2009 Jun 2009 Mar 2010 May 2013 Jul 2014
10. For at least a year now, people have been suggesting to me that South should be in Django core. “ “ June 2010
11. The Initial Plan Django Schema backend ORM Hooks South 2 Migration handling User interface
12. The Revised Plan Django Schema backend ORM Hooks South 2 Migration handling User interface Backport for 1.4 - 1.6
13. The Revised Revised Plan Django Schema backend ORM Hooks Migration handling User interface
14. Moving South into Django instead, "Adding migrations to Django"
15. Logically Separate SchemaEditor Schema Migrations field.deconstruct() ModelOptions.apps Operations Loader / Graph Executor Autodetector Optimiser State
16. Operations & State
17. Operations are a declarative representation of model changes State is an in-memory representation of entire project state
18. State State Operation AddModel No Models 1 Model
19. State State Operation State Operation State Operation State Operation Migration 1 Migration 2
20. AddModel DeleteModel RenameModel AddField DeleteField AlterModelOptions RenameField AlterModelTable AlterField AlterUniqueTogether AlterIndexTogether RunSQL RunPython
21. SchemaEditor
22. connection.schema_editor() Abstraction over database DDL Takes Django models and fields
23. with connection.schema_editor() as ed: ed.create_model(Author) ed.add_field( Book, "author", models.ForeignKey(Author), )
24. Field Deconstruction
25. Every field has deconstruct() Returns arguments passed to __init__ to duplicate itself
26. >>> field = CharField(max_length=255) >>> field.deconstruct() ( None, 'django.db.models.CharField', [], {'max_length': 255} )
27. The Graph
28. Represents migrations and dependencies as directed graph Loop detection, leaf/root selection
29. myapp/0001 myapp/0002 otherapp/0001 contenttypes/0001 appthree/0001 myapp/0003 otherapp/0002
30. The Autodetector
31. Compares two States and outputs a set of operations to perform Operations have own dependencies and resolver
32. Model A Model A w/FK, Model B diff AddModel(B) AddField(A, "b", FK(B)) dependency sort AddModel(B) AddField(A, "b", FK(B)) BEFORE AFTER
33. The Optimiser
34. Takes one set of Operations and outputs another with the same effect Steps over pairs and checks if they combine and what they pass over
35. can't reduce reduce possible check conflicts reduced, reset
36. A New Format More concise Declarative Introspectable
37. Migration actions Frozen ORM
39. makemigrations field.deconstruct() Loader / Graph Autodetector Optimiser State Writer 1 2 State 3 5 4
40. migrate SchemaEditor ModelOptions.apps Operations Loader / Graph Executor State 1 2 Recorder 3
41. In-memory running Creates models from migration sets Autodetector diffs created from on-disk Used to feed SchemaEditor / ORM
42. State State Operation State Operation State Operation State Operation Migration 1 Migration 2 Models Models Render Render
43. But what went wrong?
44. Swappable Models
45. Your migration dependencies myapp/0001 myapp/0002 otherapp/0001 auth/0001 contenttypes/0001
46. myapp/0001 myapp/0002 otherapp/0001 auth/0001 contenttypes/0001 thirdapp/0001 Your migration dependencies on swappable models
47. what? Your migration dependencies on swappable models myapp/0001 myapp/0002 otherapp/0001 auth/0001 contenttypes/0001 thirdapp/0001 ???
48. what? Your migration dependencies on swappable models myapp/0001 myapp/0002 otherapp/0001 auth/0001 contenttypes/0001 thirdapp/0001 ???
49. Unmigrated Apps
50. Test persistence
51. Test persistence on MySQL
52. Random Meta options order_with_respect_to? Really?
53. Proxy Models
54. But we're there! Django 1.7, out now!
55. Thanks! Andrew Godwin andrewgodwin@eventbrite.com