Digging Into Django's Migrations

077e9a0cb34fa3eba2699240c9509717?s=47 Andrew Godwin
September 03, 2014

Digging Into Django's Migrations

My keynote from DjangoCon US 2014

077e9a0cb34fa3eba2699240c9509717?s=128

Andrew Godwin

September 03, 2014
Tweet

Transcript

  1. 's migrations digging into

  2. Andrew Godwin Author of South migrations library Hi, I'm Author

    of 1.7 Django migrations Senior Software Engineer at
  3. south migrations for django south.aeracode.org DjangoCon 2008 https://speakerdeck.com/andrewgodwin/south-migrations-for-django

  4. Migrations: They're pretty good.

  5. South 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 1.0

    Aug 2008 Aug 2008 Sep 2008 Jan 2009 Apr 2009 Jun 2009 Mar 2010 May 2013 Jul 2014
  6. The Early Years 0.1 0.2 0.3 0.4 0.5 0.6 0.7

    0.8 1.0 Aug 2008 Aug 2008 Sep 2008 Jan 2009 Apr 2009 Jun 2009 Mar 2010 May 2013 Jul 2014
  7. Approaching Stability 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8

    1.0 Aug 2008 Aug 2008 Sep 2008 Jan 2009 Apr 2009 Jun 2009 Mar 2010 May 2013 Jul 2014 Well, in the short term, South 1.0 will be released. “ “ June 2010
  8. The Long Gap & Kickstarter 0.1 0.2 0.3 0.4 0.5

    0.6 0.7 0.8 1.0 Aug 2008 Aug 2008 Sep 2008 Jan 2009 Apr 2009 Jun 2009 Mar 2010 May 2013 Jul 2014
  9. Today 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 1.0

    Aug 2008 Aug 2008 Sep 2008 Jan 2009 Apr 2009 Jun 2009 Mar 2010 May 2013 Jul 2014
  10. For at least a year now, people have been suggesting

    to me that South should be in Django core. “ “ June 2010
  11. The Initial Plan Django Schema backend ORM Hooks South 2

    Migration handling User interface
  12. The Revised Plan Django Schema backend ORM Hooks South 2

    Migration handling User interface Backport for 1.4 - 1.6
  13. The Revised Revised Plan Django Schema backend ORM Hooks Migration

    handling User interface
  14. Moving South into Django instead, "Adding migrations to Django"

  15. Logically Separate SchemaEditor Schema Migrations field.deconstruct() ModelOptions.apps Operations Loader /

    Graph Executor Autodetector Optimiser State
  16. Operations & State

  17. Operations are a declarative representation of model changes State is

    an in-memory representation of entire project state
  18. State State Operation AddModel No Models 1 Model

  19. State State Operation State Operation State Operation State Operation Migration

    1 Migration 2
  20. AddModel DeleteModel RenameModel AddField DeleteField AlterModelOptions RenameField AlterModelTable AlterField AlterUniqueTogether

    AlterIndexTogether RunSQL RunPython
  21. SchemaEditor

  22. connection.schema_editor() Abstraction over database DDL Takes Django models and fields

  23. with connection.schema_editor() as ed: ed.create_model(Author) ed.add_field( Book, "author", models.ForeignKey(Author), )

  24. Field Deconstruction

  25. Every field has deconstruct() Returns arguments passed to __init__ to

    duplicate itself
  26. >>> field = CharField(max_length=255) >>> field.deconstruct() ( None, 'django.db.models.CharField', [],

    {'max_length': 255} )
  27. The Graph

  28. Represents migrations and dependencies as directed graph Loop detection, leaf/root

    selection
  29. myapp/0001 myapp/0002 otherapp/0001 contenttypes/0001 appthree/0001 myapp/0003 otherapp/0002

  30. The Autodetector

  31. Compares two States and outputs a set of operations to

    perform Operations have own dependencies and resolver
  32. Model A Model A w/FK, Model B diff AddModel(B) AddField(A,

    "b", FK(B)) dependency sort AddModel(B) AddField(A, "b", FK(B)) BEFORE AFTER
  33. The Optimiser

  34. Takes one set of Operations and outputs another with the

    same effect Steps over pairs and checks if they combine and what they pass over
  35. can't reduce reduce possible check conflicts reduced, reset

  36. A New Format More concise Declarative Introspectable

  37. Migration actions Frozen ORM

  38. None
  39. makemigrations field.deconstruct() Loader / Graph Autodetector Optimiser State Writer 1

    2 State 3 5 4
  40. migrate SchemaEditor ModelOptions.apps Operations Loader / Graph Executor State 1

    2 Recorder 3
  41. In-memory running Creates models from migration sets Autodetector diffs created

    from on-disk Used to feed SchemaEditor / ORM
  42. State State Operation State Operation State Operation State Operation Migration

    1 Migration 2 Models Models Render Render
  43. But what went wrong?

  44. Swappable Models

  45. Your migration dependencies myapp/0001 myapp/0002 otherapp/0001 auth/0001 contenttypes/0001

  46. myapp/0001 myapp/0002 otherapp/0001 auth/0001 contenttypes/0001 thirdapp/0001 Your migration dependencies on

    swappable models
  47. what? Your migration dependencies on swappable models myapp/0001 myapp/0002 otherapp/0001

    auth/0001 contenttypes/0001 thirdapp/0001 ???
  48. what? Your migration dependencies on swappable models myapp/0001 myapp/0002 otherapp/0001

    auth/0001 contenttypes/0001 thirdapp/0001 ???
  49. Unmigrated Apps

  50. Test persistence

  51. Test persistence on MySQL

  52. Random Meta options order_with_respect_to? Really?

  53. Proxy Models

  54. But we're there! Django 1.7, out now!

  55. Thanks! Andrew Godwin andrewgodwin@eventbrite.com