Everybody Loves Migrations

077e9a0cb34fa3eba2699240c9509717?s=47 Andrew Godwin
September 04, 2013

Everybody Loves Migrations

A talk I gave at DjangoCon US 2013 about Django's new built-in migrations, South 2, and schema alteration.

077e9a0cb34fa3eba2699240c9509717?s=128

Andrew Godwin

September 04, 2013
Tweet

Transcript

  1. Andrew Godwin @andrewgodwin everybody loves migrations

  2. Django core developer Andrew Godwin South author Database wrangler extraordinare

  3. None
  4. What is schema alteration? Background Why wasn't it in Django

    before? Why are you doing all this anyway?
  5. Adding a field What is schema alteration? Changing unique_together Setting

    db_index=True Changing the type of a field Renaming a field Setting null=True Adding a model Deleting a model
  6. Adding a field What Django did before Changing unique_together Setting

    db_index=True Changing the type of a field Renaming a field Setting null=True Adding a model Deleting a model
  7. Why is it so hard? Versioning Different databases Not necessary

    up front
  8. South

  9. Released 2008 Most popular solution Not without issues

  10. Basic Layout

  11. Basic Layout schemamigration: Creates migration files in apps

  12. Basic Layout schemamigration: Creates migration files in apps --auto: No,

    really, make them for me.
  13. Basic Layout schemamigration: Creates migration files in apps datamigration: Also

    creates migration files in apps --auto: No, really, make them for me.
  14. Basic Layout schemamigration: Creates migration files in apps datamigration: Also

    creates migration files in apps migrate: Applies migrations only --auto: No, really, make them for me.
  15. Basic Layout schemamigration: Creates migration files in apps datamigration: Also

    creates migration files in apps migrate: Applies migrations only syncdb: Does no-migration apps. --auto: No, really, make them for me.
  16. Basic Layout schemamigration: Creates migration files in apps datamigration: Also

    creates migration files in apps migrate: Applies migrations only syncdb: Does no-migration apps. Occasionally does migrations. --auto: No, really, make them for me.
  17. Basic Layout schemamigration: Creates migration files in apps datamigration: Also

    creates migration files in apps migrate: Applies migrations only syncdb: Does no-migration apps. Occasionally does migrations. --auto: No, really, make them for me. test: Good luck with that one.
  18. Issues Migrations build up over time VCS merges suck That

    file format...
  19. Migration actions Frozen ORM

  20. django.db.migrations A rewrite five years in the making

  21. Design Goals

  22. Clean migrations Readable diffs are really important

  23. Squashable migrations No need for those hundreds of old ones

  24. Better merge detection Because most teams don't read commits

  25. Better commands --auto is just so pointless now

  26. Automatic dependencies Stops those silent errors a year later

  27. Reuseable Schema API There are valid reasons to change tables

  28. Third-party compatability If nobody uses it, it's pointless

  29. Third-party compatability Squashable migrations Better merge detection Better commands Automatic

    dependencies Reuseable Schema API Clean migrations
  30. Design Overview

  31. Schema alteration Migration engine

  32. Schema alteration with connection.schema_editor() as editor: editor.create_model(Foo) editor.delete_field(Bar, "baz")

  33. Model and Field based Will eventually replace .creation

  34. Migrations makemigrations: Makes migrations. migrate: Applies migrations + legacy creation

  35. Migrations makemigrations: Makes migrations. migrate: Applies migrations + legacy creation

    syncdb: Gone!
  36. Autodetector Executor Makes new migrations Plans and runs migrations

  37. Better format

  38. Compound History Create Model Page with fields content, date Create

    Model Author with fields name, age Add title to Page Remove age from Author Migration 1 Migration 2 Migration 3 Model Page with fields title, content, date Model Author with field name
  39. Operations CreateModel DeleteModel AlterModelTable AlterUniqueTogether AlterIndexTogether AddField AlterField RenameField RemoveField

    RunSQL RunPython
  40. None
  41. BONUS CODE

  42. Field.deconstruct() Nicer way of serialising fields Vital to both this

    and the upcoming composite fields project
  43. MORE BONUS CODE

  44. Multiple AppCaches If you know what this means, well done.

    Basically, you can make multiple versions of the same model in memory at once.
  45. Progress Report Where am I spending those Kickstarter pounds?

  46. Schema backends PostgreSQL MySQL SQLite Oracle Migration Engine Dependency resolver

    Basic operations makemigrations migrate squashmigrations DONE DONE DONE DONE DONE DONE DONE DONE NOT YET Advanced operations NOT YET Autodetector PARTIAL
  47. How does it affect YOU?

  48. Don't Panic! syncdb and signals still work Upgrade path from

    South migrations South 2 to backport new format
  49. Don't Panic! syncdb and signals still work Upgrade path from

    South migrations South 2 to backport new format Slightly Panic! Custom fields will need some work
  50. Future Ideas

  51. Better autodetection Automatic data migrations?

  52. Versioned model interface True downtimeless deployments?

  53. Percona support I almost feel sorry for the MySQL users.

  54. Nonrelational support Oh, they still need migrations.

  55. Things to Remember

  56. New migration format

  57. New migration format Even easier to use

  58. New migration format Django models through and through Even easier

    to use
  59. New migration format Django models through and through Please use

    a proper database Even easier to use
  60. New migration format Django models through and through Please use

    a proper database Schemas are your friend. Even easier to use
  61. Thanks. Questions welcome, especially from почемучка Andrew Godwin @andrewgodwin