Everybody Loves Migrations

A talk I gave at DjangoCon US 2013 about Django's new built-in migrations, South 2, and schema alteration.

Transcript

1. Andrew Godwin @andrewgodwin everybody loves migrations
2. Django core developer Andrew Godwin South author Database wrangler extraordinare
4. What is schema alteration? Background Why wasn't it in Django before? Why are you doing all this anyway?
5. Adding a field What is schema alteration? Changing unique_together Setting db_index=True Changing the type of a field Renaming a field Setting null=True Adding a model Deleting a model
6. Adding a field What Django did before Changing unique_together Setting db_index=True Changing the type of a field Renaming a field Setting null=True Adding a model Deleting a model
7. Why is it so hard? Versioning Different databases Not necessary up front
8. South
9. Released 2008 Most popular solution Not without issues
10. Basic Layout
11. Basic Layout schemamigration: Creates migration files in apps
12. Basic Layout schemamigration: Creates migration files in apps --auto: No, really, make them for me.
13. Basic Layout schemamigration: Creates migration files in apps datamigration: Also creates migration files in apps --auto: No, really, make them for me.
14. Basic Layout schemamigration: Creates migration files in apps datamigration: Also creates migration files in apps migrate: Applies migrations only --auto: No, really, make them for me.
15. Basic Layout schemamigration: Creates migration files in apps datamigration: Also creates migration files in apps migrate: Applies migrations only syncdb: Does no-migration apps. --auto: No, really, make them for me.
16. Basic Layout schemamigration: Creates migration files in apps datamigration: Also creates migration files in apps migrate: Applies migrations only syncdb: Does no-migration apps. Occasionally does migrations. --auto: No, really, make them for me.
17. Basic Layout schemamigration: Creates migration files in apps datamigration: Also creates migration files in apps migrate: Applies migrations only syncdb: Does no-migration apps. Occasionally does migrations. --auto: No, really, make them for me. test: Good luck with that one.
18. Issues Migrations build up over time VCS merges suck That file format...
19. Migration actions Frozen ORM
20. django.db.migrations A rewrite five years in the making
21. Design Goals
22. Clean migrations Readable diffs are really important
23. Squashable migrations No need for those hundreds of old ones
24. Better merge detection Because most teams don't read commits
25. Better commands --auto is just so pointless now
26. Automatic dependencies Stops those silent errors a year later
27. Reuseable Schema API There are valid reasons to change tables
28. Third-party compatability If nobody uses it, it's pointless
29. Third-party compatability Squashable migrations Better merge detection Better commands Automatic dependencies Reuseable Schema API Clean migrations
30. Design Overview
31. Schema alteration Migration engine
32. Schema alteration with connection.schema_editor() as editor: editor.create_model(Foo) editor.delete_field(Bar, "baz")
33. Model and Field based Will eventually replace .creation
34. Migrations makemigrations: Makes migrations. migrate: Applies migrations + legacy creation
35. Migrations makemigrations: Makes migrations. migrate: Applies migrations + legacy creation syncdb: Gone!
36. Autodetector Executor Makes new migrations Plans and runs migrations
37. Better format
38. Compound History Create Model Page with fields content, date Create Model Author with fields name, age Add title to Page Remove age from Author Migration 1 Migration 2 Migration 3 Model Page with fields title, content, date Model Author with field name
39. Operations CreateModel DeleteModel AlterModelTable AlterUniqueTogether AlterIndexTogether AddField AlterField RenameField RemoveField RunSQL RunPython
41. BONUS CODE
42. Field.deconstruct() Nicer way of serialising fields Vital to both this and the upcoming composite fields project
43. MORE BONUS CODE
44. Multiple AppCaches If you know what this means, well done. Basically, you can make multiple versions of the same model in memory at once.
45. Progress Report Where am I spending those Kickstarter pounds?
46. Schema backends PostgreSQL MySQL SQLite Oracle Migration Engine Dependency resolver Basic operations makemigrations migrate squashmigrations DONE DONE DONE DONE DONE DONE DONE DONE NOT YET Advanced operations NOT YET Autodetector PARTIAL
47. How does it affect YOU?
48. Don't Panic! syncdb and signals still work Upgrade path from South migrations South 2 to backport new format
49. Don't Panic! syncdb and signals still work Upgrade path from South migrations South 2 to backport new format Slightly Panic! Custom fields will need some work
50. Future Ideas
51. Better autodetection Automatic data migrations?
52. Versioned model interface True downtimeless deployments?
53. Percona support I almost feel sorry for the MySQL users.
54. Nonrelational support Oh, they still need migrations.
55. Things to Remember
56. New migration format
57. New migration format Even easier to use
58. New migration format Django models through and through Even easier to use
59. New migration format Django models through and through Please use a proper database Even easier to use
60. New migration format Django models through and through Please use a proper database Schemas are your friend. Even easier to use
61. Thanks. Questions welcome, especially from почемучка Andrew Godwin @andrewgodwin