Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Everybody Loves Migrations

Andrew Godwin
September 04, 2013

Everybody Loves Migrations

A talk I gave at DjangoCon US 2013 about Django's new built-in migrations, South 2, and schema alteration.

Andrew Godwin

September 04, 2013
Tweet

More Decks by Andrew Godwin

Other Decks in Programming

Transcript

  1. Andrew Godwin
    @andrewgodwin
    everybody loves
    migrations

    View full-size slide

  2. Django core developer
    Andrew Godwin
    South author
    Database wrangler extraordinare

    View full-size slide

  3. What is schema alteration?
    Background
    Why wasn't it in Django before?
    Why are you doing all this anyway?

    View full-size slide

  4. Adding a field
    What is schema alteration?
    Changing unique_together
    Setting db_index=True
    Changing the type of a field
    Renaming a field
    Setting null=True
    Adding a model
    Deleting a model

    View full-size slide

  5. Adding a field
    What Django did before
    Changing unique_together
    Setting db_index=True
    Changing the type of a field
    Renaming a field
    Setting null=True
    Adding a model
    Deleting a model

    View full-size slide

  6. Why is it so hard?
    Versioning
    Different databases
    Not necessary up front

    View full-size slide

  7. Released 2008
    Most popular solution
    Not without issues

    View full-size slide

  8. Basic Layout

    View full-size slide

  9. Basic Layout
    schemamigration: Creates migration files in apps

    View full-size slide

  10. Basic Layout
    schemamigration: Creates migration files in apps
    --auto: No, really, make them for me.

    View full-size slide

  11. Basic Layout
    schemamigration: Creates migration files in apps
    datamigration: Also creates migration files in apps
    --auto: No, really, make them for me.

    View full-size slide

  12. Basic Layout
    schemamigration: Creates migration files in apps
    datamigration: Also creates migration files in apps
    migrate: Applies migrations only
    --auto: No, really, make them for me.

    View full-size slide

  13. Basic Layout
    schemamigration: Creates migration files in apps
    datamigration: Also creates migration files in apps
    migrate: Applies migrations only
    syncdb: Does no-migration apps.
    --auto: No, really, make them for me.

    View full-size slide

  14. Basic Layout
    schemamigration: Creates migration files in apps
    datamigration: Also creates migration files in apps
    migrate: Applies migrations only
    syncdb: Does no-migration apps.
    Occasionally does migrations.
    --auto: No, really, make them for me.

    View full-size slide

  15. Basic Layout
    schemamigration: Creates migration files in apps
    datamigration: Also creates migration files in apps
    migrate: Applies migrations only
    syncdb: Does no-migration apps.
    Occasionally does migrations.
    --auto: No, really, make them for me.
    test: Good luck with that one.

    View full-size slide

  16. Issues
    Migrations build up over time
    VCS merges suck
    That file format...

    View full-size slide

  17. Migration actions
    Frozen ORM

    View full-size slide

  18. django.db.migrations
    A rewrite five years in the making

    View full-size slide

  19. Design Goals

    View full-size slide

  20. Clean migrations
    Readable diffs are really important

    View full-size slide

  21. Squashable migrations
    No need for those hundreds of old ones

    View full-size slide

  22. Better merge detection
    Because most teams don't read commits

    View full-size slide

  23. Better commands
    --auto is just so pointless now

    View full-size slide

  24. Automatic dependencies
    Stops those silent errors a year later

    View full-size slide

  25. Reuseable Schema API
    There are valid reasons to change tables

    View full-size slide

  26. Third-party compatability
    If nobody uses it, it's pointless

    View full-size slide

  27. Third-party compatability
    Squashable migrations
    Better merge detection
    Better commands
    Automatic dependencies
    Reuseable Schema API
    Clean migrations

    View full-size slide

  28. Design Overview

    View full-size slide

  29. Schema alteration
    Migration engine

    View full-size slide

  30. Schema alteration
    with connection.schema_editor() as editor:
    editor.create_model(Foo)
    editor.delete_field(Bar, "baz")

    View full-size slide

  31. Model and Field based
    Will eventually replace .creation

    View full-size slide

  32. Migrations
    makemigrations: Makes migrations.
    migrate: Applies migrations + legacy creation

    View full-size slide

  33. Migrations
    makemigrations: Makes migrations.
    migrate: Applies migrations + legacy creation
    syncdb: Gone!

    View full-size slide

  34. Autodetector
    Executor
    Makes new migrations
    Plans and runs migrations

    View full-size slide

  35. Better format

    View full-size slide

  36. Compound History
    Create Model Page with fields content, date
    Create Model Author with fields name, age
    Add title to Page
    Remove age from Author
    Migration 1
    Migration 2
    Migration 3
    Model Page with fields title, content, date
    Model Author with field name

    View full-size slide

  37. Operations
    CreateModel
    DeleteModel
    AlterModelTable
    AlterUniqueTogether
    AlterIndexTogether
    AddField
    AlterField
    RenameField
    RemoveField
    RunSQL
    RunPython

    View full-size slide

  38. Field.deconstruct()
    Nicer way of serialising fields
    Vital to both this and the upcoming
    composite fields project

    View full-size slide

  39. MORE
    BONUS CODE

    View full-size slide

  40. Multiple AppCaches
    If you know what this means, well done.
    Basically, you can make multiple versions
    of the same model in memory at once.

    View full-size slide

  41. Progress Report
    Where am I spending those Kickstarter pounds?

    View full-size slide

  42. Schema backends
    PostgreSQL
    MySQL
    SQLite
    Oracle
    Migration Engine
    Dependency resolver
    Basic operations
    makemigrations
    migrate
    squashmigrations
    DONE
    DONE
    DONE
    DONE
    DONE
    DONE
    DONE
    DONE
    NOT YET
    Advanced operations
    NOT YET
    Autodetector
    PARTIAL

    View full-size slide

  43. How does it affect
    YOU?

    View full-size slide

  44. Don't Panic!
    syncdb and signals still work
    Upgrade path from South migrations
    South 2 to backport new format

    View full-size slide

  45. Don't Panic!
    syncdb and signals still work
    Upgrade path from South migrations
    South 2 to backport new format
    Slightly Panic!
    Custom fields will need some work

    View full-size slide

  46. Future Ideas

    View full-size slide

  47. Better autodetection
    Automatic data migrations?

    View full-size slide

  48. Versioned model interface
    True downtimeless deployments?

    View full-size slide

  49. Percona support
    I almost feel sorry for the MySQL users.

    View full-size slide

  50. Nonrelational support
    Oh, they still need migrations.

    View full-size slide

  51. Things to Remember

    View full-size slide

  52. New migration format

    View full-size slide

  53. New migration format
    Even easier to use

    View full-size slide

  54. New migration format
    Django models through and through
    Even easier to use

    View full-size slide

  55. New migration format
    Django models through and through
    Please use a proper database
    Even easier to use

    View full-size slide

  56. New migration format
    Django models through and through
    Please use a proper database
    Schemas are your friend.
    Even easier to use

    View full-size slide

  57. Thanks.
    Questions welcome, especially from почемучка
    Andrew Godwin
    @andrewgodwin

    View full-size slide