Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Models and Migrations and Schemas - oh my!

Models and Migrations and Schemas - oh my!

A talk from DjangoCon US 2012 on South, Django, and schema migrations.

Andrew Godwin

September 06, 2012
Tweet

More Decks by Andrew Godwin

Other Decks in Programming

Transcript

  1. Models & Migrations &
    Models & Migrations &
    Schemas – oh my!
    Schemas – oh my!
    Andrew Godwin
    Andrew Godwin
    @andrewgodwin
    @andrewgodwin
    flickr.com/anders_young
    flickr.com/anders_young

    View Slide

  2. · Django core dev
    · Django core dev
    · South
    · South
    · Lanyrd
    · Lanyrd

    View Slide

  3. · The Past
    · The Past
    django-evolution, south 0.1
    django-evolution, south 0.1
    · The Present
    · The Present
    south 0.7
    south 0.7
    · The Future
    · The Future
    django 1.6? south 2.0?
    django 1.6? south 2.0?

    View Slide

  4. · The Past
    · The Past
    django-evolution, south 0.1
    django-evolution, south 0.1
    · The Present
    · The Present
    south 0.7
    south 0.7
    · The Future
    · The Future
    django 1.6? south 2.0?
    django 1.6? south 2.0?

    View Slide

  5. Databases hate schema changes
    Databases hate schema changes

    “All that optimising gone to waste...”
    All that optimising gone to waste...”
    flickr.com/96dpi
    flickr.com/96dpi

    View Slide

  6. · Locks whole tables
    · Locks whole tables
    · Hammers disk I/O
    · Hammers disk I/O
    · Causes inconsistent results
    · Causes inconsistent results

    View Slide

  7. There's a code/schema split
    There's a code/schema split
    Your database is going to ignore git
    Your database is going to ignore git
    flickr.com/nataliejohnson
    flickr.com/nataliejohnson

    View Slide

  8. · Extra fields are fine
    · Extra fields are fine
    · Missing fields are not
    · Missing fields are not
    · Painful/slow to sync
    · Painful/slow to sync

    View Slide

  9. django-evolution
    django-evolution
    dmigrations
    dmigrations

    View Slide

  10. South 0.1
    South 0.1

    View Slide

  11. View Slide

  12. View Slide

  13. · South 0.2
    · South 0.2
    mysql support
    mysql support

    View Slide

  14. · South 0.3
    · South 0.3
    dependencies
    dependencies

    View Slide

  15. · South 0.4
    · South 0.4
    altering columns
    altering columns
    SQLite
    SQLite

    View Slide

  16. · South 0.5
    · South 0.5
    ORM freezing
    ORM freezing
    Automatic change detection
    Automatic change detection

    View Slide

  17. · South 0.6
    · South 0.6
    Field introspection
    Field introspection
    Dependency solving speed
    Dependency solving speed

    View Slide

  18. · South 0.7
    · South 0.7
    data/schema split
    data/schema split
    missing defaults for NOT NULL
    missing defaults for NOT NULL
    multidb
    multidb
    custom fields ignored
    custom fields ignored

    View Slide

  19. · The Past
    · The Past
    django-evolution, south 0.1
    django-evolution, south 0.1
    · The Present
    · The Present
    south 0.7
    south 0.7
    · The Future
    · The Future
    django 1.6? south 2.0?
    django 1.6? south 2.0?

    View Slide

  20. · Mostly stable
    · Mostly stable
    Core API hasn't changed for 2 years
    Core API hasn't changed for 2 years

    View Slide

  21. · No rebase/collapse
    · No rebase/collapse
    Needs a big change to migration tracking
    Needs a big change to migration tracking

    View Slide

  22. · Opaque migrations
    · Opaque migrations
    Impossible to peek inside migrations
    Impossible to peek inside migrations

    View Slide

  23. · The Past
    · The Past
    django-evolution, south 0.1
    django-evolution, south 0.1
    · The Present
    · The Present
    south 0.7
    south 0.7
    · The Future
    · The Future
    django 1.6? south 2.0?
    django 1.6? south 2.0?

    View Slide

  24. django.db.backends.schema
    django.db.backends.schema

    View Slide

  25. · Database abstraction layer
    · Database abstraction layer
    Has no idea migrations exist
    Has no idea migrations exist

    View Slide

  26. from django.db import connection
    from django.db import connection
    editor = connection.schema_editor()
    editor = connection.schema_editor()
    editor.start()
    editor.start()
    editor.create_model(Author)
    editor.create_model(Author)
    editor.create_model(Book)
    editor.create_model(Book)
    editor.commit()
    editor.commit()

    View Slide

  27. from django.db import connection
    from django.db import connection
    editor = connection.schema_editor()
    editor = connection.schema_editor()
    editor.start()
    editor.start()
    editor.alter_field(
    editor.alter_field(
    Book,
    Book,
    CharField(max_length=100, db_index=True),
    CharField(max_length=100, db_index=True),
    TextField(),
    TextField(),
    )
    )
    editor.delete_field(Author, ”age”)
    editor.delete_field(Author, ”age”)
    editor.commit()
    editor.commit()

    View Slide

  28. github.com/andrewgodwin/
    github.com/andrewgodwin/
    django/tree/schema-alteration
    django/tree/schema-alteration

    View Slide

  29. contrib.
    contrib.migrations
    migrations

    View Slide

  30. · Migration creation/running
    · Migration creation/running
    Would replace South
    Would replace South

    View Slide

  31. · Declarative migrations
    · Declarative migrations
    Combinable, optimisable
    Combinable, optimisable

    View Slide

  32. · No frozen ORM
    · No frozen ORM
    Derived from the migration history
    Derived from the migration history

    View Slide

  33. · Raw SQL support
    · Raw SQL support
    For those with beards
    For those with beards

    View Slide

  34. · SQL output support
    · SQL output support
    For those with full-flowing beards
    For those with full-flowing beards

    View Slide

  35. from migrations.api import *
    from migrations.api import *
    class Migration(BaseMigration):
    class Migration(BaseMigration):
    actions = [
    actions = [
    CreateModel(name = "Author", fields = [
    CreateModel(name = "Author", fields = [
    ("name",
    ("name",
    Field(
    Field(
    "django.db.models.fields.CharField",
    "django.db.models.fields.CharField",
    [], {"max_length": "100"}
    [], {"max_length": "100"}
    )),
    )),
    ])
    ])
    ]
    ]

    View Slide

  36. github.com/andrewgodwin/migrations/
    github.com/andrewgodwin/migrations/
    Incomplete and broken code!
    Incomplete and broken code!

    View Slide

  37. south 1.0
    south 1.0

    View Slide

  38. · Python 3 support
    · Python 3 support
    To coincide with Django 1.5
    To coincide with Django 1.5

    View Slide

  39. · Will require Python 2.6
    · Will require Python 2.6
    As otherwise Python 3 is a PITA
    As otherwise Python 3 is a PITA

    View Slide

  40. All subject to change!
    All subject to change!
    This isn't set in stone.
    This isn't set in stone.

    View Slide

  41. · Feedback on schema-alt branch
    · Feedback on schema-alt branch
    How to help
    How to help
    · Weird and wacky use cases
    · Weird and wacky use cases
    · Come talk to me during sprints
    · Come talk to me during sprints

    View Slide

  42. Questions?
    Questions?
    Andrew Godwin / @andrewgodwin
    Andrew Godwin / @andrewgodwin
    flickr.com/oimax
    flickr.com/oimax

    View Slide