Models and Migrations and Schemas - oh my!

A talk from DjangoCon US 2012 on South, Django, and schema migrations.

Transcript

1. Models & Migrations &
   Models & Migrations &
   Schemas – oh my!
   Schemas – oh my!
   Andrew Godwin
   Andrew Godwin
   @andrewgodwin
   @andrewgodwin
   flickr.com/anders_young
   flickr.com/anders_young
   

   View Slide

2. · Django core dev
   · Django core dev
   · South
   · South
   · Lanyrd
   · Lanyrd
   

   View Slide

3. · The Past
   · The Past
   django-evolution, south 0.1
   django-evolution, south 0.1
   · The Present
   · The Present
   south 0.7
   south 0.7
   · The Future
   · The Future
   django 1.6? south 2.0?
   django 1.6? south 2.0?
   

   View Slide

4. · The Past
   · The Past
   django-evolution, south 0.1
   django-evolution, south 0.1
   · The Present
   · The Present
   south 0.7
   south 0.7
   · The Future
   · The Future
   django 1.6? south 2.0?
   django 1.6? south 2.0?
   

   View Slide

5. Databases hate schema changes
   Databases hate schema changes
   “
   “All that optimising gone to waste...”
   All that optimising gone to waste...”
   flickr.com/96dpi
   flickr.com/96dpi
   

   View Slide

6. · Locks whole tables
   · Locks whole tables
   · Hammers disk I/O
   · Hammers disk I/O
   · Causes inconsistent results
   · Causes inconsistent results
   

   View Slide

7. There's a code/schema split
   There's a code/schema split
   Your database is going to ignore git
   Your database is going to ignore git
   flickr.com/nataliejohnson
   flickr.com/nataliejohnson
   

   View Slide

8. · Extra fields are fine
   · Extra fields are fine
   · Missing fields are not
   · Missing fields are not
   · Painful/slow to sync
   · Painful/slow to sync
   

   View Slide

9. django-evolution
   django-evolution
   dmigrations
   dmigrations
   

   View Slide

10. South 0.1
    South 0.1
    

    View Slide

11. View Slide

12. View Slide

13. · South 0.2
    · South 0.2
    mysql support
    mysql support
    

    View Slide

14. · South 0.3
    · South 0.3
    dependencies
    dependencies
    

    View Slide

15. · South 0.4
    · South 0.4
    altering columns
    altering columns
    SQLite
    SQLite
    

    View Slide

16. · South 0.5
    · South 0.5
    ORM freezing
    ORM freezing
    Automatic change detection
    Automatic change detection
    

    View Slide

17. · South 0.6
    · South 0.6
    Field introspection
    Field introspection
    Dependency solving speed
    Dependency solving speed
    

    View Slide

18. · South 0.7
    · South 0.7
    data/schema split
    data/schema split
    missing defaults for NOT NULL
    missing defaults for NOT NULL
    multidb
    multidb
    custom fields ignored
    custom fields ignored
    

    View Slide

19. · The Past
    · The Past
    django-evolution, south 0.1
    django-evolution, south 0.1
    · The Present
    · The Present
    south 0.7
    south 0.7
    · The Future
    · The Future
    django 1.6? south 2.0?
    django 1.6? south 2.0?
    

    View Slide

20. · Mostly stable
    · Mostly stable
    Core API hasn't changed for 2 years
    Core API hasn't changed for 2 years
    

    View Slide

21. · No rebase/collapse
    · No rebase/collapse
    Needs a big change to migration tracking
    Needs a big change to migration tracking
    

    View Slide

22. · Opaque migrations
    · Opaque migrations
    Impossible to peek inside migrations
    Impossible to peek inside migrations
    

    View Slide

23. · The Past
    · The Past
    django-evolution, south 0.1
    django-evolution, south 0.1
    · The Present
    · The Present
    south 0.7
    south 0.7
    · The Future
    · The Future
    django 1.6? south 2.0?
    django 1.6? south 2.0?
    

    View Slide

24. django.db.backends.schema
    django.db.backends.schema
    

    View Slide

25. · Database abstraction layer
    · Database abstraction layer
    Has no idea migrations exist
    Has no idea migrations exist
    

    View Slide

26. from django.db import connection
    from django.db import connection
    editor = connection.schema_editor()
    editor = connection.schema_editor()
    editor.start()
    editor.start()
    editor.create_model(Author)
    editor.create_model(Author)
    editor.create_model(Book)
    editor.create_model(Book)
    editor.commit()
    editor.commit()
    

    View Slide

27. from django.db import connection
    from django.db import connection
    editor = connection.schema_editor()
    editor = connection.schema_editor()
    editor.start()
    editor.start()
    editor.alter_field(
    editor.alter_field(
    Book,
    Book,
    CharField(max_length=100, db_index=True),
    CharField(max_length=100, db_index=True),
    TextField(),
    TextField(),
    )
    )
    editor.delete_field(Author, ”age”)
    editor.delete_field(Author, ”age”)
    editor.commit()
    editor.commit()
    

    View Slide

28. github.com/andrewgodwin/
    github.com/andrewgodwin/
    django/tree/schema-alteration
    django/tree/schema-alteration
    

    View Slide

29. contrib.
    contrib.migrations
    migrations
    

    View Slide

30. · Migration creation/running
    · Migration creation/running
    Would replace South
    Would replace South
    

    View Slide

31. · Declarative migrations
    · Declarative migrations
    Combinable, optimisable
    Combinable, optimisable
    

    View Slide

32. · No frozen ORM
    · No frozen ORM
    Derived from the migration history
    Derived from the migration history
    

    View Slide

33. · Raw SQL support
    · Raw SQL support
    For those with beards
    For those with beards
    

    View Slide

34. · SQL output support
    · SQL output support
    For those with full-flowing beards
    For those with full-flowing beards
    

    View Slide

35. from migrations.api import *
    from migrations.api import *
    class Migration(BaseMigration):
    class Migration(BaseMigration):
    actions = [
    actions = [
    CreateModel(name = "Author", fields = [
    CreateModel(name = "Author", fields = [
    ("name",
    ("name",
    Field(
    Field(
    "django.db.models.fields.CharField",
    "django.db.models.fields.CharField",
    [], {"max_length": "100"}
    [], {"max_length": "100"}
    )),
    )),
    ])
    ])
    ]
    ]
    

    View Slide

36. github.com/andrewgodwin/migrations/
    github.com/andrewgodwin/migrations/
    Incomplete and broken code!
    Incomplete and broken code!
    

    View Slide

37. south 1.0
    south 1.0
    

    View Slide

38. · Python 3 support
    · Python 3 support
    To coincide with Django 1.5
    To coincide with Django 1.5
    

    View Slide

39. · Will require Python 2.6
    · Will require Python 2.6
    As otherwise Python 3 is a PITA
    As otherwise Python 3 is a PITA
    

    View Slide

40. All subject to change!
    All subject to change!
    This isn't set in stone.
    This isn't set in stone.
    

    View Slide

41. · Feedback on schema-alt branch
    · Feedback on schema-alt branch
    How to help
    How to help
    · Weird and wacky use cases
    · Weird and wacky use cases
    · Come talk to me during sprints
    · Come talk to me during sprints
    

    View Slide

42. Questions?
    Questions?
    Andrew Godwin / @andrewgodwin
    Andrew Godwin / @andrewgodwin
    flickr.com/oimax
    flickr.com/oimax
    

    View Slide