Pro Yearly is on sale from $80 to $50! »

Upgrading Django (to 1.7)

767450215aa2f669a3dcaefa5beb4643?s=47 Andrew Pinkham
September 03, 2014

Upgrading Django (to 1.7)

Slides for my talk at DjangoCon 2014.

767450215aa2f669a3dcaefa5beb4643?s=128

Andrew Pinkham

September 03, 2014
Tweet

Transcript

  1. ANDREW PINKHAM TOPIC DATE AUTHOR SEPTEMBER 2014 UPGRADING DJANGO (TO

    1.7) DJANGOCON 2014 ORIGINAL ART BY ERIK DEPRINCE WALLPAPER CREATED BY BRYAN VELOSO AND CHRISTIAN METTS
 WALLPAPER SITE HOSTED BY JEFF TRIPLETT
  2. $ whoami Software Engineer Freelance Consultant Technical Instructor Andrew Pinkham

    Hi, I’m
  3. AUTHOR OF DJANGO UNLEASHED SET TO PUBLISH BEGINNING OF 2015

  4. $ less updj17 Django Releases Versioning and Support Summary of

    Previous Releases Django 1.7 (v1.6 vs v1.7) Migrations App Loading Static Systems Check
  5. None
  6. None
  7. afrg.co/updj17

  8. DJANGO VERSIONS UNDERSTANDING DJANGO’S RELEASE PROCESS

  9. M.m.μ

  10. 1.7.0

  11. Pre-Releases Alpha Beta Release Candidate

  12. M.m.μ_#

  13. 1.7.0a1

  14. 1.7b1

  15. 1.7c2

  16. Version Support Two Versions Supported Deprecations over Two Versions

  17. 1.3 㱺 1.5 1.4 㱺 1.6 1.5 㱺 1.7 Deprecations

  18. Support Prior to v1.7 ! 1.6.6 1.6.5 1.6.4 1.6.3 …

    1.6
 1.5.9 1.5.8 1.5.7 1.5.6 … 1.5
  19. With Arrival of v1.7 1.7
 1.6.7 1.6.6 1.6.5 1.6.4 …

    1.6
 1.5.10 1.5.9 1.5.8 1.5.7 … 1.5
  20. Long Term Support (LTS) Django 1.4

  21. With Arrival of v1.7 1.7
 1.6.7 1.6.6 1.6.5 1.6.4 …

    1.6
 1.4.15 1.4.14 1.4.13 1.4.12 ! 1.4
  22. PREVIOUS DJANGO VERSIONS A TRIP DOWN MEMORY LANE

  23. 1.7 1.6 1.5 (Deprecations) 1.4 (LTS) Relevant (Minor) Versions

  24. Django 1.4 Timezone Support ORM Additions Cookie-Based Session Handling Auth

    Security Functional Browser Testing New Project Creation and Directory Structure
  25. Django 1.5 Python 2.6+ (2.5 Dropped) Experimental Support for Python

    3.2 Custom Django User Generic Class-Based Views JSON
  26. Django 1.6 Python 2.6+, Python 3.2 Transactions Simpler, Smarter Defaults

    django-admin.py validate
  27. “Django 1.7 is shaping up to be
 the biggest Django

    release since 1.0.” – Russell Keith-Magee
  28. Django 1.7 Migrations App Loading Static Systems Check Custom QuerySet

    Custom Lookups Prefetch
  29. 1.7 DJANGO

  30. 2.7 (AND 3.2+) PYTHON

  31. None
  32. DJANGO 1.7 COMPARING DJANGO 1.7 TO DJANGO 1.6 CAMELOT!

  33. Django 1.7 Migrations App Loading Static Systems Check

  34. None
  35. None
  36. $ django-admin.py startproject camelot # dj16 ! $ django-admin startproject

    camelot # dj17 ! ! ! /manage.py /camelot/ __init__.py settings.py urls.py wsgi.py
  37. South in Django 1.6 Best Practice: run syncdb now

  38. $ ./manage.py startapp roundtable ! /camelot/roundtable/ __init__.py admin.py migrations/ __init.py__

    models.py tests.py views.py
  39. # roundtable/models.py ! from django.db import models ! ! !

    ! class Knight(models.Model): name = models.CharField(max_length=63) ! def __str__(self): return self.name
  40. # roundtable/models.py from __future__ import unicode_literals from django.db import models

    from django.utils.encoding import python_2_unicode_compatible ! @python_2_unicode_compatible class Knight(models.Model): name = models.CharField(max_length=63) ! def __str__(self): return self.name
  41. Migrations History South Kickstarter

  42. What is a migration? Version Control for you database schema

    Migrations file are instructions for predictably altering the database Change model
 => create migrations file
 => migrate (alter) database
  43. South Migrations in Django 1.6

  44. Migrations in 1.6 New Project Recommendations: Create project Add ‘south’,

    to settings.py Start with call to syncdb Create app
  45. $ ./manage.py schemamigration roundtable --initial Creating migrations directory at ‘camelot/

    roundtable/migrations'... Creating __init__.py in 'camelot/roundtable/ migrations'... + Added model roundtable.Knight Created 0001_initial.py. You can now apply this migration with: ./manage.py migrate roundtable
  46. # roundtable/migrations/0001_initial.py # -*- coding: utf-8 -*- from south.utils import

    datetime_utils as datetime from south.db import db from south.v2 import SchemaMigration from django.db import models ! ! class Migration(SchemaMigration): ! def forwards(self, orm): ... def backwards(self, orm): ... models = {...} complete_apps = ['roundtable']
  47. # roundtable/migrations/0001_initial.py from south.db import db ! class Migration(SchemaMigration): !

    def forwards(self, orm): # Adding model 'Knight' db.create_table('roundtable_knight', ( ('id', self.gf('django.db.models.fields.AutoField') (primary_key=True)), ('name', self.gf('django.db.models.fields.CharField') (max_length=63)), )) db.send_create_signal('roundtable', ['Knight'])
  48. # roundtable/migrations/0001_initial.py from south.db import db ! class Migration(SchemaMigration): !

    def backwards(self, orm): # Deleting model 'Knight' db.delete_table('roundtable_knight')
  49. # roundtable/migrations/0001_initial.py class Migration(SchemaMigration): ! models = { 'roundtable.knight': {

    'Meta': {'object_name': 'Knight'}, 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), 'name': ('django.db.models.fields.CharField', [], {'max_length': '63'}) } }
  50. $ ./manage.py migrate roundtable 0001 - Soft matched migration 0001

    to 0001_initial. Running migrations for roundtable: - Migrating forwards to 0001_initial. > roundtable:0001_initial - Loading initial data for roundtable. Installed 0 object(s) from 0 fixture(s)
  51. >>> Knight.objects.bulk_create([ ... Knight(name='Bedevere'), ... Knight(name='Bors'), ... Knight(name='Ector'), ... Knight(name='Galahad'),

    ... Knight(name='Gawain'), ... Knight(name='Lancelot'), ... Knight(name='Robin'), ... ]) [<Knight: Bedevere>, <Knight: Bors>, <Knight: Ector>, <Knight: Galahad>, <Knight: Gawain>, <Knight: Lancelot>, <Knight: Robin>]
  52. $ ./manage.py dumpdata --indent=2 roundtable > \ > roundtable/fixtures/initial_data.json

  53. $ ./manage.py migrate roundtable zero

  54. # roundtable/migrations/0001_initial.py def backwards(self, orm): # Deleting model 'Knight' db.delete_table('roundtable_knight')

  55. $ ./manage.py migrate roundtable 0001 - Soft matched migration 0001

    to 0001_initial. Running migrations for roundtable: - Migrating forwards to 0001_initial. > roundtable:0001_initial - Loading initial data for roundtable. Installed 7 object(s) from 1 fixture(s)
  56. DAMN IT, LANCELOT

  57. # roundtable/models.py from __future__ import unicode_literals from django.db import models

    from django.utils.encoding import python_2_unicode_compatible ! @python_2_unicode_compatible class Knight(models.Model): name = models.CharField(max_length=63) traitor = models.BooleanField() ! def __str__(self): return self.name
  58. $ ./manage.py schemamigration roundtable --auto

  59. # roundtable/migrations/0001_initial.py models = { 'roundtable.knight': { 'Meta': {'object_name': 'Knight'},

    'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), 'name': ('django.db.models.fields.CharField', [], {'max_length': '63'}) } }
  60. $ ./manage.py schemamigration roundtable --auto ? The field 'Knight.traitor' does

    not have a default specified, yet is NOT NULL. ? Since you are adding this field, you MUST specify a default ? value to use for existing rows. Would you like to: ? 1. Quit now, and add a default to the field in models.py ? 2. Specify a one-off value to use for existing columns now ? Please select a choice: 2 ? Please enter Python code for your one-off default value. ? The datetime module is available, so you can do e.g. datetime.date.today() >>> False + Added field traitor on roundtable.Knight Created 0002_auto__add_field_knight_traitor.py. You can now apply this migration with:
  61. $ ./manage.py migrate roundtable

  62. None
  63. Step back! Delete:
 0002_auto__add_field_knight_traitor.py Remove:
 traitor = models.BooleanField()

  64. $ ./manage.py migrate roundtable zero ! $ mv roundtable/fixtures/initial_data.json \

    roundtable/fixtures/0002_add_knight_data.json ! $ ./manage.py migrate roundtable 0001 - Soft matched migration 0001 to 0001_initial. Running migrations for roundtable: - Migrating forwards to 0001_initial. > roundtable:0001_initial - Loading initial data for roundtable. Installed 0 object(s) from 0 fixture(s)
  65. $ ./manage.py datamigration roundtable add_knight_data Created 0002_add_knight_data.py.

  66. # roundtable/migrations/0002_add_knight_data.py class Migration(DataMigration): ! def forwards(self, orm): "Write your

    forwards methods here." ! def backwards(self, orm): "Write your backwards methods here."
  67. # roundtable/migrations/0002_add_knight_data.py class Migration(SchemaMigration): ! def forwards(self, orm): "Write your

    forwards methods here." from django.core.management import call_command call_command('loaddata', '0002_add_knight_data.json')
  68. # roundtable/migrations/0002_add_knight_data.py class Migration(SchemaMigration): ! def forwards(self, orm): "Write your

    forwards methods here." orm.Knight.objects.bulk_create([ orm.Knight(name='Bedevere'), orm.Knight(name='Bors'), orm.Knight(name='Ector'), orm.Knight(name='Galahad'), orm.Knight(name='Gawain'), orm.Knight(name='Lancelot'), orm.Knight(name='Robin'), ])
  69. $ ./manage.py migrate roundtable zero $ ./manage.py migrate roundtable Running

    migrations for roundtable: - Migrating forwards to 0002_add_knight_data. > roundtable:0001_initial > roundtable:0002_add_knight_data - Migration 'roundtable:0002_add_knight_data' is marked for no-dry-run. - Loading initial data for roundtable. Installed 0 object(s) from 0 fixture(s)
  70. # roundtable/migrations/0002_add_knight_data.py def backwards(self, orm): orm.Knight.objects.all().delete()

  71. $ ./manage.py migrate roundtable 0001 - Soft matched migration 0001

    to 0001_initial. Running migrations for roundtable: - Migrating backwards to just after 0001_initial. < roundtable:0002_add_knight_data - Migration 'roundtable:0002_add_knight_data' is marked for no-dry-run.
  72. >>> from roundtable.models import Knight >>> Knight.objects.all() []

  73. # roundtable/models.py class Knight(models.Model): ... traitor = models.BooleanField()

  74. $ ./manage.py schemamigration roundtable --auto ? The field 'Knight.traitor' does

    not have a default specified, yet is NOT NULL. ? Since you are adding this field, you MUST specify a default ? value to use for existing rows. Would you like to: ? 1. Quit now, and add a default to the field in models.py ? 2. Specify a one-off value to use for existing columns now ? Please select a choice: 2 ? Please enter Python code for your one-off default value. ? The datetime module is available, so you can do e.g. datetime.date.today() >>> False + Added field traitor on roundtable.Knight Created 0003_auto__add_field_knight_traitor.py. You can now apply this migration with: ./manage.py migrate roundtable
  75. $ ./manage.py migrate roundtable Running migrations for roundtable: - Migrating

    forwards to 0003_auto__add_field_knight_traitor. > roundtable:0002_add_knight_data - Migration 'roundtable:0002_add_knight_data' is marked for no-dry-run. > roundtable:0003_auto__add_field_knight_traitor - Loading initial data for roundtable. Installed 0 object(s) from 0 fixture(s)
  76. >>> from roundtable.models import Knight >>> Knight.objects.get(name__iexact="Lancelot").traitor False

  77. $ ./manage.py datamigration roundtable label_lancelot_traitor Created 0004_label_lancelot_traitor.py.

  78. # roundtable/migrations/0004_label_lancelot_traitor.py class Migration(DataMigration): ! def forwards(self, orm): lancelot =

    orm.Knight.objects.get( name__iexact="Lancelot") lancelot.traitor = True lancelot.save()
  79. # roundtable/migrations/0004_label_lancelot_traitor.py class Migration(DataMigration): ! def backwards(self, orm): lancelot =

    orm.Knight.objects.get( name__iexact="Lancelot") lancelot.traitor = False lancelot.save()
  80. $ ./manage.py migrate roundtable Running migrations for roundtable: - Migrating

    forwards to 0004_label_lancelot_traitor. > roundtable:0004_label_lancelot_traitor - Migration 'roundtable: 0004_label_lancelot_traitor' is marked for no-dry- run. - Loading initial data for roundtable. Installed 0 object(s) from 0 fixture(s)
  81. >>> from roundtable.models import Knight >>> Knight.objects.get(name__iexact="Lancelot").traitor True

  82. Django 1.7 Migrations

  83. $ django-admin startproject camelot $ cd camelot/ $ ./manage.py startapp

    roundtable
  84. # roundtable/models.py from __future__ import unicode_literals from django.db import models

    from django.utils.encoding import python_2_unicode_compatible ! @python_2_unicode_compatible class Knight(models.Model): name = models.CharField(max_length=63) ! def __str__(self): return self.name
  85. $ ./manage.py makemigrations Migrations for 'roundtable': 0001_initial.py: - Create model

    Knight
  86. # roundtable/migrations/0001_initial.py # -*- coding: utf-8 -*- from __future__ import

    unicode_literals from django.db import models, migrations ! ! class Migration(migrations.Migration): ! dependencies = [] operations = [...]
  87. # roundtable/migrations/0001_initial.py class Migration(migrations.Migration): ! operations = [ migrations.CreateModel( name='Knight',

    fields=[ ('id', models.AutoField( primary_key=True, verbose_name='ID', serialize=False, auto_created=True)), ('name', models.CharField( max_length=63)), ], options={}, bases=(models.Model,), ), ]
  88. $ ./manage.py migrate Operations to perform: Apply all migrations: auth,

    admin, contenttypes, roundtable, sessions Running migrations: Applying contenttypes.0001_initial... OK Applying auth.0001_initial... OK Applying admin.0001_initial... OK Applying roundtable.0001_initial... OK Applying sessions.0001_initial... OK
  89. $ ./manage.py makemigrations --empty roundtable Migrations for 'roundtable': 0002_auto_20140903_1600.py:

  90. $ mv roundtable/migrations/0002_auto_20140903_1600.py \ roundtable/migrations/0002_auto_add_knight_data.py

  91. # roundtable/migrations/0002_auto_add_knight_data.py ! ! ! ! ! ! ! !

    ! class Migration(migrations.Migration): ... operations = [ ! ! ! ]
  92. # roundtable/migrations/0002_auto_add_knight_data.py ! def add_knight_data(apps, schema_editor): pass ! ! def

    remove_knight_data(apps, schema_editor): pass ! ! class Migration(migrations.Migration): ... operations = [ migrations.RunPython( add_knight_data, reverse_code=remove_knight_data), ]
  93. # roundtable/migrations/0002_auto_add_knight_data.py from django.core.management import call_command ! ! def add_knight_data(apps,

    schema_editor): call_command('loaddata', 'knight_data.json')
  94. # roundtable/migrations/0002_auto_add_knight_data.py def add_knight_data(apps, schema_editor): Knight = apps.get_model('roundtable', 'Knight') Knight.objects.bulk_create([

    Knight(name='Bedevere'), Knight(name='Bors'), Knight(name='Ector'), Knight(name='Galahad'), Knight(name='Gawain'), Knight(name='Lancelot'), Knight(name='Robin'), ])
  95. # roundtable/migrations/0002_auto_add_knight_data.py def remove_knight_data(apps, schema_editor): Knight = apps.get_model('roundtable', 'Knight') Knight.objects.all().delete()

  96. DAMN IT LANCELOT (AGAIN)

  97. # roundtable/models.py from __future__ import unicode_literals from django.db import models

    from django.utils.encoding import python_2_unicode_compatible ! @python_2_unicode_compatible class Knight(models.Model): name = models.CharField(max_length=63) traitor = models.BooleanField() ! def __str__(self): return self.name
  98. $ ./manage.py check System check identified some issues: ! WARNINGS:

    roundtable.Knight.traitor: (1_6.W002) BooleanField does not have a default value. HINT: Django 1.6 changed the default value of BooleanField from False to None. See [DOCS LINK] for more information. ! System check identified 1 issue (0 silenced).
  99. $ ./manage.py makemigrations System check identified some issues: ! WARNINGS:

    [Removed from Slides] You are trying to add a non-nullable field 'traitor' to knight without a default; we can't do that (the database needs something to populate existing rows). Please select a fix: 1) Provide a one-off default now (will be set on all existing rows) 2) Quit, and let me add a default in models.py Select an option: 1 Please enter the default value now, as valid Python The datetime module is available, so you can do e.g. datetime.date.today()
  100. >>> False Migrations for 'roundtable': 0003_knight_traitor.py: - Add field traitor

    to knight
  101. $ ./manage.py migrate System check identified some issues: ! WARNINGS:

    roundtable.Knight.traitor: (1_6.W002) BooleanField does not have a default value. HINT: Django 1.6 changed the default value of BooleanField from False to None. See [DOCS LINK] for more information. Operations to perform: Apply all migrations: auth, sessions, admin, roundtable, contenttypes Running migrations: Applying roundtable.0003_knight_traitor... OK
  102. Don’t Do That. Listen to check.

  103. # roundtable/models.py from __future__ import unicode_literals from django.db import models

    from django.utils.encoding import python_2_unicode_compatible ! @python_2_unicode_compatible class Knight(models.Model): name = models.CharField(max_length=63) traitor = models.BooleanField(default=False) ! def __str__(self): return self.name
  104. $ ./manage.py makemigrations $ ./manage.py migrate

  105. >>> from roundtable.models import Knight >>> Knight.objects.get(name__iexact="Lancelot").traitor False

  106. $ ./manage.py makemigrations --empty roundtable Migrations for 'roundtable': 0004_auto_20140903_2045.py:

  107. $ mv roundtable/migrations/0004_auto_20140903_2045.py \ roundtable/migrations/0004_label_lancelot_traitor.py

  108. # roundtable/migrations/0004_label_lancelot_traitor.py # -*- coding: utf-8 -*- from __future__ import

    unicode_literals from django.db import models, migrations ! ... ! class Migration(migrations.Migration): ! dependencies = [ ('roundtable', '0003_knight_traitor'), ] ! operations = [ migrations.RunPython( set_lancelot_traitor, reverse_code=unset_lancelot_traitor) ]
  109. # roundtable/migrations/0004_label_lancelot_traitor.py def set_lancelot_traitor(apps, schema_editor): Knight = apps.get_model('roundtable', 'Knight') lancelot

    = Knight.objects.get( name__iexact='Lancelot') lancelot.traitor = True lancelot.save()
  110. # roundtable/migrations/0004_label_lancelot_traitor.py def unset_lancelot_traitor(apps, schema_editor): Knight = apps.get_model('roundtable', 'Knight') lancelot

    = Knight.objects.get( name__iexact='Lancelot') lancelot.traitor = False lancelot.save()
  111. $ ./manage.py migrate

  112. >>> from roundtable.models import Knight >>> Knight.objects.get(name__iexact="Lancelot").traitor True

  113. REVIEW OF EXAMPLE

  114. Migrations

  115. Migrations in 1.7 initial_data system deprecated Fixtures still valid Frozen

    Model gone
  116. None
  117. Migrations in 1.7 Historical Model built and kept in memory

    Integrated with App Loader Utility of Dependency Caveat: Python Classes do not serialize!
 ModelA.save() will not be called. May be slow…
  118. Migrations in 1.7 Workflow Identical change model create migration apply

    migration
  119. South will NOT run in 1.7 Upgrade from South: Latest

    Migration Create Native Delete South Settings: MIGRATION_MODULES and SOUTH_MIGRATION_MODULES South and 1.7
  120. Legacy Behavior:
 rm -R appname/migrations/ Remove Migrations

  121. App Loader

  122. App Cache Likened to Borg by Andrew Godwin Shared State

    Incompatible with migrations
  123. New App Registry Allow apps without a models module or

    package apps.get_model() app/apps.py Calls made in AppConfig admin.autodiscover()
  124. Systems Check

  125. Systems Check check in 1.6 verified settings response to 1.5

    upgrade now extendable and modular validate deprecated (avoid confusion with field and forms use check
  126. from django.core.checks import register ! @register('security') def my_check(app_configs, **kwargs): #

    ... perform checks, collect errors return errors
  127. Other Changes

  128. Customized ORM Interaction QuerySet objects = KnightQuerySet.as_manager() Lookups Prefetch object

    for prefetch_related()
  129. PREPARING TO UPGRADE

  130. None
  131. Upgrade Process Dependencies Release Notes Test Suite

  132. Upgrade Schedule Schedule Time for Upgrade Upgrade Early. Upgrade Often.

    Start with Release Candidate
  133. 1.0 1.1 1.2 1.3 1.4 1.5 1.6 R² = 0.8733

    190d 253d 340d 366d 310d 292d 329d Duration Until Next Release Polynomial Regression (Order 3) Expected
  134. 1.0 1.1 1.2 1.3 1.4 1.5 1.6 R² = 0.1971

    300d 253d 340d 366d 310d 292d 329d Duration Until Next Release Polynomial Regression (Order 3) Actual
  135. Practical Knowledge Django 1.7 means death of 1.5 Django 1.5

    has not been added to LTS support Django 1.4 has less than a year of support Migrate to Django 1.6 now
  136. Practical Upgrade Knowledge Iterate Through Versions Password Gotcha: base36 (1.5)

    -> base64 (1.6) syncdb is dead. Long live migrate Initial data fixtures are dead.
 Long live data migrations. Fields need deconstruct()
  137. The Future? LTS Status for 1.8? Look ahead at deprecations

    URL Configurations lists no patterns direct imports
  138. Thank you! (Dev Documentation)

  139. QUESTIONS? AFRG.CO/UPDJ17 @ANDREWSFORGE ANDREWSFORGE.COM AFRG.CO/UPDJ17 @ANDREWSFORGE ANDREWSFORGE.COM QUESTIONS?