Upgrading Django (to 1.7)

Slides for my talk at DjangoCon 2014.

Transcript

1. ANDREW PINKHAM TOPIC DATE AUTHOR SEPTEMBER 2014 UPGRADING DJANGO (TO 1.7) DJANGOCON 2014 ORIGINAL ART BY ERIK DEPRINCE WALLPAPER CREATED BY BRYAN VELOSO AND CHRISTIAN METTS
 WALLPAPER SITE HOSTED BY JEFF TRIPLETT
2. $ whoami Software Engineer Freelance Consultant Technical Instructor Andrew Pinkham Hi, I’m
3. AUTHOR OF DJANGO UNLEASHED SET TO PUBLISH BEGINNING OF 2015
4. $ less updj17 Django Releases Versioning and Support Summary of Previous Releases Django 1.7 (v1.6 vs v1.7) Migrations App Loading Static Systems Check
7. afrg.co/updj17
8. DJANGO VERSIONS UNDERSTANDING DJANGO’S RELEASE PROCESS
9. M.m.μ
10. 1.7.0
11. Pre-Releases Alpha Beta Release Candidate
12. M.m.μ_#
13. 1.7.0a1
14. 1.7b1
15. 1.7c2
16. Version Support Two Versions Supported Deprecations over Two Versions
17. 1.3 㱺 1.5 1.4 㱺 1.6 1.5 㱺 1.7 Deprecations
18. Support Prior to v1.7 ! 1.6.6 1.6.5 1.6.4 1.6.3 … 1.6
 1.5.9 1.5.8 1.5.7 1.5.6 … 1.5
19. With Arrival of v1.7 1.7
 1.6.7 1.6.6 1.6.5 1.6.4 … 1.6
 1.5.10 1.5.9 1.5.8 1.5.7 … 1.5
20. Long Term Support (LTS) Django 1.4
21. With Arrival of v1.7 1.7
 1.6.7 1.6.6 1.6.5 1.6.4 … 1.6
 1.4.15 1.4.14 1.4.13 1.4.12 ! 1.4
22. PREVIOUS DJANGO VERSIONS A TRIP DOWN MEMORY LANE
23. 1.7 1.6 1.5 (Deprecations) 1.4 (LTS) Relevant (Minor) Versions
24. Django 1.4 Timezone Support ORM Additions Cookie-Based Session Handling Auth Security Functional Browser Testing New Project Creation and Directory Structure
25. Django 1.5 Python 2.6+ (2.5 Dropped) Experimental Support for Python 3.2 Custom Django User Generic Class-Based Views JSON
26. Django 1.6 Python 2.6+, Python 3.2 Transactions Simpler, Smarter Defaults django-admin.py validate
27. “Django 1.7 is shaping up to be
 the biggest Django release since 1.0.” – Russell Keith-Magee
28. Django 1.7 Migrations App Loading Static Systems Check Custom QuerySet Custom Lookups Prefetch
29. 1.7 DJANGO
30. 2.7 (AND 3.2+) PYTHON
32. DJANGO 1.7 COMPARING DJANGO 1.7 TO DJANGO 1.6 CAMELOT!
33. Django 1.7 Migrations App Loading Static Systems Check
36. $ django-admin.py startproject camelot # dj16 ! $ django-admin startproject camelot # dj17 ! ! ! /manage.py /camelot/ __init__.py settings.py urls.py wsgi.py
37. South in Django 1.6 Best Practice: run syncdb now
38. $ ./manage.py startapp roundtable ! /camelot/roundtable/ __init__.py admin.py migrations/ __init.py__ models.py tests.py views.py
39. # roundtable/models.py ! from django.db import models ! ! ! ! class Knight(models.Model): name = models.CharField(max_length=63) ! def __str__(self): return self.name
40. # roundtable/models.py from __future__ import unicode_literals from django.db import models from django.utils.encoding import python_2_unicode_compatible ! @python_2_unicode_compatible class Knight(models.Model): name = models.CharField(max_length=63) ! def __str__(self): return self.name
41. Migrations History South Kickstarter
42. What is a migration? Version Control for you database schema Migrations file are instructions for predictably altering the database Change model
 => create migrations file
 => migrate (alter) database
43. South Migrations in Django 1.6
44. Migrations in 1.6 New Project Recommendations: Create project Add ‘south’, to settings.py Start with call to syncdb Create app
45. $ ./manage.py schemamigration roundtable --initial Creating migrations directory at ‘camelot/ roundtable/migrations'... Creating __init__.py in 'camelot/roundtable/ migrations'... + Added model roundtable.Knight Created 0001_initial.py. You can now apply this migration with: ./manage.py migrate roundtable
46. # roundtable/migrations/0001_initial.py # -*- coding: utf-8 -*- from south.utils import datetime_utils as datetime from south.db import db from south.v2 import SchemaMigration from django.db import models ! ! class Migration(SchemaMigration): ! def forwards(self, orm): ... def backwards(self, orm): ... models = {...} complete_apps = ['roundtable']
47. # roundtable/migrations/0001_initial.py from south.db import db ! class Migration(SchemaMigration): ! def forwards(self, orm): # Adding model 'Knight' db.create_table('roundtable_knight', ( ('id', self.gf('django.db.models.fields.AutoField') (primary_key=True)), ('name', self.gf('django.db.models.fields.CharField') (max_length=63)), )) db.send_create_signal('roundtable', ['Knight'])
48. # roundtable/migrations/0001_initial.py from south.db import db ! class Migration(SchemaMigration): ! def backwards(self, orm): # Deleting model 'Knight' db.delete_table('roundtable_knight')
49. # roundtable/migrations/0001_initial.py class Migration(SchemaMigration): ! models = { 'roundtable.knight': { 'Meta': {'object_name': 'Knight'}, 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), 'name': ('django.db.models.fields.CharField', [], {'max_length': '63'}) } }
50. $ ./manage.py migrate roundtable 0001 - Soft matched migration 0001 to 0001_initial. Running migrations for roundtable: - Migrating forwards to 0001_initial. > roundtable:0001_initial - Loading initial data for roundtable. Installed 0 object(s) from 0 fixture(s)
51. >>> Knight.objects.bulk_create([ ... Knight(name='Bedevere'), ... Knight(name='Bors'), ... Knight(name='Ector'), ... Knight(name='Galahad'), ... Knight(name='Gawain'), ... Knight(name='Lancelot'), ... Knight(name='Robin'), ... ]) [<Knight: Bedevere>, <Knight: Bors>, <Knight: Ector>, <Knight: Galahad>, <Knight: Gawain>, <Knight: Lancelot>, <Knight: Robin>]
52. $ ./manage.py dumpdata --indent=2 roundtable > \ > roundtable/fixtures/initial_data.json
53. $ ./manage.py migrate roundtable zero
54. # roundtable/migrations/0001_initial.py def backwards(self, orm): # Deleting model 'Knight' db.delete_table('roundtable_knight')
55. $ ./manage.py migrate roundtable 0001 - Soft matched migration 0001 to 0001_initial. Running migrations for roundtable: - Migrating forwards to 0001_initial. > roundtable:0001_initial - Loading initial data for roundtable. Installed 7 object(s) from 1 fixture(s)
56. DAMN IT, LANCELOT
57. # roundtable/models.py from __future__ import unicode_literals from django.db import models from django.utils.encoding import python_2_unicode_compatible ! @python_2_unicode_compatible class Knight(models.Model): name = models.CharField(max_length=63) traitor = models.BooleanField() ! def __str__(self): return self.name
58. $ ./manage.py schemamigration roundtable --auto
59. # roundtable/migrations/0001_initial.py models = { 'roundtable.knight': { 'Meta': {'object_name': 'Knight'}, 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), 'name': ('django.db.models.fields.CharField', [], {'max_length': '63'}) } }
60. $ ./manage.py schemamigration roundtable --auto ? The field 'Knight.traitor' does not have a default specified, yet is NOT NULL. ? Since you are adding this field, you MUST specify a default ? value to use for existing rows. Would you like to: ? 1. Quit now, and add a default to the field in models.py ? 2. Specify a one-off value to use for existing columns now ? Please select a choice: 2 ? Please enter Python code for your one-off default value. ? The datetime module is available, so you can do e.g. datetime.date.today() >>> False + Added field traitor on roundtable.Knight Created 0002_auto__add_field_knight_traitor.py. You can now apply this migration with:
61. $ ./manage.py migrate roundtable
63. Step back! Delete:
 0002_auto__add_field_knight_traitor.py Remove:
 traitor = models.BooleanField()
64. $ ./manage.py migrate roundtable zero ! $ mv roundtable/fixtures/initial_data.json \ roundtable/fixtures/0002_add_knight_data.json ! $ ./manage.py migrate roundtable 0001 - Soft matched migration 0001 to 0001_initial. Running migrations for roundtable: - Migrating forwards to 0001_initial. > roundtable:0001_initial - Loading initial data for roundtable. Installed 0 object(s) from 0 fixture(s)
65. $ ./manage.py datamigration roundtable add_knight_data Created 0002_add_knight_data.py.
66. # roundtable/migrations/0002_add_knight_data.py class Migration(DataMigration): ! def forwards(self, orm): "Write your forwards methods here." ! def backwards(self, orm): "Write your backwards methods here."
67. # roundtable/migrations/0002_add_knight_data.py class Migration(SchemaMigration): ! def forwards(self, orm): "Write your forwards methods here." from django.core.management import call_command call_command('loaddata', '0002_add_knight_data.json')
68. # roundtable/migrations/0002_add_knight_data.py class Migration(SchemaMigration): ! def forwards(self, orm): "Write your forwards methods here." orm.Knight.objects.bulk_create([ orm.Knight(name='Bedevere'), orm.Knight(name='Bors'), orm.Knight(name='Ector'), orm.Knight(name='Galahad'), orm.Knight(name='Gawain'), orm.Knight(name='Lancelot'), orm.Knight(name='Robin'), ])
69. $ ./manage.py migrate roundtable zero $ ./manage.py migrate roundtable Running migrations for roundtable: - Migrating forwards to 0002_add_knight_data. > roundtable:0001_initial > roundtable:0002_add_knight_data - Migration 'roundtable:0002_add_knight_data' is marked for no-dry-run. - Loading initial data for roundtable. Installed 0 object(s) from 0 fixture(s)
70. # roundtable/migrations/0002_add_knight_data.py def backwards(self, orm): orm.Knight.objects.all().delete()
71. $ ./manage.py migrate roundtable 0001 - Soft matched migration 0001 to 0001_initial. Running migrations for roundtable: - Migrating backwards to just after 0001_initial. < roundtable:0002_add_knight_data - Migration 'roundtable:0002_add_knight_data' is marked for no-dry-run.
72. >>> from roundtable.models import Knight >>> Knight.objects.all() []
73. # roundtable/models.py class Knight(models.Model): ... traitor = models.BooleanField()
74. $ ./manage.py schemamigration roundtable --auto ? The field 'Knight.traitor' does not have a default specified, yet is NOT NULL. ? Since you are adding this field, you MUST specify a default ? value to use for existing rows. Would you like to: ? 1. Quit now, and add a default to the field in models.py ? 2. Specify a one-off value to use for existing columns now ? Please select a choice: 2 ? Please enter Python code for your one-off default value. ? The datetime module is available, so you can do e.g. datetime.date.today() >>> False + Added field traitor on roundtable.Knight Created 0003_auto__add_field_knight_traitor.py. You can now apply this migration with: ./manage.py migrate roundtable
75. $ ./manage.py migrate roundtable Running migrations for roundtable: - Migrating forwards to 0003_auto__add_field_knight_traitor. > roundtable:0002_add_knight_data - Migration 'roundtable:0002_add_knight_data' is marked for no-dry-run. > roundtable:0003_auto__add_field_knight_traitor - Loading initial data for roundtable. Installed 0 object(s) from 0 fixture(s)
76. >>> from roundtable.models import Knight >>> Knight.objects.get(name__iexact="Lancelot").traitor False
77. $ ./manage.py datamigration roundtable label_lancelot_traitor Created 0004_label_lancelot_traitor.py.
78. # roundtable/migrations/0004_label_lancelot_traitor.py class Migration(DataMigration): ! def forwards(self, orm): lancelot = orm.Knight.objects.get( name__iexact="Lancelot") lancelot.traitor = True lancelot.save()
79. # roundtable/migrations/0004_label_lancelot_traitor.py class Migration(DataMigration): ! def backwards(self, orm): lancelot = orm.Knight.objects.get( name__iexact="Lancelot") lancelot.traitor = False lancelot.save()
80. $ ./manage.py migrate roundtable Running migrations for roundtable: - Migrating forwards to 0004_label_lancelot_traitor. > roundtable:0004_label_lancelot_traitor - Migration 'roundtable: 0004_label_lancelot_traitor' is marked for no-dry- run. - Loading initial data for roundtable. Installed 0 object(s) from 0 fixture(s)
81. >>> from roundtable.models import Knight >>> Knight.objects.get(name__iexact="Lancelot").traitor True
82. Django 1.7 Migrations
83. $ django-admin startproject camelot $ cd camelot/ $ ./manage.py startapp roundtable
84. # roundtable/models.py from __future__ import unicode_literals from django.db import models from django.utils.encoding import python_2_unicode_compatible ! @python_2_unicode_compatible class Knight(models.Model): name = models.CharField(max_length=63) ! def __str__(self): return self.name
85. $ ./manage.py makemigrations Migrations for 'roundtable': 0001_initial.py: - Create model Knight
86. # roundtable/migrations/0001_initial.py # -*- coding: utf-8 -*- from __future__ import unicode_literals from django.db import models, migrations ! ! class Migration(migrations.Migration): ! dependencies = [] operations = [...]
87. # roundtable/migrations/0001_initial.py class Migration(migrations.Migration): ! operations = [ migrations.CreateModel( name='Knight', fields=[ ('id', models.AutoField( primary_key=True, verbose_name='ID', serialize=False, auto_created=True)), ('name', models.CharField( max_length=63)), ], options={}, bases=(models.Model,), ), ]
88. $ ./manage.py migrate Operations to perform: Apply all migrations: auth, admin, contenttypes, roundtable, sessions Running migrations: Applying contenttypes.0001_initial... OK Applying auth.0001_initial... OK Applying admin.0001_initial... OK Applying roundtable.0001_initial... OK Applying sessions.0001_initial... OK
89. $ ./manage.py makemigrations --empty roundtable Migrations for 'roundtable': 0002_auto_20140903_1600.py:
90. $ mv roundtable/migrations/0002_auto_20140903_1600.py \ roundtable/migrations/0002_auto_add_knight_data.py
91. # roundtable/migrations/0002_auto_add_knight_data.py ! ! ! ! ! ! ! ! ! class Migration(migrations.Migration): ... operations = [ ! ! ! ]
92. # roundtable/migrations/0002_auto_add_knight_data.py ! def add_knight_data(apps, schema_editor): pass ! ! def remove_knight_data(apps, schema_editor): pass ! ! class Migration(migrations.Migration): ... operations = [ migrations.RunPython( add_knight_data, reverse_code=remove_knight_data), ]
93. # roundtable/migrations/0002_auto_add_knight_data.py from django.core.management import call_command ! ! def add_knight_data(apps, schema_editor): call_command('loaddata', 'knight_data.json')
94. # roundtable/migrations/0002_auto_add_knight_data.py def add_knight_data(apps, schema_editor): Knight = apps.get_model('roundtable', 'Knight') Knight.objects.bulk_create([ Knight(name='Bedevere'), Knight(name='Bors'), Knight(name='Ector'), Knight(name='Galahad'), Knight(name='Gawain'), Knight(name='Lancelot'), Knight(name='Robin'), ])
95. # roundtable/migrations/0002_auto_add_knight_data.py def remove_knight_data(apps, schema_editor): Knight = apps.get_model('roundtable', 'Knight') Knight.objects.all().delete()
96. DAMN IT LANCELOT (AGAIN)
97. # roundtable/models.py from __future__ import unicode_literals from django.db import models from django.utils.encoding import python_2_unicode_compatible ! @python_2_unicode_compatible class Knight(models.Model): name = models.CharField(max_length=63) traitor = models.BooleanField() ! def __str__(self): return self.name
98. $ ./manage.py check System check identified some issues: ! WARNINGS: roundtable.Knight.traitor: (1_6.W002) BooleanField does not have a default value. HINT: Django 1.6 changed the default value of BooleanField from False to None. See [DOCS LINK] for more information. ! System check identified 1 issue (0 silenced).
99. $ ./manage.py makemigrations System check identified some issues: ! WARNINGS: [Removed from Slides] You are trying to add a non-nullable field 'traitor' to knight without a default; we can't do that (the database needs something to populate existing rows). Please select a fix: 1) Provide a one-off default now (will be set on all existing rows) 2) Quit, and let me add a default in models.py Select an option: 1 Please enter the default value now, as valid Python The datetime module is available, so you can do e.g. datetime.date.today()
100. >>> False Migrations for 'roundtable': 0003_knight_traitor.py: - Add field traitor to knight
101. $ ./manage.py migrate System check identified some issues: ! WARNINGS: roundtable.Knight.traitor: (1_6.W002) BooleanField does not have a default value. HINT: Django 1.6 changed the default value of BooleanField from False to None. See [DOCS LINK] for more information. Operations to perform: Apply all migrations: auth, sessions, admin, roundtable, contenttypes Running migrations: Applying roundtable.0003_knight_traitor... OK
102. Don’t Do That. Listen to check.
103. # roundtable/models.py from __future__ import unicode_literals from django.db import models from django.utils.encoding import python_2_unicode_compatible ! @python_2_unicode_compatible class Knight(models.Model): name = models.CharField(max_length=63) traitor = models.BooleanField(default=False) ! def __str__(self): return self.name
104. $ ./manage.py makemigrations $ ./manage.py migrate
105. >>> from roundtable.models import Knight >>> Knight.objects.get(name__iexact="Lancelot").traitor False
106. $ ./manage.py makemigrations --empty roundtable Migrations for 'roundtable': 0004_auto_20140903_2045.py:
107. $ mv roundtable/migrations/0004_auto_20140903_2045.py \ roundtable/migrations/0004_label_lancelot_traitor.py
108. # roundtable/migrations/0004_label_lancelot_traitor.py # -*- coding: utf-8 -*- from __future__ import unicode_literals from django.db import models, migrations ! ... ! class Migration(migrations.Migration): ! dependencies = [ ('roundtable', '0003_knight_traitor'), ] ! operations = [ migrations.RunPython( set_lancelot_traitor, reverse_code=unset_lancelot_traitor) ]
109. # roundtable/migrations/0004_label_lancelot_traitor.py def set_lancelot_traitor(apps, schema_editor): Knight = apps.get_model('roundtable', 'Knight') lancelot = Knight.objects.get( name__iexact='Lancelot') lancelot.traitor = True lancelot.save()
110. # roundtable/migrations/0004_label_lancelot_traitor.py def unset_lancelot_traitor(apps, schema_editor): Knight = apps.get_model('roundtable', 'Knight') lancelot = Knight.objects.get( name__iexact='Lancelot') lancelot.traitor = False lancelot.save()
111. $ ./manage.py migrate
112. >>> from roundtable.models import Knight >>> Knight.objects.get(name__iexact="Lancelot").traitor True
113. REVIEW OF EXAMPLE
114. Migrations
115. Migrations in 1.7 initial_data system deprecated Fixtures still valid Frozen Model gone
117. Migrations in 1.7 Historical Model built and kept in memory Integrated with App Loader Utility of Dependency Caveat: Python Classes do not serialize!
 ModelA.save() will not be called. May be slow…
118. Migrations in 1.7 Workflow Identical change model create migration apply migration
119. South will NOT run in 1.7 Upgrade from South: Latest Migration Create Native Delete South Settings: MIGRATION_MODULES and SOUTH_MIGRATION_MODULES South and 1.7
120. Legacy Behavior:
 rm -R appname/migrations/ Remove Migrations
121. App Loader
122. App Cache Likened to Borg by Andrew Godwin Shared State Incompatible with migrations
123. New App Registry Allow apps without a models module or package apps.get_model() app/apps.py Calls made in AppConfig admin.autodiscover()
124. Systems Check
125. Systems Check check in 1.6 verified settings response to 1.5 upgrade now extendable and modular validate deprecated (avoid confusion with field and forms use check
126. from django.core.checks import register ! @register('security') def my_check(app_configs, **kwargs): # ... perform checks, collect errors return errors
127. Other Changes
128. Customized ORM Interaction QuerySet objects = KnightQuerySet.as_manager() Lookups Prefetch object for prefetch_related()
129. PREPARING TO UPGRADE
131. Upgrade Process Dependencies Release Notes Test Suite
132. Upgrade Schedule Schedule Time for Upgrade Upgrade Early. Upgrade Often. Start with Release Candidate
133. 1.0 1.1 1.2 1.3 1.4 1.5 1.6 R² = 0.8733 190d 253d 340d 366d 310d 292d 329d Duration Until Next Release Polynomial Regression (Order 3) Expected
134. 1.0 1.1 1.2 1.3 1.4 1.5 1.6 R² = 0.1971 300d 253d 340d 366d 310d 292d 329d Duration Until Next Release Polynomial Regression (Order 3) Actual
135. Practical Knowledge Django 1.7 means death of 1.5 Django 1.5 has not been added to LTS support Django 1.4 has less than a year of support Migrate to Django 1.6 now
136. Practical Upgrade Knowledge Iterate Through Versions Password Gotcha: base36 (1.5) -> base64 (1.6) syncdb is dead. Long live migrate Initial data fixtures are dead.
 Long live data migrations. Fields need deconstruct()
137. The Future? LTS Status for 1.8? Look ahead at deprecations URL Configurations lists no patterns direct imports
138. Thank you! (Dev Documentation)
139. QUESTIONS? AFRG.CO/UPDJ17 @ANDREWSFORGE ANDREWSFORGE.COM AFRG.CO/UPDJ17 @ANDREWSFORGE ANDREWSFORGE.COM QUESTIONS?