Migrating The Future

A talk I gave at DjangoCon EU 2013 about South, Django, and the future of migrations in Django core.

Transcript

1. Andrew Godwin @andrewgodwin Migrating THE Future
2. Andrew Godwin Django core developer South author Database Wrangler
3. South
4. South django.db.migrations
5. South django.db.migrations Kickstarter
6. £2,500
7. £2,500 Raised!
8. £2,500 Raised! 65 minutes in.
9. £17,952 From 502 backers
11. Tom Christie Alex Gaynor Pixtape.com Ulrich Petri (@ulope) Sam Wilson Dan Fairs Nyaruka Ltd Jorge Bastida Aaron Bassett Stefan Foulis Aymeric Augustin Rich Leland Ray Schamp CookDrop Inc. fitclipboard.com Type/Code Bruno Renié Bryan Veloso Smashing Ideas John P. Neumann Filipe Regadas Simon Law (sfllaw@sfllaw.ca) Ryan Kaskel CYeong Baptiste Mispelon Eric Florenzano Albert O'Connor, Web Developer Jeremy Dunck Daniel Lindsley Dan Cundiff Automatique another one GmbH Vincent Driessen (@nvie) Daniel Greenfeld Joey Wilhelm Marek Stępniowski Sil Westerveld (SURFsara) http://Scal.io RMSA - the makers of Fresco Nitty Gritty Software, LLC Greg Aker Christopher Dickes SJM Steffann Messiah Studio (New Zealand) Dirty Monkey Łukasz Langa (ambv) Dougal Matthews Joel Crocker
12. Aaron Bassett Aaron Ho Aaron Larner Aaron Malone Adam Connor Adam Fast Adam Foster Adam Gross Adam Joseph Cook Adam Rich Adrian Espinosa Adrian Rego Adrian Smith Agostino Albert O'Connor Alec Perkins Ales Alexander Rudakov Alexandre Lessard Alexey Sveshnikov Alex Gaynor Alex Hung Alex Louden Alex Tziallas Alex Yuzhakov Alfredo Aguirre Ames Cornish Anatoly Popov Andreas Louca Andreas Madsack Andreas Rieger André Simões Andrew Austin Andrew Dunn Andrew Gorcester Andrew Jesaitis Andrzej Voss Andy Baker Andy Dirnberger Angel Freire Anselm Lingnau Arild Martin Møller Haugstad Arnaud Art van Scheppingen Athanassios Diacakis August Bigelow Aymeric Augustin Backer Name Backer Name Backer Name Backer Name Backer Name Backer Name Backer Name Backer Name Baldur Þór Emilsson Baptiste Millou Baptiste Mispelon Barbara Shaurette Bayo opadeyi Ben Ben Firshman Ben Graham Benjamin Wohlwend Ben Lopatin Ben Spaulding BeProud Bernhard Janetzki bitprophet Bob Barcklay Bob Haugen Bob Lozano Bolster Boris Chervenkov Bouke Haarsma Bozidar Benko Brad Montgomery Brad Wright Brandon Bennett Brandon Konkle Brett Epps Brian O'Dell Brian Reigle Brian Rosner Brian Wisti Bruno Renié Bruno Tikami Bryan Koroleski Bryan Veloso Carl Meyer Carlos Ribeiro Carny Cheng Casper S. Jensen Chad Catlett Chris Adams Chris Beaven Chris Drackett Chris Johnston Chris Kleinknecht Chris Streeter Christian Bertschy Christian Lange Christian Toivola Christian Treutler Christophe Pettus Christopher Dickes Christopher Grebs Christopher J. Hasenpflug Christopher Reeves Christoph Heer Chris Vigelius "Chris ""Wraithan"" McDonald" Cícero Raupp Rolim Claes Ström Colin Woods Cuauhtemoc Ramirez Damon Ciarelli Dan Cundiff DanEE Dan Fairs Daniel Gagnon Daniel Greenfeld Daniel Lindsley Daniel Perry Daniel Watkins Danilo Dan Koch Dan Langer Danny Roa Dan Tyan Darian Moody Dave Arter Dave Masse David David Cramer David Feinzeig David Grant David Jeffrey David Novák David Sinsky Denis Darii Dennis Seymour Derek Leverenz Deric Crago Devin Hunt Dj Gilcrease Dmitriy Narkevich Dong Hee Kim Dougal Matthews Doug Tabacco Dustin Davis Edgars Jēkabsons Elliot Murphy Enrico Berti Ensorceled Eric Eric Florenzano Ernesto Rico-Schmidt Esa Evan Culver Ezequiel Pochiero Federico Giordano Felipe Coury Felipe Kellermann Felix Yuan Fernando Gutierrez Fernando Rocha Filipe Regadas Filipp Lepalaan Florian Apolloner fm 10.4 Frederik De Bleser Fusionbox Gabriel Gamil Gabriel Grant g chow George Hickman Gert Van Gool Giles Thomas Giovanni Totaro Graham Ullrich Greg Aker Guilherme Gondim Håkon Erichsen Hans Lawrenz harijay Herbert Poul Horst Gutmann Hrishikesh Ballal Hugo Rodger-Brown Hugo Wang Ian Ildar Samit Ivan Habunek Iván Raskovsky Ivan Yurchenko Ivo Teel Jacek Jacob Briggs Jacob Kaplan-Moss Jaime Irurzun Jakub Wiśniowski James Cleveland James Johnson James Robert James Tiplady James Walker Jamie Wilson Jannis Leidel Jason Jason Cartwright Jason Mayfield Jason Simeone Jasper den Hertog Javed Khan Jeff jens quade Jérémie Blaser Jeremy Cantrell Jeremy Dunck Jeremy Keeshin Jesse Michelsen Jesus Noland Jim Ray Jirka Vejrazka JMatt Peterson João Oliveira Joel Crocker Joel Shapiro Joel Squire Joey Castillo Joey Leingang Joey Wilhelm John Paulett John P. Neumann John Sykora John Wegis Jonas Nockert Jonathan Biddle Jonathan Morgan Jon Danao Jon Fedyk Joonathan Mägi Jorge Bastida Jorge Camargo Jörg Kreß Jose Padilla Joseph Wenninger Josh Ourisman Josh Steiner Joshua Blount Joshua Johanan Josh Wright Julen Ruiz Aizpuru Julian Wachholz Julio Oña Julius Welby Jure Ham Kalle Karjalainen Katie Cunningham Katsumi Honda Keith Gray Kenneth Love Kenneth Reitz Kirill Sibirev Kiwi Koh Chuan Yeong kortina Kura Kuratko Lachlan Musicman Larry Hamel Larry Osborn Lars Wikman Laust M. Ladefoged Leo Leo Hourvitz Leonidas Tsementzis Leon Matthews Leon Smith Le Thanh Duong Logit internet services Ltd. Loic Bistuer Lucas Eisenzimmer Łukasz Langa Luke Hatcher Maciej Konieczny Maciek Gryka Marc Egli Marcelo Alves Marcel Rudolph Marco Bonetti Marc Tamlyn Marek Olszewski Marek Stępniowski Margus Holland Marianne Marielle Volz Mark Hepburn Mark Hurley Mark Longair Mark Lyon Mark Ng Mark Norman Francis Marko Samastur Markus Zapke-Gründemann marteinn Martin Brochhaus Mathieu Fenniak Mathieu Meylan Mathieu Richardoz Matt Austin Matthew Tretter Mattias Linnap Matt Lenc Matt McGlincy Matt Stevens Matt Woodward Matus Moravcik Mauro Doglio Max Arnold Meir Kriheli Michael Blatherwick Michael Fogel Michael Gisi Michael Sandford Michael Twomey Miguel Pilar Mihael Konjević Mike Gallagher MIKe O'Toole Mirus Research Mitch Berkson Mladen Kovacevic moberley Nate Graves Nate Meyvis Nate Thelen Nephila Nick Albright Nick Siemsen Nicolas Perriault Nicolas Pottier Nicole Foster niels Niels Sandholt Busch Nikolai Prokoschenko Noah Kantrowitz Noemi notanumber Numan Sachwani numbsafari Nurzhan Bakibayev Ola Sitarska Oliver Beattie Oliver George Oliver Steele Omen Ondrej Slintak ORDOQUY Xavier Ori Oskar Jakieła Ousmane Wilane Parham Pascal Hartig Pathwright Press Patrick Arminio Patrick Craston Patrick Taylor Paul Bersch Paul Ebreo Paul Mattal Paul Medlock-Walton Peppe Bergqvist Peter Baumgartner + Lincoln Loop Peter Evans Peter Farrell Peter Hyben Peter Inglesby Peter Kropf Peter Sanchez Petr Marhoun Petr Timofeev Philip Kimmey Philipp Bosch Philipp Kruse Philipp Läubli Philipp Weidenhiller Philip Roche Pierre-Julien Grizel pmdhazy Preston Holmes Promeritum quinox Raj Jalan Ralph Ramji Srinivasan Ray Remco Wendt Renato Garcia Pedigoni Renato Oliveira riccardo magliocchetti Richard Beales Rich Leland Rinat Akhtamov Rishabh Manocha Rivo Laks RJ Walsh Rob Berwick Robert Medeiros Roberto Rosario Rod Afshar Rodrigo Chacon Rogelio Rolo Mawlabaux Ross Crawford-d'Heureuse Ross Duggan Russell Keith-Magee Ryan Kaskel Ryan McDevitt Ryan van Roode Sævar Öfjörð Magnússon Sahas Katta Sam Bull Sam Hartsfield Sam Johnson Sam Kitonyi Samuel Clay Samuel Linde Sander Steffann Sandor Weisz Sau Ramirez Scott Henley Scott Nixon Sean O'Connor Sean Powell Sebastian Brandt Sebastian Desimone Sebastian vetter Sebastian Wain seibert Senko Rašić Sergey Leschina Sergey Silaev Sergey Vilgelm Seth Burke Shawn Milochik Silvan Spross Sil Westerveld Simeon Franklin Simon Bernier simon hayward Simon Hewitt Simon Law Simon Pantzare Simon Willison Skirmantas Jurgaitis Solomon Matthews Sören Schlömer Stefan Foulis Stefano Delli Ponti Stein Magnus Jodal Stephan Huiser Stephen Steve Steve Holden Steve Jalim Sumit Chachraa Sungjoo Ha Sven Grunewaldt Taavi Taijala Takuya Tozawa Taymon Thijs van Dien Thomas Gruner Tijs Teulings Tim Carter Timothy Zwolak TinFoil Tom Christie Tom Dyson Tomek Paczkowski Tommy Gudmundsen Tom Usher Tony Hauber Tony Meyer Trevor Riles TrialReach Tuk Bredsdorff Tunde Ashafa Twidi Type/Code Ulrich Petri Valentin V. Bartenev Victor Allen Vidir Gudmundsson ViKing Ville Alatalo Vincent Driessen Visa Kopu Vitaliy Fuks Vladimir Volodymyr Hotsyk Wade Austin Walker Hamilton Warren Packard Wiley Kestner William Arnold William Hogan Williams Mendez Wil Linssen Yarden Michael Shem-Tov Yury Yurevich zaebee Павел Тявин
13. Why replace South?
14. 5 years of learning Poor VCS branching Huge migration file size Migration sets get too large
15. The Future New! Exciting! 95% less migration noise!
16. django.db.migrations django.db.backends.schema
17. django.db.migrations Migration commands Autodetection Dependencies
18. django.db.backends.schema SQL generation Database abstraction
19. ed = connection.schema_editor() ed.start() ed.create_model(MyModel) ed.add_field(OtherModel, "foo", BooleanField()) ed.commit()
20. PostgreSQL MySQL SQLite Oracle MSSQL Yep! Hopefully! DB2 MongoDB Wait, what?
21. New migration format Finally! Please stop complaining now.
22. Old Format
23. Old Format Migration actions Frozen ORM
24. New Format
25. New Format
26. How? Create Model Page with fields content, date Create Model Author with fields name, age Add title to Page Remove age from Author Migration 1 Migration 2 Migration 3 Model Page with fields title, content, date Model Author with field name
27. Dependencies Better. Faster. Stronger. Not harder.
28. 0001_initial Before 0002_ponies 0003_foo 0003_bar
29. 0001_initial Now 0002_ponies 0003_foo 0003_bar
30. 0001_initial Now 0002_ponies 0003_foo 0003_bar 0004_merge
31. Squashing Saves precious, precious bytes.
32. 0001 0002 0003 0004 0005 0006
33. 0001 0002 0003 0004 0005 0006 0005_alt
34. 0006 0005_alt
35. 0006 0001 0002 0003 0004 0005 0006 0001 0002 0003 0004 0005 APP A APP B
36. 0006 0001 0002 0003 0004 0005 0006 0001 0002 0003 0004 0005 APP A APP B
37. How's it going? Kickstarter is funding so many holidays work days
38. Schema Backends Mostly done, ready for merge
39. Migration code Still going, most complex part
40. Field API changes Incoming. Watch django-developers if interested.
41. The Code https://github.com/andrewgodwin/ django/tree/schema-alteration/
42. Blog Posts aeracode.org/category/django-diaries/
43. Thoughts, feedback and fully- complete migration frameworks to: andrew@aeracode.org
44. Fin. Andrew Godwin Thank you. @andrewgodwin andrew@aeracode.org http://www.aeracode.org