Migrating The Future

Migrating The Future

A talk I gave at DjangoCon EU 2013 about South, Django, and the future of migrations in Django core.

077e9a0cb34fa3eba2699240c9509717?s=128

Andrew Godwin

May 15, 2013
Tweet

Transcript

  1. Andrew Godwin @andrewgodwin Migrating THE Future

  2. Andrew Godwin Django core developer South author Database Wrangler

  3. South

  4. South django.db.migrations

  5. South django.db.migrations Kickstarter

  6. £2,500

  7. £2,500 Raised!

  8. £2,500 Raised! 65 minutes in.

  9. £17,952 From 502 backers

  10. None
  11. Tom Christie Alex Gaynor Pixtape.com Ulrich Petri (@ulope) Sam Wilson

    Dan Fairs Nyaruka Ltd Jorge Bastida Aaron Bassett Stefan Foulis Aymeric Augustin Rich Leland Ray Schamp CookDrop Inc. fitclipboard.com Type/Code Bruno Renié Bryan Veloso Smashing Ideas John P. Neumann Filipe Regadas Simon Law (sfllaw@sfllaw.ca) Ryan Kaskel CYeong Baptiste Mispelon Eric Florenzano Albert O'Connor, Web Developer Jeremy Dunck Daniel Lindsley Dan Cundiff Automatique another one GmbH Vincent Driessen (@nvie) Daniel Greenfeld Joey Wilhelm Marek Stępniowski Sil Westerveld (SURFsara) http://Scal.io RMSA - the makers of Fresco Nitty Gritty Software, LLC Greg Aker Christopher Dickes SJM Steffann Messiah Studio (New Zealand) Dirty Monkey Łukasz Langa (ambv) Dougal Matthews Joel Crocker
  12. Aaron Bassett Aaron Ho Aaron Larner Aaron Malone Adam Connor

    Adam Fast Adam Foster Adam Gross Adam Joseph Cook Adam Rich Adrian Espinosa Adrian Rego Adrian Smith Agostino Albert O'Connor Alec Perkins Ales Alexander Rudakov Alexandre Lessard Alexey Sveshnikov Alex Gaynor Alex Hung Alex Louden Alex Tziallas Alex Yuzhakov Alfredo Aguirre Ames Cornish Anatoly Popov Andreas Louca Andreas Madsack Andreas Rieger André Simões Andrew Austin Andrew Dunn Andrew Gorcester Andrew Jesaitis Andrzej Voss Andy Baker Andy Dirnberger Angel Freire Anselm Lingnau Arild Martin Møller Haugstad Arnaud Art van Scheppingen Athanassios Diacakis August Bigelow Aymeric Augustin Backer Name Backer Name Backer Name Backer Name Backer Name Backer Name Backer Name Backer Name Baldur Þór Emilsson Baptiste Millou Baptiste Mispelon Barbara Shaurette Bayo opadeyi Ben Ben Firshman Ben Graham Benjamin Wohlwend Ben Lopatin Ben Spaulding BeProud Bernhard Janetzki bitprophet Bob Barcklay Bob Haugen Bob Lozano Bolster Boris Chervenkov Bouke Haarsma Bozidar Benko Brad Montgomery Brad Wright Brandon Bennett Brandon Konkle Brett Epps Brian O'Dell Brian Reigle Brian Rosner Brian Wisti Bruno Renié Bruno Tikami Bryan Koroleski Bryan Veloso Carl Meyer Carlos Ribeiro Carny Cheng Casper S. Jensen Chad Catlett Chris Adams Chris Beaven Chris Drackett Chris Johnston Chris Kleinknecht Chris Streeter Christian Bertschy Christian Lange Christian Toivola Christian Treutler Christophe Pettus Christopher Dickes Christopher Grebs Christopher J. Hasenpflug Christopher Reeves Christoph Heer Chris Vigelius "Chris ""Wraithan"" McDonald" Cícero Raupp Rolim Claes Ström Colin Woods Cuauhtemoc Ramirez Damon Ciarelli Dan Cundiff DanEE Dan Fairs Daniel Gagnon Daniel Greenfeld Daniel Lindsley Daniel Perry Daniel Watkins Danilo Dan Koch Dan Langer Danny Roa Dan Tyan Darian Moody Dave Arter Dave Masse David David Cramer David Feinzeig David Grant David Jeffrey David Novák David Sinsky Denis Darii Dennis Seymour Derek Leverenz Deric Crago Devin Hunt Dj Gilcrease Dmitriy Narkevich Dong Hee Kim Dougal Matthews Doug Tabacco Dustin Davis Edgars Jēkabsons Elliot Murphy Enrico Berti Ensorceled Eric Eric Florenzano Ernesto Rico-Schmidt Esa Evan Culver Ezequiel Pochiero Federico Giordano Felipe Coury Felipe Kellermann Felix Yuan Fernando Gutierrez Fernando Rocha Filipe Regadas Filipp Lepalaan Florian Apolloner fm 10.4 Frederik De Bleser Fusionbox Gabriel Gamil Gabriel Grant g chow George Hickman Gert Van Gool Giles Thomas Giovanni Totaro Graham Ullrich Greg Aker Guilherme Gondim Håkon Erichsen Hans Lawrenz harijay Herbert Poul Horst Gutmann Hrishikesh Ballal Hugo Rodger-Brown Hugo Wang Ian Ildar Samit Ivan Habunek Iván Raskovsky Ivan Yurchenko Ivo Teel Jacek Jacob Briggs Jacob Kaplan-Moss Jaime Irurzun Jakub Wiśniowski James Cleveland James Johnson James Robert James Tiplady James Walker Jamie Wilson Jannis Leidel Jason Jason Cartwright Jason Mayfield Jason Simeone Jasper den Hertog Javed Khan Jeff jens quade Jérémie Blaser Jeremy Cantrell Jeremy Dunck Jeremy Keeshin Jesse Michelsen Jesus Noland Jim Ray Jirka Vejrazka JMatt Peterson João Oliveira Joel Crocker Joel Shapiro Joel Squire Joey Castillo Joey Leingang Joey Wilhelm John Paulett John P. Neumann John Sykora John Wegis Jonas Nockert Jonathan Biddle Jonathan Morgan Jon Danao Jon Fedyk Joonathan Mägi Jorge Bastida Jorge Camargo Jörg Kreß Jose Padilla Joseph Wenninger Josh Ourisman Josh Steiner Joshua Blount Joshua Johanan Josh Wright Julen Ruiz Aizpuru Julian Wachholz Julio Oña Julius Welby Jure Ham Kalle Karjalainen Katie Cunningham Katsumi Honda Keith Gray Kenneth Love Kenneth Reitz Kirill Sibirev Kiwi Koh Chuan Yeong kortina Kura Kuratko Lachlan Musicman Larry Hamel Larry Osborn Lars Wikman Laust M. Ladefoged Leo Leo Hourvitz Leonidas Tsementzis Leon Matthews Leon Smith Le Thanh Duong Logit internet services Ltd. Loic Bistuer Lucas Eisenzimmer Łukasz Langa Luke Hatcher Maciej Konieczny Maciek Gryka Marc Egli Marcelo Alves Marcel Rudolph Marco Bonetti Marc Tamlyn Marek Olszewski Marek Stępniowski Margus Holland Marianne Marielle Volz Mark Hepburn Mark Hurley Mark Longair Mark Lyon Mark Ng Mark Norman Francis Marko Samastur Markus Zapke-Gründemann marteinn Martin Brochhaus Mathieu Fenniak Mathieu Meylan Mathieu Richardoz Matt Austin Matthew Tretter Mattias Linnap Matt Lenc Matt McGlincy Matt Stevens Matt Woodward Matus Moravcik Mauro Doglio Max Arnold Meir Kriheli Michael Blatherwick Michael Fogel Michael Gisi Michael Sandford Michael Twomey Miguel Pilar Mihael Konjević Mike Gallagher MIKe O'Toole Mirus Research Mitch Berkson Mladen Kovacevic moberley Nate Graves Nate Meyvis Nate Thelen Nephila Nick Albright Nick Siemsen Nicolas Perriault Nicolas Pottier Nicole Foster niels Niels Sandholt Busch Nikolai Prokoschenko Noah Kantrowitz Noemi notanumber Numan Sachwani numbsafari Nurzhan Bakibayev Ola Sitarska Oliver Beattie Oliver George Oliver Steele Omen Ondrej Slintak ORDOQUY Xavier Ori Oskar Jakieła Ousmane Wilane Parham Pascal Hartig Pathwright Press Patrick Arminio Patrick Craston Patrick Taylor Paul Bersch Paul Ebreo Paul Mattal Paul Medlock-Walton Peppe Bergqvist Peter Baumgartner + Lincoln Loop Peter Evans Peter Farrell Peter Hyben Peter Inglesby Peter Kropf Peter Sanchez Petr Marhoun Petr Timofeev Philip Kimmey Philipp Bosch Philipp Kruse Philipp Läubli Philipp Weidenhiller Philip Roche Pierre-Julien Grizel pmdhazy Preston Holmes Promeritum quinox Raj Jalan Ralph Ramji Srinivasan Ray Remco Wendt Renato Garcia Pedigoni Renato Oliveira riccardo magliocchetti Richard Beales Rich Leland Rinat Akhtamov Rishabh Manocha Rivo Laks RJ Walsh Rob Berwick Robert Medeiros Roberto Rosario Rod Afshar Rodrigo Chacon Rogelio Rolo Mawlabaux Ross Crawford-d'Heureuse Ross Duggan Russell Keith-Magee Ryan Kaskel Ryan McDevitt Ryan van Roode Sævar Öfjörð Magnússon Sahas Katta Sam Bull Sam Hartsfield Sam Johnson Sam Kitonyi Samuel Clay Samuel Linde Sander Steffann Sandor Weisz Sau Ramirez Scott Henley Scott Nixon Sean O'Connor Sean Powell Sebastian Brandt Sebastian Desimone Sebastian vetter Sebastian Wain seibert Senko Rašić Sergey Leschina Sergey Silaev Sergey Vilgelm Seth Burke Shawn Milochik Silvan Spross Sil Westerveld Simeon Franklin Simon Bernier simon hayward Simon Hewitt Simon Law Simon Pantzare Simon Willison Skirmantas Jurgaitis Solomon Matthews Sören Schlömer Stefan Foulis Stefano Delli Ponti Stein Magnus Jodal Stephan Huiser Stephen Steve Steve Holden Steve Jalim Sumit Chachraa Sungjoo Ha Sven Grunewaldt Taavi Taijala Takuya Tozawa Taymon Thijs van Dien Thomas Gruner Tijs Teulings Tim Carter Timothy Zwolak TinFoil Tom Christie Tom Dyson Tomek Paczkowski Tommy Gudmundsen Tom Usher Tony Hauber Tony Meyer Trevor Riles TrialReach Tuk Bredsdorff Tunde Ashafa Twidi Type/Code Ulrich Petri Valentin V. Bartenev Victor Allen Vidir Gudmundsson ViKing Ville Alatalo Vincent Driessen Visa Kopu Vitaliy Fuks Vladimir Volodymyr Hotsyk Wade Austin Walker Hamilton Warren Packard Wiley Kestner William Arnold William Hogan Williams Mendez Wil Linssen Yarden Michael Shem-Tov Yury Yurevich zaebee Павел Тявин
  13. Why replace South?

  14. 5 years of learning Poor VCS branching Huge migration file

    size Migration sets get too large
  15. The Future New! Exciting! 95% less migration noise!

  16. django.db.migrations django.db.backends.schema

  17. django.db.migrations Migration commands Autodetection Dependencies

  18. django.db.backends.schema SQL generation Database abstraction

  19. ed = connection.schema_editor() ed.start() ed.create_model(MyModel) ed.add_field(OtherModel, "foo", BooleanField()) ed.commit()

  20. PostgreSQL MySQL SQLite Oracle MSSQL Yep! Hopefully! DB2 MongoDB Wait,

    what?
  21. New migration format Finally! Please stop complaining now.

  22. Old Format

  23. Old Format Migration actions Frozen ORM

  24. New Format

  25. New Format

  26. How? Create Model Page with fields content, date Create Model

    Author with fields name, age Add title to Page Remove age from Author Migration 1 Migration 2 Migration 3 Model Page with fields title, content, date Model Author with field name
  27. Dependencies Better. Faster. Stronger. Not harder.

  28. 0001_initial Before 0002_ponies 0003_foo 0003_bar

  29. 0001_initial Now 0002_ponies 0003_foo 0003_bar

  30. 0001_initial Now 0002_ponies 0003_foo 0003_bar 0004_merge

  31. Squashing Saves precious, precious bytes.

  32. 0001 0002 0003 0004 0005 0006

  33. 0001 0002 0003 0004 0005 0006 0005_alt

  34. 0006 0005_alt

  35. 0006 0001 0002 0003 0004 0005 0006 0001 0002 0003

    0004 0005 APP A APP B
  36. 0006 0001 0002 0003 0004 0005 0006 0001 0002 0003

    0004 0005 APP A APP B
  37. How's it going? Kickstarter is funding so many holidays work

    days
  38. Schema Backends Mostly done, ready for merge

  39. Migration code Still going, most complex part

  40. Field API changes Incoming. Watch django-developers if interested.

  41. The Code https://github.com/andrewgodwin/ django/tree/schema-alteration/

  42. Blog Posts aeracode.org/category/django-diaries/

  43. Thoughts, feedback and fully- complete migration frameworks to: andrew@aeracode.org

  44. Fin. Andrew Godwin Thank you. @andrewgodwin andrew@aeracode.org http://www.aeracode.org