Data migration to Drupal using the migrate module

Data migration is the concept of moving the data from a platform A to a platform B. Often, in between, you want to transform the data.

Migrate module is a great tool to migrate data into native Drupal data.

My talk from the Drupalcamp Kyoto in Japan, on April 2014.

Transcript

1. Data Migration with Drupal using Migrate Drupal を使ったデータ移行について Data Migration with Drupal using Migrate, Drupalcamp Kyoto. 12th April 2014 - Luc Bézier, @Luukyb Drupalcamp Kyoto, Japan   h#ps://www.flickr.com/photos/lifes26/4509331619/  
2. Who am I Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb
3. Who am I Freelance web, exclusively Drupal projects since ~ 4 years. I moved to Cebu City, Philippines ~ 1 year. Twitter @Luukyb luc@webizat.com Luukyb on drupal.org Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb
4. Who am I Contributed to several large data migration to drupal using migrate module. Oracle to drupal : BBC Good Food UK public sector organisation Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb
5. h#p://www.flickr.com/photos/larahsphotography/2795859728/   1.  Introduction to data migration 2.  Understanding the migrate module 3.  Organise your Data migration with Migrate 4.  Migrate in Drupal 8 Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb
6. Data migration Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb h#ps://www.flickr.com/photos/usace_albuquerque/5547589951/  
7. Data migration Moving data from A to B (and often, in the middle you transform the data.) Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb h#ps://www.flickr.com/photos/usace_albuquerque/5547589951/  
8. Data migration Simple migration? Feeds module Data mapping using UI. drupal.org/project/feeds Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb h#ps://www.flickr.com/photos/usace_albuquerque/5547589951/  
9. Data migration Use migrate module for a complex migration. Complex source or destination. Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb h#ps://www.flickr.com/photos/usace_albuquerque/5547589951/  
10. Data migration Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb h#ps://www.flickr.com/photos/kieranlane/6635502419/  
11. Data migration Using the migrate module can require code, coffee and patience. Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb h#ps://www.flickr.com/photos/kieranlane/6635502419/  
12. Data migration Good news: Migrate module is very flexible, powerful and well documented. Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb h#ps://www.flickr.com/photos/usace_albuquerque/5547589951/  
13. Introduction to Migrate module h#p://www.flickr.com/photos/zigazou76/9104028602   Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb
14. Introduction to Migrate module h#p://www.flickr.com/photos/zigazou76/9104028602   Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb Drupal module Install and enable like any other contrib module drupal.org/project/migrate
15. Introduction to Migrate module h#p://www.flickr.com/photos/zigazou76/9104028602   Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb Created by Mike Ryan. drupal.org/user/4420 Senior Technical Consultant at Acquia
16. Introduction to Migrate module h#p://www.flickr.com/photos/zigazou76/9104028602   Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb “framework for migrating content into Drupal from other sources”
17. Introduction to Migrate module h#p://www.flickr.com/photos/zigazou76/9104028602   Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb What’s inside ? -  OOP Architecture -  An API -  Some drush commands -  Migrate UI
18. Introduction to Migrate module h#p://www.flickr.com/photos/zigazou76/9104028602   Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb Built-in support of core items (Nodes, Users, Files, Taxonomy and comments)
19. Introduction to Migrate module h#p://www.flickr.com/photos/zigazou76/9104028602   Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb Built-in support for PDO, XML, CSV, JSON, and native MSSQL and Oracle API sources
20. Introduction to Migrate module h#p://www.flickr.com/photos/zigazou76/9104028602   Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb You can extend all sources, destinations.
21. Introduction to Migrate module h#p://www.flickr.com/photos/zigazou76/9104028602   Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb Migration: -  100% native drupal -  iterative
22. Introduction to Migrate module h#p://www.flickr.com/photos/zigazou76/9104028602   Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb Related modules: -  Drupal-to-Drupal data migration (migrate_d2d) -  WordPress Migrate -  TYPO3_migrate
23. Understanding Migrate module h#p://www.flickr.com/photos/zigazou76/9104028602   Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb
24. Understanding Migrate module h#p://www.flickr.com/photos/zigazou76/9104028602   Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb Your migration classes extends the abstract class Migration
25. Understanding Migrate module h#p://www.flickr.com/photos/zigazou76/9104028602   Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb Your source. $this->source = new MigrateSourceSQL($query);
26. Understanding Migrate module h#p://www.flickr.com/photos/zigazou76/9104028602   Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb Your destination. $this->destination = new MigrateDestinationNode(’article');
27. Understanding Migrate module h#p://www.flickr.com/photos/zigazou76/9104028602   Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb Your mapping. $this->addFieldMapping ('field_summary', 'summary');
28. Understanding Migrate module h#p://www.flickr.com/photos/zigazou76/9104028602   Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb Transform your data. $this->addFieldMapping ('field_summary', 'summary’) ->callbacks(array($this, ’summaryFormat'));
29. Understanding Migrate module h#p://www.flickr.com/photos/zigazou76/9104028602   Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb Extra transformation and processing can be done using methods like: function prepareRow($row)
30. Understanding Migrate module h#p://www.flickr.com/photos/zigazou76/9104028602   Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb Some cool features ! Source dependency definition. $this->addFieldMapping('field_image', ’imagepath') ->sourceMigration(’MediaImages');
31. Understanding Migrate module h#p://www.flickr.com/photos/zigazou76/9104028602   Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb Easy to rollback the data. When you run a migration, migrate module stores the keys.
32. Understanding Migrate module h#p://www.flickr.com/photos/zigazou76/9104028602   Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb If you import into a contributed module: 2 cases.
33. Understanding Migrate module h#p://www.flickr.com/photos/zigazou76/9104028602   Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb First case, my contrib module does not support migrate.
34. Understanding Migrate module h#p://www.flickr.com/photos/zigazou76/9104028602   Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb Example: Media module. Migrating content for the media module requires migrate_extras.
35. Understanding Migrate module h#p://www.flickr.com/photos/zigazou76/9104028602   Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb Migrate extras module implements extra support. drupal.org/project/migrate_extras
36. Understanding Migrate module h#p://www.flickr.com/photos/zigazou76/9104028602   Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb Module maintainers are now recommended to implement migration directly in the modules.
37. Understanding Migrate module h#p://www.flickr.com/photos/zigazou76/9104028602   Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb So second case, my contrib module implements migrate.
38. Understanding Migrate module h#p://www.flickr.com/photos/zigazou76/9104028602   Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb Example: field collection. Comes with a file dedicated to migrate: field_collection.migrate.inc
39. Understanding Migrate module h#p://www.flickr.com/photos/zigazou76/9104028602   Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb Use drush ! Limit the number of items to import, Rollback the data, Audit your migration, …
40. Understanding Migrate module h#p://www.flickr.com/photos/zigazou76/9104028602   Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb
41. Organise your Data migration Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb h#ps://www.flickr.com/photos/usace_albuquerque/5547589951/  
42. Organise your Data migration Study the source data, keep track of the changes. Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb h#ps://www.flickr.com/photos/usace_albuquerque/5547589951/  
43. Organise your Data migration Use a separate migration environment. Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb h#ps://www.flickr.com/photos/usace_albuquerque/5547589951/  
44. Organise your Data migration Use a script to build this environment from scratch. Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb h#ps://www.flickr.com/photos/usace_albuquerque/5547589951/  
45. Organise your Data migration Install your site with a profile $ drush si profile Migrate the data $ drush mi MyMigration Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb h#ps://www.flickr.com/photos/usace_albuquerque/5547589951/  
46. Migrate to D8 Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb h#ps://www.flickr.com/photos/michaeljohn/306759922/  
47. Migrate to D8 Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb h#ps://www.flickr.com/photos/michaeljohn/306759922/   Drupal 8 will integrate a Migration API to migrate your site from Drupal 6 and Drupal 7 to Drupal 8.
48. Migrate to D8 Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb h#ps://www.flickr.com/photos/michaeljohn/306759922/   This API will be based on migrate module. (currently still in progress)
49. Migrate to D8 Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb h#ps://www.flickr.com/photos/michaeljohn/306759922/   Migration API in D8: drupal.org/node/2127611 Migration modules list : drupal.org/documentation/migrate
50. Questions? Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb h#ps://www.flickr.com/photos/michaeljohn/306759922/  
51. Coming camps in PHL Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb h#ps://www.flickr.com/photos/michaeljohn/306759922/   Drupalcamp Manila : 26th April Drupalcamp Cebu : October 2014 (TBC)
52. Thank you ! Data Migration with Drupal using Migrate, Drupalcamp Kyoto. Luc Bézier, @Luukyb h#ps://www.flickr.com/photos/michaeljohn/306759922/   ありがとう luc@webizat.com @luukyb