Drupal Migration API By Spreadsheet

The presentation taken by Jayesh Makwana in the second Drupal meetup on 19 Aug 2023
https://www.meetup.com/ahmedabad-drupal-meetup/events/294822150/

Transcript

1. Migration API by Spreadsheet/CSV PRESENTED BY Jayesh Makwana
2. Agenda • Why Drupal 7 is still running and D7 needs migration? • Ways Of Migration in Drupal • Introduction of Drupal Migration API • Key Concepts in Drupal Migration • Migration Plugins & Handlers • Required Modules • Use case, Migration commands & Debugging • Best Practices, Tips & Benefits of Migration API 2
3. Why Drupal 7 is still running? • Migration Complexity (For large and huge websites) • Less availability of modules and themes • Budget issue • Feel Scared of Consistency • No awareness about Drupal 9/10 features. 3
4. Why does D7 website need migration? • Security: Drupal 7 no longer receives support or security updates after deadline. It increases the risk in your site. • Enhanced Features: Upgrading to a newer version of Drupal improves performance, scalability, and user experience. • Easy Upgrade: Easy to upgrade for next few years. • Future Approach: Take a step into Headless environment. 4
5. Ways Of Migration • Custom SQL Queries (Not recommended) • Core migration process • Migration API 5
7. Introduction of Drupal Migration API • The Migration API provides services for migrating data from a source system (OLD Drupal version/other CMS) to Drupal New version (D9/D10). • It uses ETL process. Extract-Transform-Load 7
8. Key Concepts in Drupal Migration API •Source and Destination Systems •Migration Plugins and Configuration 8
9. Migration Plugins • Types of Migration Plugins 9 Source Process Destination
10. Source Plugins • It defines to source of the data and helps to fetch the data from source system. • It supports, sourceURL, sourceDatabase, sourceCSV. 10
11. Process Plugins • It handles the transformation of data during migration process. • For Ex, skip_on_emtpy, explode, callback, entity_lookup, etc.. 11
12. Destination Plugins • It gives the target location where data will be stored within Drupal. • For Ex, destinationEntity, destinationFile, destinationConfig, etc.. 12
13. Process Diagram 13
14. Migration Handlers • Migration Groups • Migration Dependency • ID Mapping • Migration Runners 14
15. Required modules • Core Modules • Migrate (Core) • Migrate Drupal UI (Optional) • Contrib Modules • Migrate Plus • Migrate Source CSV/Migrate Spreadsheet • View Data Export 15 Additional Tools: drush
16. Export the Data from Drupal 7 16
17. Excel Sheet • Save this file in project folder or suitable folder in a system. • Use the prefix for primary key to avoid the duplication. • Mention Worksheet name 17
18. Create A YAML 18
19. Save and Import YAML File • Save this YAML file in your config folder. (where we store the configuration files. Default path `sites/default/files/config/sync/`) • Then import this file configuration by using `drush cim`. • Check the migration status: `drush ms` 19
20. Migration Status (drush ms) 20
21. Migration Import (drush mim) 21
22. Migration Rollback (drush mr) 22
23. Running Migration and Debugging • Commands: ◦ Migration status: drush ms ◦ Migration import: drush mim migration_id ◦ Migration rollback: drush mr migration_id ◦ Migration state: drush mrs migration_id ◦ Multiple migration import: drush mim migration_id1,migration_id2 ◦ Migration message: drush mmsg • Debugging ◦ Custom plugins ◦ Error Logs ▪ Drush mmsg ▪ migration _map table in database 23
24. Best Practice and Tips • Check the Source File Mapping • Try to start migrating with single field, and then one by one • Use --limit=1 • Use custom plugins for debugging • Use migrate_map for checking failed status • Remove unnecessary code 24
25. Benefits of Migration API • Flexible • Extensibility • Update existing Entity • Easy to target single entity type • Controllable process • Easy to rollback • Less number of code (in ideal case) 25
26. 26 After finishing the migration
27. 27 “Everyone needs a new home, even DATA too. ”
28. Questions? 28
29. Useful References • Source Plugin List: https://www.drupal.org/docs/8/api/migrate-api/migrate-source-plugins • Process Plugin List: https://www.drupal.org/docs/8/api/migrate-api/migrate-process-plugi ns/list-of-core-migrate-process-plugins • Destination Plugin List: https://www.drupal.org/docs/drupal-apis/migrate-api/migrate-destina tion-plugins-examples 29