MAGE1 >> MAGE2 NOT EVERYTHING THAT CAN BE COUNTED COUNTS, AND NOT EVERYTHING THAT COUNTS CAN BE COUNTED. ALBERT EINSTEIN, PHYSICIST (16.03.2016) LONDON MAGENTO MEETUP MARCIN SZTERLING REDBOX DIGITAL @php4u

Image: http://fishki.pl/fishka,Syzyfowa-praca,31405.html MARCIN SZTERLING REDBOX DIGITAL @php4u

▸ Data migration is the process of importing legacy data, from one or more old systems, to a new system or systems. ▸ Data migration is typically part of a larger programme ▸ “Big bang” and “trickle” migrations WHAT IS DATA MIGRATION? MARCIN SZTERLING REDBOX DIGITAL @php4u

MIGRATE DATA ▸ Should we bring all or part of the data? ▸ If just part, which parts? - based on creation date? based on status of product, customer? or some combination of criteria? ▸ If we choose to bring over data on or after a certain date, which should it be - last 3 months, last 6 months, last year?...should it be all the data? ▸ Should we clean data at source or destination? BEFORE ALL STARTS MARCIN SZTERLING REDBOX DIGITAL @php4u

CHANGING TIRES AT 60 MPH THIS IS NOT A 1 DAY TASK ▸ CE -> CE, CE -> EE, EE -> EE ▸ There is no silver-bullet solution ▸ There will be issues with source data ▸ There will be issues with migrated data ▸ Amount of data matters ▸ Reserve time for it and prepare for battle! ▸ http://alankent.me/2015/01/31/ magento-1-to-2-migration-tool-plans/ MARCIN SZTERLING REDBOX DIGITAL @php4u

WHAT DATA DO WE NEED? MAIN DATA TO BE MIGRATED (WHAT THE HELL IS EAV??) ▸ Products ▸ Categories (and its products relations) ▸ Customers ▸ Orders ▸ Configurations ▸ CMS elements ▸ Websites, Store groups, Stores ▸ Media assets for Products, Categories, CMS MARCIN SZTERLING REDBOX DIGITAL @php4u

WHAT EXACTLY HAS CHANGED FROM M1 TO M2? ▸ Tables can be deleted ▸ Tables can be renamed ▸ New tables can exist in Magento 2 but not in Magento 1 ▸ Table columns can be deleted ▸ Table columns can be renamed ▸ New table columns can exist in Magento 2 but not in Magento 1 MARCIN SZTERLING REDBOX DIGITAL @php4u

SOME OF THE DATA MUST BE TRANSFORMED ▸ Values must be changed to match Magento 2 new classes namespaces (models) ▸ Values must be nulled even if are empty ▸ Names must be prefixed to avoid naming issues WHAT EXACTLY HAS CHANGED FROM M1 TO M2? MARCIN SZTERLING REDBOX DIGITAL @php4u

DO YOUR HOMEWORK! RIGHT TOOLS TO DO RIGHT JOB MARCIN SZTERLING REDBOX DIGITAL @php4u

WASHING AND IRONING AS WELL?? CRITERIA ▸ Ease of use ▸ Speed ▸ Security ▸ Accuracy ▸ Repeatable ▸ Multi-project MARCIN SZTERLING REDBOX DIGITAL @php4u

IN THE WILD WHAT IS AVAILABLE? ▸ Ubertheme Magento 2 data migration (1.0.4)
 https://github.com/ubertheme/module-ubdatamigration ▸ Magento data migration tool (2.0.2)
 https://github.com/magento/data-migration-tool ▸ CSV based import ▸ Custom own written solution (API?) ▸ https://github.com/SchumacherFM/Magento2-Data- Migration MARCIN SZTERLING REDBOX DIGITAL @php4u

UBERTHEME MIGRATION TOOL MARCIN SZTERLING REDBOX DIGITAL @php4u

JUST WORKS? UBERTHEME MIGRATION TOOL ▸ Yii framework based browser tool ▸ Not secured at all so you must not use it on production unless secured ▸ Almost zero configuration required ▸ Slow and dependant on server settings ▸ Supports delta changes and can be migrated on top of existing data ▸ Does the job! ▸ More info how to use it here
 https://github.com/ubertheme/module-ubdatamigration/blob/master/ README.md MARCIN SZTERLING REDBOX DIGITAL @php4u

SHOW ME SHOW ME… UBERTHEME MIGRATION TOOL MARCIN SZTERLING REDBOX DIGITAL @php4u

MAGENTO DATA MIGRATION TOOL MARCIN SZTERLING REDBOX DIGITAL @php4u

CARTE BLANCHE MAGENTO DATA MIGRATION TOOL ▸ Now one tool (2.0.2) for both EE and CE ▸ Preconfigured for all magento versions and editions ▸ Works with magento sample data (!) ▸ Supports delta changes ▸ Cannot be migrated on top of existing data ▸ Requires “perfect” source data integrity (ex. eav_entity_type) ▸ Blazing fast (with exceptions) ▸ xml based configuration - which can be stored in VCS, can have multiple profiles ▸ No way to specify database connection details in CLI ▸ Very flexible configuration to adjust to needs, easy to extend ▸ Progress bar with estimated time per task (but not for overall task) ▸ No automated data verification after import MARCIN SZTERLING REDBOX DIGITAL @php4u

COMPOSER WHAT? INSTALLATION ▸ http://devdocs.magento.com/guides/v2.0/migration/ migration-tool-install.html ▸ composer config repositories.data-migration-tool git https://github.com/magento/data-migration-tool ▸ composer require magento/data-migration-tool:2.0.2 ▸ Installed in vendor/magento/data-migration ▸ bin/magento migrate:data [-r|--reset] {} MARCIN SZTERLING REDBOX DIGITAL @php4u

WHERE TO START? MAGENTO DATA MIGRATION TOOL ▸ 3 main types of run modes in Migration/Console 
 Data, 
 Delta, 
 Settings ▸ Data is read, transformed and then saved in batches ▸ CPU and memory heavy MARCIN SZTERLING REDBOX DIGITAL @php4u

STEPS Migration\Step\Settings\Integrity Migration\Step\Settings\Data Migration\Step\Stores\Integrity Migration\Step\Stores\Data Migration\Step\Stores\Volume MARCIN SZTERLING REDBOX DIGITAL @php4u

WHERE, FROM AND MORE SETTINGS… etc/ce-to-ee/1.7.0.2/map.xml.dist etc/ce-to-ee/map-eav.xml.dist etc/ce-to-ee/eav-document-groups.xml.dist etc/ce-to-ee/eav-attribute-groups.xml.dist etc/ce-to-ee/map-log.xml.dist etc/ce-to-ee/log-document-groups.xml.dist etc/ce-to-ee/settings.xml.dist etc/ce-to-ee/map-customer.xml.dist etc/ce-to-ee/customer-document-groups.xml.dist etc/ce-to-ee/customer-attribute-groups.xml.dist etc/ce-to-ee/deltalog.xml.dist etc/ce-to-ee/order-grids-document-groups.xml.dist etc/ce-to-ee/map-document-groups.xml.dist etc/ce-to-ee/class-map.xml.dist 0 0 0 migration.log %percent%% [%bar%] Remaining Time: %remaining% 1 http://devdocs.magento.com/guides/v2.0/migration/migration-tool-internal-spec.html MARCIN SZTERLING REDBOX DIGITAL @php4u

WHEN DOCUMENT IS NOT DOCUMENT ANYMORE DEFINITIONS ▸ Source - your old Magento database ▸ Destination - your new Magento 2 database ▸ Document - mysql table ▸ Field - mysql column ▸ https://github.com/magento/data-migration-tool/blob/ master/etc/ce-to-ee/map-eav.xml.dist MARCIN SZTERLING REDBOX DIGITAL @php4u

ETC/CE-TO-EE/MAP-EAV.XML.DIST MARCIN SZTERLING REDBOX DIGITAL @php4u

RUN, NOTE ERRORS, FIX, REPEAT… ERRORS ERRORS ERRORS ▸ [ERROR]: Source fields not mapped. Document: sales_flat_order. Fields: field1, field2 ▸ [ERROR]: Destination fields not mapped. Document: customer_entity. Fields: field1, field2 ▸ Interesting script - http://oyenetwork.com/articles/data-migration- to-magento2-case-study/ ▸ Problem with EAV relationships after migration ▸ Error messages are not detailed enough (Mismatch of entities in the document) MARCIN SZTERLING REDBOX DIGITAL @php4u

LIFE SAVERS! TIPS ▸ Divide and Conquer - prepare config.xml per task group ▸ Try with smaller data samples - it is easier to debug it ▸ Use -vvv for extended debugging ▸ Use -r to restart process ▸ Source and Destination resource classes are disabling foreign keys check ▸ Password rehash is performance killer ▸ Create your own steps, create your own handlers ▸ Some of the attributes might be required in M2 and are not in M1 (telephone) MARCIN SZTERLING REDBOX DIGITAL @php4u

LIFE SAVERS! TIPS ▸ Data migration should be your first task - copy data on top of clean Magento 2 installation ▸ Ideally try to have two databases on same server ▸ Magento 1 instance can be live during migration ▸ Docs - http://devdocs.magento.com/guides/v2.0/migration/bk-migration- guide.html ▸ Logging can be extended ▸ The versions of the Data Migration Tool and your Magento software must exactly match. For example, if you’re using version 2.0.2 of the Magento software, you must use version 2.0.2 of the Data Migration Tool. MARCIN SZTERLING REDBOX DIGITAL @php4u

LIFE SAVERS! TIPS ▸ Use scripting to have repeatable process ▸ Do many mysql dumps in the process, so data can be reverted to previous state ▸ DROP DATABASE …, CREATE DATABASE … ▸ Keep your files separate from core data-migration files MARCIN SZTERLING REDBOX DIGITAL @php4u

HOW QUICK IS QUICK? RESULTS ▸ 20k customers - minutes ▸ 300k products - 1 hour MARCIN SZTERLING REDBOX DIGITAL @php4u

HOPE YOU LIKED IT! THANKS! @php4u Image: http://justsomething.co/ MARCIN SZTERLING REDBOX DIGITAL