Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Migration, everything I don't know about Migration but probably should

Migration, everything I don't know about Migration but probably should

mikebell

May 31, 2014
Tweet

More Decks by mikebell

Other Decks in Technology

Transcript

  1. Migration
    Everything I don't know about
    Migration but probably should*
    0

    View full-size slide

  2. Who?
    Name - Mike Bell
    Twitter -
    Drupal.org -
    Github -
    Where - CTI Digital - Drupal Technical Architect
    @mikebell_
    mikebell_
    mikebell

    View full-size slide

  3. In the beginning
    Migration options were pretty slim
    Hand crafted sql scripts
    Some bash thrown in for good measure

    View full-size slide

  4. Then came the light
    Migrate Module
    Drupal 6 & 7
    Staple for Drupal to Drupal
    ... and everything else
    Good foundation to start with

    View full-size slide

  5. Migrate Features in D7/6
    Migration framework
    - Source and Destination Plugins
    - Nice UI

    View full-size slide

  6. Migrate D2D
    Contrib module
    Drupal to Drupal
    Common field destinations for contrib modules
    Provides a lot of the heavy lifting for you

    View full-size slide

  7. A small (/big) Caveat
    All subject to change
    A small Apology
    I'm not an expert on D8 Migrate
    Plenty of Drupal 7 migrations though
    ... which helps a lot.

    View full-size slide

  8. Say hello to Migrate in
    Drupal 8 Core
    D6 to D8 already done
    D7 to D8 on it's way
    Base sources are in, MySQL, CSV etc

    View full-size slide

  9. Why include Migrate in core?
    The New Upgrade Path
    - Reliable and easy way of updating
    - A major selling point
    - A new skill set for developers

    View full-size slide

  10. Why Migrate instead of
    Updating(.php)?
    No hit or miss update.php runs
    Did I mention it's repeatable?
    It's repeatable
    Easier to test by taking smaller chunks first (and the above bit)

    View full-size slide

  11. How will it help upgrading?
    End Users - not much - yet...
    - Migrate UI? Hopefully in contrib
    Developers - xYES
    - Re-usable components
    - Predefined destinations and sources

    View full-size slide

  12. How will it help
    companies?
    Easily port third party cmss to Drupal
    Creates a skill set pool
    - The more people the better
    - Major selling point in the commercial process

    View full-size slide

  13. How will it help you?
    Migration is cool and/or fun
    May cause insanity

    View full-size slide

  14. Migrate 8 vs 7
    Plugins/Yaml everywhere!
    D8 architecture
    Builds on existing proven features of Migrate 7

    View full-size slide

  15. What you get
    Migrate - core files, general api, plugin manager
    Migrate Drupal - d2d, generic migrations for D6 (soon D7)

    View full-size slide

  16. Down to the details
    Migration is split into:
    - Source
    - Process
    - Destination

    View full-size slide

  17. Things you need to know
    PSR4
    Yaml

    View full-size slide

  18. PSR4 Is here to stay
    Bye bye PSR-0
    It just makes sense
    PSR-4 example -

    View full-size slide

  19. Fully Qualified Class Name Namespace
    Prefix
    Base
    Directory
    Resulting File Path
    \Acme\Log\Writer\File_Writer Acme\Log\Writer ./acme-
    log-
    writer/src/
    ./acme-log-
    writer/src/File_Writer.ph

    View full-size slide

  20. No More
    ./lib/src/Vendor/Badger/ClassName/Lib/Vendor/BadgerClass*
    *May not be PSR-0

    View full-size slide

  21. Config Files
    Much of the configuration has been moved into yaml files
    Easier to edit and read
    Harder to debug, spelling mistakes can be a pain

    View full-size slide

  22. Yaml and You
    What is it?
    - Data orientated
    - Human readable
    Why do you need it?
    - Drupal 8 config is Yaml
    - Widely adopted across many different languages

    View full-size slide

  23. Source Plugins
    Defines source rows
    Migrate Drupal comes with D6 SQL sources
    Lots of common one

    View full-size slide

  24. Process Plugins
    Uses plugins
    Different functionality via small utilities

    View full-size slide

  25. Destination Plugins
    Entities
    - User
    - Nodes

    View full-size slide

  26. More on Migrate
    Process Plugins
    Additional helpers to process data before importing
    Some useful examples:
    - concat - concatination plugin
    - dedupe_entity - de-duplicates entities

    View full-size slide

  27. Useful Process Examples
    Getting/Setting nested Values
    Setting default values
    Extracting values from multilevel arrays

    View full-size slide

  28. Process: Setting/Getting
    Nested Values
    If you want to set $destination['display_settings']['label']['format'] or read from
    $source['display_settings']['label']['format'] you need to use
    display_settings:label:format. Example:
    p
    r
    o
    c
    e
    s
    s
    :
    s
    o
    u
    r
    c
    e
    : '
    d
    i
    s
    p
    l
    a
    y
    _
    s
    e
    t
    t
    i
    n
    g
    s
    :
    l
    a
    b
    e
    l
    :
    f
    o
    r
    m
    a
    t
    '

    View full-size slide

  29. Process: Setting
    default_value
    p
    r
    o
    c
    e
    s
    s
    :
    u
    i
    d
    :
    -
    p
    l
    u
    g
    i
    n
    : m
    i
    g
    r
    a
    t
    i
    o
    n
    i
    d
    : u
    s
    e
    r
    s
    s
    o
    u
    r
    c
    e
    : a
    u
    t
    h
    o
    r
    -
    p
    l
    u
    g
    i
    n
    : d
    e
    f
    a
    u
    l
    t
    _
    v
    a
    l
    u
    e
    d
    e
    f
    a
    u
    l
    t
    _
    v
    a
    l
    u
    e
    : 4
    4

    View full-size slide

  30. Process: Extracting values
    p
    r
    o
    c
    e
    s
    s
    :
    n
    e
    w
    _
    t
    e
    x
    t
    _
    f
    i
    e
    l
    d
    :
    p
    l
    u
    g
    i
    n
    : e
    x
    t
    r
    a
    c
    t
    s
    o
    u
    r
    c
    e
    : s
    o
    m
    e
    _
    t
    e
    x
    t
    _
    f
    i
    e
    l
    d
    i
    n
    d
    e
    x
    :
    - u
    n
    d
    - 0
    - v
    a
    l
    u
    e
    $destination['new_text_field'] = $source['some_text_field']['und'][0]['value']

    View full-size slide

  31. User Migration Example Pt1
    i
    d
    : m
    i
    g
    r
    a
    t
    e
    _
    e
    x
    a
    m
    p
    l
    e
    _
    p
    e
    o
    p
    l
    e
    s
    o
    u
    r
    c
    e
    :
    p
    l
    u
    g
    i
    n
    : c
    u
    s
    t
    o
    m
    _
    u
    s
    e
    r
    d
    e
    s
    t
    i
    n
    a
    t
    i
    o
    n
    :
    p
    l
    u
    g
    i
    n
    : e
    n
    t
    i
    t
    y
    :
    u
    s
    e
    r
    m
    d
    5
    _
    p
    a
    s
    s
    w
    o
    r
    d
    s
    : t
    r
    u
    e

    View full-size slide

  32. User Migration Example Pt2
    p
    r
    o
    c
    e
    s
    s
    :
    n
    a
    m
    e
    :
    -
    p
    l
    u
    g
    i
    n
    : c
    o
    n
    c
    a
    t
    d
    e
    l
    i
    m
    i
    t
    e
    r
    : .
    s
    o
    u
    r
    c
    e
    :
    - f
    i
    r
    s
    t
    _
    n
    a
    m
    e
    - l
    a
    s
    t
    _
    n
    a
    m
    e
    -
    p
    l
    u
    g
    i
    n
    : c
    a
    l
    l
    b
    a
    c
    k
    c
    a
    l
    l
    a
    b
    l
    e
    :
    - '
    \
    D
    r
    u
    p
    a
    l
    \
    C
    o
    m
    p
    o
    n
    e
    n
    t
    \
    U
    t
    i
    l
    i
    t
    y
    \
    U
    n
    i
    c
    o
    d
    e
    '
    - s
    t
    r
    t
    o
    l
    o
    w
    e
    r

    View full-size slide

  33. User Migration Example Pt3
    -
    p
    l
    u
    g
    i
    n
    : c
    a
    l
    l
    b
    a
    c
    k
    c
    a
    l
    l
    a
    b
    l
    e
    : t
    r
    i
    m
    -
    p
    l
    u
    g
    i
    n
    : d
    e
    d
    u
    p
    e
    _
    e
    n
    t
    i
    t
    y
    e
    n
    t
    i
    t
    y
    _
    t
    y
    p
    e
    : u
    s
    e
    r
    f
    i
    e
    l
    d
    : n
    a
    m
    e
    m
    a
    i
    l
    : e
    m
    a
    i
    l
    p
    a
    s
    s
    : p
    a
    s
    s
    r
    o
    l
    e
    s
    :
    -
    p
    l
    u
    g
    i
    n
    : e
    x
    p
    l
    o
    d
    e
    d
    e
    l
    i
    m
    i
    t
    e
    r
    : '
    ;
    '
    s
    o
    u
    r
    c
    e
    : g
    r
    o
    u
    p
    s

    View full-size slide

  34. Defining the source Pt1
    /
    *
    *
    * D
    r
    u
    p
    a
    l 6 u
    s
    e
    r s
    o
    u
    r
    c
    e f
    r
    o
    m d
    a
    t
    a
    b
    a
    s
    e
    .
    *
    * @
    M
    i
    g
    r
    a
    t
    e
    S
    o
    u
    r
    c
    e
    (
    * i
    d = "
    c
    u
    s
    t
    o
    m
    _
    u
    s
    e
    r
    "
    * )
    *
    /
    c
    l
    a
    s
    s U
    s
    e
    r e
    x
    t
    e
    n
    d
    s D
    r
    u
    p
    a
    l
    S
    q
    l
    B
    a
    s
    e i
    m
    p
    l
    e
    m
    e
    n
    t
    s S
    o
    u
    r
    c
    e
    E
    n
    t
    i
    t
    y
    I
    n
    t
    e
    r
    f
    a
    c
    e {
    p
    u
    b
    l
    i
    c f
    u
    n
    c
    t
    i
    o
    n g
    e
    t
    I
    d
    s
    (
    ) {
    r
    e
    t
    u
    r
    n a
    r
    r
    a
    y
    (
    '
    i
    d
    ' =
    > a
    r
    r
    a
    y
    (
    '
    t
    y
    p
    e
    ' =
    > '
    i
    n
    t
    e
    g
    e
    r
    '
    )
    )
    ;
    }

    View full-size slide

  35. Defining the source Pt2
    p
    u
    b
    l
    i
    c f
    u
    n
    c
    t
    i
    o
    n f
    i
    e
    l
    d
    s
    (
    ) {
    r
    e
    t
    u
    r
    n a
    r
    r
    a
    y
    (
    '
    i
    d
    ' =
    > $
    t
    h
    i
    s
    -
    >
    t
    (
    '
    U
    s
    e
    r I
    D
    '
    )
    ,
    '
    f
    i
    r
    s
    t
    _
    n
    a
    m
    e
    ' =
    > $
    t
    h
    i
    s
    -
    >
    t
    (
    '
    F
    i
    r
    s
    t n
    a
    m
    e
    '
    )
    ,
    '
    l
    a
    s
    t
    _
    n
    a
    m
    e
    ' =
    > $
    t
    h
    i
    s
    -
    >
    t
    (
    '
    L
    a
    s
    t n
    a
    m
    e
    '
    )
    ,
    '
    e
    m
    a
    i
    l
    ' =
    > $
    t
    h
    i
    s
    -
    >
    t
    (
    '
    E
    -
    m
    a
    i
    l
    '
    )
    ,
    '
    p
    a
    s
    s
    ' =
    > $
    t
    h
    i
    s
    -
    >
    t
    (
    '
    M
    D
    5 h
    a
    s
    h
    e
    d p
    a
    s
    s
    w
    o
    r
    d
    '
    )
    ,
    '
    g
    r
    o
    u
    p
    s
    ' =
    > $
    t
    h
    i
    s
    -
    >
    t
    (
    '
    U
    s
    e
    r g
    r
    o
    u
    p
    s
    '
    )
    ,
    )
    ;
    }

    View full-size slide

  36. Defining the source Pt3
    p
    u
    b
    l
    i
    c f
    u
    n
    c
    t
    i
    o
    n q
    u
    e
    r
    y
    (
    ) {
    r
    e
    t
    u
    r
    n $
    t
    h
    i
    s
    -
    >
    s
    e
    l
    e
    c
    t
    (
    '
    m
    i
    g
    r
    a
    t
    e
    _
    e
    x
    a
    m
    p
    l
    e
    _
    p
    e
    o
    p
    l
    e
    '
    , '
    p
    '
    )
    -
    >
    f
    i
    e
    l
    d
    s
    (
    '
    p
    '
    , a
    r
    r
    a
    y
    (
    '
    i
    d
    '
    , '
    f
    i
    r
    s
    t
    _
    n
    a
    m
    e
    '
    , '
    l
    a
    s
    t
    _
    n
    a
    m
    e
    '
    , '
    e
    m
    a
    i
    l
    '
    , '
    p
    a
    s
    s
    '
    , '
    g
    r
    o
    u
    p
    s
    '
    )
    )
    ;
    }

    View full-size slide

  37. Executing your migration
    Drush is the only way at the moment
    Requires a manifest file
    It's yaml (no surprise!)

    View full-size slide

  38. Example Manifest File
    - d
    6
    _
    u
    s
    e
    r
    - d
    6
    _
    u
    s
    e
    r
    _
    r
    o
    l
    e

    View full-size slide

  39. Running it through drush
    d
    r
    u
    s
    h m
    i
    g
    r
    a
    t
    e
    -
    m
    a
    n
    i
    f
    e
    s
    t m
    a
    n
    i
    f
    e
    s
    t
    -
    n
    a
    m
    e -
    -
    d
    b
    -
    u
    r
    l
    =
    m
    y
    s
    q
    l
    :
    /
    /
    r
    o
    o
    t
    :
    p
    a
    s
    s
    @
    1
    2
    7
    .
    0
    .
    0
    .
    1
    /
    d
    b

    View full-size slide

  40. Any Questions?

    View full-size slide

  41. Resources
    Pedro Cambra - @pcambra
    webikon.com - http://webikon.com/cases/migrating-to-drupal-8
    The IMP Team
    Migrate API Documentation - https://drupal.org/node/2127611
    *Thanks to Mark Jones for the title!

    View full-size slide