Migrating Anything with Baleen

Several migration frameworks and libraries have been created over the years in order to help developers manage their data and schema changes in a reliable, collaborative manner. But even well-adopted solutions like Phinx and Doctrine Migrations have either struggled to keep-up with modern PHP due to architectural issues, or failed to provide enough framework integration to be actually practical for everyday use.

In this talk I'll guide you through Baleen: an open-source PHP framework that synthesizes the very core of the migrations mechanism into a “domain library”, which can be re-used to power migrations for virtually any modern PHP framework (not kidding!). I'll also reveal plans of re-building Doctrine Migrations using the Baleen framework. But most importantly, you'll see an example of how separating common logic into small domain models can help you invent and contribute relevant modules to the open-source community.

Transcript

1. Migrating Anything with Baleen @gabriel_somoza
2. About Me • Started coding around 2000 • Consultant PHP Architect (Strategery) @ Cu.be • ZCE & Magento Certified Developer (PLUS) • Organizer of the PHP Limburg meetup in Belgium • Regularly contribute to open-source
3. Born Migrated a Lot About Me
4. Born About Me Lots of Migrations baleen.org
5. Raised Born About Me Lots of Migrations
6. Raised Born About Me Lots of Migrations
7. Raised Born About Me Lots of Migrations baleen.org
8. Raised Born About Me Lots of Migrations Live
9. Overview About Migrations Ecosystem & Use-cases Baleen Domain Model Demo
10. About Migrations
11. Migration Definition Describes a process that transforms SOMETHING from one STATE to another. baleen.org
12. Describing Change Q1 Q2 Q3 d1 d2 Git Commits Qn = States dn = Deltas Source: http://gá.be/1PrKEAh STATE baleen.org
13. Describing Change Q1 Q2 Q3 d1 d2 SVN, Mercurial, etc. Qn = States dn = Deltas Source: http://gá.be/1PrKEAh DELTAS baleen.org
14. Delta (Migration) Class Example
15. Some Migration Best-Practices About Migrations Idempotent Reversible (optional) Atomic baleen.org
16. Delta (Migration) Class Example
17. Collaborating with Deltas About Migrations Feature A Feature B Production Continuous Integration Stage D1 D2 D1 D2 D1 D2 baleen.org
18. Migration Ecosystem
19. The Migrations Ecosystem FRAGME Doctrine Migrations Phinx Laravel Migrations Magento (Install Scripts) CakePHP Yii Migrations NTED baleen.org
20. Challenges of Fragmentation • No unified vision / roadmap • Features cannot be ported • Incomplete tests (or none at all) • Repeated business logic = Poor Community Experience baleen.org
21. The Idea 21 You! Anyone can contribute, there’s lots to do Mike Simonson Maintainer for Doctrine Migrations & Baleen co-author Gabriel Somoza Baleen Architect baleen.org baleen.org
22. The Baleen Ecosystem Baleen Migrations Baleen CLI Flysystem Pimcore Doctrine Packages Overview (Domain Model) (Framework)
23. Packages Overview Baleen Migrations Baleen CLI Doctrine Laravel Phinx Etc. Web App Entities Business Logic Value Objects Generic Tasks (Commands) Config Specialization Config Views Views baleen.org The Baleen Ecosystem
24. Community Effort Cycle Enriches Gives Back Baleen Migrations Baleen CLI Doctrine Laravel Phinx Etc. Web App baleen.org The Baleen Ecosystem
25. What Baleen IS • A framework- and infrastructure- agnostic domain model • Designed to empower migration tools • With sensible defaults but highly customizable • An open-source project for the PHP community (MIT) • Very well tested baleen.org
26. What Baleen is NOT • Another migration library • An API / DSL that with a similar workflow to Phinx or Java’s Liquibase • A magic bullet baleen.org
27. Our Sample Scenarios I. You are using a DAL, like Doctrine. II. It has no tools for managing deltas (migrations). III. You build a tool from scratch. IV. It has only the features you need. I. Probably not enough tests. II. Its not generic enough to redistribute. V. You work a lot to get it to a point where you can distribute it. (or use-cases) #1 baleen.org
28. Our Sample Scenarios I. You are using a DAL, like Doctrine. II. It has no tools for managing deltas (migrations). III. You build a tool from scratch. (or use-cases) #1 baleen.org
29. Our Sample Scenarios I. You are using a DAL, like Doctrine. II. It has no tools for managing deltas (migrations). III. You build a tool from scratch. (or use-cases) #1 baleen.org
30. Our Sample Scenarios I. You are using a DAL, like Doctrine. II. It has no tools for managing deltas (migrations). III. You build a tool using Baleen. IV. You get most features out of the box. V. Test coverage for the core mechanisms. VI. Detailed documentation for end-users. VII. Your end-users already know how Baleen works. (or use-cases) #1 baleen.org
31. Our Sample Scenarios I. You’re using a tool that’s already based on Baleen. II. You’d like to have a “status” command (new feature). III. You contribute that feature to the core library. IV. Every other tool that uses Baleen can now get that feature. Not only yours. (or use-cases) #2 baleen.org
32. Migrations Domain
33. “All models are wrong, but some are useful.” George Box (probably)
34. The Migrations Domain Who are the Domain Experts ? • Framework Authors & Contributors • Database Administrators • Deployment Managers baleen.org
35. The Migrations Domain Who are the End-Users ? • Developers that use your migration tool to manage deltas (migrations). baleen.org
36. The Migrations Domain (how we see it) Migration Tool (e.g. Baleen CLI) Target Domain Model Infrastructure Data-Access Layer (ORM / ODM / etc.) baleen.org
37. Baleen Architectural Overview 37 Delta (Aggregate) Domain Model Migration Class Business Logic Baleen Core Commands Events State Storage DAL Target Domain Migration Tool Target Domain Infrastructure DAL Config Infrastructure baleen.org
38. Baleen Architectural Overview 38 Interaction baleen.org Framework Infrastructure Application Query (up) Event Event Event Baleen Core
39. Value Objects
40. Value Objects VersionId baleen.org
41. Value Objects Direction UP DOWN baleen.org
42. Value Objects Options DIRECTION FORCE? DRY RUN? EXCEPTION ON SKIP? baleen.org
43. Anatomy of a Delta (Migration)
44. Commands Baleen Architectural Overview 44 Domain Model Business Logic Baleen Core baleen.org Events State Storage DAL Target Domain Migration Tool Target Domain Infrastructure DAL Config Infrastructure Migration Class Delta (Aggregate)
45. Delta Aggregate Uniquely IDENTIFIES a migration file, STATE and can INVOKE it knows its when commanded. baleen.org
46. Delta Aggregate Ubiquitous Language Migration Delta 1 migrate(options) baleen.org
47. Commands Baleen Architectural Overview 47 Delta (Aggregate) Domain Model Business Logic Baleen Core baleen.org Events State Storage DAL Target Domain Migration Tool Target Domain Infrastructure DAL Config Infrastructure Migration Class
48. Migration Class Example
49. Migration Class Example
50. Migration Class Example
51. Migration Class Life-Cycle Migration Class Requested Factory Injects •Application •Infrastructure Instance migrate(direction) •Customizable process •Fires events Update state Done baleen.org
52. Migration Factory Baleen CLI Your Migrations Tool Target Application / Framework / Domain Model baleen.org
53. “Migrate” Command ./vendor/bin/baleen migrate
54. Baleen Architectural Overview 54 Delta (Aggregate) Domain Model Migration Class Business Logic Baleen Core baleen.org Events State Storage DAL Target Domain Migration Tool Target Domain Infrastructure DAL Config Infrastructure Commands
55. Migrate Command Migrates through one or more DELTAS towards a target using one of the following strategies: UP DOWN CONVERGE baleen.org
56. UP Strategy Ubiquitous Language v1 v2 v3 v4 v5 target baleen.org
57. DOWN Strategy Ubiquitous Language v1 v2 v3 v4 v5 target baleen.org
58. Converge Strategy Ubiquitous Language v1 v2 v3 v4 v5 target baleen.org
59. Domain Events
60. Domain Events Ubiquitous Language DomainEvent Sequence Id Occurred On baleen.org
61. Domain Events Ubiquitous Language v1 v2 v3 v4 v5 target head BeforeCollection Deltas to Migrate Target version Options baleen.org
62. Domain Events Ubiquitous Language v1 v2 v3 v4 v5 target head BeforeMigration Delta Options Collection Context baleen.org
63. Domain Events Ubiquitous Language v1 v2 v3 v4 v5 target head AfterMigration Delta Options Collection Context baleen.org
64. Domain Events Ubiquitous Language v1 v2 v3 v4 v5 target head AfterCollection Modified Deltas Target Options baleen.org
65. Baleen Architectural Overview 65 Interaction baleen.org Framework Infrastructure Application Query (up) Event Event Event Baleen Core
66. Baleen CLI: What is the Bare Minimum?
67. Nothing! composer require baleen/cli
68. Minimum for Rolling Your Own
69. Migration Factory Baleen CLI Your Migrations Tool Target Application / Framework / Domain Model baleen.org
70. DEMO: Baleen CLI
71. DEMO: Pimcore Migrations
72. Remaining Challenges The Migration Ecosystem Tomorrow Adoption Integration Collaboration baleen.org
73. Roadmap 1. Finish DDD changes 2. Release stable 1.0 version 3. Doctrine Migrations
74. How YOU Can Help The Migration Ecosystem Tomorrow 1. Review Code 2. Spread the Word 3. Fork and Contribute! 4. Integrate baleen.org
75. The Links Migrating Anythingwith DDD Baleen Migrations Baleen CLI github.com/baleen/cli baleen.org Read the Docs Read the Docs
76. QUESTIONS TIME! Have a question? Ask NOW! https://joind.in/ @gabriel_somoza @BaleenPHP 16817
77. Thanks for Listening! @gabriel_somoza @BaleenPHP https://joind.in/ 16817
78. DEMO: Baleen CLI
79. Migrations List Command Baleen CLI baleen.org
80. Status Command Baleen CLI baleen.org
81. Migrate Command Baleen CLI baleen.org
82. Migrate Command Baleen CLI baleen.org
83. Create Command Baleen CLI