$30 off During Our Annual Pro Sale. View Details »

Employing CQRS and Event Sourcing to Build an MVP (Symfony Live San Francisco 2017)

Employing CQRS and Event Sourcing to Build an MVP (Symfony Live San Francisco 2017)

Learning about Command Query Responsibility Segregation (CQRS) and Event Sourcing can be both exciting and confusing. It sounds great, but how does it work in the real world? Won't it be a lot of work before you even get started? Will it be worth the investment and when will that investment start paying off? Are these technologies appropriate for building a minimum viable product (MVP)? Find out how these questions were answered for one team as they set out to build their MVP.

Beau Simensen

October 19, 2017
Tweet

More Decks by Beau Simensen

Other Decks in Programming

Transcript

  1. Employing
    CQRS and Event Sourcing
    to Build an MVP
    Beau Simensen • @beausimensen • beau.io
    joind.in/talk/3d016

    View Slide

  2. What is
    CQRS?

    View Slide

  3. Command / Query
    Responsibility Segregation

    View Slide

  4. View Slide

  5. View Slide

  6. What is
    Event Sourcing?

    View Slide

  7. View Slide

  8. View Slide

  9. View Slide

  10. View Slide

  11. View Slide

  12. Why do
    ES + CQRS?

    View Slide

  13. View Slide

  14. What is an
    MVP?

    View Slide

  15. View Slide

  16. Fintech!
    Accounting + Big Data

    View Slide

  17. View Slide

  18. View Slide

  19. MVP Criteria

    View Slide

  20. Address Book

    View Slide

  21. Deals

    View Slide

  22. Solid
    No rewrites every two years

    View Slide

  23. Solid
    Jump on features quickly

    View Slide

  24. Multi-Tenant

    View Slide

  25. Single Page
    Application
    (SPA)

    View Slide

  26. View Slide

  27. React + Redux
    Flux

    View Slide

  28. Material Design
    Material-UI

    View Slide

  29. Internal Framework
    API + CLI + Container

    View Slide

  30. Internal
    Infrastructure
    Event Sourcing + CQRS

    View Slide

  31. View Slide

  32. Me

    View Slide

  33. The Cowboy

    View Slide

  34. The Junior

    View Slide

  35. (that's it)

    View Slide

  36. Testing

    View Slide

  37. Modern PHP

    View Slide

  38. We built our
    ES+CQRS Infrastructure
    as a team

    View Slide

  39. View Slide

  40. Choose Boring Technology
    Dan McKinley
    http://mcfunley.com/choose-boring-technology

    View Slide

  41. "Boring" should not be
    conflated with "bad."
    — Dan McKinley

    View Slide

  42. ... three innovation tokens

    View Slide

  43. React
    1 Token

    View Slide

  44. React for our team?
    2 - 3 Tokens

    View Slide

  45. Redux
    1 Token

    View Slide

  46. Redux for our team?
    2 - 3 Tokens

    View Slide

  47. Material UI
    1 Token

    View Slide

  48. Material UI for our team?
    2 - 3 Tokens

    View Slide

  49. Event Sourcing
    1 Token

    View Slide

  50. Event Sourcing for our team?
    2 - 3 Tokens

    View Slide

  51. CQRS
    1 Token

    View Slide

  52. CQRS for our team?
    2 - 3 Tokens

    View Slide

  53. PSR-7
    1 Token

    View Slide

  54. PSR-7 for our team?
    Boring!

    View Slide

  55. PHP
    Boring!

    View Slide

  56. MySQL
    Boring!

    View Slide

  57. Envoyer + Forge
    Boring!

    View Slide

  58. Best case?
    5 Tokens

    View Slide

  59. Worst cases?
    10-15 Tokens

    View Slide

  60. It took our team
    SIX MONTHS
    to reach MVP

    View Slide

  61. Similar functionality could have been built in
    ONE MONTH
    using a "boring" framework & HTML templates

    View Slide

  62. We had a solid
    foundation

    View Slide

  63. It felt like a desktop
    application

    View Slide

  64. Six months from MVP
    we threw in the towel

    View Slide

  65. View Slide

  66. Was it the fault of the
    stack?
    — me, every day for the next few months

    View Slide

  67. Surprise! We're just going
    to build a CRM.
    — two weeks after MVP was ready

    View Slide

  68. PIVOT

    View Slide

  69. Sales and marketing
    could not sell the
    product

    View Slide

  70. Saturated and
    Competitive Market

    View Slide

  71. Knowing that sooner would have
    saved us a ton of cash

    View Slide

  72. Catch-22

    View Slide

  73. If the plan was to just
    build a CRM I wouldn't
    have gotten involved.

    View Slide

  74. View Slide

  75. Reflection Properties
    Serializer
    Optimized for DX

    View Slide

  76. Reflection Properties
    Serializer
    Started to hurt after 3 months in production...

    View Slide

  77. Search Read Model
    Awesome!

    View Slide

  78. History Read Models
    Neat, but limited...

    View Slide

  79. History Read Models
    Required augmenting events with more info

    View Slide

  80. Timeline
    Full power of Event Sourcing + CQRS

    View Slide

  81. Designed for async;
    never implemented
    No need to worry about eventual consistency

    View Slide

  82. Events change
    Especially if your domain isn't stable

    View Slide

  83. Version

    View Slide

  84. Upcast

    View Slide

  85. Lazy Upcast

    View Slide

  86. In Place Migration

    View Slide

  87. Copy Migration

    View Slide

  88. In Place / Copy Migration

    View Slide

  89. Required downtime for safety

    View Slide

  90. Rebuilt all read models from scratch

    View Slide

  91. 30 seconds
    For the first month

    View Slide

  92. 10 minutes
    By middle of second month

    View Slide

  93. Replaying events
    broke on some
    changes

    View Slide

  94. Rebuilding Read Models
    (not just when migrating the event stream...)

    View Slide

  95. Nuclear Option
    Rebuild ALL Read Models

    View Slide

  96. Rebuilding Individual
    Read Models
    Helped a lot with downtime, smaller sets of events

    View Slide

  97. Mostly optimized for
    Read
    Since write load was generally low...

    View Slide

  98. False sense of
    security
    Rebuilding Read Models became expensive

    View Slide

  99. Rebuilding Read Models while
    app was active
    Dangerous!

    View Slide

  100. Rebuilding Read Models while
    app was active
    Deemed acceptable risk while load was low

    View Slide

  101. Rebuilding Read Models while
    app was active
    Weird side effects as rebuild started to take longer

    View Slide

  102. Solution?
    Queue events so that new events
    would not be missed

    View Slide

  103. Solution?
    Replay old events into to new
    read model repository

    View Slide

  104. Solution?
    Swap new repository for old
    repository

    View Slide

  105. Solution?
    Start reading events from queue
    again and start projecting

    View Slide

  106. Solution!
    Follow Store

    View Slide

  107. ChatOps

    View Slide

  108. Migrations

    View Slide

  109. Rebuilding Read Models!

    View Slide

  110. Is Read Model up to date?

    View Slide

  111. Private State

    View Slide

  112. Too strict CQRS led to
    extra Read Models we
    probably didn't need...

    View Slide

  113. Private State = Private Read Models

    View Slide

  114. Eventually moved toward private state for
    Model

    View Slide

  115. Eventually moved toward private state for
    Read Model

    View Slide

  116. Eventually moved toward private state for
    Services

    View Slide

  117. View Slide

  118. Biggest Takeaway?
    Slower Development.

    View Slide

  119. Command -> Command Bus -> Command Handler -> Model -> Events -> Event Store -> Event Bus -> Projector -> Read Model!

    View Slide

  120. Adding one new field...

    View Slide

  121. Search
    huge win

    View Slide

  122. Timeline
    huge win

    View Slide

  123. Retroactive Features

    View Slide

  124. Natural Evolution

    View Slide

  125. Event Sourcing makes sense if the
    Model is known and stable
    How often is that the case for an MVP?

    View Slide

  126. CQRS makes sense if Read Model
    is not known / flexible
    Good fit for an MVP!

    View Slide

  127. CQRS makes sense if Read Model
    is not known / flexible
    At a cost...

    View Slide

  128. Can you use an
    off-the-shelf
    framework?

    View Slide

  129. Event Store

    View Slide

  130. Test Harness

    View Slide

  131. Good Fit!

    View Slide

  132. Rolling an ES+CQRS
    framework ourself was
    expensive but was a great
    investment for the team.

    View Slide

  133. Not sure I'd make the
    same decision today...

    View Slide

  134. It DID work though!
    (But development pace was considerably slower...)

    View Slide

  135. Support From
    Management

    View Slide

  136. Had that support initially

    View Slide

  137. Unrealistic expectations

    View Slide

  138. Eventually just a feature race

    View Slide

  139. Eventually just wanted CRUD

    View Slide

  140. Pivot changed everything

    View Slide

  141. We couldn't sell the MVP
    This made ES+CQRS infinitely more expensive

    View Slide

  142. View Slide

  143. Thanks!
    @sensiolabs • @thatpodcast
    Beau Simensen • @beausimensen • beau.io
    joind.in/talk/3d016

    View Slide

  144. • Stop overthinking…Just stop!

    View Slide