Employing CQRS and Event Sourcing to Build an MVP (Symfony Live San Francisco 2017)

Learning about Command Query Responsibility Segregation (CQRS) and Event Sourcing can be both exciting and confusing. It sounds great, but how does it work in the real world? Won't it be a lot of work before you even get started? Will it be worth the investment and when will that investment start paying off? Are these technologies appropriate for building a minimum viable product (MVP)? Find out how these questions were answered for one team as they set out to build their MVP.

Transcript

1. Employing
   CQRS and Event Sourcing
   to Build an MVP
   Beau Simensen • @beausimensen • beau.io
   joind.in/talk/3d016
   

   View Slide

2. What is
   CQRS?
   

   View Slide

3. Command / Query
   Responsibility Segregation
   

   View Slide

4. View Slide

5. View Slide

6. What is
   Event Sourcing?
   

   View Slide

7. View Slide

8. View Slide

9. View Slide

10. View Slide

11. View Slide

12. Why do
    ES + CQRS?
    

    View Slide

13. View Slide

14. What is an
    MVP?
    

    View Slide

15. View Slide

16. Fintech!
    Accounting + Big Data
    

    View Slide

17. View Slide

18. View Slide

19. MVP Criteria
    

    View Slide

20. Address Book
    

    View Slide

21. Deals
    

    View Slide

22. Solid
    No rewrites every two years
    

    View Slide

23. Solid
    Jump on features quickly
    

    View Slide

24. Multi-Tenant
    

    View Slide

25. Single Page
    Application
    (SPA)
    

    View Slide

26. View Slide

27. React + Redux
    Flux
    

    View Slide

28. Material Design
    Material-UI
    

    View Slide

29. Internal Framework
    API + CLI + Container
    

    View Slide

30. Internal
    Infrastructure
    Event Sourcing + CQRS
    

    View Slide

31. View Slide

32. Me
    

    View Slide

33. The Cowboy
    

    View Slide

34. The Junior
    

    View Slide

35. (that's it)
    

    View Slide

36. Testing
    

    View Slide

37. Modern PHP
    

    View Slide

38. We built our
    ES+CQRS Infrastructure
    as a team
    

    View Slide

39. View Slide

40. Choose Boring Technology
    Dan McKinley
    http://mcfunley.com/choose-boring-technology
    

    View Slide

41. "Boring" should not be
    conflated with "bad."
    — Dan McKinley
    

    View Slide

42. ... three innovation tokens
    

    View Slide

43. React
    1 Token
    

    View Slide

44. React for our team?
    2 - 3 Tokens
    

    View Slide

45. Redux
    1 Token
    

    View Slide

46. Redux for our team?
    2 - 3 Tokens
    

    View Slide

47. Material UI
    1 Token
    

    View Slide

48. Material UI for our team?
    2 - 3 Tokens
    

    View Slide

49. Event Sourcing
    1 Token
    

    View Slide

50. Event Sourcing for our team?
    2 - 3 Tokens
    

    View Slide

51. CQRS
    1 Token
    

    View Slide

52. CQRS for our team?
    2 - 3 Tokens
    

    View Slide

53. PSR-7
    1 Token
    

    View Slide

54. PSR-7 for our team?
    Boring!
    

    View Slide

55. PHP
    Boring!
    

    View Slide

56. MySQL
    Boring!
    

    View Slide

57. Envoyer + Forge
    Boring!
    

    View Slide

58. Best case?
    5 Tokens
    

    View Slide

59. Worst cases?
    10-15 Tokens
    

    View Slide

60. It took our team
    SIX MONTHS
    to reach MVP
    

    View Slide

61. Similar functionality could have been built in
    ONE MONTH
    using a "boring" framework & HTML templates
    

    View Slide

62. We had a solid
    foundation
    

    View Slide

63. It felt like a desktop
    application
    

    View Slide

64. Six months from MVP
    we threw in the towel
    

    View Slide

65. View Slide

66. Was it the fault of the
    stack?
    — me, every day for the next few months
    

    View Slide

67. Surprise! We're just going
    to build a CRM.
    — two weeks after MVP was ready
    

    View Slide

68. PIVOT
    

    View Slide

69. Sales and marketing
    could not sell the
    product
    

    View Slide

70. Saturated and
    Competitive Market
    

    View Slide

71. Knowing that sooner would have
    saved us a ton of cash
    

    View Slide

72. Catch-22
    

    View Slide

73. If the plan was to just
    build a CRM I wouldn't
    have gotten involved.
    

    View Slide

74. View Slide

75. Reflection Properties
    Serializer
    Optimized for DX
    

    View Slide

76. Reflection Properties
    Serializer
    Started to hurt after 3 months in production...
    

    View Slide

77. Search Read Model
    Awesome!
    

    View Slide

78. History Read Models
    Neat, but limited...
    

    View Slide

79. History Read Models
    Required augmenting events with more info
    

    View Slide

80. Timeline
    Full power of Event Sourcing + CQRS
    

    View Slide

81. Designed for async;
    never implemented
    No need to worry about eventual consistency
    

    View Slide

82. Events change
    Especially if your domain isn't stable
    

    View Slide

83. Version
    

    View Slide

84. Upcast
    

    View Slide

85. Lazy Upcast
    

    View Slide

86. In Place Migration
    

    View Slide

87. Copy Migration
    

    View Slide

88. In Place / Copy Migration
    

    View Slide

89. Required downtime for safety
    

    View Slide

90. Rebuilt all read models from scratch
    

    View Slide

91. 30 seconds
    For the first month
    

    View Slide

92. 10 minutes
    By middle of second month
    

    View Slide

93. Replaying events
    broke on some
    changes
    

    View Slide

94. Rebuilding Read Models
    (not just when migrating the event stream...)
    

    View Slide

95. Nuclear Option
    Rebuild ALL Read Models
    

    View Slide

96. Rebuilding Individual
    Read Models
    Helped a lot with downtime, smaller sets of events
    

    View Slide

97. Mostly optimized for
    Read
    Since write load was generally low...
    

    View Slide

98. False sense of
    security
    Rebuilding Read Models became expensive
    

    View Slide

99. Rebuilding Read Models while
    app was active
    Dangerous!
    

    View Slide

100. Rebuilding Read Models while
     app was active
     Deemed acceptable risk while load was low
     

     View Slide

101. Rebuilding Read Models while
     app was active
     Weird side effects as rebuild started to take longer
     

     View Slide

102. Solution?
     Queue events so that new events
     would not be missed
     

     View Slide

103. Solution?
     Replay old events into to new
     read model repository
     

     View Slide

104. Solution?
     Swap new repository for old
     repository
     

     View Slide

105. Solution?
     Start reading events from queue
     again and start projecting
     

     View Slide

106. Solution!
     Follow Store
     

     View Slide

107. ChatOps
     

     View Slide

108. Migrations
     

     View Slide

109. Rebuilding Read Models!
     

     View Slide

110. Is Read Model up to date?
     

     View Slide

111. Private State
     

     View Slide

112. Too strict CQRS led to
     extra Read Models we
     probably didn't need...
     

     View Slide

113. Private State = Private Read Models
     

     View Slide

114. Eventually moved toward private state for
     Model
     

     View Slide

115. Eventually moved toward private state for
     Read Model
     

     View Slide

116. Eventually moved toward private state for
     Services
     

     View Slide

117. View Slide

118. Biggest Takeaway?
     Slower Development.
     

     View Slide

119. Command -> Command Bus -> Command Handler -> Model -> Events -> Event Store -> Event Bus -> Projector -> Read Model!
     

     View Slide

120. Adding one new field...
     

     View Slide

121. Search
     huge win
     

     View Slide

122. Timeline
     huge win
     

     View Slide

123. Retroactive Features
     

     View Slide

124. Natural Evolution
     

     View Slide

125. Event Sourcing makes sense if the
     Model is known and stable
     How often is that the case for an MVP?
     

     View Slide

126. CQRS makes sense if Read Model
     is not known / flexible
     Good fit for an MVP!
     

     View Slide

127. CQRS makes sense if Read Model
     is not known / flexible
     At a cost...
     

     View Slide

128. Can you use an
     off-the-shelf
     framework?
     

     View Slide

129. Event Store
     

     View Slide

130. Test Harness
     

     View Slide

131. Good Fit!
     

     View Slide

132. Rolling an ES+CQRS
     framework ourself was
     expensive but was a great
     investment for the team.
     

     View Slide

133. Not sure I'd make the
     same decision today...
     

     View Slide

134. It DID work though!
     (But development pace was considerably slower...)
     

     View Slide

135. Support From
     Management
     

     View Slide

136. Had that support initially
     

     View Slide

137. Unrealistic expectations
     

     View Slide

138. Eventually just a feature race
     

     View Slide

139. Eventually just wanted CRUD
     

     View Slide

140. Pivot changed everything
     

     View Slide

141. We couldn't sell the MVP
     This made ES+CQRS infinitely more expensive
     

     View Slide

142. View Slide

143. Thanks!
     @sensiolabs • @thatpodcast
     Beau Simensen • @beausimensen • beau.io
     joind.in/talk/3d016
     

     View Slide

144. • Stop overthinking…Just stop!
     

     View Slide