Event Sourcing in practice @ Qandidate.com

Transcript

1. Event sourcing in production Willem-Jan Zijderveld @willemjanz wjzijderveld@gmail.com joind.in/16227

2. Event sourcing

3. Short introduction into our domain

4. Job Marketing Platform

5. Choosing the best channels to find your candidates

6. Online Job Marketing

7. 1. Create a campaign for your target group

8. 2. You get a recommendation

9. 3. You can customize your campaign

10. Why event sourcing?

11. You are throwing away data!

12. By only saving last known state

13. You don't know the previous state

14. You don't know why something changed

15. How does event sourcing help?

16. Using events to store changes

17. Describe with your events what happened

18. Describe with your events why something happened

19. Use the events to calculate the current state

20. Allows you to go back in time

21. Helps to understand the flow in your application

22. Example: Discarding the customizations

23. CRUD

24. Keep track if channel was part of recommendation

25. Keep track if channel was part of recommendation Remove the

    non-recommended channels

26. Keep track if channel was part of recommendation How about

    removed channels?

27. Keep track if channel was part of recommendation Implement soft-delete

28. Keep track if channel was part of recommendation Implement soft-delete

    for new campaigns

29. Event sourcing

30. Just rewind

31. Just rewind Campaign Created For Target Group Channel Removed From

    Campaign Channel Added To Campaign Customization Discarded

32. Just rewind Campaign Created For Target Group Channel Removed From

    Campaign Channel Added To Campaign Customization Discarded

33. Just rewind Campaign Created For Target Group Channel Removed From

    Campaign Channel Added To Campaign Customization Discarded

34. Just rewind Campaign Created For Target Group Channel Removed From

    Campaign Channel Added To Campaign Customization Discarded

35. How?

36. With the help of CQRS and Domain Driven Design

37. And the help of Broadway github.com/qandidate-labs/broadway

38. Creating a new campaign

39. Campaign Event Store Create Campaign For Target Group Campaign Created

    For Target Group

40. Dispatching the command /** @Route("/campaigns", methods={"POST"}) */ public function createCampaignAction(CreateCampaignDto

    $dto) { $campaignId = Webshop\CampaignId::generate(); $this->commandBus->dispatch(new CreateCampaignForTargetGroup( $campaignId, $this->companyId, $this->accountId, TargetGroup::describe( TargetGroup\JobTitle::fromString($dto->title), TargetGroup\Region::fromId($dto->region), TargetGroup\JobCategory::fromId($dto->jobCategory), TargetGroup\EducationLevel::fromId($dto->educationLevel), TargetGroup\JobLevel::fromId($dto->jobLevel), TargetGroup\Industry::fromId($dto->industry) ) )); return new JsonResponse( ['id' => (string) $campaignId], JsonResponse::HTTP_CREATED ); }

41. Dispatching the command /** @Route("/campaigns", methods={"POST"}) */ public function createCampaignAction(CreateCampaignDto

    $dto) { $campaignId = Webshop\CampaignId::generate(); $this->commandBus->dispatch(new CreateCampaignForTargetGroup( $campaignId, $this->companyId, $this->accountId, TargetGroup::describe( TargetGroup\JobTitle::fromString($dto->title), TargetGroup\Region::fromId($dto->region), TargetGroup\JobCategory::fromId($dto->jobCategory), TargetGroup\EducationLevel::fromId($dto->educationLevel), TargetGroup\JobLevel::fromId($dto->jobLevel), TargetGroup\Industry::fromId($dto->industry) ) )); return new JsonResponse( ['id' => (string) $campaignId], JsonResponse::HTTP_CREATED ); }

42. Dispatching the command /** @Route("/campaigns", methods={"POST"}) */ public function createCampaignAction(CreateCampaignDto

    $dto) { $campaignId = Webshop\CampaignId::generate(); $this->commandBus->dispatch(new CreateCampaignForTargetGroup( $campaignId, $this->companyId, $this->accountId, TargetGroup::describe( TargetGroup\JobTitle::fromString($dto->title), TargetGroup\Region::fromId($dto->region), TargetGroup\JobCategory::fromId($dto->jobCategory), TargetGroup\EducationLevel::fromId($dto->educationLevel), TargetGroup\JobLevel::fromId($dto->jobLevel), TargetGroup\Industry::fromId($dto->industry) ) )); return new JsonResponse( ['id' => (string) $campaignId], JsonResponse::HTTP_CREATED ); }

43. Handling the command // CampaignCommandHandler.php public function handleCreateCampaignForTargetGroup( CreateCampaignForTargetGroup $command

    ) { $campaign = Campaign::createForTargetGroup( $command->getCampaignId(), $command->getCompanyId(), $command->getRecruiterId(), $command->getTargetGroup(), $this->recommender ); $this->campaignRepository->save($campaign); }

44. Handling the command // CampaignCommandHandler.php public function handleCreateCampaignForTargetGroup( CreateCampaignForTargetGroup $command

    ) { $campaign = Campaign::createForTargetGroup( $command->getCampaignId(), $command->getCompanyId(), $command->getRecruiterId(), $command->getTargetGroup(), $this->recommender ); $this->campaignRepository->save($campaign); }

45. Handling the command // CampaignCommandHandler.php public function handleCreateCampaignForTargetGroup( CreateCampaignForTargetGroup $command

    ) { $campaign = Campaign::createForTargetGroup( $command->getCampaignId(), $command->getCompanyId(), $command->getRecruiterId(), $command->getTargetGroup(), $this->recommender ); $this->campaignRepository->save($campaign); }

46. Action → Reaction // Campaign.php public static function createForTargetGroup(/* args

    */) { $campaign = new Campaign(); $campaignChannels = $recommender->recommend(/* args */); $campaign->apply(new CampaignCreatedForTargetGroup( $campaignId, $companyId, $accountId, Channels::fromCollection($campaignChannels), $targetGroup, $campaign->calculateCampaignPrice($campaignChannels), $campaign->calculateEstimatedOutcome($campaignChannels) )); return $campaign; }

47. Action → Reaction // Campaign.php public static function createForTargetGroup(/* args

    */) { $campaign = new Campaign(); $campaignChannels = $recommender->recommend(/* args */); $campaign->apply(new CampaignCreatedForTargetGroup( $campaignId, $companyId, $accountId, Channels::fromCollection($campaignChannels), $targetGroup, $campaign->calculateCampaignPrice($campaignChannels), $campaign->calculateEstimatedOutcome($campaignChannels) )); return $campaign; }

48. Action → Reaction // Campaign.php public static function createForTargetGroup(/* args

    */) { $campaign = new Campaign(); $campaignChannels = $recommender->recommend(/* args */); $campaign->apply(new CampaignCreatedForTargetGroup( $campaignId, $companyId, $accountId, Channels::fromCollection($campaignChannels), $targetGroup, $campaign->calculateCampaignPrice($campaignChannels), $campaign->calculateEstimatedOutcome($campaignChannels) )); return $campaign; }

49. Action → Reaction // Campaign.php public static function createForTargetGroup(/* args

    */) { $campaign = new Campaign(); $campaignChannels = $recommender->recommend(/* args */); $campaign->apply(new CampaignCreatedForTargetGroup( $campaignId, $companyId, $accountId, Channels::fromCollection($campaignChannels), $targetGroup, $campaign->calculateCampaignPrice($campaignChannels), $campaign->calculateEstimatedOutcome($campaignChannels) )); return $campaign; }

50. // Broadway: EventSourcedAggregateRoot.php public function apply($event) { $this->handleRecursively($event); $this->playhead++; $this->uncommittedEvents[]

    = DomainMessage::recordNow( $this->getAggregateRootId(), $this->playhead, new Metadata(), $event ); }

51. // Broadway: EventSourcedAggregateRoot.php public function apply($event) { $this->handleRecursively($event); $this->playhead++; $this->uncommittedEvents[]

    = DomainMessage::recordNow( $this->getAggregateRootId(), $this->playhead, new Metadata(), $event ); }

52. Campaign Event Store Create Campaign For Target Group Campaign Created

    For Target Group

53. // Campaign.php protected function applyCampaignCreatedForTargetGroup( CampaignCreatedForTargetGroup $event ) { $this->id

    = $event->getCampaignId(); $this->channels = $event->getRecommendedChannels()->toArray(); $this->campaignPrice = $event->getCampaignPrice(); $this->estimatedOutcome = $event->getEstimatedOutcome(); $this->targetGroup = $event->getTargetGroup(); $this->setInitialRecommendation( $this->channels, $this->estimatedOutcome, $this->campaignPrice ); }

54. // Broadway: EventSourcedAggregateRoot.php public function apply($event) { $this->handleRecursively($event); $this->playhead++; $this->uncommittedEvents[]

    = DomainMessage::recordNow( $this->getAggregateRootId(), $this->playhead, new Metadata(), $event ); }

55. // Broadway: EventSourcedAggregateRoot.php public function apply($event) { $this->handleRecursively($event); $this->playhead++; $this->uncommittedEvents[]

    = DomainMessage::recordNow( $this->getAggregateRootId(), $this->playhead, new Metadata(), $event ); }

56. // Broadway: EventSourcedAggregateRoot.php public function apply($event) { $this->handleRecursively($event); $this->playhead++; $this->uncommittedEvents[]

    = DomainMessage::recordNow( $this->getAggregateRootId(), $this->playhead, new Metadata(), $event ); }

57. Handling the command // CampaignCommandHandler.php public function handleCreateCampaignForTargetGroup( CreateCampaignForTargetGroup $command

    ) { $campaign = Campaign::createForTargetGroup( $command->getCampaignId(), $command->getCompanyId(), $command->getRecruiterId(), $command->getTargetGroup(), $this->recommender, $this->portfolio, $this->aidaMetricsCalculator ); $this->campaignRepository->save($campaign); }

58. Saving the events // Broadway: EventSourcingRepository.php public function save(AggregateRoot $aggregate)

    { // maybe we can get generics one day.... ;) Assert::isInstanceOf($aggregate, $this->aggregateClass); $eventStream = $aggregate->getUncommittedEvents(); $this->eventStore->append( $aggregate->getAggregateRootId(), $eventStream ); $this->eventBus->publish($eventStream); }

59. Saving the events // Broadway: EventSourcingRepository.php public function save(AggregateRoot $aggregate)

    { // maybe we can get generics one day.... ;) Assert::isInstanceOf($aggregate, $this->aggregateClass); $eventStream = $aggregate->getUncommittedEvents(); $this->eventStore->append( $aggregate->getAggregateRootId(), $eventStream ); $this->eventBus->publish($eventStream); }

60. Saving the events // Broadway: EventSourcingRepository.php public function save(AggregateRoot $aggregate)

    { // maybe we can get generics one day.... ;) Assert::isInstanceOf($aggregate, $this->aggregateClass); $eventStream = $aggregate->getUncommittedEvents(); $this->eventStore->append( $aggregate->getAggregateRootId(), $eventStream ); $this->eventBus->publish($eventStream); }

61. It happened in the past

62. Append only

63. Campaign Event Store Create Campaign For Target Group Campaign Created

    For Target Group

64. How would we load a campaign?

65. Campaign Event Store Campaign Created For Target Group

66. // Broadway: EventSourcingRepository.php public function load($id) { try { $domainEventStream

    = $this->eventStore->load($id); return $this->aggregateFactory->create( $this->aggregateClass, $domainEventStream ); } catch (EventStreamNotFoundException $e) { throw AggregateNotFoundException::create($id, $e); } } // Broadway: EventSourcedAggregateRoot.php public function initializeState(DomainEventStreamInterface $stream) { foreach ($stream as $message) { $this->playhead++; $this->handleRecursively($message->getPayload()); } }

67. // Broadway: EventSourcingRepository.php public function load($id) { try { $domainEventStream

    = $this->eventStore->load($id); return $this->aggregateFactory->create( $this->aggregateClass, $domainEventStream ); } catch (EventStreamNotFoundException $e) { throw AggregateNotFoundException::create($id, $e); } } // Broadway: EventSourcedAggregateRoot.php public function initializeState(DomainEventStreamInterface $stream) { foreach ($stream as $message) { $this->playhead++; $this->handleRecursively($message->getPayload()); } }

68. // Broadway: EventSourcingRepository.php public function load($id) { try { $domainEventStream

    = $this->eventStore->load($id); return $this->aggregateFactory->create( $this->aggregateClass, $domainEventStream ); } catch (EventStreamNotFoundException $e) { throw AggregateNotFoundException::create($id, $e); } } // Broadway: EventSourcedAggregateRoot.php public function initializeState(DomainEventStreamInterface $stream) { foreach ($stream as $message) { $this->playhead++; $this->handleRecursively($message->getPayload()); } }

69. // Broadway: EventSourcingRepository.php public function load($id) { try { $domainEventStream

    = $this->eventStore->load($id); return $this->aggregateFactory->create( $this->aggregateClass, $domainEventStream ); } catch (EventStreamNotFoundException $e) { throw AggregateNotFoundException::create($id, $e); } } // Broadway: EventSourcedAggregateRoot.php public function initializeState(DomainEventStreamInterface $stream) { foreach ($stream as $message) { $this->playhead++; $this->handleRecursively($message->getPayload()); } }

70. Let's add a channel

71. Campaign Event Store Add Channel To Campaign Channel Added To

    Campaign

72. // CampaignCommandHandler.php public function handleAddChannelToCampaignCommand( AddChannelToCampaignCommand $command ) { $channel

    = $this->portfolio->getById($command->getChannelId()); $campaign = $this->campaignRepository->load( $command->getCampaignId() ); $campaign->addChannel($channel, $this->aidaMetricsCalculator); $this->campaignRepository->save($campaign); }

73. // CampaignCommandHandler.php public function handleAddChannelToCampaignCommand( AddChannelToCampaignCommand $command ) { $channel

    = $this->portfolio->getById($command->getChannelId()); $campaign = $this->campaignRepository->load( $command->getCampaignId() ); $campaign->addChannel($channel, $this->aidaMetricsCalculator); $this->campaignRepository->save($campaign); }

74. // CampaignCommandHandler.php public function handleAddChannelToCampaignCommand( AddChannelToCampaignCommand $command ) { $channel

    = $this->portfolio->getById($command->getChannelId()); $campaign = $this->campaignRepository->load( $command->getCampaignId() ); $campaign->addChannel($channel, $this->aidaMetricsCalculator); $this->campaignRepository->save($campaign); }

75. // CampaignCommandHandler.php public function handleAddChannelToCampaignCommand( AddChannelToCampaignCommand $command ) { $channel

    = $this->portfolio->getById($command->getChannelId()); $campaign = $this->campaignRepository->load( $command->getCampaignId() ); $campaign->addChannel($channel, $this->aidaMetricsCalculator); $this->campaignRepository->save($campaign); }

76. // Campaign.php public function addChannel( ChannelInPortfolio $channel, AidaMetricsCalculator $aidaMetricsCalculator )

    { $this->guardCannotModifyOrderedCampaign(); $channelInCampaign = new ChannelInCampaign(/* $args */); $channels = array_merge($this->channels, [$channelInCampaign]); $campaignPrice = $this->calculateCampaignPrice($channels); $estimatedOutcome = $this->calculateEstimatedOutcome($channels); $this->apply(new ChannelAddedToCampaign( $this->id, $channelInCampaign, $campaignPrice, $estimatedOutcome )); }

77. // Campaign.php public function addChannel( ChannelInPortfolio $channel, AidaMetricsCalculator $aidaMetricsCalculator )

    { $this->guardCannotModifyOrderedCampaign(); $channelInCampaign = new ChannelInCampaign(/* $args */); $channels = array_merge($this->channels, [$channelInCampaign]); $campaignPrice = $this->calculateCampaignPrice($channels); $estimatedOutcome = $this->calculateEstimatedOutcome($channels); $this->apply(new ChannelAddedToCampaign( $this->id, $channelInCampaign, $campaignPrice, $estimatedOutcome )); }

78. // Campaign.php public function addChannel( ChannelInPortfolio $channel, AidaMetricsCalculator $aidaMetricsCalculator )

    { $this->guardCannotModifyOrderedCampaign(); $channelInCampaign = new ChannelInCampaign(/* $args */); $channels = array_merge($this->channels, [$channelInCampaign]); $campaignPrice = $this->calculateCampaignPrice($channels); $estimatedOutcome = $this->calculateEstimatedOutcome($channels); $this->apply(new ChannelAddedToCampaign( $this->id, $channelInCampaign, $campaignPrice, $estimatedOutcome )); }

79. // Campaign.php protected function applyChannelAddedToCampaign( ChannelAddedToCampaign $event ) { $this->channels[]

    = $event->getChannel(); $this->campaignPrice = $event->getCampaignPrice(); $this->estimatedOutcome = $event- >getEstimatedOutcomeOfCampaign(); $this->isCustomized = true; }

80. How do we get a listing of the campaigns?

81. We'll create a read model

82. Campaign Event Store Campaign Created For Target Group

83. Campaign Event Store Campaign Created For Target Group Read Model

    Store

84. Campaign Event Store Campaign Created For Target Group Read Model

    Store MySQL Read Model Store Redis

85. Saving the events // Broadway: EventSourcingRepository.php public function save(AggregateRoot $aggregate)

    { // maybe we can get generics one day.... ;) Assert::isInstanceOf($aggregate, $this->aggregateClass); $eventStream = $aggregate->getUncommittedEvents(); $this->eventStore->append( $aggregate->getAggregateRootId(), $eventStream ); $this->eventBus->publish($eventStream); }

86. Creating read models // CampaignOverviewProjector.php public function applyCampaignCreatedForTargetGroup( CampaignCreatedForTargetGroup $event,

    DomainMessage $message ) { $this->repository->save(new CampaignOverview( $event->getCampaignId(), $event->getCompanyId(), $event->getTargetGroup()->getJobTitle(), $message->getRecordedOn() )); }

87. Creating read models // CampaignOverviewProjector.php public function applyCampaignCreatedForTargetGroup( CampaignCreatedForTargetGroup $event,

    DomainMessage $message ) { $this->repository->save(new CampaignOverview( $event->getCampaignId(), $event->getCompanyId(), $event->getTargetGroup()->getJobTitle(), $message->getRecordedOn() )); }

88. Creating read models // CampaignOverviewProjector.php public function applyCampaignCreatedForTargetGroup( CampaignCreatedForTargetGroup $event,

    DomainMessage $message ) { $this->repository->save(new CampaignOverview( $event->getCampaignId(), $event->getCompanyId(), $event->getTargetGroup()->getJobTitle(), $message->getRecordedOn() )); }

89. Creating read models // CampaignDetailsProjector.php public function applyCampaignCreatedForTargetGroup( CampaignCreatedForTargetGroup $event

    ) { $campaignDetails = new CampaignDetails( $event->getCampaignId(), $event->getRecruiterId(), $event->getTargetGroup(), $event->getCompanyId(), $event->getRecommendedChannels()->toArray() ); $campaignDetails->updatePrice($event->getCampaignPrice()); $campaignDetails->updateEstimatedOutcome( $event->getEstimatedOutcome() ); $this->repository->save($campaignDetails); }

90. Creating read models // CampaignDetailsProjector.php public function applyCampaignCreatedForTargetGroup( CampaignCreatedForTargetGroup $event

    ) { $campaignDetails = new CampaignDetails( $event->getCampaignId(), $event->getRecruiterId(), $event->getTargetGroup(), $event->getCompanyId(), $event->getRecommendedChannels()->toArray() ); $campaignDetails->updatePrice($event->getCampaignPrice()); $campaignDetails->updateEstimatedOutcome( $event->getEstimatedOutcome() ); $this->repository->save($campaignDetails); }

91. Creating read models // CampaignDetailsProjector.php public function applyCampaignCreatedForTargetGroup( CampaignCreatedForTargetGroup $event

    ) { $campaignDetails = new CampaignDetails( $event->getCampaignId(), $event->getRecruiterId(), $event->getTargetGroup(), $event->getCompanyId(), $event->getRecommendedChannels()->toArray() ); $campaignDetails->updatePrice($event->getCampaignPrice()); $campaignDetails->updateEstimatedOutcome( $event->getEstimatedOutcome() ); $this->repository->save($campaignDetails); }

92. Using the history to your advantage

93. Discard Customizations

94. // Campaign.php protected function applyCampaignCreatedForTargetGroup( CampaignCreatedForTargetGroup $event ) { $this->id

    = $event->getCampaignId(); $this->channels = $event->getRecommendedChannels()->toArray(); $this->campaignPrice = $event->getCampaignPrice(); $this->estimatedOutcome = $event->getEstimatedOutcome(); $this->targetGroup = $event->getTargetGroup(); $this->setInitialRecommendation( $this->channels, $this->estimatedOutcome, $this->campaignPrice ); }

95. // Campaign.php public function discardCustomization() { $this->guardCannotModifyOrderedCampaign(); if ($this->isCustomized ===

    false) { return; } $this->apply(new CustomizationDiscarded( $this->id, $this->initialRecommendation['channels'], $this->initialRecommendation['campaignPrice'], $this->initialRecommendation['estimatedOutcome'] )); }

96. // Campaign.php protected function applyCustomizationDiscarded( CustomizationDiscarded $event ) { $this->channels

    = $event->getRecommendedChannels(); $this->campaignPrice = $event->getCampaignPrice(); $this->estimatedOutcome = $event->getEstimatedOutcome(); $this->isCustomized = false; }

97. Where are the tests?

98. Scenario testing

99. Given When Then

100. Campaign Created For Target Group Add Channel To Campaign Channel

     Added To Campaign

101. $channelId = new ChannelId('42'); $this->scenario ->given([ new CampaignCreatedForTargetGroup( $this->campaignId, //

     $args ) ]) ->when(new AddChannelToCampaignCommand( $this->campaignId, $channelId )) ->then([ new ChannelAddedToCampaign( $this->campaignId, $this->stubbedChannel($channelId), new CampaignPrice(Money::EUR(40000), Money::EUR(0)), new EstimatedOutcome(/* snip */) ) ]);

102. $channelId = new ChannelId('42'); $this->scenario ->given([ new CampaignCreatedForTargetGroup( $this->campaignId, //

     $args ) ]) ->when(new AddChannelToCampaignCommand( $this->campaignId, $channelId )) ->then([ new ChannelAddedToCampaign( $this->campaignId, $this->stubbedChannel($channelId), new CampaignPrice(Money::EUR(40000), Money::EUR(0)), new EstimatedOutcome(/* snip */) ) ]);

103. $channelId = new ChannelId('42'); $this->scenario ->given([ new CampaignCreatedForTargetGroup( $this->campaignId, //

     $args ) ]) ->when(new AddChannelToCampaignCommand( $this->campaignId, $channelId )) ->then([ new ChannelAddedToCampaign( $this->campaignId, $this->stubbedChannel($channelId), new CampaignPrice(Money::EUR(40000), Money::EUR(0)), new EstimatedOutcome(/* snip */) ) ]);

104. $channelId = new ChannelId('42'); $this->scenario ->given([ new CampaignCreatedForTargetGroup( $this->campaignId, //

     $args ) ]) ->when(new AddChannelToCampaignCommand( $this->campaignId, $channelId )) ->then([ new ChannelAddedToCampaign( $this->campaignId, $this->stubbedChannel($channelId), new CampaignPrice(Money::EUR(40000), Money::EUR(0)), new EstimatedOutcome(/* snip */) ) ]);

105. Alternative scenario test $channelId = new ChannelId('42'); $this->scenario ->given([ new

     CampaignCreatedForTargetGroup( $this->campaignId, // $args ) ]) ->when(function (Campaign $campaign) { $campaign->addChannel($channelId) }) ->then([ new ChannelAddedToCampaign( $this->campaignId, $this->stubbedChannel($channelId), new CampaignPrice(Money::EUR(40000), Money::EUR(0)), new EstimatedOutcome(/* snip */) ) ]);

106. You can do something similar to test your read models

107. Using the history to correct mistakes

108. Replaying events to fix read models

109. Replaying events to create new read models

110. Very powerful

111. Very tricky

112. Replaying events Event Store Read Model Store e v e

     n t b u s Read Model Store

113. Replaying events Event Store Read Model Store e v e

     n t b u s Email Notification Processor

114. Find a solution for your specific problem

115. Questions? joind.in/16227 freenode: #qandidate github.com/qandidate-labs/broadway

116. Symfony User Group The Renaissance 10 december http://www.meetup.com/Symfony-User-Group-NL