Pro Yearly is on sale from $80 to $50! »

CQRS & Event Sourcing in the wild (PHP Benelux 2017)

2f4800411154a8c66dde489448a044d2?s=47 Michiel Rook
January 28, 2017

CQRS & Event Sourcing in the wild (PHP Benelux 2017)

CQRS & event sourcing are currently very popular topics in the PHP community, with good reason. However, most blogs and talks focus on the theory, simple applications or introductions to one of the frameworks currently available, not necessarily the challenges of using and maintaining it in production.

This session bridges that gap and looks at some of the pitfalls of a real-world deployment. I’ll discuss topics like concurrency & scale, refactoring events and updating read models. Attend this talk to learn from my experiences and be better prepared when you face these challenges.

2f4800411154a8c66dde489448a044d2?s=128

Michiel Rook

January 28, 2017
Tweet

Transcript

  1. CQRS & EVENT SOURCING IN THE WILD Michiel Rook -

    @michieltcs
  2. ➤ Java, PHP & Scala developer ➤ Consultant, trainer, speaker

    ➤ Dutch Web Alliance ➤ make.io ➤ Maintainer of Phing ➤ @michieltcs
  3. YOU

  4. RAISE YOUR HAND IF YOU HAVE

  5. know CQRS / Event Sourcing theory RAISE YOUR HAND IF

    YOU HAVE
  6. know CQRS / Event Sourcing theory followed a tutorial, built

    a hobby project RAISE YOUR HAND IF YOU HAVE
  7. know CQRS / Event Sourcing theory followed a tutorial, built

    a hobby project used it in production RAISE YOUR HAND IF YOU HAVE
  8. TOPICS ➤ Quick recap ➤ Replays and rebuilds ➤ Event

    versioning ➤ Concurrency ➤ Scale @michieltcs
  9. QUICK RECAP

  10. ' Event Sourcing ensures that all changes to application state

    are stored as a sequence of events. -Martin Fowler
  11. ACTIVE RECORD VS. EVENT SOURCING Account number Balance 12345678 €€

    50,00 ... ... Money Withdrawn Account number 12345678 Amount €€ 50,00 Money Deposited Account number 12345678 Amount €€ 100,00 Account Created Account number 12345678 @michieltcs
  12. COMMANDS TO EVENTS Deposit Money Account number 12345678 Amount €€

    100,00 class DepositMoney {
 public $accountNumber;
 public $amount;
 } @michieltcs
  13. COMMANDS TO EVENTS Deposit Money Account number 12345678 Amount €€

    100,00 function depositMoney(DepositMoney $command) {
 $this->apply(new MoneyDeposited(
 $command->accountNumber,
 $command->amount,
 date()));
 } command
 handler @michieltcs
  14. COMMANDS TO EVENTS Deposit Money Account number 12345678 Amount €€

    100,00 Money Deposited Account number 12345678 Amount €€ 100,00 class MoneyDeposited {
 public $accountNumber;
 public $amount;
 public $timestamp;
 } command
 handler @michieltcs
  15. AGGREGATES class BankAccount {
 public $accountNumber;
 public $balance;
 
 //

    ...
 
 public function applyMoneyDeposited(
 MoneyDeposited $event) {
 $this->balance += $event->amount;
 }
 @michieltcs
  16. AGGREGATE STATE Account number Balance 12345678 €€ 0,00 Money Withdrawn

    Account number 12345678 Amount €€ 50,00 Money Deposited Account number 12345678 Amount €€ 100,00 Account Created Account number 12345678 Account number Balance 12345678 €€ 100,00 Account number Balance 12345678 €€ 50,00 @michieltcs
  17. CQRS + EVENT SOURCING Domain UI Event Bus Event Handlers

    Command Repository Data Layer Database Database Event Store commands events events queries DTOs Aggregates @michieltcs
  18. PROS AND CONS ➤ Domain fit ➤ Testing ➤ Audit

    trail ➤ Scalability ➤ Complexity ➤ Library support / maturity ➤ Infrastructure @michieltcs
  19. DISCLAIMER

  20. REPLAYS AND REBUILDS

  21. PROJECTIONS AND READ MODELS User Registered User Registered User Unregistered

    Number of active users? @michieltcs
  22. PROJECTIONS AND READ MODELS User Registered User Deactivated User Reactivated

    Number of active users? User Registered User Unregistered @michieltcs
  23. PROJECTIONS AND READ MODELS User Registered Event Handler Number of

    active users +1 User Unregistered Event Handler Number of active users -1 @michieltcs
  24. PROJECTIONS AND READ MODELS Events Event Handler(s) Storage @michieltcs

  25. ELASTICSEARCH class CompanyRegistered {
 public $companyId;
 public $name;
 public $street;


    public $city;
 } @michieltcs
  26. ELASTICSEARCH function handleCompanyRegistered(
 CompanyRegistered $event) {
 
 $this->elasticsearch->index([
 'index' =>

    'companies',
 'type' => 'company',
 'id' => $event->companyId,
 'body' => [
 'name' => $event->name,
 'address' => [
 'street' => $event->street,
 'city' => $event->city
 ]
 ]
 ]);
 } @michieltcs
  27. READ MODEL UPDATES ➤ New type ➤ New structure ➤

    Based on existing events ➤ Generate from scratch? @michieltcs
  28. REBUILDING Stop app Remove old read model Loop over events

    Apply to read model Start app @michieltcs
  29. ZERO DOWNTIME New events Queue @michieltcs Loop over existing events

    Apply to new read model Apply queued events Use new read model
  30. CHALLENGE: LONG RUNNING REBUILDS ➤ Alternatives: ➤ In memory ➤

    Distributed ➤ Partial ➤ Background @michieltcs
  31. CHALLENGE: SIDE EFFECTS User Registered User Id 123abc Email Address

    test@example.net Event Handler @michieltcs
  32. CHALLENGE: SIDE EFFECTS User Registered User Id 123abc Email Address

    test@example.net Event Handler Exclude during replays! @michieltcs
  33. CHALLENGE: TRANSACTIONS Event Handler Event Handler Event Event Handler Event

    Handler ? @michieltcs
  34. CHALLENGE: EVENTUAL CONSISTENCY ➤ Asynchronous event handlers ➤ Reads eventually

    return the same value ➤ Projections in event handlers? ➤ UI? @michieltcs
  35. EVENT VERSIONING

  36. DILEMMA ➤ New business requirements ➤ Refactoring ➤ New view

    on events @michieltcs
  37. NEW EVENTS / VERSIONS ➤ No longer relevant ➤ Renamed

    ➤ Additional or renamed field(s) ➤ Too coarse, too fine @michieltcs
  38. SUPPORT YOUR LEGACY? ➤ Commands can be renamed ➤ Events

    are immutable ➤ Correct (incorrect) old events with new events @michieltcs
  39. UPCASTING Event Store UserRegistered_V1 Upcaster UserRegistered_V2 Event Handler @michieltcs

  40. UPCASTING class UserRegistered_V1 {
 public $userId;
 public $name;
 public $timestamp;


    } @michieltcs
  41. UPCASTING class UserRegistered_V2 {
 public $userId;
 public $name;
 public $date;


    } @michieltcs
  42. UPCASTING function upcast($event): array {
 if (!$event instanceof UserRegistered_V1) {


    return [];
 }
 return [
 new UserRegistered_V2(
 $event->userId, $event->name,
 $event->timestamp->format("Y-m-d"))
 ];
 } @michieltcs
  43. VERSIONED EVENT STORE Loop over existing events Apply conversion Add

    queued events Use new event store New events Queue @michieltcs
  44. REWRITING HISTORY Load (subset of) events Deserialize Modify Serialize Save/replace

    @michieltcs
  45. THINGS TO BE AWARE OF Upcasting ➤ Performance ➤ Complexity

    Rewriting events ➤ Rewriting history ➤ Code that depends on old structure ➤ Serialization ➤ Changing wrong events @michieltcs
  46. THINGS TO BE AWARE OF Upcasting ➤ Performance ➤ Complexity

    Rewriting events ➤ Rewriting history ➤ Code that depends on old structure ➤ Serialization ➤ Changing wrong events @michieltcs Existing projections not automatically updated!
  47. CONCURRENCY

  48. CONCURRENT COMMANDS Withdraw Money Account number 12345678 Amount €€ 50,00

    Deposit Money Account number 12345678 Amount €€ 100,00 ? @michieltcs
  49. PESSIMISTIC LOCKING Withdraw Money Account number 12345678 Amount €€ 50,00

    Deposit Money Account number 12345678 Amount €€ 100,00 Account number Balance 12345678 €€ 100,00 Account number Balance 12345678 €€ 50,00 wait for lock lock @michieltcs
  50. OPTIMISTIC LOCKING Withdraw Money Account number 12345678 Amount €€ 50,00

    version 1 Deposit Money Account number 12345678 Amount €€ 100,00 version 1 Account number Balance 12345678 €€ 100,00 version 2 ConcurrencyException @michieltcs
  51. SCALE

  52. PERFORMANCE ➤ Server ➤ Database ➤ Framework ➤ Language ➤

    Serializer @michieltcs
  53. STORAGE ➤ #events ➤ #aggregates ➤ #events_per_aggregate ➤ Serializer ➤

    Event payloads ➤ Costs @michieltcs
  54. SNAPSHOTS Events 1 Account Created 2 Money Deposited 3 Money

    Withdrawn 4 Money Deposited SNAPSHOT 5 Money Withdrawn Events 1 Account Created 2 Money Deposited 3 Money Withdrawn 4 Money Deposited 5 Money Withdrawn @michieltcs
  55. SHARDING ➤ Aggregate Id, Type, Event Timestamp, ... ➤ Rebalancing

    ➤ Distribution @michieltcs
  56. ARCHIVING EVENTS ➤ Reduce working set ➤ Inactive / deleted

    aggregates ➤ Historic / irrelevant events ➤ Cheaper storage @michieltcs
  57. FRAMEWORK COMPARISON @michieltcs Upcasting Snapshots Replaying Conflict detection Broadway (PHP)

    No (PR) No (PR) Not in core Not in core Prooph (PHP) MessageFactory Yes, triggers on event count Example code, off line No Axon (Java/Scala) Upcaster / UpcasterChain Yes, triggers on event count Replaying Cluster / Tracking Processor Yes Akka Persistence (Java/Scala) Event Adapter Yes, decided by actor Yes Yes
  58. CLOSING WORDS

  59. LITERATURE

  60. QUESTIONS @michieltcs / michiel@make.io www.michielrook.nl https://joind.in/talk/1ee46

  61. THANK YOU! @michieltcs / michiel@make.io www.michielrook.nl https://joind.in/talk/1ee46