CQRS & Event Sourcing in the wild (ScotlandPHP 2016)

CQRS & Event Sourcing in the wild (ScotlandPHP 2016)

CQRS & event sourcing are currently very popular topics in the PHP community, with good reason. However, most blogs and talks focus on the theory, simple applications or introductions to one of the frameworks currently available, not necessarily the challenges of using and maintaining it in production.

This session bridges that gap and looks at some of the pitfalls of a real-world deployment. I'll discuss topics like concurrency & scale, refactoring events and updating read models. Attend this talk to learn from my experiences and be better prepared when you face these challenges.

2f4800411154a8c66dde489448a044d2?s=128

Michiel Rook

October 29, 2016
Tweet

Transcript

  1. CQRS & EVENT SOURCING IN THE WILD Michiel Rook -

    @michieltcs
  2. ➤ Java, PHP & Scala developer ➤ Consultant, trainer, speaker

    ➤ Dutch Web Alliance ➤ make.io ➤ Maintainer of Phing ➤ @michieltcs
  3. YOU

  4. RAISE YOUR HAND IF YOU HAVE

  5. heard about CQRS / Event Sourcing RAISE YOUR HAND IF

    YOU HAVE
  6. heard about CQRS / Event Sourcing followed a tutorial, built

    a hobby project RAISE YOUR HAND IF YOU HAVE
  7. heard about CQRS / Event Sourcing followed a tutorial, built

    a hobby project used it in production RAISE YOUR HAND IF YOU HAVE
  8. TOPICS ➤ Quick recap ➤ Replays and rebuilds ➤ Event

    versioning ➤ Concurrency ➤ Scale
  9. QUICK RECAP

  10. ' Event Sourcing ensures that all changes to application state

    are stored as a sequence of events. -Martin Fowler
  11. ACTIVE RECORD VS. EVENT SOURCING Account number Balance 12345678 €€

    50,00 ... ... Money Withdrawn Account number 12345678 Amount €€ 50,00 Money Deposited Account number 12345678 Amount €€ 100,00 Account Created Account number 12345678
  12. COMMANDS TO EVENTS Deposit Money Account number 12345678 Amount €€

    100,00 class DepositMoney {
 public $accountNumber;
 public $amount;
 }
  13. COMMANDS TO EVENTS Deposit Money Account number 12345678 Amount €€

    100,00 function depositMoney(DepositMoney $command) {
 $this->apply(new MoneyDeposited(
 $command->accountNumber,
 $command->amount,
 date()));
 } command
 handler
  14. COMMANDS TO EVENTS Deposit Money Account number 12345678 Amount €€

    100,00 Money Deposited Account number 12345678 Amount €€ 100,00 class MoneyDeposited {
 public $accountNumber;
 public $amount;
 public $timestamp;
 } command
 handler
  15. AGGREGATES class BankAccount {
 public $accountNumber;
 public $balance;
 
 //

    ...
 
 public function applyMoneyDeposited(
 MoneyDeposited $event) {
 $this->balance += $event->amount;
 }

  16. AGGREGATE STATE Account number Balance 12345678 €€ 0,00 Money Withdrawn

    Account number 12345678 Amount €€ 50,00 Money Deposited Account number 12345678 Amount €€ 100,00 Account Created Account number 12345678 Account number Balance 12345678 €€ 100,00 Account number Balance 12345678 €€ 50,00
  17. CQRS + EVENT SOURCING Domain UI Event Bus Event Handlers

    Command Repository Data Layer Database Database Event Store commands events events queries DTOs Aggregates
  18. PROS AND CONS ➤ Domain fit ➤ Testing ➤ Audit

    trail ➤ Scalability ➤ Complexity ➤ Library support / maturity
  19. DISCLAIMER

  20. REPLAYS AND REBUILDS

  21. PROJECTIONS AND READ MODELS User Registered User Registered User Unregistered

    Number of active users?
  22. PROJECTIONS AND READ MODELS User Registered User Deactivated User Reactivated

    Number of active users? User Registered User Unregistered
  23. PROJECTIONS AND READ MODELS User Registered Event Handler Number of

    active users +1 User Unregistered Event Handler Number of active users -1
  24. PROJECTIONS AND READ MODELS Events Event Handler(s) Storage

  25. ELASTICSEARCH class CompanyRegistered {
 public $companyId;
 public $name;
 public $street;


    public $city;
 }
  26. ELASTICSEARCH function handleCompanyRegistered(
 CompanyRegistered $event) {
 
 $this->elasticsearch->index([
 'index' =>

    'companies',
 'type' => 'company',
 'id' => $event->companyId,
 'body' => [
 'name' => $event->name,
 'address' => [
 'street' => $event->street,
 'city' => $event->city
 ]
 ]
 ]);
 }
  27. READ MODEL UPDATES ➤ New type ➤ New structure ➤

    Based on existing events ➤ Generate from scratch?
  28. REBUILDING Stop application Remove old read model Loop over events

    Apply to read model Start application
  29. ZERO DOWNTIME Loop over existing events Apply to new read

    model Apply queued events Start using new read model New events Queue
  30. CHALLENGE: LONG RUNNING REBUILDS ➤ Alternatives: ➤ In memory ➤

    Distributed ➤ Partial ➤ Background
  31. CHALLENGE: SIDE EFFECTS User Registered User Id 123abc Email Address

    test@example.net Event Handler Exclude during replays!
  32. CHALLENGE: TRANSACTIONS Event Handler Event Handler Event Event Handler Event

    Handler ?
  33. CHALLENGE: EVENTUAL CONSISTENCY ➤ Asynchronous event handlers ➤ Reads eventually

    return the same value ➤ Compare with ACID ➤ UI?
  34. EVENT VERSIONING

  35. DILEMMA ➤ New business requirements ➤ Refactoring ➤ New view

    on events
  36. NEW EVENTS / VERSIONS ➤ No longer relevant ➤ Renamed

    ➤ Additional or renamed field(s) ➤ Too coarse, too fine
  37. SUPPORT YOUR LEGACY? ➤ Commands can be renamed ➤ Events

    are immutable ➤ Correct (incorrect) old events with new events
  38. UPCASTING Event Store UserRegistered_V1 Upcaster UserRegistered_V2 Event Handler

  39. UPCASTING class UserRegistered_V1 {
 public $userId;
 public $name;
 public $timestamp;


    }
  40. UPCASTING class UserRegistered_V2 {
 public $userId;
 public $name;
 public $date;


    }
  41. UPCASTING function upcast($event): array {
 if (!$event instanceof UserRegistered_V1) {


    return [];
 }
 return [
 new UserRegistered_V2(
 $event->userId, $event->name,
 $event->timestamp->format("Y-m-d"))
 ];
 }
  42. REWRITING HISTORY Load (subset of) events Deserialize Modify Serialize Save/replace

  43. THINGS TO BE AWARE OF Upcasting ➤ Performance ➤ Complexity

    ➤ Existing projections not automatically updated Rewriting events ➤ Running code that depends on old structure ➤ Breaking serialization ➤ Changing wrong events
  44. CONCURRENCY

  45. CONCURRENT COMMANDS Withdraw Money Account number 12345678 Amount €€ 50,00

    Deposit Money Account number 12345678 Amount €€ 100,00 ?
  46. PESSIMISTIC LOCKING Withdraw Money Account number 12345678 Amount €€ 50,00

    Deposit Money Account number 12345678 Amount €€ 100,00 Account number Balance 12345678 €€ 100,00 Account number Balance 12345678 €€ 50,00 wait for lock lock
  47. OPTIMISTIC LOCKING Withdraw Money Account number 12345678 Amount €€ 50,00

    version 1 Deposit Money Account number 12345678 Amount €€ 100,00 version 1 Account number Balance 12345678 €€ 100,00 version 2 ConcurrencyException
  48. SCALE

  49. PERFORMANCE ➤ Server ➤ Database ➤ Framework ➤ Language ➤

    Serializer
  50. STORAGE ➤ #events ➤ #aggregates ➤ #events_per_aggregate ➤ Serializer ➤

    Event payloads ➤ Costs
  51. SNAPSHOTS Events 1 Account Created 2 Money Deposited 3 Money

    Withdrawn 4 Money Deposited SNAPSHOT 5 Money Withdrawn Events 1 Account Created 2 Money Deposited 3 Money Withdrawn 4 Money Deposited 5 Money Withdrawn
  52. SHARDING ➤ Aggregate Id, Type, Event Timestamp, ... ➤ Rebalancing

    ➤ Distribution
  53. ARCHIVING EVENTS ➤ Reduce working set ➤ Inactive / deleted

    aggregates ➤ Historic / irrelevant events ➤ Cheaper storage
  54. FRAMEWORK COMPARISON Framework Upcasting Snapshots Replaying Broadway (PHP) No (PR)

    No (PR) Not in core Prooph (PHP) MessageFactory Yes, triggers on event count Example code, off line Axon (Java/Scala) Upcaster / UpcasterChain Yes, triggers on event count Yes, ReplayingCluster Akka Persistence (Java/Scala) Event Adapter Yes, decided by actor Yes
  55. QUESTIONS @michieltcs / michiel@make.io www.touchdownconsulting.nl https://joind.in/talk/584c0

  56. THANK YOU! @michieltcs / michiel@make.io www.touchdownconsulting.nl https://joind.in/talk/584c0