Event Sourcing, CQRS and Best Practices

This is the presentation I gave during the Leipzig Software Development Meetup on May 3, 2017. See https://www.meetup.com/de-DE/Leipzig-Software-Development-Meetup/events/238405300/.

Transcript

1. Event Sourcing, CQRS and Best Practices 2017-05-03 Leipzig Software Development Meetup
2. Email: torsten.heinrich@trivago.com Twitter: https://twitter.com/heinrichtorsten Senior Software Engineer trivago Leipzig Torsten Heinrich
3. Part 1 Event Sourcing Capture all changes to an application state as sequence of events
4. 4 Definition SELECT INSERT UPDATE DELETE How many of these statements are actually deleting data?
5. 5 Definition https://twitter.com/jboner/status/857173298811207680
6. 6 Definition
7. 7 Definition Event Sourcing ensures that all changes to application state are stored as a sequence of events. Not just can we query these events, we can also use the event log to reconstruct past states, and as a foundation to automatically adjust the state to cope with retroactive changes. (Fowler, 2005)
8. 8 Sequence of events
9. 9 Sequence of events
10. 10 Sequence of events • Events are first-class citizens • Events track a change in state • The order of events is essential • Track the version of an event, track when an event occurred (VCS) • Events are append-only • History must not be re-written • Use compensating events to correct issues (AccountBilled > AccountRefunded)
11. 11 Domain Driven Design • Event sourcing and Domain Driven Design complement each other well • Events should be part of the ubiquitous language • Domain events are an ideal stepping stone • Aggregates and Entities are ideal candidates • No setters allowed
12. 12 Aggregates public function bill(array $subscriptions, System $system) : Invoice { // check invariants $this->ensureAccountActive(); $this->ensureSubscriptionsExist($subscriptions); $this->ensureSubscriptionsOfAccount($subscriptions); $lineItems = []; foreach ($subscriptions as $subscription) { $lineItems[] = InvoiceLineItem::createFromSubscription($subscription); } // create a new invoice $invoice = new Invoice(Uuid::uuid4(), $this->id(), $lineItems, $this->balance()->getCurrency()); // adjust the balance of the account $this->balance = $this->balance()->subtract($invoice->amount()); $this->updatedBy = $system; return $invoice; }
13. 13 Aggregates public function bill(array $subscriptions, System $system) : Invoice { ... $balance = $this->balance()->subtract($invoice->amount()); $event = new AccountBilled($this->id(), $invoice->id(), $balance, $system, new \DateTimeImmutable()); $this->applyUncommittedEvent($event); return $invoice; } private function onAccountBilled(AccountBilled $event) { // adjust the balance of the account $this->balance = $event->balance(); $this->updatedBy = $event->system(); }
14. 14 Events • Include the name of the affected aggregate/entity, e.g. Account • If necessary, include the name of the sub-entity/value object, e.g. AccountInvoice • Describe what exactly has happened • Use past tense, e.g. AccountInvoiceGenerated • Include all the properties modifying the state of the aggregate/entity in the payload of the event • Good: AccountCreated, AccountInvoiceGenerated, CustomerPaymentVerified • Bad: DomainObjectUpdated, AccountInserted, AccountInvoiceEvent
15. 15 Concurrency
16. 16 Concurrency • Pessimistic locking vs. Optimistic locking • Pessimistic locking prevents any concurrent access • Safe, but slow • Optimistic locking tries to detect concurrent access • Keep track of the version of an aggregate • Throw an exception if there’s a mismatch between the aggregate and the event version • Further conflict resolution possible
17. 17 Performance • Event stores are (usually) pretty fast • If possible, use proven technology or a NoSQL database • https://geteventstore.com • https://www.mongodb.com • If possible, use one collection/stream per aggregate instance • Think about sharding • Serializers can be a bottleneck • Performance can degrade over time
18. 18 Snapshots
19. 19 Snapshots • Snapshots capture the current state of an aggregate • Create them at regular intervals • Can be created on-the-fly using a trigger condition • Can be created in the background using a scheduled job • Start without it and see how far you can get • Once necessary, start with larger intervals
20. 20 Versioning • Don’t do it! • Avoid, if possible! • Changing requirements, deeper insights, refactoring • Events are immutable • Multiple solutions: • Replace all the previous versions • Upcasting
21. 21 Upcasting public function upcast(DomainEvent $event) : array { // we're only interesting in the first version if (!$event instanceof AccountClosed) { return []; } // transform to the current version $upcasted = new AccountClosedV2( $event->accountId(), ‘no reason given', $event->status(), $event->customer(), $event->occurredOn() ); return [$upcasted]; }
22. 22 Upcasting • Responsibility of the event store/infrastructure • Older versions of an event can be ignored completely • Performance can degrade over time • Increases complexity
23. 23 Summary • Audit trial • Debugging, tracing issues is easy • No ORM required • Getting Event Sourcing right (the first time) is not an easy task • Invest in monitoring, logging • KPI: # of events, # of events/s, # of events per aggregate, … • It requires a different mind set • It might require different infrastructure • It might require refactoring your business logic
24. 24 Summary
25. Part 2 CQRS Use a different model to update information than to read information
26. 26 Definition CQRS stands for Command Query Responsibility Segregation. It's a pattern that I first heard described by GregYoung. At its heart is the notion that you can use a different model to update information than the model you use to read information. For some situations, this separation can be valuable, but beware that for most systems CQRS adds risky complexity. (Fowler, 2011)
27. 27 Definition • Command side with one write model • Emits events when application state changes • Events are dispatched asynchronously (RabbitMQ, ZeroMQ) • Query side with multiple different read models (MySQL, MongoDB, ElasticSearch, …) • Be aware of eventual consistency
28. 28 Definition
29. 29 Commands class BillAccountCommand { /** @var string */ private $accountId; /** @var string */ private $systemId; public function __construct(string $accountId, string $systemId) { $this->accountId = $accountId; $this->systemId = $systemId; } ... }
30. 30 Commands public function billAccount(BillAccountCommand $command) { $accountId = Uuid::fromString($command->accountId()); $subscriptions = $this->subscriptionRepository->subscriptionsOfAccount($accountId); $system = $this->userRepository->systemOfId(Uuid::fromString($command->systemId())); $account = $this->accountRepository->accountOfId($accountId); $account->bill($subscriptions, $system); $this->accountRepository->save($account); }
31. 31 Command Sourcing • Commands and events have a different intend: • Command: state -> command -> event • Event: state -> event -> state • Commands have to be persisted before they are executed • Might not be side-effect free
32. 32 Projections public function onAccountBilled(AccountBilled $event) { $update = [ '$set' => [ 'balance' => $event->balance()->getAmount(), 'updatedBy' => $event->system()->id()->toString(), 'updatedOn' => new UTCDateTime($event->occurredOn()->format('U') * 1000), ] ]; $this->client() ->selectCollection($this->database(), $this->collection()) ->updateOne(['_id' => $event->accountId()->toString()], $update); }
33. 33 Projections public function accountDataOfId(string $id) :? AccountData { $document = $this->client() ->selectCollection($this->database(), $this->collection()) ->findOne(['_id' => $id]); if (!$document) { return null; } $accountData = new AccountData( $document->_id, $document->balance, $document->currency, $document->status, $document->createdBy, $document->createdOn->toDateTime(), $document->updatedBy, $document->updatedOn->toDateTime() ); return $accountData; }
34. 34 Summary • CQRS is easy without Event Sourcing, Event Sourcing is hard without CQRS • CQRS offers good Performance/Scalability • Complexity and learning curve are challenges • Separation of concerns should be a side effect • Only apply CQRS where it’s most valuable
35. 35 Frameworks • Broadway (PHP) https://github.com/broadway/broadway • Prooph (PHP) https://github.com/prooph, http://getprooph.org • Axon (Java + Scala) http://www.axonframework.org • Akka Persistence (Java + Scala) http://doc.akka.io/docs/akka/current/scala/persistence.html • Eventuate (Java + Scala) http://rbmhtechnology.github.io/eventuate/
36. Thank you! https://github.com/torstenheinrich/lej-es-sample https://speakerdeck.com/theinrich/event-sourcing-cqrs-and-best-practices