On to choreography 2.0!

@martinschimak -> May 29th, 2019 -> µCon London

Transcript

1. On to choreography 2.0! May 29th, 2019 > London > µCon @martinschimak
2. "Everything fails all the time” -- Werner Vogels, CTO Amazon (since 2005) @martinschimak
3. plexiti.com @martinschimak
5. Vodafone, Liongate & WDW in Berlin 2018
6. Event-driven architecture is NOT about technology
7. Eric Evans, 2004 Transactions & Asynchrony Boundaries & Autonomy Language & Context and much more
8. 2004 2006 2008 2018 2010 2012 2014 2016 ? @martinschimak DDD is NOT a book
9. Distributed Transactions? Missing and Duplicates? Naïve Eventual Consistency? Overly Central Controllers? Temporal Coupling? Sea of Events? Sharing Everything? Unidirectional Event Chains? Overly Vertical Decomposition? Central Reactors? Implicit Flows?
10. Hardware account #123 Book 10£ for a new mouse Acc. #123 increased by 10£ Intention Outcome Actor Behavior Always! Bookkeeping system Example 1: Simple Profit and Loss Accounting @martinschimak
11. Book 10£ for a new mouse Hardware account #123 Acc. #123 increased by 10£ Consistency Boundary Cash account #654 Example 2: Double Entry Accounting Intention Outcome Actor Behavior Acc. #654 decreased by 10£ Provided balance > = 0 Nothing changed or Accounting Service @martinschimak
12. Potentially dangerous pattern Book 10£ for a new mouse Hardware account #123 Acc. #123 increased by 10£ Cash account #654 Example 3: Double Entry Accounting Intention Outcome Actor Behavior Acc. #654 decreased by 10£ Always! Provided balance > = 0 Nothing changed or Consistency Boundary Accounting Service Network Boundary
13. ”Entities and Activities”
14. Intention Actor Book 10£ for a new mouse Hardware account #123 Consistency Boundary Entity & its activities @martinschimak
15. Intention Outcome Actor Book 10£ for a new mouse Hardware account #123 Consistency Boundary Cash account #654 Acc. #123 increased by 10£ Acc. #654 decreased by 10£ Eventual Consistency Async after persistence @martinschimak
16. Events are safely recorded historical facts Amount credited { "name":"AmountCredited", "id":{ … }, "fact":{ "amount":10 … } } @martinschimak
17. Hardware account #123 Aggregate Boundary Acc. #123 increased by 10£ Persisting the (last) state When persisting the structural state an event can be a transactional “by- product” of persisting a new state! The aggregate consistency is e.g. protected by transactions and optimistic locking @martinschimak
18. Hardware account #123 Aggregate Boundary Acc. #123 increased by 10£ Persisting (all) the change The aggregate consistency is e.g. protected by append only logs guaranteeing unique ids With Event Sourcing events are the only persistent “source of truth”, state is dynamically derived in memory Acc. #123 increased by 10£ Acc. #123 increased by 10£ @martinschimak
19. Distributed Transactions? Missing and Duplicates? Naïve Eventual Consistency? Overly Central Controllers? Temporal Coupling? Sea of Events? Sharing Everything? Unidirectional Event Chains? Overly Vertical Decomposition? Central Reactors? Missing and Duplicates? Naïve Eventual Consistency? Overly Central Controllers? Temporal Coupling? Sea of Events? Sharing Everything? Unidirectional Event Chains? Overly Vertical Decomposition? Central Reactors? Eventually Consistent eactions Implicit Flows?
20. Intention Outcome Actor Book 10£ for a new mouse Hardware account #123 Cash account #654 Acc. #123 increased by 10£ Acc. #654 decreased by 10£ Leo Actor Eventual Consistency @martinschimak
21. Intention Outcome Actor Book 10£ for a new mouse Hardware account #123 Cash account #654 Acc. #123 increase requested Acc. #654 decreased by 10£ Leo Actor Eventual Consistency Acc. #123 increased by 10£ @martinschimak
22. Intention Outcome Actor Book 10£ for a new mouse Hardware account #123 Cash account #654 Acc. #123 increase requested Acc. #654 decrease rejected Leo Actor Eventual Consistency Acc. #123 increase rejected @martinschimak
23. ”Fine-grained workflow with entities as the participants”
24. Missing and Duplicates? Naïve Eventual Consistency? Overly Central Controllers? Temporal Coupling? Sea of Events? Sharing Everything? Unidirectional Event Chains? Overly Vertical Decomposition? Central Reactors? Eventually Consistent eactions Missing and Duplicates? Overly Central Controllers? Temporal Coupling? Sea of Events? Sharing Everything? Unidirectional Event Chains? Overly Vertical Decomposition? Central Reactors? Eventually Consistent eactions Event Driven Workflows Implicit Flows?
25. Intention Actor Book 10£ for a new mouse Hardware account #123 Cash account #654 Book 10£ for a new mouse Sync. within orig. request Network Boundary Consistency Boundary Example 3: Double Entry Accounting Potentially dangerous pattern
26. Intention Outcome Actor Book 10£ for a new mouse Hardware account #123 Consistency Boundary Cash account #654 Acc. #123 increased by 10£ Acc. #654 decreased by 10£ Eventual Consistency Immutability and Idempotency @martinschimak
27. Missing and Duplicates? Overly Central Controllers? Temporal Coupling? Sea of Events? Sharing Everything? Unidirectional Event Chains? Overly Vertical Decomposition? Central Reactors? Eventually Consistent eactions Event Driven Workflows Overly Central Controllers? Temporal Coupling? Sea of Events? Sharing Everything? Unidirectional Event Chains? Overly Vertical Decomposition? Central Reactors? Eventually Consistent eactions Event Driven Workflows Immutability and Idempotency Implicit Flows?
28. Temporal coupling with queries Intention Actor Order Payment Inventory Shipment Resource Order Summary Request Request Request Request Potentially dangerous pattern Eventual Consistency Customer Customer
29. @martinschimak Intention Outcome Actor Resource Knowing about change early on Order Payment Inventory Shipment Order Summary Eventual Consistency Customer Payment retrieved Goods fetched Goods shipped
30. Overly Central Controllers? Temporal Coupling? Sea of Events? Sharing Everything? Unidirectional Event Chains? Overly Vertical Decomposition? Central Reactors? Eventually Consistent eactions Event Driven Workflows Immutability and Idempotency Overly Central Controllers? Sea of Events? Sharing Everything? Unidirectional Event Chains? Overly Vertical Decomposition? Central Reactors? Eventually Consistent eactions Event Driven Workflows Immutability and Idempotency Knowing Change Early On Implicit Flows?
31. Marketing Promote customer Intention Payment Retrieve payment Intention Intention Actor Intention Fulfillment Shipment Ship goods Intention Overly central control? Potentially dangerous pattern
32. @martinschimak Order placed Goods shipped Payment retrieved More decentral choreography? Payment Marketing Shipment Order Intention Actor Outcome
33. Intention Outcome @martinschimak
34. Overly Central Controllers? Sea of Events? Sharing Everything? Unidirectional Event Chains? Overly Vertical Decomposition? Central Reactors? Eventually Consistent eactions Event Driven Workflows Immutability and Idempotency Knowing Change Early On Sea of Events? Sharing Everything? Unidirectional Event Chains? Overly Vertical Decomposition? Central Reactors? Eventually Consistent eactions Event Driven Workflows Immutability and Idempotency Knowing Change Early On More Decentral Choreography Implicit Flows?
35. Bounded Context is not equal to Deployment Unit @martinschimak
36. Make the context boundary explicit Amount credited { "context": "Bookkeeping" "name":"AmountCredited", "id":{ … }, … } @martinschimak Delimiting bounded contexts on moving parts ... makes sense both 1. across multiple fine-grained deployment units <= bounded context 2. within a coarse-grained monolith deployment unit > = bounded context
37. Sea of Events? Sharing Everything? Unidirectional Event Chains? Overly Vertical Decomposition? Central Reactors? Eventually Consistent eactions Event Driven Workflows Immutability and Idempotency Knowing Change Early On More Decentral Choreography Sharing Everything? Unidirectional Event Chains? Overly Vertical Decomposition? Central Reactors? Eventually Consistent eactions Event Driven Workflows Immutability and Idempotency Knowing Change Early On More Decentral Choreography Explicitly Bounded Contexts Implicit Flows?
38. Make the visibility boundary explicit @martinschimak Amount credited Internal Event Bus if internal if public Public Event Bus Head over to Mathias Verraes' blog!
39. For "public" events, consider to share "less" @martinschimak Account closed if public { "context": "BookKeeping" "name":"AccountClosed", "id":{ … } } Public integration events. Consider - a dedicated bounded context - explicit public event properties - coarse-grained summary events Public Event Bus Head over to Mathias Verraes' blog!
40. Sharing Everything? Unidirectional Event Chains? Overly Vertical Decomposition? Central Reactors? Eventually Consistent eactions Event Driven Workflows Immutability and Idempotency Knowing Change Early On More Decentral Choreography Explicitly Bounded Contexts Unidirectional Event Chains? Overly Vertical Decomposition? Central Reactors? Eventually Consistent eactions Event Driven Workflows Immutability and Idempotency Knowing Change Early On More Decentral Choreography Explicitly Bounded Contexts Explicit Visibility Layers Implicit Flows?
41. Auth "Central reactors" Asynchronous Bus Actor Outcome Account closing requested Account closing requested Account closing requested Account closing requested Account closed Potentially dangerous pattern Order placed Payment Operation authorized Payment retrieved
42. Payment Actor Outcome Binding to outcomes Receiver binds to language defined by sender! Order Order placed Language Context of Sender @martinschimak
43. Potentially dangerous pattern Asynchronous Bus Actor "Intentions in disguise" Payment Outcome { "context": "Order" "name":"PaymentRequired", "id":{ … } } Order Payment required Language Context of Sender Billable Order generated Order placed Billable Order generated
44. Asynchronous Bus Actor Intention Explicit intentions Payment { "context": "Payment" "name":"RetrievePayment", "id":{ … } … } Language Context of Receiver Order Retrieve payment @martinschimak
45. Asynchronous Bus Actor Intention Promising reactor Payment Language Context of Receiver Payment retrieved Outcome Retrieve payment { "context": "Payment" "name":"PaymentRetrieved", "id":{ … } … } Order @martinschimak
46. Asynchronous Bus Actor Intention Payment Language Context of Receiver Payment retrieved Outcome Retrieve payment Payment failed Outcome may be a business success or failure Promising? Yes, some outcome! Order @martinschimak
47. Asynchronous Bus Actor Intention Payment Language Context of Receiver Payment retrieved Outcome Retrieve payment Payment failed API modelling a reactive promise @martinschimak
48. Asynchronous Bus Actor Intention Payment Language Context of Receiver Payment retrieved Outcome Retrieve payment Payment failed Explicit intention { "context":"Payment" "type":"Intention", "name":"RetrievePayment", "id":{ … } … } @martinschimak
49. Asynchronous Bus Actor Intention Payment Language Context of Receiver Payment retrieved Outcome Retrieve payment Payment failed Explicit outcome { "context":"Payment" "type":"Success", "name":"PaymentRetrieved", "intention_id": { … } "id":{ … } … } @martinschimak
50. Asynchronous Bus Payment Language Context of Receiver Payment retrieved Retrieve payment Payment failed Full transparency of reactive promises o Location o Composition o Execution Time @martinschimak
51. Unidirectional Event Chains? Overly Vertical Decomposition? Central Reactors? Eventually Consistent eactions Event Driven Workflows Immutability and Idempotency Knowing Change Early On More Decentral Choreography Explicitly Bounded Contexts Explicit Visibility Layers Unidirectional Event Chains? Overly Vertical Decomposition? Eventually Consistent eactions Event Driven Workflows Immutability and Idempotency Knowing Change Early On More Decentral Choreography Explicitly Bounded Contexts Explicit Visibility Layers Reactive Promises Implicit Flows?
52. @martinschimak Order placed Goods shipped Payment retrieved "Implicit, unidirectional event chain" Payment Marketing Shipment Order Actor Outcome Potentially dangerous pattern "Premium customers pay after shipment" "Check identity for new customers"
53. To insert a link you must open two, to move a link even more
54. Eventual consistency Binding to intentions Order Retrieve payment Sender binds to language defined by receiver! Data Flow Direction Payment Sender Receiver Language Context of Receiver Sender decides that a payment needs to be retrieved! depends on @martinschimak
55. Payment Binding to outcomes Receiver binds to language defined by sender! Order Order placed Language Context of Sender Eventual consistency Data Flow Direction depends on Sender Receiver Receiver decides that a payment needs to be retrieved! @martinschimak
56. @martinschimak Order placed Goods shipped Payment retrieved Explicit, bidirectional choreography Payment Marketing Shipment Fulfillment "Premium customers pay after shipment" "Check identity for new customers" Actor Intention Outcome Retrieve payment Ship goods
57. Unidirectional Event Chains? Overly Vertical Decomposition? Eventually Consistent eactions Event Driven Workflows Immutability and Idempotency Knowing Change Early On More Decentral Choreography Explicitly Bounded Contexts Explicit Visibility Layers Reactive Promises Overly Vertical Decomposition? Eventually Consistent eactions Event Driven Workflows Immutability and Idempotency Knowing Change Early On More Decentral Choreography Explicitly Bounded Contexts Explicit Visibility Layers Reactive Promises Bidirectional Choreography Implicit Flows?
58. Shipment Ship goods Intention Intention Actor Intention Payment Fulfillment Outcome Payment retrieved Retrieve payment Intention Goods shipped Outcome Outcome Vertical decomposition: intention + outcomes Intentions travel vertically downwards Outcomes travel vertically upwards
59. Shipment Ship goods Intention Payment Outcome Payment retrieved Retrieve payment Intention Goods shipped Outcome Marketing Customer promoted Intention Actor Intention Outcome Horizontal decomposition: mainly outcomes Fulfillment Mainly outcomes travel horizontally intentions may travel horizontally, e.g. for monitoring purposes
60. Balanced decomposition Vertical (functional) decomposition Horizontal (capability) decomposition may foster too much hierarchy may foster too much implicitness A balanced decomposition leverages both options as appropriate
61. Downwards & (seldom) sidewards Upwards & (often) sidewards @martinschimak
62. Overly Vertical Decomposition? Eventually Consistent eactions Event Driven Workflows Immutability and Idempotency Knowing Change Early On More Decentral Choreography Explicitly Bounded Contexts Explicit Visibility Layers Reactive Promises Bidirectional Choreography Eventually Consistent eactions Event Driven Workflows Immutability and Idempotency Knowing Change Early On More Decentral Choreography Explicitly Bounded Contexts Explicit Visibility Layers Reactive Promises Bidirectional Choreography Balanced Decomposition Implicit Flows?
63. Business Function Retrieve payment Credit card charged create on Payment Order placed on Charge credit card create Credit Card create on Payment create @martinschimak Delegation of (part of) promise Execution of promise Interpretation of observation on Consistency Boundaries Payment retrieved on Fulfillment Consequence of observation Vertical decomposition
64. Ship goods create on create @martinschimak Interpretation of observation Payment retrieved Fulfillment Consequence of observation Payment Promote customer on Promotions Consequence of observation create Horizontal decomposition Business Capability Consistency Boundaries
65. Aggregate Intention Outcome create on Aggregate Outcome on Intention create Aggregate create on Aggregate create @martinschimak Consequence of observation Delegation of (part of) promise Execution of promise Interpretation of observation on Consistency Boundaries Intention on Aggregate Consequence of observation create Asynchronous and reactive aggregates
66. Outcome Intention Aggregate create @martinschimak Execution of promise A bit often discussed Aggregates handling commands (intention)
67. Aggregate Intention create on Aggregate Outcome on Intention create Aggregate create @martinschimak Consequence of observation Delegation of (part of) promise Interpretation of observation Consistency Boundaries Intention on Aggregate Consequence of observation create A bit seldom discussed Outcome on Aggregates handling sagas and processes
68. ”Fine-grained workflow with entities as the participants”
69. Actor Intention Consistency Boundary create Outcome create Intention Outcome either or and/or How does look like in domain code? Behavior Behavior @martinschimak
70. Fulfillment Retrieve payment Consistency Boundary create Order placed Consequence of observation fun handle(outcome: OrderPlaced): Fact { // Consequence decision logic here return RetrievePayment( /* Constructor params here */ ) } @martinschimak
71. Payment Charge credit card Consistency Boundary create Retrieve payment Delegation of (part of) promise fun handle(intention: RetrievePayment): Fact { // Delegation decision logic here return ChargeCreditCard( /* Constructor params here */ ) } @martinschimak
72. Credit Card Consistency Boundary Credit card charged create Charge credit card Execution of promise fun handle(intention: ChargeCreditCard): Fact { // (in real life execution would still be composite, see later) return CreditCardCharged( /* Constructor params here */ ) } @martinschimak
73. Payment Consistency Boundary Payment retrieved create Credit card charged Interpretation of observation fun handle (success: CreditCardCharged): Fact { // Interpretation logic here return PaymentRetrieved( /* Constructor params here */ ) } @martinschimak
74. Payment Eventually Consistent Saga or Business Process Payment retrieved create Retrieve payment Composite execution of promise How could look like in domain code? Composite behavior @martinschimak
75. Handlers implementing state transitions class Payment { fun handle(fact: RetrievePayment): Fact { // Business decision logic here return ChargeCreditCard( /* Constructor params here */ ) } fun handle(fact: CreditCardCharged): Fact { // Business decision logic here return PaymentRetrieved( /* Constructor params here */ ) } fun handle(fact: CreditCardFailed): Fact { // … } @martinschimak
76. Make the implicit saga or process flow explicit @martinschimak
77. A typed DSL declaring a saga or process flow define <Payment> { on message(RetrievePayment::class) … execute service { create intention(ChargeCreditCard::class) by { … } on message(CreditCardCharged::class) having "reference" match { paymentId } … } create success(PaymentRetrieved::class) by { PaymentRetrieved(paymentId, covered) } } @martinschimak
78. Eventually Consistent eactions Event Driven Workflows Immutability and Idempotency Knowing Change Early On More Decentral Choreography Explicitly Bounded Contexts Explicit Visibility Layers Reactive Promises Bidirectional Choreography Balanced Decomposition Explicit Flows
79. A typed DSL declaring a saga or process flow + Clean, readable code for composite saga/process behavior + Options to execute declared state machine with different means + Automatic generation of (living, graphical) documentation + Everything needed in one place (flow, data, correlation, …) MVP coming soon! https://github.com/factdriven @martinschimak
80. medium.com/plexiti @martinschimak Thank you!