Lessons learned Implementing an Event Sourced Microservice Architecture on Azure door Cornell Knulst en Pascal Naber

devNetNoord #13

Transcript

1. Think ahead. Act now. Think ahead. Act now. Lessons learned: Implementing an event sourced, microservices architecture on Azure Pascal Naber & Cornell Knulst
2. Think ahead. Act now. Pascal Naber Cornell Knulst Cornell Knulst Pascal Naber
3. Think ahead. Act now. 2 1 3 4 Exploring the customer domain The right set of microservices Cross-cutting concerns Reliable communication in a MSA 5 Recap and Q & A
4. Think ahead. Act now. 2 1 3 4 Exploring the customer domain The right set of microservices Cross-cutting concerns 5 Recap and Q & A Reliable communication in a MSA
5. Think ahead. Act now. How to effectively Model COMPLEXITY in an AGILE world? SPEED COMPLEXITY
6. Think ahead. Act now.
7. Think ahead. Act now. Focusses on functional behavior Facts Happened in the past
8. Think ahead. Act now. Payment Completed Time Payment Failed Merchant Registered Payment Initiated Buyer Customer Accepted Underwriter Payment Method Registered Employee Payment Method Enabled Merchant MonthlyInvoice Sent Clock
9. Think ahead. Act now. Request changes to the domain May be refused Wish for the future
10. Think ahead. Act now. Payment Completed Time Payment Initiated Payment Failed Payment Method Registered Payment Method Enabled MonthlyInvoice Sent Merchant Registered Customer Accepted Buyer Underwriter Employee Clock Merchant Register PaymentMethod Register Merchant Enable PaymentMethod Initiate Payment Complete Payment KYC-Check Customer
11. Think ahead. Act now. Event Storming experience First time overall view Define scope of the discussion/model for next times
12. Think ahead. Act now. 2 1 3 4 Exploring the customer domain The right set of microservices Cross-cutting concerns 5 Recap and Q & A Reliable communication in a MSA
13. Think ahead. Act now. AUTONOMOUS BUSINESS CAPABILITIES SELF ORGANIZING TEAMS
14. Think ahead. Act now. Horizontal teams API team Business Logic team Data team Infrastructure team Microservice A Microservice B Microservice C
15. Think ahead. Act now. Microservice A Self organizing teams API Business Logic Data Infrastructure Microservice B API Business Logic Data Infrastructure Microservice C API Business Logic Data Infrastructure API team Business Logic team Data team Infrastructure team You think it, you build it, you run it Team Red Team Blue
16. Think ahead. Act now. How big is a MICROSERVICE?
17. Think ahead. Act now. Is a MICROSERVICE this big? 1 function
18. Think ahead. Act now. THIS big is a MICROSERVICE It depends
19. Think ahead. Act now. 1 Spot the Bounded Contexts 2 Find the Autonomous Business Capabilities 3 Define the set of microservices Bounded Context ABC ABC µ µ µ µ µ µ µ µ USER USERACCOUNT, USERPROFILE UserAccountRemoval UserAccountSynchronization
20. Think ahead. Act now. Payment Completed Time Payment Initiated Payment Failed Payment Method Registered Payment Method Enabled MonthlyInvoice Sent Merchant Registered Customer Accepted Transactions Billing Customer Buyer Underwriter Employee Clock Merchant Register PaymentMethod Register Merchant Enable PaymentMethod Initiate Payment Complete Payment Fail Payment KYC-Check Customer 1: Spot the BOUNDED CONTEXTS
21. Think ahead. Act now. 2: Find the Autonomous Business Capabilities User Account User Profile User Context User Account Removal Contact Contract Customer Context
22. Think ahead. Act now. Autonomous Business Capability Timer/Job Process
23. Think ahead. Act now. Timer/Job Process User Context User Account WebApi User User Account sync User Account removal User Profile WebApi Profile Autonomous Business Capability Timer/Job Process WebApi User Account sync User Account removal User
24. Think ahead. Act now. 2 1 3 4 Exploring the customer domain The right set of microservices Cross-cutting concerns 5 Recap and Q & A Reliable communication in a MSA
25. Think ahead. Act now. CAP THEOREM Eric Brewer Consistency Availability Partition Tolerance Service A Service C Service B SOA MSA Service A Service C Bus Service B
26. Think ahead. Act now. Transactions Billing Customer Payment Service Contact Service Contract Service Invoice Service Bus EVENTUALLY CONSISTENCY
27. Think ahead. Act now. How to ensure RELIABLE MESSAGING within a MSA?
28. Think ahead. Act now. Controller Logic Storage 1. Persist 2. Publish Azure Service Bus Controller Logic 1. Persist 2. Publish Storage EventHandler Command Command Event X CustomerService PolicyService
29. Think ahead. Act now. Controller Logic Storage 1. Persist 2. Publish Azure Service Bus Controller Logic 1. Persist 2. Publish Storage Transactional Scope ServiceX ServiceY EventHandler
30. Think ahead. Act now. TRANSACTIONAL SCOPE SOLUTION ----- ----- ----- Transaction log scraping Event outbox Target table ServiceX Event Publisher 2. GetAll 3. Publish 4. Remove Unpub. events 1. Add At-least once Delivery Recovery flow Unprocessed Requests Execute() { } Recover() { } 1. Insert 2. Process 3. Pub 4. Remove 1. Retry 3. Remove 2. Pub Event Sourcing
31. Think ahead. Act now. EVENT SOURCING
32. Think ahead. Act now. EVENT SOURCING Out-of-the box event storage and publishing (EventStore) Stored event != published event Replay Audit trail Snapshot in time Scalable (Kafka) Status quo CQRS GDPR
33. Think ahead. Act now. Controller Logic Storage 1. Persist 2. Publish Azure Service Bus Controller Logic 1. Persist 2. Publish Storage EventHandler Command guarding Command ServiceX ServiceY
34. Think ahead. Act now. COMMAND GUARDING Unique ID Expiration Digital Footprint
35. Think ahead. Act now. Controller Logic Storage 1. Persist 2. Publish Azure Service Bus Controller Logic 1. Persist 2. Publish Storage EventHandler Event ordering/ deduplication ServiceX ServiceY
36. Think ahead. Act now. EVENT ORDERING Timestamp
37. Think ahead. Act now. Timestamp Server clock times EVENT ORDERING
38. Think ahead. Act now. Unique number Unique version per entity EVENT ORDERING EventType CustomerId Version Data CustomerCreated 123 0 John Doe, Kerkstraat 4, Zwolle CustomerMoved 123 1 John Doe, Agnietenweg 5, Zwolle CustomerUpdated 123 2 Jane Doe, Agnietenweg 5, Zwolle EventVersion EventType BankAccountId Version Data 0 MoneyDebited 123 0 FromAccount: 123, ToAccount: 124, Amount: 10 1 MoneyCredited 124 0 FromAccount: 123, ToAccount: 124, Amount: 10 2 MoneyCredited 123 0 FromAccount: 101, ToAccount: 123, Amount: 10 Balance 123 and 124 = 0 Balance 123 = -10 Balance 124 = 20 Balance 123 = 0
39. Think ahead. Act now. EVENT DEDUPLICATION GUID based
40. Think ahead. Act now. 2 1 3 4 Exploring the customer domain The right set of microservices Cross-cutting concerns 5 Recap and Q & A Reliable communication in a MSA
41. Think ahead. Act now. CROSS-CUTTING CONCERNS Decentralized Authorization Education School Student Teacher LearningPath Organizatio n Learner Expert Bundle Community Member
42. Think ahead. Act now. Functional Application Infrastructure MONITORING Polyglot X,but….
43. Think ahead. Act now. DATA DUPLICATION Not necessarily bad, as long as you have one source of truth. Be aware of PII!
44. Think ahead. Act now. 2 1 3 4 Exploring the customer domain The right set of microservices Cross-cutting concerns 5 Recap and Q & A Reliable communication in a MSA
45. Think ahead. Act now. Applying a MICROSERVICES ARCHITECTURE is NOT straightforward Self organizing Teams Identify Bounded Contexts Autonomous Business Capabilities Agile Business Focused High Availability Independent Scalable Message Deduplication Message Ordering Transactional Scope Reliable Messaging Efficient Modelling Independent Deployable Scope is understandable
46. Xebia Stelvio Challenge! 46 Ons doel: € 25.000 Om via de Stichting Pim ernstig zieke of meervoudig gehandicapte kinderen te helpen en extra zorg te geven. Waardeer je onze sessie, waardeer je ons goede doel? Help ons op: http://xpir.it/stelviopascal BIJ VOORBAAT BEDANKT !
47. Think ahead. Act now. Thanks Cornell Knulst Cloud Architect DevOps Consultant Xpirit Netherlands @CornellKnulst – cknulst@xpirit.com Pascal Naber Coding Azure Architect Microsoft Azure MVP Xpirit Netherlands @pascalnaber – pnaber@xpirit.com http://xpir.it/stelviopascal