Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Lessons learned Implementing an Event Sourced M...

Lessons learned Implementing an Event Sourced Microservice Architecture on Azure door Cornell Knulst en Pascal Naber

devNetNoord #13

devNetNoord

March 22, 2018
Tweet

More Decks by devNetNoord

Other Decks in Technology

Transcript

  1. Think ahead. Act now. Think ahead. Act now. Lessons learned:

    Implementing an event sourced, microservices architecture on Azure Pascal Naber & Cornell Knulst
  2. Think ahead. Act now. 2 1 3 4 Exploring the

    customer domain The right set of microservices Cross-cutting concerns Reliable communication in a MSA 5 Recap and Q & A
  3. Think ahead. Act now. 2 1 3 4 Exploring the

    customer domain The right set of microservices Cross-cutting concerns 5 Recap and Q & A Reliable communication in a MSA
  4. Think ahead. Act now. Payment Completed Time Payment Failed Merchant

    Registered Payment Initiated Buyer Customer Accepted Underwriter Payment Method Registered Employee Payment Method Enabled Merchant MonthlyInvoice Sent Clock
  5. Think ahead. Act now. Payment Completed Time Payment Initiated Payment

    Failed Payment Method Registered Payment Method Enabled MonthlyInvoice Sent Merchant Registered Customer Accepted Buyer Underwriter Employee Clock Merchant Register PaymentMethod Register Merchant Enable PaymentMethod Initiate Payment Complete Payment KYC-Check Customer
  6. Think ahead. Act now. Event Storming experience First time overall

    view Define scope of the discussion/model for next times
  7. Think ahead. Act now. 2 1 3 4 Exploring the

    customer domain The right set of microservices Cross-cutting concerns 5 Recap and Q & A Reliable communication in a MSA
  8. Think ahead. Act now. Horizontal teams API team Business Logic

    team Data team Infrastructure team Microservice A Microservice B Microservice C
  9. Think ahead. Act now. Microservice A Self organizing teams API

    Business Logic Data Infrastructure Microservice B API Business Logic Data Infrastructure Microservice C API Business Logic Data Infrastructure API team Business Logic team Data team Infrastructure team You think it, you build it, you run it Team Red Team Blue
  10. Think ahead. Act now. 1 Spot the Bounded Contexts 2

    Find the Autonomous Business Capabilities 3 Define the set of microservices Bounded Context ABC ABC µ µ µ µ µ µ µ µ USER USERACCOUNT, USERPROFILE UserAccountRemoval UserAccountSynchronization
  11. Think ahead. Act now. Payment Completed Time Payment Initiated Payment

    Failed Payment Method Registered Payment Method Enabled MonthlyInvoice Sent Merchant Registered Customer Accepted Transactions Billing Customer Buyer Underwriter Employee Clock Merchant Register PaymentMethod Register Merchant Enable PaymentMethod Initiate Payment Complete Payment Fail Payment KYC-Check Customer 1: Spot the BOUNDED CONTEXTS
  12. Think ahead. Act now. 2: Find the Autonomous Business Capabilities

    User Account User Profile User Context User Account Removal Contact Contract Customer Context
  13. Think ahead. Act now. Timer/Job Process User Context User Account

    WebApi User User Account sync User Account removal User Profile WebApi Profile Autonomous Business Capability Timer/Job Process WebApi User Account sync User Account removal User
  14. Think ahead. Act now. 2 1 3 4 Exploring the

    customer domain The right set of microservices Cross-cutting concerns 5 Recap and Q & A Reliable communication in a MSA
  15. Think ahead. Act now. CAP THEOREM Eric Brewer Consistency Availability

    Partition Tolerance Service A Service C Service B SOA MSA Service A Service C Bus Service B
  16. Think ahead. Act now. Transactions Billing Customer Payment Service Contact

    Service Contract Service Invoice Service Bus EVENTUALLY CONSISTENCY
  17. Think ahead. Act now. Controller Logic Storage 1. Persist 2.

    Publish Azure Service Bus Controller Logic 1. Persist 2. Publish Storage EventHandler Command Command Event X CustomerService PolicyService
  18. Think ahead. Act now. Controller Logic Storage 1. Persist 2.

    Publish Azure Service Bus Controller Logic 1. Persist 2. Publish Storage Transactional Scope ServiceX ServiceY EventHandler
  19. Think ahead. Act now. TRANSACTIONAL SCOPE SOLUTION ----- ----- -----

    Transaction log scraping Event outbox Target table ServiceX Event Publisher 2. GetAll 3. Publish 4. Remove Unpub. events 1. Add At-least once Delivery Recovery flow Unprocessed Requests Execute() { } Recover() { } 1. Insert 2. Process 3. Pub 4. Remove 1. Retry 3. Remove 2. Pub Event Sourcing
  20. Think ahead. Act now. EVENT SOURCING Out-of-the box event storage

    and publishing (EventStore) Stored event != published event Replay Audit trail Snapshot in time Scalable (Kafka) Status quo CQRS GDPR
  21. Think ahead. Act now. Controller Logic Storage 1. Persist 2.

    Publish Azure Service Bus Controller Logic 1. Persist 2. Publish Storage EventHandler Command guarding Command ServiceX ServiceY
  22. Think ahead. Act now. Controller Logic Storage 1. Persist 2.

    Publish Azure Service Bus Controller Logic 1. Persist 2. Publish Storage EventHandler Event ordering/ deduplication ServiceX ServiceY
  23. Think ahead. Act now. Unique number Unique version per entity

    EVENT ORDERING EventType CustomerId Version Data CustomerCreated 123 0 John Doe, Kerkstraat 4, Zwolle CustomerMoved 123 1 John Doe, Agnietenweg 5, Zwolle CustomerUpdated 123 2 Jane Doe, Agnietenweg 5, Zwolle EventVersion EventType BankAccountId Version Data 0 MoneyDebited 123 0 FromAccount: 123, ToAccount: 124, Amount: 10 1 MoneyCredited 124 0 FromAccount: 123, ToAccount: 124, Amount: 10 2 MoneyCredited 123 0 FromAccount: 101, ToAccount: 123, Amount: 10 Balance 123 and 124 = 0 Balance 123 = -10 Balance 124 = 20 Balance 123 = 0
  24. Think ahead. Act now. 2 1 3 4 Exploring the

    customer domain The right set of microservices Cross-cutting concerns 5 Recap and Q & A Reliable communication in a MSA
  25. Think ahead. Act now. CROSS-CUTTING CONCERNS Decentralized Authorization Education School

    Student Teacher LearningPath Organizatio n Learner Expert Bundle Community Member
  26. Think ahead. Act now. DATA DUPLICATION Not necessarily bad, as

    long as you have one source of truth. Be aware of PII!
  27. Think ahead. Act now. 2 1 3 4 Exploring the

    customer domain The right set of microservices Cross-cutting concerns 5 Recap and Q & A Reliable communication in a MSA
  28. Think ahead. Act now. Applying a MICROSERVICES ARCHITECTURE is NOT

    straightforward Self organizing Teams Identify Bounded Contexts Autonomous Business Capabilities Agile Business Focused High Availability Independent Scalable Message Deduplication Message Ordering Transactional Scope Reliable Messaging Efficient Modelling Independent Deployable Scope is understandable
  29. Xebia Stelvio Challenge! 46 Ons doel: € 25.000 Om via

    de Stichting Pim ernstig zieke of meervoudig gehandicapte kinderen te helpen en extra zorg te geven. Waardeer je onze sessie, waardeer je ons goede doel? Help ons op: http://xpir.it/stelviopascal BIJ VOORBAAT BEDANKT !
  30. Think ahead. Act now. Thanks Cornell Knulst Cloud Architect DevOps

    Consultant Xpirit Netherlands @CornellKnulst – [email protected] Pascal Naber Coding Azure Architect Microsoft Azure MVP Xpirit Netherlands @pascalnaber – [email protected] http://xpir.it/stelviopascal