Slide 1

Slide 1 text

@crichardson Evolving a microservice architecture: how to right-size your services Chris Richardson Microservice architecture consultant and trainer Founder of the original CloudFoundry.com Author of POJOs in Action and Microservices Patterns Founder of Eventuate.io @crichardson [email protected] adopt.microservices.io Copyright © 2023. Chris Richardson Consulting, Inc. All rights reserved

Slide 2

Slide 2 text

@crichardson Let’s imagine… Your microservices-based application + New feature: Coupons

Slide 3

Slide 3 text

@crichardson Implement a new MICROservice, right?

Slide 4

Slide 4 text

@crichardson Anti-pattern: The more the merrier Excessively fi ne-grained microservice architecture: Reduced maintainability, performance, availability, …. https://microservices.io/post/antipatterns/2019/05/21/antipattern-more-the- merrier.html

Slide 5

Slide 5 text

@crichardson Presentation goal How to rightsize your services and avoid creating an overly complex, fi ne-grained architecture

Slide 6

Slide 6 text

@crichardson About Chris http://adopt.microservices.io Late 80s 2006 2008 2009 2012-

Slide 7

Slide 7 text

@crichardson About Chris https://chrisrichardson.net/

Slide 8

Slide 8 text

@crichardson Agenda How micro is a microservice? Designing a microservice architecture Dark matter forces: reasons to not use services Dark energy forces: reasons to use services

Slide 9

Slide 9 text

Pattern: microservice architecture A service is: Loosely coupled Independently deployable Implements a business capability Owned by a small team

Slide 10

Slide 10 text

@crichardson But how big is a service? Microservice architecture Must be small, right?!

Slide 11

Slide 11 text

A common conversation Client: We are having problems testing our services. Can you help us? Me: Sure. How many services? Client: 5 Me: Oh ok. BTW How many developers? Client: 5 Me: Oh. How about merging into one service? 1 service/developer is remarkably common

Slide 12

Slide 12 text

@crichardson Anti-pattern: The more the merrier Excessively fi ne-grained microservice architecture: Reduced maintainability, performance, availability, …. https://microservices.io/post/antipatterns/2019/05/21/antipattern-more-the- merrier.html

Slide 13

Slide 13 text

@crichardson Agenda How micro is a microservice? Designing a microservice architecture Dark matter forces: reasons to not use services Dark energy forces: reasons to use services

Slide 14

Slide 14 text

@crichardson How to de fi ne an Architecture… Application ≪subdomain≫ Customer management ≪aggregate≫ Customer ≪subdomain≫ Order management ≪aggregate≫ Order createCustomer() createOrder() fi ndOrder() fi ndOrderHistory() System operations Distill Requirements The “requests” that the application implements Have SLOs Customer Team Order Team About Subdomains • Business capability/function/etc • Logical view: packages and classes • Team-sized • Loosely coupled (Conways law) 1 2 Functional requirements As a consumer I want to place an Order So that I can …. As a Restaurant I want to accept an Order So that I can …. Event storming Wireframe/UI mockups Available Restaurants Restaurant Menu System quality attributes • SLA: Reliability/Latency • Scalability • … https://microservices.io/post/architecture/2023/02/09/assemblage-architecture-de fi nition-process.html

Slide 15

Slide 15 text

@crichardson Subdomains: Eg. Java classes and packages that implement business logic Customer Management Order Management

Slide 16

Slide 16 text

@crichardson Kitchen Service Delivery Service Order Service createOrder() … how to de fi ne an Architecture createOrder() <> Order Management Order System operations <> Order Management <> Kitchen Management <> Delivery Management <> Courier Management Group subdomains into services Application Collaboration Design collaborations for distributed operations createOrder() 3

Slide 17

Slide 17 text

@crichardson Grouping subdomains into components: together or separate? ≪subdomain≫ Customer Mgmt. ≪aggregate≫ Customer ≪subdomain≫ Order Mgmt. ≪aggregate≫ Order Attraction Repulsion Simple components Team-sized services Fast deployment pipeline … Dark energy: an anti- gravity that’s accelerating the expansion of the universe Dark matter: an invisible matter that has a gravitational effect on stars and galaxies. https://www.nasa.gov/feature/goddard/2020/new-hubble-data-explains-missing-dark-matter Simple, ef fi cient interactions Prefer ACID over BASE Minimize runtime coupling … https://chrisrichardson.net/post/microservices/2021/04/15/mucon-2021-dark-energy-dark-matter.html Generate systemOperation()

Slide 18

Slide 18 text

@crichardson Dark energy and dark matter forces Subdomain A «Aggregate» X Subdomain B «Aggregate» Y Service A Service B Attraction Simple interactions Efficient interactions Prefer ACID over BASE Minimize runtime coupling Minimize design time coupling Simple components Team autonomy Fast deployment pipeline Support multiple technology stacks Segregate by characteristics Repulsion Dark energy Dark matter Metaphor for Metaphor for

Slide 19

Slide 19 text

@crichardson Let’s imagine you want to implement Coupons… Order Service <> Order Management Customer Service <> Customer Management createCustomer() createOrder(…, couponId) reserve Credit() As a Vendor I want to create a coupon So that I can … As a Customer I want to place an order redeeming a coupon So that I can … AS-IS architecture New/modi fi ed user stories

Slide 20

Slide 20 text

@crichardson Application Distill requirements to operations and subdomains As a Vendor I want to create a coupon So that I can … As a Customer I want to place an order redeeming a coupon So that I can … createCoupon() createOrder(..couponId) … <> Coupon Management <> Order Management Distill New and modi fi ed operations and subdomains

Slide 21

Slide 21 text

@crichardson Key decision: New service or existing service?… Order Service <> Order Management Customer Service <> Customer Management createCustomer() createOrder(…, couponId) <> Coupon Management createCoupon(discount, …) redeem(couponID) Which service? reserve Credit()

Slide 22

Slide 22 text

@crichardson … Key decision: New service or existing service?… Coupon Service Order Service <> Order Management Customer Service <> Customer Management createCustomer() createOrder(…, couponId) <> Coupon Management createCoupon(discount, …) redeem(couponID) reserve Credit() Distributed Order Service <> Order Management Customer Service <> Customer Management createCustomer() createCoupon(discount, …) createOrder(…, couponId) <> Coupon Management reserve Credit() Local

Slide 23

Slide 23 text

@crichardson … Key decision: New service or existing service? Coupon Service Order Service <> Order Management <> Coupon Management redeem(couponID) Service Service Service Transaction Compensating transaction Transaction Compensating transaction Transaction Compensating transaction command() Saga Service Service command() Replica Source Event Command-side replica API Composition Provider Service Service query() Composer Provider Service Provider Service CQRS Provider Service Service query() View Provider Service Provider Service Event Event Event Collaboration design createOrder()

Slide 24

Slide 24 text

@crichardson Evaluate alternative designs using dark energy and dark matter forces Within Order Service Coupon Service Dark energy, repulsive forces Simple components ? ? Team autonomy Fast deployment pipeline Support multiple technology stacks Segregate by characteristics Dark matter, attractive forces Simple interactions ? ? Ef fi cient interactions Prefer ACID over BASE Minimize runtime coupling Minimize design-time coupling

Slide 25

Slide 25 text

@crichardson Agenda How micro is a microservice? Designing a microservice architecture Dark matter forces: reasons to not use services Dark energy forces: reasons to use services

Slide 26

Slide 26 text

@crichardson Dark matter attractive forces 㱺 subdomains in same service https://chrisrichardson.net/post/microservices/2021/04/15/mucon-2021-dark-energy-dark-matter.html Subdomain A «Aggregate» X Subdomain B «Aggregate» Y Service A Service B Simple interactions Efficient interactions Prefer ACID over BASE Minimize runtime coupling Minimize design time coupling Generates SystemOperation() Collaboration

Slide 27

Slide 27 text

@crichardson Dark matter forces: reasons to colocate Order and Coupon management Coupon Service Order Service Order Service <> Order Management <> Order Management <> Coupon Management <> Coupon Management 㱺 But do they apply in this situation? Does de fi ning a new service create a problem?

Slide 28

Slide 28 text

@crichardson Simple interactions Create Order() Service Subdomain A Subdomain B Service B Service A Subdomain A Subdomain B Create Order() Complex distributed operation Simple local operation: easier to develop, test, understand, troubleshoot, … vs.

Slide 29

Slide 29 text

@crichardson Question: is each distributed operation simple? Additional interaction Additional participant Minimal increase in complexity but eventually …

Slide 30

Slide 30 text

@crichardson Distributed invocations are expensive Local method call: customerService.reserveCredit(customerID, amount) Testing with mock objects vs. Distributed invocation: CustomerServiceProxy CustomerController Sagas, compensating transactions, partial failure Consumer-driven contract tests …

Slide 31

Slide 31 text

@crichardson Time to troubleshoot ∝ Depth of distributed call chain

Slide 32

Slide 32 text

@crichardson Ef fi cient interactions Create Order() Service Subdomain A Subdomain B Service B Service A Subdomain A Subdomain B Create Order() Network latency, limited bandwidth In-memory, fast! vs. Must satisfy SLOs

Slide 33

Slide 33 text

@crichardson Question: is each distributed operation ef fi cient enough? Additional sequential interaction Minimal reduction in ef fi ciency but eventually …

Slide 34

Slide 34 text

@crichardson Prefer ACID over BASE System Operation() Service Subdomain A Subdomain B Service B Service A Subdomain A Subdomain B System Operation() Distributed, eventually consistent transaction Simple, Local ACID transaction vs. ACID txn ACID txn ACID txn

Slide 35

Slide 35 text

@crichardson Question: is the distributed operation “suf fi ciently” ACID? Step Participant Transaction Compensating Transaction 1 Order Service createOrder() rejectOrder() 2 Coupon Service redeemCoupon() unredeemCoupon() 3 Customer Service reserveCredit() - 4 Order Service approveOrder() - Create Order Saga Doable but much more complex…

Slide 36

Slide 36 text

@crichardson Minimize runtime coupling System Operation() Service Subdomain A Subdomain B Service B Service A Subdomain A Subdomain B System Operation() Risk of runtime coupling No runtime coupling: higher availability, lower latency vs. Must satisfy SLOs

Slide 37

Slide 37 text

@crichardson Question: does the distributed operation meet its latency/availability SLO? All must be available More available More complex Wait No Wait

Slide 38

Slide 38 text

@crichardson Risk: Silo’d teams have dif fi culty identifying excessive runtime coupling Payment Team: “We just call the Fraud Service” Fraud Team: “We have lots of services”

Slide 39

Slide 39 text

@crichardson Minimize design time coupling Order Subdomain Customer Subdomain reserveCredit() createOrder() Customer Order Design-time coupling Minimize with careful design BUT You can’t always eliminate it 㱺 Risk of lock step changes API Risk proportional to: • API instability • API complexity • …

Slide 40

Slide 40 text

@crichardson Question: are the two subdomains suf fi ciently design-time decoupled? interface CouponManagement { redeemCoupon(couponID, amount) interface CouponManagement { redeemCoupon(couponID, amount, orderLineItems) Not bad ✅ More complex, coupled to order concept ❌ vs.

Slide 41

Slide 41 text

@crichardson Summary: dark matter forces and Coupon Management Within Order Service Coupon Service Dark energy, repulsive forces Simple components ? ? Team autonomy Fast deployment pipeline Support multiple technology stacks Segregate by characteristics Dark matter, attractive forces Simple interactions ✅ ✅❌ Ef fi cient interactions ✅ Prefer ACID over BASE ❌ Minimize runtime coupling ❌✅ Minimize design-time coupling ✅ Creates problems

Slide 42

Slide 42 text

@crichardson Agenda How micro is a microservice? Designing a microservice architecture Dark matter forces: reasons to not use services Dark energy forces: reasons to use services

Slide 43

Slide 43 text

@crichardson Dark energy repulsive forces 㱺 subdomains in different services https://chrisrichardson.net/post/microservices/2021/04/15/mucon-2021-dark-energy-dark-matter.html Service Service «Subdomain» A «Aggregate» X «Subdomain» B «Aggregate» Y Simple components Team autonomy Fast deployment pipeline Support multiple technology stacks Segregate by characteristics Repulsive dark energy forces

Slide 44

Slide 44 text

@crichardson Dark energy forces: reasons to create a Coupon Service Coupon Service Order Service Order Service <> Order Management <> Order Management <> Coupon Management <> Coupon Management 㱺 But do they apply in this situation? Does de fi ning a new service solve a problem?

Slide 45

Slide 45 text

@crichardson Simpler components/services Service Service Service Subdomain A Subdomain A Subdomain B Subdomain B More complex service Simpler services: easier to understand, develop, test, … versus dependencies

Slide 46

Slide 46 text

Question: is the Order+Coupon Service excessively complex? Coupon management: reasonably simple, no new dependencies, … Minimal additional complexity Therefore Separate Coupon Service is not required But If Coupon management becomes complex then separate Order Service main main Order Management orders.web couponAPI orders. domain Coupon Management coupons. persistence orders. persistence Coupon team Order team coupons. domain coupons.web coupons.api Modular Order Service

Slide 47

Slide 47 text

@crichardson Team autonomy = service per team Service Service Service Subdomain A Subdomain A Subdomain B Subdomain B Coordination required Contention for resources Develop, push, build, test and deploy independently vs. Team A Team B Team A Team B

Slide 48

Slide 48 text

@crichardson Question: impact of embedding Coupon Management in Order Service on team autonomy? Who develops Coupon Management? Orders team Team autonomy is not an issue Different team, e.g. Coupons Team: How much autonomy would they lose? A few teams = probably ok But there’s a limit

Slide 49

Slide 49 text

@crichardson Fast deployment pipeline… @mipsytipsy https://speakerdeck.com/charity/cd?slide=17

Slide 50

Slide 50 text

@crichardson … fast deployment pipeline Service Subdomain Subdomain Service Subdomain Shorter lead time Simpler build Longer lead time More complex build* Service Subdomain vs. Deployment pipeline Deployment pipeline Deployment pipeline * Parallelizing building/testing partially helps

Slide 51

Slide 51 text

Question: Impact of adding Customer Management to the Order Service? Increase on test execution time? testExecutionTime(Coupon Management)? Incremental build and test? Worst case: changing one subdomain requires testing the other Increase in commit frequency? More developers = more commits Possibility of delays due to queuing Order Service main main Order Management orders.web couponAPI orders. domain Coupon Management coupons. persistence orders. persistence coupons. domain coupons.web coupons.api Stable?

Slide 52

Slide 52 text

@crichardson Support multiple technology stacks Service Python Service Java Service JVM Subdomain A Subdomain A Subdomain B Subdomain B Single technology stack Upgrade together Separate technology stacks Right tool for the job Upgrade independently Experiment easily versus

Slide 53

Slide 53 text

@crichardson Question: does Coupon Management introduce technology stack issues? Does Coupon Management use the same technology stack as Order Management? Same language, framework Compatible transitive dependencies Does Coupon Management introduce new dependencies that would complicate technology upgrades? Service upgrade effort proportional # dependencies A dependency might not support newer versions of libraries, JDK, etc

Slide 54

Slide 54 text

@crichardson Separate subdomains by characteristics Subdomain characteristic Issue Resource requirements Cost-effective, scalability Regulations, e.g. SaMD/ PCI DevOps vs. Slower regulated process Business criticality/tier Maximize availability Security, e.g. PII, … Improve security DDD core/supporting/ generic Focus on being competitive

Slide 55

Slide 55 text

@crichardson Cost effective scaling Service Service Service Subdomain A Subdomain A Subdomain B Subdomain B versus CPU MEM GPU Scale together • Wasteful • Costly CPU MEM GPU Scale separately • Ef fi cient • Cheaper Load Load Load Load EC2: p4d.24xlarge EC2: p4d.24xlarge EC2: m5.24xlarge 8x cost! Unit of scaling

Slide 56

Slide 56 text

@crichardson Example: Segregate by business criticality Service Service Service Payment Processing Payment Processing Merchant management Merchant management Shared infrastructure Shared code base Risk of interference Separate infrastructure Separate code base Isolated vs. chargeCard() 2.9% + 30c/ request Revenue loss and penalties chargeCard() Critical Important

Slide 57

Slide 57 text

@crichardson Question: How does Coupon Management compare to Order Management? Subdomain characteristic Same? Resource requirements ✅ Regulations, e.g. SaMD/PCI/… ✅ Business criticality/tier ✅ Security, e.g. PII, … ✅ DDD core/supporting/generic ✅

Slide 58

Slide 58 text

@crichardson Summary: designing Coupon Management Within Order Service Coupon Service Dark energy, repulsive forces Simple components ✅ Doesn’t solve a problem Team autonomy Fast deployment pipeline Support multiple technology stacks Segregate by characteristics Dark matter, attractive forces Simple interactions ✅ ✅❌ Ef fi cient interactions ✅ Prefer ACID over BASE ❌ Minimize runtime coupling ❌✅ Minimize design-time coupling ✅ Creates problems

Slide 59

Slide 59 text

@crichardson Chosen design: Order Service <> Order Management Customer Service <> Customer Management createCustomer() createCoupon(discount, …) createOrder(…, couponId) <> Coupon Management reserve Credit() Service Service «Subdomain» A «Aggregate» X «Subdomain» B «Aggregate» Y Simple components Team autonomy Fast deployment pipeline Support multiple technology stacks Segregate by characteristics Repulsive dark energy forces Continual growth => eventually won’t resolve

Slide 60

Slide 60 text

@crichardson Summary: beware of Zombie ideas

Slide 61

Slide 61 text

@crichardson Summary Don’t take MICROservices literally THINK! Brainstorm and evaluate candidate designs Designing a microservice architecture Dark energy forces = reasons to use services Dark matter forces = reasons to not use services Con fl icting forces => must make trade-offs Implementing subdomains: JARs by default As a service to solve a tangible dark energy-related problem https://www.nasa.gov/feature/goddard/2019/nasa-s-james-webb-space-telescope-has-been-assembled-for-the- fi rst-time

Slide 62

Slide 62 text

@crichardson @crichardson [email protected] adopt.microservices.io Questions?