Evolving a microservice architecture: how to right-size your services

This is a talk I gave at SF JUG in May 2023.

Let’s imagine that you want to add a new feature to your microservices-based application. You might be tempted to simply define a new service. After all, it is the MICROservice architecture. The trouble, however, with blindly adding new services, is that it often leads to the More the Merrier anti-pattern. An overly complex architecture that’s difficult to maintain and, perhaps, brittle.

In this presentation, I describe the dark energy and dark matter forces, which are conflicting concerns that you must consider when designing a microservice architecture.You will learn how the dark energy forces encourage decomposition into finer-grained services in order to improve aspects of your architecture including testability, maintainability, scalability. I describe how the dark matter forces discourage decomposition in order to improve aspects such as efficiency, availability and data consistency. You will learn how to use these forces to evolve your application’s microservice architecture.

Transcript

1. @crichardson Evolving a microservice architecture: how to right-size your services Chris Richardson Microservice architecture consultant and trainer Founder of the original CloudFoundry.com Author of POJOs in Action and Microservices Patterns Founder of Eventuate.io @crichardson chris@chrisrichardson.net adopt.microservices.io Copyright © 2023. Chris Richardson Consulting, Inc. All rights reserved
2. @crichardson Let’s imagine… Your microservices-based application + New feature: Coupons
3. @crichardson Implement a new MICROservice, right?
4. @crichardson Anti-pattern: The more the merrier Excessively fi ne-grained microservice architecture: Reduced maintainability, performance, availability, …. https://microservices.io/post/antipatterns/2019/05/21/antipattern-more-the- merrier.html
5. @crichardson Presentation goal How to rightsize your services and avoid creating an overly complex, fi ne-grained architecture
6. @crichardson About Chris http://adopt.microservices.io Late 80s 2006 2008 2009 2012-
7. @crichardson About Chris https://chrisrichardson.net/
8. @crichardson Agenda How micro is a microservice? Designing a microservice architecture Dark matter forces: reasons to not use services Dark energy forces: reasons to use services
9. Pattern: microservice architecture A service is: Loosely coupled Independently deployable Implements a business capability Owned by a small team
10. @crichardson But how big is a service? Microservice architecture Must be small, right?!
11. A common conversation Client: We are having problems testing our services. Can you help us? Me: Sure. How many services? Client: 5 Me: Oh ok. BTW How many developers? Client: 5 Me: Oh. How about merging into one service? 1 service/developer is remarkably common
12. @crichardson Anti-pattern: The more the merrier Excessively fi ne-grained microservice architecture: Reduced maintainability, performance, availability, …. https://microservices.io/post/antipatterns/2019/05/21/antipattern-more-the- merrier.html
13. @crichardson Agenda How micro is a microservice? Designing a microservice architecture Dark matter forces: reasons to not use services Dark energy forces: reasons to use services
14. @crichardson How to de fi ne an Architecture… Application ≪subdomain≫ Customer management ≪aggregate≫ Customer ≪subdomain≫ Order management ≪aggregate≫ Order createCustomer() createOrder() fi ndOrder() fi ndOrderHistory() System operations Distill Requirements The “requests” that the application implements Have SLOs Customer Team Order Team About Subdomains • Business capability/function/etc • Logical view: packages and classes • Team-sized • Loosely coupled (Conways law) 1 2 Functional requirements As a consumer I want to place an Order So that I can …. As a Restaurant I want to accept an Order So that I can …. Event storming Wireframe/UI mockups Available Restaurants Restaurant Menu System quality attributes • SLA: Reliability/Latency • Scalability • … https://microservices.io/post/architecture/2023/02/09/assemblage-architecture-de fi nition-process.html
15. @crichardson Subdomains: Eg. Java classes and packages that implement business logic Customer Management Order Management
16. @crichardson Kitchen Service Delivery Service Order Service createOrder() … how to de fi ne an Architecture createOrder() <<subdomain>> Order Management Order System operations <<subdomain>> Order Management <<subdomain>> Kitchen Management <<subdomain>> Delivery Management <<subdomain>> Courier Management Group subdomains into services Application Collaboration Design collaborations for distributed operations createOrder() 3
17. @crichardson Grouping subdomains into components: together or separate? ≪subdomain≫ Customer Mgmt. ≪aggregate≫ Customer ≪subdomain≫ Order Mgmt. ≪aggregate≫ Order Attraction Repulsion Simple components Team-sized services Fast deployment pipeline … Dark energy: an anti- gravity that’s accelerating the expansion of the universe Dark matter: an invisible matter that has a gravitational effect on stars and galaxies. https://www.nasa.gov/feature/goddard/2020/new-hubble-data-explains-missing-dark-matter Simple, ef fi cient interactions Prefer ACID over BASE Minimize runtime coupling … https://chrisrichardson.net/post/microservices/2021/04/15/mucon-2021-dark-energy-dark-matter.html Generate systemOperation()
18. @crichardson Dark energy and dark matter forces Subdomain A «Aggregate» X Subdomain B «Aggregate» Y Service A Service B Attraction Simple interactions Efficient interactions Prefer ACID over BASE Minimize runtime coupling Minimize design time coupling Simple components Team autonomy Fast deployment pipeline Support multiple technology stacks Segregate by characteristics Repulsion Dark energy Dark matter Metaphor for Metaphor for
19. @crichardson Let’s imagine you want to implement Coupons… Order Service <<subdomain>> Order Management Customer Service <<subdomain>> Customer Management createCustomer() createOrder(…, couponId) reserve Credit() As a Vendor I want to create a coupon So that I can … As a Customer I want to place an order redeeming a coupon So that I can … AS-IS architecture New/modi fi ed user stories
20. @crichardson Application Distill requirements to operations and subdomains As a Vendor I want to create a coupon So that I can … As a Customer I want to place an order redeeming a coupon So that I can … createCoupon() createOrder(..couponId) … <<subdomain>> Coupon Management <<subdomain>> Order Management Distill New and modi fi ed operations and subdomains
21. @crichardson Key decision: New service or existing service?… Order Service <<subdomain>> Order Management Customer Service <<subdomain>> Customer Management createCustomer() createOrder(…, couponId) <<subdomain>> Coupon Management createCoupon(discount, …) redeem(couponID) Which service? reserve Credit()
22. @crichardson … Key decision: New service or existing service?… Coupon Service Order Service <<subdomain>> Order Management Customer Service <<subdomain>> Customer Management createCustomer() createOrder(…, couponId) <<subdomain>> Coupon Management createCoupon(discount, …) redeem(couponID) reserve Credit() Distributed Order Service <<subdomain>> Order Management Customer Service <<subdomain>> Customer Management createCustomer() createCoupon(discount, …) createOrder(…, couponId) <<subdomain>> Coupon Management reserve Credit() Local
23. @crichardson … Key decision: New service or existing service? Coupon Service Order Service <<subdomain>> Order Management <<subdomain>> Coupon Management redeem(couponID) Service Service Service Transaction Compensating transaction Transaction Compensating transaction Transaction Compensating transaction command() Saga Service Service command() Replica Source Event Command-side replica API Composition Provider Service Service query() Composer Provider Service Provider Service CQRS Provider Service Service query() View Provider Service Provider Service Event Event Event Collaboration design createOrder()
24. @crichardson Evaluate alternative designs using dark energy and dark matter forces Within Order Service Coupon Service Dark energy, repulsive forces Simple components ? ? Team autonomy Fast deployment pipeline Support multiple technology stacks Segregate by characteristics Dark matter, attractive forces Simple interactions ? ? Ef fi cient interactions Prefer ACID over BASE Minimize runtime coupling Minimize design-time coupling
25. @crichardson Agenda How micro is a microservice? Designing a microservice architecture Dark matter forces: reasons to not use services Dark energy forces: reasons to use services
26. @crichardson Dark matter attractive forces 㱺 subdomains in same service https://chrisrichardson.net/post/microservices/2021/04/15/mucon-2021-dark-energy-dark-matter.html Subdomain A «Aggregate» X Subdomain B «Aggregate» Y Service A Service B Simple interactions Efficient interactions Prefer ACID over BASE Minimize runtime coupling Minimize design time coupling Generates SystemOperation() Collaboration
27. @crichardson Dark matter forces: reasons to colocate Order and Coupon management Coupon Service Order Service Order Service <<subdomain>> Order Management <<subdomain>> Order Management <<subdomain>> Coupon Management <<subdomain>> Coupon Management 㱺 But do they apply in this situation? Does de fi ning a new service create a problem?
28. @crichardson Simple interactions Create Order() Service Subdomain A Subdomain B Service B Service A Subdomain A Subdomain B Create Order() Complex distributed operation Simple local operation: easier to develop, test, understand, troubleshoot, … vs.
29. @crichardson Question: is each distributed operation simple? Additional interaction Additional participant Minimal increase in complexity but eventually …
30. @crichardson Distributed invocations are expensive Local method call: customerService.reserveCredit(customerID, amount) Testing with mock objects vs. Distributed invocation: CustomerServiceProxy CustomerController Sagas, compensating transactions, partial failure Consumer-driven contract tests …
31. @crichardson Time to troubleshoot ∝ Depth of distributed call chain
32. @crichardson Ef fi cient interactions Create Order() Service Subdomain A Subdomain B Service B Service A Subdomain A Subdomain B Create Order() Network latency, limited bandwidth In-memory, fast! vs. Must satisfy SLOs
33. @crichardson Question: is each distributed operation ef fi cient enough? Additional sequential interaction Minimal reduction in ef fi ciency but eventually …
34. @crichardson Prefer ACID over BASE System Operation() Service Subdomain A Subdomain B Service B Service A Subdomain A Subdomain B System Operation() Distributed, eventually consistent transaction Simple, Local ACID transaction vs. ACID txn ACID txn ACID txn
35. @crichardson Question: is the distributed operation “suf fi ciently” ACID? Step Participant Transaction Compensating Transaction 1 Order Service createOrder() rejectOrder() 2 Coupon Service redeemCoupon() unredeemCoupon() 3 Customer Service reserveCredit() - 4 Order Service approveOrder() - Create Order Saga Doable but much more complex…
36. @crichardson Minimize runtime coupling System Operation() Service Subdomain A Subdomain B Service B Service A Subdomain A Subdomain B System Operation() Risk of runtime coupling No runtime coupling: higher availability, lower latency vs. Must satisfy SLOs
37. @crichardson Question: does the distributed operation meet its latency/availability SLO? All must be available More available More complex Wait No Wait
38. @crichardson Risk: Silo’d teams have dif fi culty identifying excessive runtime coupling Payment Team: “We just call the Fraud Service” Fraud Team: “We have lots of services”
39. @crichardson Minimize design time coupling Order Subdomain Customer Subdomain reserveCredit() createOrder() Customer Order Design-time coupling Minimize with careful design BUT You can’t always eliminate it 㱺 Risk of lock step changes API Risk proportional to: • API instability • API complexity • …
40. @crichardson Question: are the two subdomains suf fi ciently design-time decoupled? interface CouponManagement { redeemCoupon(couponID, amount) interface CouponManagement { redeemCoupon(couponID, amount, orderLineItems) Not bad ✅ More complex, coupled to order concept ❌ vs.
41. @crichardson Summary: dark matter forces and Coupon Management Within Order Service Coupon Service Dark energy, repulsive forces Simple components ? ? Team autonomy Fast deployment pipeline Support multiple technology stacks Segregate by characteristics Dark matter, attractive forces Simple interactions ✅ ✅❌ Ef fi cient interactions ✅ Prefer ACID over BASE ❌ Minimize runtime coupling ❌✅ Minimize design-time coupling ✅ Creates problems
42. @crichardson Agenda How micro is a microservice? Designing a microservice architecture Dark matter forces: reasons to not use services Dark energy forces: reasons to use services
43. @crichardson Dark energy repulsive forces 㱺 subdomains in different services https://chrisrichardson.net/post/microservices/2021/04/15/mucon-2021-dark-energy-dark-matter.html Service Service «Subdomain» A «Aggregate» X «Subdomain» B «Aggregate» Y Simple components Team autonomy Fast deployment pipeline Support multiple technology stacks Segregate by characteristics Repulsive dark energy forces
44. @crichardson Dark energy forces: reasons to create a Coupon Service Coupon Service Order Service Order Service <<subdomain>> Order Management <<subdomain>> Order Management <<subdomain>> Coupon Management <<subdomain>> Coupon Management 㱺 But do they apply in this situation? Does de fi ning a new service solve a problem?
45. @crichardson Simpler components/services Service Service Service Subdomain A Subdomain A Subdomain B Subdomain B More complex service Simpler services: easier to understand, develop, test, … versus dependencies
46. Question: is the Order+Coupon Service excessively complex? Coupon management: reasonably simple, no new dependencies, … Minimal additional complexity Therefore Separate Coupon Service is not required But If Coupon management becomes complex then separate Order Service main main Order Management orders.web couponAPI orders. domain Coupon Management coupons. persistence orders. persistence Coupon team Order team coupons. domain coupons.web coupons.api Modular Order Service
47. @crichardson Team autonomy = service per team Service Service Service Subdomain A Subdomain A Subdomain B Subdomain B Coordination required Contention for resources Develop, push, build, test and deploy independently vs. Team A Team B Team A Team B
48. @crichardson Question: impact of embedding Coupon Management in Order Service on team autonomy? Who develops Coupon Management? Orders team Team autonomy is not an issue Different team, e.g. Coupons Team: How much autonomy would they lose? A few teams = probably ok But there’s a limit
49. @crichardson Fast deployment pipeline… @mipsytipsy https://speakerdeck.com/charity/cd?slide=17
50. @crichardson … fast deployment pipeline Service Subdomain Subdomain Service Subdomain Shorter lead time Simpler build Longer lead time More complex build* Service Subdomain vs. Deployment pipeline Deployment pipeline Deployment pipeline * Parallelizing building/testing partially helps
51. Question: Impact of adding Customer Management to the Order Service? Increase on test execution time? testExecutionTime(Coupon Management)? Incremental build and test? Worst case: changing one subdomain requires testing the other Increase in commit frequency? More developers = more commits Possibility of delays due to queuing Order Service main main Order Management orders.web couponAPI orders. domain Coupon Management coupons. persistence orders. persistence coupons. domain coupons.web coupons.api Stable?
52. @crichardson Support multiple technology stacks Service Python Service Java Service JVM Subdomain A Subdomain A Subdomain B Subdomain B Single technology stack Upgrade together Separate technology stacks Right tool for the job Upgrade independently Experiment easily versus
53. @crichardson Question: does Coupon Management introduce technology stack issues? Does Coupon Management use the same technology stack as Order Management? Same language, framework Compatible transitive dependencies Does Coupon Management introduce new dependencies that would complicate technology upgrades? Service upgrade effort proportional # dependencies A dependency might not support newer versions of libraries, JDK, etc
54. @crichardson Separate subdomains by characteristics Subdomain characteristic Issue Resource requirements Cost-effective, scalability Regulations, e.g. SaMD/ PCI DevOps vs. Slower regulated process Business criticality/tier Maximize availability Security, e.g. PII, … Improve security DDD core/supporting/ generic Focus on being competitive
55. @crichardson Cost effective scaling Service Service Service Subdomain A Subdomain A Subdomain B Subdomain B versus CPU MEM GPU Scale together • Wasteful • Costly CPU MEM GPU Scale separately • Ef fi cient • Cheaper Load Load Load Load EC2: p4d.24xlarge EC2: p4d.24xlarge EC2: m5.24xlarge 8x cost! Unit of scaling
56. @crichardson Example: Segregate by business criticality Service Service Service Payment Processing Payment Processing Merchant management Merchant management Shared infrastructure Shared code base Risk of interference Separate infrastructure Separate code base Isolated vs. chargeCard() 2.9% + 30c/ request Revenue loss and penalties chargeCard() Critical Important
57. @crichardson Question: How does Coupon Management compare to Order Management? Subdomain characteristic Same? Resource requirements ✅ Regulations, e.g. SaMD/PCI/… ✅ Business criticality/tier ✅ Security, e.g. PII, … ✅ DDD core/supporting/generic ✅
58. @crichardson Summary: designing Coupon Management Within Order Service Coupon Service Dark energy, repulsive forces Simple components ✅ Doesn’t solve a problem Team autonomy Fast deployment pipeline Support multiple technology stacks Segregate by characteristics Dark matter, attractive forces Simple interactions ✅ ✅❌ Ef fi cient interactions ✅ Prefer ACID over BASE ❌ Minimize runtime coupling ❌✅ Minimize design-time coupling ✅ Creates problems
59. @crichardson Chosen design: Order Service <<subdomain>> Order Management Customer Service <<subdomain>> Customer Management createCustomer() createCoupon(discount, …) createOrder(…, couponId) <<subdomain>> Coupon Management reserve Credit() Service Service «Subdomain» A «Aggregate» X «Subdomain» B «Aggregate» Y Simple components Team autonomy Fast deployment pipeline Support multiple technology stacks Segregate by characteristics Repulsive dark energy forces Continual growth => eventually won’t resolve
60. @crichardson Summary: beware of Zombie ideas
61. @crichardson Summary Don’t take MICROservices literally THINK! Brainstorm and evaluate candidate designs Designing a microservice architecture Dark energy forces = reasons to use services Dark matter forces = reasons to not use services Con fl icting forces => must make trade-offs Implementing subdomains: JARs by default As a service to solve a tangible dark energy-related problem https://www.nasa.gov/feature/goddard/2019/nasa-s-james-webb-space-telescope-has-been-assembled-for-the- fi rst-time
62. @crichardson @crichardson chris@chrisrichardson.net adopt.microservices.io Questions?