Evolving a microservice architecture: how to right-size your services

@crichardson
Evolving a microservice
architecture: how to right-size your
services
Chris Richardson

Microservice architecture consultant and trainer

Founder of the original CloudFoundry.com

Author of POJOs in Action and Microservices Patterns

Founder of Eventuate.io

@crichardson

[email protected]

adopt.microservices.io
Copyright © 2023. Chris Richardson Consulting, Inc. All rights reserved

View Slide

@crichardson
Let’s imagine…
Your microservices-based application
+ New feature:

Coupons

View Slide

@crichardson
Implement a new
MICROservice, right?

View Slide

@crichardson
Anti-pattern: The more the merrier
Excessively
fi
ne-grained
microservice architecture:

Reduced maintainability,
performance, availability, ….
https://microservices.io/post/antipatterns/2019/05/21/antipattern-more-the-
merrier.html

View Slide

@crichardson
Presentation goal
How to rightsize your services

and

avoid creating an overly complex,

fi
ne-grained architecture

View Slide

@crichardson
About Chris
http://adopt.microservices.io
Late 80s 2006 2008 2009
2012-

View Slide

@crichardson
About Chris
https://chrisrichardson.net/

View Slide

@crichardson
Agenda
How micro is a microservice?

Designing a microservice architecture

Dark matter forces: reasons to not use services

Dark energy forces: reasons to use services

View Slide

Pattern: microservice
architecture
A service is:

Loosely coupled

Independently
deployable

Implements a business
capability

Owned by a small
team

View Slide

@crichardson
But how big is a service?
Microservice architecture
Must be small, right?!

View Slide

A common conversation
Client: We are having problems testing our services. Can you
help us?

Me: Sure. How many services?

Client: 5

Me: Oh ok. BTW How many developers?

Client: 5

Me: Oh. How about merging into one service?
1 service/developer is remarkably common

View Slide

@crichardson
Anti-pattern: The more the merrier
Excessively
fi
ne-grained
microservice architecture:

Reduced maintainability,
performance, availability, ….
https://microservices.io/post/antipatterns/2019/05/21/antipattern-more-the-
merrier.html

View Slide

@crichardson
Agenda




View Slide

@crichardson
How to de
fi
ne an
Architecture…
Application
≪subdomain≫

Customer
management
≪aggregate≫

Customer
≪subdomain≫

Order
management
≪aggregate≫

Order
createCustomer()
createOrder()
fi
ndOrder()
fi
ndOrderHistory()
System operations
Distill
Requirements The “requests” that the
application implements

Have SLOs
Customer Team
Order Team
About Subdomains
• Business capability/function/etc

• Logical view: packages and classes

• Team-sized

• Loosely coupled (Conways law)
1
2
Functional requirements
As a consumer
I want to place an Order
So that I can ….
As a Restaurant
I want to accept an Order
So that I can ….
Event storming
Wireframe/UI mockups
Available
Restaurants
Restaurant
Menu
System quality attributes
• SLA: Reliability/Latency
• Scalability
• …
https://microservices.io/post/architecture/2023/02/09/assemblage-architecture-de
fi
nition-process.html

View Slide

@crichardson
Subdomains: Eg. Java classes and
packages that implement business logic
Customer Management Order Management

View Slide

@crichardson
Kitchen Service
Delivery Service
Order Service
createOrder()
… how to de
fi
ne an Architecture
createOrder()
<>

Order Management
Order
System operations
<>

Order

Management
<>

Kitchen

Management
<>

Delivery

Management
<>

Courier

Management
Group

subdomains
into services
Application
Collaboration
Design
collaborations
for distributed

operations
createOrder()
3

View Slide

@crichardson
Grouping subdomains into
components: together or separate?
≪subdomain≫

Customer Mgmt.
≪aggregate≫

Customer
≪subdomain≫

Order Mgmt.
≪aggregate≫

Order
Attraction
Repulsion
Simple components

Team-sized services

Fast deployment pipeline

…
Dark energy: an anti-
gravity that’s accelerating
the expansion of the
universe
Dark matter: an invisible
matter that has a
gravitational effect on stars
and galaxies.
https://www.nasa.gov/feature/goddard/2020/new-hubble-data-explains-missing-dark-matter
Simple, ef
fi
cient interactions

Prefer ACID over BASE

Minimize runtime coupling

…
https://chrisrichardson.net/post/microservices/2021/04/15/mucon-2021-dark-energy-dark-matter.html
Generate
systemOperation()

View Slide

@crichardson
Dark energy and dark matter
forces
Subdomain A
«Aggregate»
X
Subdomain B
«Aggregate»
Y
Service A Service B
Attraction
Simple interactions
Eﬃcient interactions
Minimize design time coupling
Simple components
Team autonomy
Support multiple technology stacks
Segregate by characteristics
Repulsion
Dark energy
Dark matter
Metaphor for
Metaphor for

View Slide

@crichardson
Let’s imagine you want to
implement Coupons…
Order
Service
<>

Order
Management
Customer
Service
<>

Customer
Management
createCustomer() createOrder(…, couponId)
reserve

Credit()
As a Vendor

I want to create a coupon

So that I can …
As a Customer

I want to place an order

redeeming a coupon

So that I can …
AS-IS architecture New/modi
fi
ed user stories

View Slide

@crichardson
Application
Distill requirements to
operations and subdomains
As a Vendor

I want to create a coupon

So that I can …
As a Customer

I want to place an order

redeeming a coupon

So that I can …
createCoupon()

createOrder(..couponId)

…
<>

Coupon
Management
<>

Order
Management
Distill
New and modi
fi
ed

operations and
subdomains

View Slide

@crichardson
Key decision: New service or
existing service?…
Order
Service
<>

Order
Management
Customer
Service
<>

Customer
Management
<>

Coupon
Management
createCoupon(discount, …)
redeem(couponID)
Which service?
reserve

Credit()

View Slide

@crichardson
… Key decision: New service or existing
service?…
Coupon
Service
Order
Service
<>

Order
Management
Customer
Service
<>

Customer
Management
<>

Coupon
Management
redeem(couponID)
reserve

Credit()
Distributed
Order Service
<>

Order
Management
Customer
Service
<>

Customer
Management
createCustomer()

createOrder(…, couponId)
<>

Coupon
Management
reserve

Credit()
Local

View Slide

@crichardson
… Key decision: New service or existing service?
Coupon
Service
Order
Service
<>

Order
Management
<>

Coupon
Management
redeem(couponID)
Service
Service Service
Transaction
Compensating
transaction
Transaction
Compensating
transaction
Transaction
Compensating
transaction
command()
Saga
Service
Service
command()
Replica Source
Event
Command-side replica
API Composition
Provider
Service
Service
query()
Composer
Provider
Service
Provider
Service
CQRS
Provider
Service
Service
query()
View
Provider
Service
Provider
Service
Event
Event
Event
Collaboration design
createOrder()

View Slide

@crichardson
Evaluate alternative designs using dark energy and
dark matter forces
Within Order Service Coupon Service
Dark energy, repulsive forces
Simple components

? ?
Team autonomy


Dark matter, attractive forces
Simple interactions
? ?
Ef
fi
cient interactions


Minimize design-time coupling

View Slide

@crichardson
Agenda




View Slide

@crichardson
Dark matter attractive forces 㱺
subdomains in same service
Subdomain A
«Aggregate»
X
Subdomain B
«Aggregate»
Y
Service A Service B
Simple interactions
Eﬃcient interactions
Generates
SystemOperation()
Collaboration

View Slide

@crichardson
Dark matter forces: reasons to colocate
Order and Coupon management
Coupon
Service
Order
Service
Order Service
<>

Order
Management
<>

Order
Management
<>

Coupon
Management
<>

Coupon
Management
㱺
But do they apply in this situation?

Does de
fi
ning a new service create a problem?

View Slide

@crichardson
Simple interactions
Create

Order()
Service
Subdomain
A
Subdomain
B
Service B
Service A
Subdomain
A
Subdomain
B
Create

Order()
Complex distributed operation
Simple local operation: easier to
develop, test, understand, troubleshoot,
…
vs.

View Slide

@crichardson
Question: is each distributed
operation simple?
Additional
interaction
Additional
participant
Minimal increase in complexity but eventually …

View Slide

@crichardson
Distributed invocations are
expensive
Local method call: customerService.reserveCredit(customerID, amount)

Testing with mock objects

vs.
Distributed invocation:

CustomerServiceProxy

CustomerController

Sagas, compensating transactions, partial failure

Consumer-driven contract tests

…

View Slide

@crichardson
Time to troubleshoot

∝

Depth of distributed call chain

View Slide

@crichardson
Ef
fi
cient interactions
Create

Order()
Service
Subdomain
A
Subdomain
B
Service B
Service A
Subdomain
A
Subdomain
B
Create

Order()
Network latency, limited
bandwidth In-memory, fast!
vs.
Must satisfy
SLOs

View Slide

@crichardson
Question: is each distributed
operation ef
fi
cient enough?
Additional sequential interaction
Minimal reduction in ef
fi
ciency but eventually …

View Slide

@crichardson
System

Operation()
Service
Subdomain
A
Subdomain
B
Service B
Service A
Subdomain
A
Subdomain
B
System

Operation()
Distributed, eventually
consistent transaction Simple, Local ACID transaction
vs.
ACID txn ACID txn
ACID txn

View Slide

@crichardson
Question: is the distributed
operation “suf
fi
ciently” ACID?
Step Participant Transaction
Compensating
Transaction
1 Order Service createOrder() rejectOrder()
2 Coupon Service redeemCoupon() unredeemCoupon()
3 Customer Service reserveCredit() -
4 Order Service approveOrder() -
Create Order Saga
Doable but much more complex…

View Slide

@crichardson
System

Operation()
Service
Subdomain
A
Subdomain
B
Service B
Service A
Subdomain
A
Subdomain
B
System

Operation()
Risk of runtime coupling No runtime coupling: higher
availability, lower latency
vs.
Must satisfy
SLOs

View Slide

@crichardson
Question: does the distributed operation
meet its latency/availability SLO?
All must be available
More available
More complex
Wait
No Wait

View Slide

@crichardson
Risk: Silo’d teams have dif
fi
culty
identifying excessive runtime coupling
Payment Team:

“We just call the Fraud Service”
Fraud Team: “We have lots of services”

View Slide

@crichardson
Order
Subdomain
Customer
Subdomain

reserveCredit()
createOrder()
Customer
Order
Design-time coupling
Minimize with careful design

BUT

You can’t always eliminate it

㱺

Risk of lock step changes
API Risk proportional to:

• API instability

• API complexity

• …

View Slide

@crichardson
Question: are the two subdomains
suf
fi
ciently design-time decoupled?
interface CouponManagement {

redeemCoupon(couponID, amount)
interface CouponManagement {

redeemCoupon(couponID, amount, orderLineItems)
Not bad ✅
More complex, coupled to order concept ❌
vs.

View Slide

@crichardson
Summary: dark matter forces and Coupon
Management
Simple components

? ?
Team autonomy


Simple interactions
✅
✅❌
Ef
fi
cient interactions ✅

❌

❌✅
Minimize design-time coupling ✅
Creates
problems

View Slide

@crichardson
Agenda




View Slide

@crichardson
Dark energy repulsive forces 㱺
subdomains in different services
Service
Service
«Subdomain» A
«Aggregate»
X
«Subdomain» B
«Aggregate»
Y
Simple components
Team autonomy
Repulsive dark energy forces

View Slide

@crichardson
Dark energy forces: reasons
to create a Coupon Service
Coupon
Service
Order
Service
Order Service
<>

Order
Management
<>

Order
Management
<>

Coupon
Management
<>

Coupon
Management
㱺
But do they apply in this situation?

Does de
fi
ning a new service solve a problem?

View Slide

@crichardson
Simpler components/services
Service
Service
Service
Subdomain
A
Subdomain
A
Subdomain
B
Subdomain
B
More complex service
Simpler services: easier to
understand, develop, test, …
versus
dependencies

View Slide

Question: is the Order+Coupon
Service excessively complex?
Coupon management:
reasonably simple, no new
dependencies, …

Minimal additional complexity

Therefore
Separate Coupon Service is not
required

But
If Coupon management
becomes complex then separate
Order Service
main
main
Order Management
orders.web
couponAPI
orders.
domain
Coupon Management
coupons.
persistence
orders.
persistence
Coupon team
Order team
coupons.
domain
coupons.web
coupons.api
Modular Order Service

View Slide

@crichardson
Team autonomy = service per team
Service
Service
Service
Subdomain
A
Subdomain
A
Subdomain
B
Subdomain
B
Coordination required

Contention for resources
Develop, push, build, test
and deploy independently
vs.
Team A Team B Team A Team B

View Slide

@crichardson
Question: impact of embedding Coupon
Management in Order Service on

team autonomy?
Who develops Coupon Management?

Orders team

Team autonomy is not an issue

Different team, e.g. Coupons Team:

How much autonomy would they lose?

A few teams = probably ok

But there’s a limit

View Slide

@crichardson
Fast deployment pipeline…
@mipsytipsy

https://speakerdeck.com/charity/cd?slide=17

View Slide

@crichardson
… fast deployment pipeline
Service
Subdomain
Subdomain
Service
Subdomain
Shorter lead time

Simpler build
Longer lead time

More complex build*
Service
Subdomain
vs.
Deployment pipeline
Deployment pipeline
Deployment pipeline
* Parallelizing building/testing partially helps

View Slide

Question: Impact of adding Customer
Management to the Order Service?
Increase on test execution time?

testExecutionTime(Coupon
Management)?

Incremental build and test? Worst
case: changing one subdomain
requires testing the other

Increase in commit frequency?

More developers = more commits

Possibility of delays due to queuing
Order Service
main
main
Order Management
orders.web
couponAPI
orders.
domain
Coupon Management
coupons.
persistence
orders.
persistence
coupons.
domain
coupons.web
coupons.api
Stable?

View Slide

@crichardson
Support multiple technology
stacks
Service
Python
Service
Java
Service
JVM
Subdomain
A
Subdomain
A
Subdomain
B
Subdomain
B
Single technology stack

Upgrade together
Separate technology stacks

Right tool for the job

Upgrade independently

Experiment easily
versus

View Slide

@crichardson
Question: does Coupon Management
introduce technology stack issues?
Does Coupon Management use the same technology stack as
Order Management?

Same language, framework

Compatible transitive dependencies

Does Coupon Management introduce new dependencies that
would complicate technology upgrades?

Service upgrade effort proportional # dependencies

A dependency might not support newer versions of libraries,
JDK, etc

View Slide

@crichardson
Separate subdomains by
characteristics
Subdomain characteristic Issue
Resource requirements Cost-effective, scalability
Regulations, e.g. SaMD/
PCI
DevOps vs. Slower regulated process
Business criticality/tier Maximize availability
Security, e.g. PII, … Improve security
DDD core/supporting/
generic
Focus on being competitive

View Slide

@crichardson
Cost effective scaling
Service
Service
Service
Subdomain
A
Subdomain
A
Subdomain
B
Subdomain
B
versus
CPU MEM GPU
Scale together

• Wasteful

• Costly
CPU MEM GPU
Scale separately

• Ef
fi
cient

• Cheaper
Load Load Load Load
EC2: p4d.24xlarge EC2: p4d.24xlarge
EC2: m5.24xlarge
8x cost!
Unit of
scaling

View Slide

@crichardson
Example: Segregate by business criticality
Service
Service
Service
Payment
Processing
Payment
Processing
Merchant
management
Merchant
management
Shared infrastructure

Shared code base

Risk of interference
Separate infrastructure

Separate code base

Isolated
vs.
chargeCard()
2.9% + 30c/
request Revenue loss and penalties
chargeCard()
Critical
Important

View Slide

@crichardson
Question: How does Coupon
Management compare to Order
Management?
Subdomain characteristic Same?
Resource requirements ✅
Regulations, e.g. SaMD/PCI/… ✅
Business criticality/tier ✅
Security, e.g. PII, … ✅
DDD core/supporting/generic ✅

View Slide

@crichardson
Summary: designing Coupon Management
Simple components

✅ Doesn’t solve a
problem
Team autonomy


Simple interactions
✅
✅❌
Ef
fi
cient interactions ✅

❌

❌✅
Minimize design-time coupling ✅
Creates
problems

View Slide

@crichardson
Chosen design:
Order Service
<>

Order
Management
Customer
Service
<>

Customer
Management
createCustomer()

createOrder(…, couponId)
<>

Coupon
Management
reserve

Credit()
Service
Service
«Subdomain» A
«Aggregate»
X
«Subdomain» B
«Aggregate»
Y
Simple components
Team autonomy
Repulsive dark energy forces
Continual growth => eventually won’t resolve

View Slide

@crichardson
Summary: beware of Zombie
ideas

View Slide

@crichardson
Summary
Don’t take MICROservices literally

THINK! Brainstorm and evaluate candidate designs

Dark energy forces = reasons to use services

Dark matter forces = reasons to not use services

Con
fl
icting forces => must make trade-offs

Implementing subdomains:

JARs by default

As a service to solve a tangible dark energy-related problem
https://www.nasa.gov/feature/goddard/2019/nasa-s-james-webb-space-telescope-has-been-assembled-for-the-
fi
rst-time

View Slide

@crichardson
@crichardson [email protected]
adopt.microservices.io
Questions?

View Slide

Evolving a microservice architecture: how to right-size your services

Evolving a microservice architecture: how to right-size your services

Chris Richardson
PRO

More Decks by Chris Richardson

Other Decks in Programming

Featured

Transcript

Evolving a microservice architecture: how to right-size your services

Evolving a microservice architecture: how to right-size your services

Chris Richardson PRO

More Decks by Chris Richardson

Other Decks in Programming

Featured

Transcript

Chris Richardson
PRO