Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Evolving a microservice architecture: how to right-size your services

Evolving a microservice architecture: how to right-size your services

This is a talk I gave at SF JUG in May 2023.

Let’s imagine that you want to add a new feature to your microservices-based application. You might be tempted to simply define a new service. After all, it is the MICROservice architecture. The trouble, however, with blindly adding new services, is that it often leads to the More the Merrier anti-pattern. An overly complex architecture that’s difficult to maintain and, perhaps, brittle.

In this presentation, I describe the dark energy and dark matter forces, which are conflicting concerns that you must consider when designing a microservice architecture.You will learn how the dark energy forces encourage decomposition into finer-grained services in order to improve aspects of your architecture including testability, maintainability, scalability. I describe how the dark matter forces discourage decomposition in order to improve aspects such as efficiency, availability and data consistency. You will learn how to use these forces to evolve your application’s microservice architecture.

Chris Richardson

May 17, 2023
Tweet

More Decks by Chris Richardson

Other Decks in Programming

Transcript

  1. @crichardson
    Evolving a microservice
    architecture: how to right-size your
    services
    Chris Richardson


    Microservice architecture consultant and trainer


    Founder of the original CloudFoundry.com


    Author of POJOs in Action and Microservices Patterns


    Founder of Eventuate.io


    @crichardson


    [email protected]


    adopt.microservices.io
    Copyright © 2023. Chris Richardson Consulting, Inc. All rights reserved

    View full-size slide

  2. @crichardson
    Let’s imagine…
    Your microservices-based application
    + New feature:


    Coupons

    View full-size slide

  3. @crichardson
    Implement a new
    MICROservice, right?

    View full-size slide

  4. @crichardson
    Anti-pattern: The more the merrier
    Excessively
    fi
    ne-grained
    microservice architecture:


    Reduced maintainability,
    performance, availability, ….
    https://microservices.io/post/antipatterns/2019/05/21/antipattern-more-the-
    merrier.html

    View full-size slide

  5. @crichardson
    Presentation goal
    How to rightsize your services


    and


    avoid creating an overly complex,


    fi
    ne-grained architecture


    View full-size slide

  6. @crichardson
    About Chris
    http://adopt.microservices.io
    Late 80s 2006 2008 2009
    2012-

    View full-size slide

  7. @crichardson
    About Chris
    https://chrisrichardson.net/

    View full-size slide

  8. @crichardson
    Agenda
    How micro is a microservice?


    Designing a microservice architecture


    Dark matter forces: reasons to not use services


    Dark energy forces: reasons to use services

    View full-size slide

  9. Pattern: microservice
    architecture
    A service is:


    Loosely coupled


    Independently
    deployable


    Implements a business
    capability


    Owned by a small
    team

    View full-size slide

  10. @crichardson
    But how big is a service?
    Microservice architecture
    Must be small, right?!

    View full-size slide

  11. A common conversation
    Client: We are having problems testing our services. Can you
    help us?


    Me: Sure. How many services?


    Client: 5


    Me: Oh ok. BTW How many developers?


    Client: 5


    Me: Oh. How about merging into one service?
    1 service/developer is remarkably common

    View full-size slide

  12. @crichardson
    Anti-pattern: The more the merrier
    Excessively
    fi
    ne-grained
    microservice architecture:


    Reduced maintainability,
    performance, availability, ….
    https://microservices.io/post/antipatterns/2019/05/21/antipattern-more-the-
    merrier.html

    View full-size slide

  13. @crichardson
    Agenda
    How micro is a microservice?


    Designing a microservice architecture


    Dark matter forces: reasons to not use services


    Dark energy forces: reasons to use services

    View full-size slide

  14. @crichardson
    How to de
    fi
    ne an
    Architecture…
    Application
    ≪subdomain≫


    Customer
    management
    ≪aggregate≫


    Customer
    ≪subdomain≫


    Order
    management
    ≪aggregate≫


    Order
    createCustomer()
    createOrder()
    fi
    ndOrder()
    fi
    ndOrderHistory()
    System operations
    Distill
    Requirements The “requests” that the
    application implements


    Have SLOs
    Customer Team
    Order Team
    About Subdomains
    • Business capability/function/etc


    • Logical view: packages and classes


    • Team-sized


    • Loosely coupled (Conways law)
    1
    2
    Functional requirements
    As a consumer
    I want to place an Order
    So that I can ….
    As a Restaurant
    I want to accept an Order
    So that I can ….
    Event storming
    Wireframe/UI mockups
    Available
    Restaurants
    Restaurant
    Menu
    System quality attributes
    • SLA: Reliability/Latency
    • Scalability
    • …
    https://microservices.io/post/architecture/2023/02/09/assemblage-architecture-de
    fi
    nition-process.html

    View full-size slide

  15. @crichardson
    Subdomains: Eg. Java classes and
    packages that implement business logic
    Customer Management Order Management

    View full-size slide

  16. @crichardson
    Kitchen Service
    Delivery Service
    Order Service
    createOrder()
    … how to de
    fi
    ne an Architecture
    createOrder()
    <>


    Order Management
    Order
    System operations
    <>


    Order


    Management
    <>


    Kitchen


    Management
    <>


    Delivery


    Management
    <>


    Courier


    Management
    Group


    subdomains
    into services
    Application
    Collaboration
    Design
    collaborations
    for distributed


    operations
    createOrder()
    3

    View full-size slide

  17. @crichardson
    Grouping subdomains into
    components: together or separate?
    ≪subdomain≫


    Customer Mgmt.
    ≪aggregate≫


    Customer
    ≪subdomain≫


    Order Mgmt.
    ≪aggregate≫


    Order
    Attraction
    Repulsion
    Simple components


    Team-sized services


    Fast deployment pipeline



    Dark energy: an anti-
    gravity that’s accelerating
    the expansion of the
    universe
    Dark matter: an invisible
    matter that has a
    gravitational effect on stars
    and galaxies.
    https://www.nasa.gov/feature/goddard/2020/new-hubble-data-explains-missing-dark-matter
    Simple, ef
    fi
    cient interactions


    Prefer ACID over BASE


    Minimize runtime coupling



    https://chrisrichardson.net/post/microservices/2021/04/15/mucon-2021-dark-energy-dark-matter.html
    Generate
    systemOperation()

    View full-size slide

  18. @crichardson
    Dark energy and dark matter
    forces
    Subdomain A
    «Aggregate»
    X
    Subdomain B
    «Aggregate»
    Y
    Service A Service B
    Attraction
    Simple interactions
    Efficient interactions
    Prefer ACID over BASE
    Minimize runtime coupling
    Minimize design time coupling
    Simple components
    Team autonomy
    Fast deployment pipeline
    Support multiple technology stacks
    Segregate by characteristics
    Repulsion
    Dark energy
    Dark matter
    Metaphor for
    Metaphor for

    View full-size slide

  19. @crichardson
    Let’s imagine you want to
    implement Coupons…
    Order
    Service
    <>


    Order
    Management
    Customer
    Service
    <>


    Customer
    Management
    createCustomer() createOrder(…, couponId)
    reserve


    Credit()
    As a Vendor


    I want to create a coupon


    So that I can …
    As a Customer


    I want to place an order


    redeeming a coupon


    So that I can …
    AS-IS architecture New/modi
    fi
    ed user stories

    View full-size slide

  20. @crichardson
    Application
    Distill requirements to
    operations and subdomains
    As a Vendor


    I want to create a coupon


    So that I can …
    As a Customer


    I want to place an order


    redeeming a coupon


    So that I can …
    createCoupon()


    createOrder(..couponId)



    <>


    Coupon
    Management
    <>


    Order
    Management
    Distill
    New and modi
    fi
    ed


    operations and
    subdomains

    View full-size slide

  21. @crichardson
    Key decision: New service or
    existing service?…
    Order
    Service
    <>


    Order
    Management
    Customer
    Service
    <>


    Customer
    Management
    createCustomer() createOrder(…, couponId)
    <>


    Coupon
    Management
    createCoupon(discount, …)
    redeem(couponID)
    Which service?
    reserve


    Credit()

    View full-size slide

  22. @crichardson
    … Key decision: New service or existing
    service?…
    Coupon
    Service
    Order
    Service
    <>


    Order
    Management
    Customer
    Service
    <>


    Customer
    Management
    createCustomer() createOrder(…, couponId)
    <>


    Coupon
    Management
    createCoupon(discount, …)
    redeem(couponID)
    reserve


    Credit()
    Distributed
    Order Service
    <>


    Order
    Management
    Customer
    Service
    <>


    Customer
    Management
    createCustomer()
    createCoupon(discount, …)


    createOrder(…, couponId)
    <>


    Coupon
    Management
    reserve


    Credit()
    Local

    View full-size slide

  23. @crichardson
    … Key decision: New service or existing service?
    Coupon
    Service
    Order
    Service
    <>


    Order
    Management
    <>


    Coupon
    Management
    redeem(couponID)
    Service
    Service Service
    Transaction
    Compensating
    transaction
    Transaction
    Compensating
    transaction
    Transaction
    Compensating
    transaction
    command()
    Saga
    Service
    Service
    command()
    Replica Source
    Event
    Command-side replica
    API Composition
    Provider
    Service
    Service
    query()
    Composer
    Provider
    Service
    Provider
    Service
    CQRS
    Provider
    Service
    Service
    query()
    View
    Provider
    Service
    Provider
    Service
    Event
    Event
    Event
    Collaboration design
    createOrder()

    View full-size slide

  24. @crichardson
    Evaluate alternative designs using dark energy and
    dark matter forces
    Within Order Service Coupon Service
    Dark energy, repulsive forces
    Simple components


    ? ?
    Team autonomy
    Fast deployment pipeline
    Support multiple technology stacks


    Segregate by characteristics


    Dark matter, attractive forces
    Simple interactions
    ? ?
    Ef
    fi
    cient interactions
    Prefer ACID over BASE


    Minimize runtime coupling


    Minimize design-time coupling

    View full-size slide

  25. @crichardson
    Agenda
    How micro is a microservice?


    Designing a microservice architecture


    Dark matter forces: reasons to not use services


    Dark energy forces: reasons to use services

    View full-size slide

  26. @crichardson
    Dark matter attractive forces 㱺
    subdomains in same service
    https://chrisrichardson.net/post/microservices/2021/04/15/mucon-2021-dark-energy-dark-matter.html
    Subdomain A
    «Aggregate»
    X
    Subdomain B
    «Aggregate»
    Y
    Service A Service B
    Simple interactions
    Efficient interactions
    Prefer ACID over BASE
    Minimize runtime coupling
    Minimize design time coupling
    Generates
    SystemOperation()
    Collaboration

    View full-size slide

  27. @crichardson
    Dark matter forces: reasons to colocate
    Order and Coupon management
    Coupon
    Service
    Order
    Service
    Order Service
    <>


    Order
    Management
    <>


    Order
    Management
    <>


    Coupon
    Management
    <>


    Coupon
    Management

    But do they apply in this situation?


    Does de
    fi
    ning a new service create a problem?

    View full-size slide

  28. @crichardson
    Simple interactions
    Create


    Order()
    Service
    Subdomain
    A
    Subdomain
    B
    Service B
    Service A
    Subdomain
    A
    Subdomain
    B
    Create


    Order()
    Complex distributed operation
    Simple local operation: easier to
    develop, test, understand, troubleshoot,

    vs.

    View full-size slide

  29. @crichardson
    Question: is each distributed
    operation simple?
    Additional
    interaction
    Additional
    participant
    Minimal increase in complexity but eventually …

    View full-size slide

  30. @crichardson
    Distributed invocations are
    expensive
    Local method call: customerService.reserveCredit(customerID, amount)


    Testing with mock objects


    vs.
    Distributed invocation:


    CustomerServiceProxy


    CustomerController


    Sagas, compensating transactions, partial failure


    Consumer-driven contract tests



    View full-size slide

  31. @crichardson
    Time to troubleshoot





    Depth of distributed call chain

    View full-size slide

  32. @crichardson
    Ef
    fi
    cient interactions
    Create


    Order()
    Service
    Subdomain
    A
    Subdomain
    B
    Service B
    Service A
    Subdomain
    A
    Subdomain
    B
    Create


    Order()
    Network latency, limited
    bandwidth In-memory, fast!
    vs.
    Must satisfy
    SLOs

    View full-size slide

  33. @crichardson
    Question: is each distributed
    operation ef
    fi
    cient enough?
    Additional sequential interaction
    Minimal reduction in ef
    fi
    ciency but eventually …

    View full-size slide

  34. @crichardson
    Prefer ACID over BASE
    System


    Operation()
    Service
    Subdomain
    A
    Subdomain
    B
    Service B
    Service A
    Subdomain
    A
    Subdomain
    B
    System


    Operation()
    Distributed, eventually
    consistent transaction Simple, Local ACID transaction
    vs.
    ACID txn ACID txn
    ACID txn

    View full-size slide

  35. @crichardson
    Question: is the distributed
    operation “suf
    fi
    ciently” ACID?
    Step Participant Transaction
    Compensating
    Transaction
    1 Order Service createOrder() rejectOrder()
    2 Coupon Service redeemCoupon() unredeemCoupon()
    3 Customer Service reserveCredit() -
    4 Order Service approveOrder() -
    Create Order Saga
    Doable but much more complex…

    View full-size slide

  36. @crichardson
    Minimize runtime coupling
    System


    Operation()
    Service
    Subdomain
    A
    Subdomain
    B
    Service B
    Service A
    Subdomain
    A
    Subdomain
    B
    System


    Operation()
    Risk of runtime coupling No runtime coupling: higher
    availability, lower latency
    vs.
    Must satisfy
    SLOs

    View full-size slide

  37. @crichardson
    Question: does the distributed operation
    meet its latency/availability SLO?
    All must be available
    More available
    More complex
    Wait
    No Wait

    View full-size slide

  38. @crichardson
    Risk: Silo’d teams have dif
    fi
    culty
    identifying excessive runtime coupling
    Payment Team:


    “We just call the Fraud Service”
    Fraud Team: “We have lots of services”

    View full-size slide

  39. @crichardson
    Minimize design time coupling
    Order
    Subdomain
    Customer
    Subdomain


    reserveCredit()
    createOrder()
    Customer
    Order
    Design-time coupling
    Minimize with careful design


    BUT


    You can’t always eliminate it





    Risk of lock step changes
    API Risk proportional to:


    • API instability


    • API complexity


    • …

    View full-size slide

  40. @crichardson
    Question: are the two subdomains
    suf
    fi
    ciently design-time decoupled?
    interface CouponManagement {


    redeemCoupon(couponID, amount)
    interface CouponManagement {


    redeemCoupon(couponID, amount, orderLineItems)
    Not bad ✅
    More complex, coupled to order concept ❌
    vs.

    View full-size slide

  41. @crichardson
    Summary: dark matter forces and Coupon
    Management
    Within Order Service Coupon Service
    Dark energy, repulsive forces
    Simple components


    ? ?
    Team autonomy
    Fast deployment pipeline
    Support multiple technology stacks


    Segregate by characteristics


    Dark matter, attractive forces
    Simple interactions

    ✅❌
    Ef
    fi
    cient interactions ✅
    Prefer ACID over BASE



    Minimize runtime coupling


    ❌✅
    Minimize design-time coupling ✅
    Creates
    problems

    View full-size slide

  42. @crichardson
    Agenda
    How micro is a microservice?


    Designing a microservice architecture


    Dark matter forces: reasons to not use services


    Dark energy forces: reasons to use services

    View full-size slide

  43. @crichardson
    Dark energy repulsive forces 㱺
    subdomains in different services
    https://chrisrichardson.net/post/microservices/2021/04/15/mucon-2021-dark-energy-dark-matter.html
    Service
    Service
    «Subdomain» A
    «Aggregate»
    X
    «Subdomain» B
    «Aggregate»
    Y
    Simple components
    Team autonomy
    Fast deployment pipeline
    Support multiple technology stacks
    Segregate by characteristics
    Repulsive dark energy forces

    View full-size slide

  44. @crichardson
    Dark energy forces: reasons
    to create a Coupon Service
    Coupon
    Service
    Order
    Service
    Order Service
    <>


    Order
    Management
    <>


    Order
    Management
    <>


    Coupon
    Management
    <>


    Coupon
    Management

    But do they apply in this situation?


    Does de
    fi
    ning a new service solve a problem?

    View full-size slide

  45. @crichardson
    Simpler components/services
    Service
    Service
    Service
    Subdomain
    A
    Subdomain
    A
    Subdomain
    B
    Subdomain
    B
    More complex service
    Simpler services: easier to
    understand, develop, test, …
    versus
    dependencies

    View full-size slide

  46. Question: is the Order+Coupon
    Service excessively complex?
    Coupon management:
    reasonably simple, no new
    dependencies, …


    Minimal additional complexity


    Therefore
    Separate Coupon Service is not
    required


    But
    If Coupon management
    becomes complex then separate
    Order Service
    main
    main
    Order Management
    orders.web
    couponAPI
    orders.
    domain
    Coupon Management
    coupons.
    persistence
    orders.
    persistence
    Coupon team
    Order team
    coupons.
    domain
    coupons.web
    coupons.api
    Modular Order Service

    View full-size slide

  47. @crichardson
    Team autonomy = service per team
    Service
    Service
    Service
    Subdomain
    A
    Subdomain
    A
    Subdomain
    B
    Subdomain
    B
    Coordination required


    Contention for resources
    Develop, push, build, test
    and deploy independently
    vs.
    Team A Team B Team A Team B

    View full-size slide

  48. @crichardson
    Question: impact of embedding Coupon
    Management in Order Service on


    team autonomy?
    Who develops Coupon Management?


    Orders team


    Team autonomy is not an issue


    Different team, e.g. Coupons Team:


    How much autonomy would they lose?


    A few teams = probably ok


    But there’s a limit

    View full-size slide

  49. @crichardson
    Fast deployment pipeline…
    @mipsytipsy


    https://speakerdeck.com/charity/cd?slide=17

    View full-size slide

  50. @crichardson
    … fast deployment pipeline
    Service
    Subdomain
    Subdomain
    Service
    Subdomain
    Shorter lead time


    Simpler build
    Longer lead time


    More complex build*
    Service
    Subdomain
    vs.
    Deployment pipeline
    Deployment pipeline
    Deployment pipeline
    * Parallelizing building/testing partially helps

    View full-size slide

  51. Question: Impact of adding Customer
    Management to the Order Service?
    Increase on test execution time?


    testExecutionTime(Coupon
    Management)?


    Incremental build and test? Worst
    case: changing one subdomain
    requires testing the other


    Increase in commit frequency?


    More developers = more commits


    Possibility of delays due to queuing
    Order Service
    main
    main
    Order Management
    orders.web
    couponAPI
    orders.
    domain
    Coupon Management
    coupons.
    persistence
    orders.
    persistence
    coupons.
    domain
    coupons.web
    coupons.api
    Stable?

    View full-size slide

  52. @crichardson
    Support multiple technology
    stacks
    Service
    Python
    Service
    Java
    Service
    JVM
    Subdomain
    A
    Subdomain
    A
    Subdomain
    B
    Subdomain
    B
    Single technology stack


    Upgrade together
    Separate technology stacks


    Right tool for the job


    Upgrade independently


    Experiment easily
    versus

    View full-size slide

  53. @crichardson
    Question: does Coupon Management
    introduce technology stack issues?
    Does Coupon Management use the same technology stack as
    Order Management?


    Same language, framework


    Compatible transitive dependencies


    Does Coupon Management introduce new dependencies that
    would complicate technology upgrades?


    Service upgrade effort proportional # dependencies


    A dependency might not support newer versions of libraries,
    JDK, etc

    View full-size slide

  54. @crichardson
    Separate subdomains by
    characteristics
    Subdomain characteristic Issue
    Resource requirements Cost-effective, scalability
    Regulations, e.g. SaMD/
    PCI
    DevOps vs. Slower regulated process
    Business criticality/tier Maximize availability
    Security, e.g. PII, … Improve security
    DDD core/supporting/
    generic
    Focus on being competitive

    View full-size slide

  55. @crichardson
    Cost effective scaling
    Service
    Service
    Service
    Subdomain
    A
    Subdomain
    A
    Subdomain
    B
    Subdomain
    B
    versus
    CPU MEM GPU
    Scale together


    • Wasteful


    • Costly
    CPU MEM GPU
    Scale separately


    • Ef
    fi
    cient


    • Cheaper
    Load Load Load Load
    EC2: p4d.24xlarge EC2: p4d.24xlarge
    EC2: m5.24xlarge
    8x cost!
    Unit of
    scaling

    View full-size slide

  56. @crichardson
    Example: Segregate by business criticality
    Service
    Service
    Service
    Payment
    Processing
    Payment
    Processing
    Merchant
    management
    Merchant
    management
    Shared infrastructure


    Shared code base


    Risk of interference
    Separate infrastructure


    Separate code base


    Isolated
    vs.
    chargeCard()
    2.9% + 30c/
    request Revenue loss and penalties
    chargeCard()
    Critical
    Important

    View full-size slide

  57. @crichardson
    Question: How does Coupon
    Management compare to Order
    Management?
    Subdomain characteristic Same?
    Resource requirements ✅
    Regulations, e.g. SaMD/PCI/… ✅
    Business criticality/tier ✅
    Security, e.g. PII, … ✅
    DDD core/supporting/generic ✅

    View full-size slide

  58. @crichardson
    Summary: designing Coupon Management
    Within Order Service Coupon Service
    Dark energy, repulsive forces
    Simple components


    ✅ Doesn’t solve a
    problem
    Team autonomy
    Fast deployment pipeline
    Support multiple technology stacks


    Segregate by characteristics


    Dark matter, attractive forces
    Simple interactions

    ✅❌
    Ef
    fi
    cient interactions ✅
    Prefer ACID over BASE



    Minimize runtime coupling


    ❌✅
    Minimize design-time coupling ✅
    Creates
    problems

    View full-size slide

  59. @crichardson
    Chosen design:
    Order Service
    <>


    Order
    Management
    Customer
    Service
    <>


    Customer
    Management
    createCustomer()
    createCoupon(discount, …)


    createOrder(…, couponId)
    <>


    Coupon
    Management
    reserve


    Credit()
    Service
    Service
    «Subdomain» A
    «Aggregate»
    X
    «Subdomain» B
    «Aggregate»
    Y
    Simple components
    Team autonomy
    Fast deployment pipeline
    Support multiple technology stacks
    Segregate by characteristics
    Repulsive dark energy forces
    Continual growth => eventually won’t resolve

    View full-size slide

  60. @crichardson
    Summary: beware of Zombie
    ideas

    View full-size slide

  61. @crichardson
    Summary
    Don’t take MICROservices literally


    THINK! Brainstorm and evaluate candidate designs
    Designing a microservice architecture


    Dark energy forces = reasons to use services


    Dark matter forces = reasons to not use services


    Con
    fl
    icting forces => must make trade-offs


    Implementing subdomains:


    JARs by default


    As a service to solve a tangible dark energy-related problem
    https://www.nasa.gov/feature/goddard/2019/nasa-s-james-webb-space-telescope-has-been-assembled-for-the-
    fi
    rst-time

    View full-size slide

  62. @crichardson
    @crichardson [email protected]
    adopt.microservices.io
    Questions?

    View full-size slide