$30 off During Our Annual Pro Sale. View Details »

Getting (service) design right

Getting (service) design right

This slide deck basically tries to address functional service design not only from a maintainability point of view, but also form an operational point of view. How to spread your business functionality across your service landscape to maximize availability and responsiveness?

It starts with the observation that virtually all system landscapes these days, including cloud-native, are distributed systems. Due to the failure modes of distributed systems every remote communication basically is a predetermined breaking point of your system.

After briefly discussing other options to increase availability and responsiveness the biggest part of the deck discusses the question how a good functional design can positively influence the aforementioned system properties.

Using a very simple eCommerce application as an example, first a typical design is shown and why it is suboptimal in terms of availability and responsiveness. After that, we briefly dive into classical CS papers and see what we can learn from them. We also learn that while the core insights of those papers are mostly timeless, their implementation instructions are not and that we need to update them to our current context.

Then we use the learnt ideas to derive a different functional decomposition approach starting with the system behavior and apply it to our example - which leads to a lot better system behavior in terms of availability and responsiveness. Finally we discuss the most important trade-offs of the different design approach - as every design decision of course also has its downsides ... ;)

As always, the voice track is missing. Still, I hope the slides on their own also offer you a few valuable ideas.

Uwe Friedrichsen

March 07, 2019
Tweet

More Decks by Uwe Friedrichsen

Other Decks in Technology

Transcript

  1. Getting (service) design right
    or how timeless wisdom can lead the way
    Uwe Friedrichsen – codecentric AG – 2016-2019

    View Slide

  2. Uwe Friedrichsen
    IT traveller.
    Dot Connector.
    Cartographer of uncharted territory.
    Keeper of timeless wisdom.
    CTO and Fellow at codecentric.
    https://www.speakerdeck.com/ufried
    https://medium.com/@ufried
    @ufried

    View Slide

  3. The marvelous world of microservices

    View Slide

  4. So, you are doing microservices ...

    View Slide

  5. Yeah!

    View Slide

  6. ... of course, putting them in Docker ...

    View Slide

  7. Yeah!

    View Slide

  8. ... and running them on Kubernetes

    View Slide

  9. Yeah!

    View Slide

  10. So, your are doing it the cloud-native way

    View Slide

  11. “Cloud native computing uses an open source software stack
    to deploy applications as microservices,
    packaging each part into its own container,
    and dynamically orchestrating those containers
    to optimize resource utilization.”
    Source: Cloud Native Computing Foundation Homepage (https://www.cncf.io/)

    View Slide

  12. Yeah!

    View Slide

  13. Well, ever pondered the consequences?

    View Slide

  14. Eh, what?

    View Slide

  15. Why do
    you ask?

    View Slide

  16. Consequences of cloud-native computing

    View Slide

  17. Different processes at runtime
    Remote communication between services
    Distributed system
    Cloud-native computing
    Services packaged in containers

    View Slide

  18. Distributed systems are not limited to cloud-native computing

    View Slide

  19. (Almost) every system is a distributed system
    -- Chas Emerick
    http://www.infoq.com/presentations/problems-distributed-systems

    View Slide

  20. The software you develop and maintain is most likely
    part of a (big) distributed system landscape

    View Slide

  21. Consequences of distributed systems

    View Slide

  22. Everything fails, all the time.
    -- Werner Vogels

    View Slide

  23. Failures in distributed systems ...
    • Crash failure
    • Omission failure
    • Timing failure
    • Response failure
    • Byzantine failure

    View Slide

  24. ... lead to a variety of effects
    • Lost messages
    • Incomplete messages
    • Duplicate messages
    • Distorted messages
    • Out-of-order message arrival
    • Partial, out-of-sync local memory
    • ...

    View Slide

  25. These effects are based on the non-determinism
    introduced by remote communication
    due to distributed failure modes

    View Slide

  26. Understand that remote communication points are
    predetermined breaking points of your application
    Accept that the effects will hit you at the application level

    View Slide

  27. What can we do against it?

    View Slide

  28. Typical measures
    • High-availability (HA) hardware/software
    • Only applicable for very small installations
    • Usually not available in cloud environments
    • Delegate failure handling to infrastructure level
    • Partial relief, will not solve all problems
    • Implement resilient software design patterns
    • Very important, still will not fix a bad design
    • Minimize number of remote communication points
    • Minimize problem surface by design

    View Slide

  29. Reducing remote communication points

    View Slide

  30. Reducing remote communication points
    Reduce the number of
    distributed runtime artifacts
    i.e., create coarser grained services
    Reduce the need to
    communicate across
    runtime artifact boundaries
    i.e., get the functional design right

    View Slide

  31. Reducing remote communication points
    Reduce the number of
    distributed runtime artifacts
    i.e., create coarser grained services
    Reduce the need to
    communicate across
    runtime artifact boundaries
    i.e., get the functional design right

    View Slide

  32. Remember?
    https://www.martinfowler.com/articles/
    distributed-objects-microservices.html

    View Slide

  33. If you can get away with a monolith, just do it!
    You are still allowed to structure it well ... ;)

    View Slide

  34. Still, your functional domain usually will be too big
    to put it all in a single monolith
    Thus, you will need to distribute your system landscape to a certain degree

    View Slide

  35. Rule of thumb
    Always think (at least) twice before distributing functionality
    Only do it if you really need independent deployments
    Addendum
    You may also want to distribute your functionality if you have
    very disparate NFRs for different parts of your functionality
    (e.g., security *, availability, reliability, scalability **, portability,
    resource utilization)
    * forgotten most of the time
    ** less often needed than most people assume

    View Slide

  36. Reducing remote communication points
    Reduce the number of
    distributed runtime artifacts
    i.e., create coarser grained services
    Reduce the need to
    communicate across
    runtime artifact boundaries
    i.e., get the functional design right

    View Slide

  37. Case study
    (Very simple) eCommerce shop
    • Implements the core functionalities
    • Search & show
    • Add to shopping cart
    • Checkout
    • Shipment
    • Payments only touched as black box
    • No recommendations, etc.

    View Slide

  38. The typical design approach ...
    a.k.a. “the counter-example”

    View Slide

  39. Typical design approach
    Focus on avoiding redundancy and maximizing reuse
    1. Start with a comprehensive domain (actually: E/R) model

    View Slide

  40. *
    *
    *
    *
    Customer
    • name
    • address
    • payment methods
    E
    Order
    • customer
    • payment information
    • shipping information
    • order items
    • product
    • quantity
    E
    Product
    • name
    • description
    • image(s)
    • price
    • items in stock
    • packaging information
    (size, weight, special care)
    E

    View Slide

  41. Typical design approach
    Focus on avoiding redundancy and maximizing re-use
    1. Start with a comprehensive domain (actually: E/R) model
    2. Wrap entities with services

    View Slide

  42. CustomerService
    S
    Customer OrderService
    S
    Order
    ProductService
    S
    Product

    View Slide

  43. Typical design approach
    Focus on avoiding redundancy and maximizing re-use
    1. Start with a comprehensive domain (actually: E/R) model
    2. Wrap entities with services
    3. Spread functionality over services

    View Slide

  44. ProductService
    OrderService
    CustomerService
    S
    Customer OrderService
    • Add to shopping cart
    S
    ProductService
    • Search/show
    S
    Order
    Product

    View Slide

  45. Typical design approach
    Focus on avoiding redundancy and maximizing re-use
    1. Start with a comprehensive domain (actually: E/R) model
    2. Wrap entities with services
    3. Spread functionality over services
    4. Add “process services” for “complex use cases”
    • i.e., use cases that touch more than one data service

    View Slide

  46. CustomerService
    S
    Customer OrderService
    • Add to shopping cart
    S
    ProductService
    • Search/show
    S
    Order
    Product
    CheckoutService
    • Checkout
    S
    ShipmentService
    • Shipment
    S

    View Slide

  47. Typical design approach
    Focus on avoiding redundancy and maximizing re-use
    1. Start with a comprehensive domain (actually: E/R) model
    2. Wrap entities with services
    3. Spread functionality over services
    4. Add “process services” for “complex use cases”
    • i.e., use cases that touch more than one data service
    5. Add missing data maintenance use cases

    View Slide

  48. CustomerService
    ProductService
    • Search/show
    CustomerService
    • Customer self care
    S
    Customer OrderService
    • Add to shopping cart
    S
    ProductService
    • Search/show
    • Product catalog
    maintenance
    S
    CheckoutService
    • Checkout
    S
    ShipmentService
    • Shipment
    S
    Order
    Product

    View Slide

  49. This (familiar) design looks innocuous at first sight
    But how good is it in terms of remote communication?

    View Slide

  50. CheckoutService OrderService ProductService CustomerService Payment Provider

    read order
    read price
    loop [order items]
    calculate price
    read payment methods
    for payment method>

    pay
    mark paid

    View Slide

  51. ShipmentService OrderService ProductService CustomerService Delivery Provider

    read order
    read product and packaging information
    loop [order items]
    read delivery address
    information>
    initiate delivery>
    inform delivery provider
    mark dispatched
    update items in stock
    loop [order items]

    View Slide

  52. Findings
    • Core business use cases are failure-prone and slow
    • Data maintenance use cases are robust and fast

    View Slide

  53. Congratulations!
    You designed a system for a company
    that's core business purpose is
    to maintain data, not to make money!

    View Slide

  54. Properties of the design
    • Focus on avoiding redundancy and maximizing reuse
    • Based on traditional OO design practices
    • Results in high coupling between services
    • Results in moderate cohesion inside services
    • Okay for CRUD applications
    • But then better use a generator, scaffolding framework, ...
    • Okay-ish for single-process applications
    • Tends to affect maintainability negatively
    • Not okay for distributed services
    • Big failure surface, bad response times

    View Slide

  55. How can we do better?

    View Slide

  56. Let’s do a bit of research ...

    View Slide

  57. Structured design
    by W. P. Stevens, G. J. Myers and L. L. Constantine
    [Stev 1974]

    View Slide

  58. “The fewer and simpler the connections between modules, the easier it is to
    understand each module without reference to other modules.
    Minimizing connections between modules also minimizes the paths
    along which changes and errors can propagate into other parts of the
    system, thus eliminating disastrous ‘ripple’ effects, where changes in one
    part cause errors in another, necessitating additional changes elsewhere,
    giving rise to new errors, etc.”
    [Ste 1974]

    View Slide

  59. “Coupling is the measure of the strength of association
    established by a connection from one module to another.”
    [Ste 1974]

    View Slide

  60. Coupling
    High
    Low
    Contributing factors
    Interface
    Complexity
    Type of
    Connection
    Type of
    Communication
    Simple,
    obvious
    Complicated,
    obscure
    To module
    by name
    (depending
    on interface)
    To internal
    elements
    (depending on
    implementation)
    Data
    (control flow handled
    by environment)
    Control
    (Explicit passing
    of control)
    Hybrid
    (Manipulation of internal
    control flow by parameters)
    [Ste 1974]

    View Slide

  61. Realize that this paper was written at a very different time
    and in a very different context than we face today
    While the core concepts are timeless and still valid,
    we usually need to rethink the concrete instructions

    View Slide

  62. Contributing factors
    Interface
    Complexity
    Type of
    Connection
    Type of
    Communication
    Simple,
    obvious
    Complicated,
    obscure
    To module
    by name
    (depending
    on interface)
    To internal
    elements
    (depending on
    implementation)
    Data
    (control flow handled
    by environment)
    Control
    (Explicit passing
    of control)
    Hybrid
    (Manipulation of internal
    control flow by parameters)
    * Ability of a service to complete its task without the other service being present
    Functional
    Independence *
    Independent
    (does not need other
    service to work)
    Fully dependent
    (does not work
    without other service)
    Partly dependent
    (graceful degradation
    of service)
    Coupling
    High
    Low

    View Slide

  63. “Coupling is reduced when the relationships among elements not in the
    same module are minimized.
    There are two ways of achieving this – minimizing the relationships among
    modules and maximizing relationships among elements in the same
    module. In practice, both ways are used. [...]
    Binding is the measure of the cohesiveness of a module. The objective
    here is to reduce coupling by striving for high binding.”
    [Ste 1974]

    View Slide

  64. On the criteria to be
    used in decomposing
    systems into modules
    by David L. Parnas
    [Par 1972]

    View Slide

  65. Separation of concerns
    One concept/decision per module
    Information hiding
    Reveal as little as possible about
    internal implementation
    +
    Better changeability
    Changes are kept local
    Independent teams
    Teams can easier work
    independently on different
    modules
    Easier to comprehend
    Modules can be understood on
    their own easier

    View Slide

  66. Research findings
    • High cohesion, low coupling leads the right way
    • Separation of Concerns and Information hiding
    support implementing them
    • Concrete paper instructions should not be followed blindly
    • Different context (single process, very limited hardware)
    • Would lead to nano or pico services à lots of remote calls
    • You need to rethink instructions in the current context
    • Required for all CS papers from a different time & context
    • Leads to concept of Functional Independence in this context
    • Reduces risk of “vertical decomposition” (i.e., layered design)

    View Slide


  67. Functionality
    vertical decomposition
    (layer design, composition)
    In practice, you typically
    use a combination
    of both approaches
    Core functional decomposition approaches
    horizontal decomposition
    (pillar design, segregation)

    View Slide

  68. Vertical decomposition
    • Based on “uses” relation
    • Typical drivers are reuse and avoidance of redundancy
    • Creates strong coupling (high functional dependence)
    • Often useful pattern inside a process boundary
    • Due to deterministic communication behavior
    • Problematic across process boundaries
    à Should be avoided in service design

    View Slide

  69. Horizontal decomposition
    • Based on functional segregation
    • Typical drivers are autonomy and independence
    • Creates low coupling (high functional independence)
    • Useful pattern across process boundaries
    • Can also be useful inside a process boundary
    • Less accidental "ripple" effects due to changes
    à Should be preferred in service design

    View Slide

  70. Watch out!
    • Vertical decomposition is our default design approach
    • We’ve learned it in our CS education (divide and conquer, ...)
    • It’s emphasized in our daily work (DRY, reusability, ...)
    • Even our IDEs support it (“Extract method”)
    • It's everywhere! It's predominant!
    • It takes energy not to apply vertical decomposition
    • Most people never learned horizontal decomposition

    View Slide

  71. How to learn horizontal decomposition?

    View Slide

  72. Domain-Driven Design
    by Eric Evans
    [Eva 2004]

    View Slide

  73. DDD to the rescue?
    • Naive application of building block patterns leads
    to the counter-example design we have seen before
    • Not useful in our context due to high coupling
    • “Service” pattern leads to process service working on entities
    • Anti-pattern in our context due to high coupling
    • “Conceptual contours” supports high cohesion
    • Yet, tends to be too fine grained for our context
    • “Bounded contexts” supports low coupling
    • Yet, tends to be too coarse grained for our context
    à Mixed emotions: Good, but not the expected panacea

    View Slide

  74. And now?

    View Slide

  75. For good service design, look at the behavior first, not the data

    View Slide

  76. Case study
    (Very simple) eCommerce shop
    • Implements the core functionalities
    • Search & show
    • Add to shopping cart
    • Checkout
    • Shipment
    • Customer self-care
    • Product catalog maintenance
    • Payments only touched as black box
    • No recommendations, etc.

    View Slide

  77. Core reasoning
    To reduce the number of remote calls needed for a given functionality,
    we need to spread the functionality between the services in a way
    that a single use case/user interaction less often needs to cross service
    boundaries.
    Therefore, we try to organize services around use cases/user interactions.

    View Slide

  78. Search & Show
    Add to
    shopping cart
    Checkout
    Shipment
    Customer
    self-care
    Product catalog
    maintenance
    eCommerce shop
    Customer
    Back office
    employee
    Warehouse
    employee

    View Slide

  79. Search & Show
    Add to
    shopping cart
    Checkout
    Shipment
    Customer
    self-care
    Product catalog
    maintenance
    eCommerce shop
    Customer
    Back office
    employee
    Warehouse
    employee
    Three different actors
    • Indicator for cohesion boundaries
    • (At least) three different UIs
    • Could be completely different architectures
    • Depending on user needs, usage patterns and other NFRs
    • As an architect this gives you additional options

    View Slide

  80. Warehouse
    employee
    Search & Show
    Add to
    shopping cart
    Checkout
    Shipment
    Customer
    self-care
    Product catalog
    maintenance
    eCommerce shop
    Customer
    Back office
    employee
    Could be a mobile-first FE
    with service-oriented backend
    Could be a special warehouse device FE
    with a monolithic backend
    Could be a rich
    desktop app
    Could be a
    desktop browser
    first FE with a
    service-oriented
    backend

    View Slide

  81. Search & Show
    Add to
    shopping cart
    Checkout
    Shipment
    Customer
    self-care
    Product catalog
    maintenance

    View Slide

  82. Behavior-based design approach
    Focus on minimum cross-service communication
    inside a use case/user interaction
    1. Each use case/user interaction is a service candidate

    View Slide

  83. ProductCatalogService
    • Product catalog
    maintenance
    S
    ShipmentService
    • Shipment
    S
    CustomerMDService *
    • Customer self care
    S
    Candidate
    Candidate
    Candidate
    ShoppingCartService
    • Add to shopping cart
    S
    CheckoutService
    • Checkout
    S
    SearchService
    • Search/show
    S
    Candidate
    Candidate
    Candidate
    * MD = Master Data

    View Slide

  84. Behavior-based design approach
    Focus on minimum cross-service communication
    inside a use case/user interaction
    1. Each use case/user interaction is a service candidate
    2. Possibly split big use cases in multiple services
    • Only if really needed (e.g., multiple teams, disparate NFRs)
    • Look for functional clusters with low coupling between them

    View Slide

  85. ShoppingCartService
    • Add to shopping cart
    S
    CheckoutService
    • Checkout
    S
    SearchService
    • Search/show
    S
    Candidate
    Candidate
    Candidate
    ProductCatalogService
    • Product catalog
    maintenance
    S
    ShipmentService
    • Shipment
    S
    CustomerMDService
    • Customer self care
    S
    Candidate
    Candidate
    Candidate
    Splitting up use cases in multiple services not needed in this example

    View Slide

  86. Behavior-based design approach
    Focus on minimum cross-service communication
    inside a use case/user interaction
    1. Each use case/user interaction is a service candidate
    2. Possibly split big use cases in multiple services
    • Only if really needed (e.g., multiple teams, disparate NFRs)
    • Look for functional clusters with low coupling between them
    3. Try to group several use cases in a single service
    • Strive for a sweet spot in terms of an overall trade-off
    • Look for service candidates that operate on the same data

    View Slide

  87. ShoppingCartService
    • Add to shopping cart
    S
    ProductCatalogService
    • Product catalog
    maintenance
    S
    CheckoutService
    • Checkout
    S
    ShipmentService
    • Shipment
    S
    SearchService
    • Search/show
    S
    CustomerMDService
    • Customer self care
    S
    Candidate
    Candidate
    Candidate
    Candidate
    Candidate
    Candidate
    Product catalog
    Customer master data
    Shopping cart Buying order
    Inventory data / shipping order Product catalog

    View Slide

  88. ShoppingCartService
    • Add to shopping cart
    S
    ProductCatalogService
    • Product catalog
    maintenance
    S
    CheckoutService
    • Checkout
    S
    ShipmentService
    • Shipment
    S
    SearchService
    • Search/show
    S
    CustomerMDService
    • Customer self care
    S
    Candidate
    Candidate
    Candidate
    Candidate
    Candidate
    Candidate
    Product catalog
    Customer master data
    Shopping cart Buying order
    Inventory data / shipping order Product catalog
    Service candidates working on the same data

    View Slide

  89. Architectural reasoning
    • Same data ...
    • ... but different actors
    • Option to work on a single product catalog database
    here outweighs different actors using a single service
    à Unite in one service
    (unless you decide to use a different architectural style
    for the back office employee application)

    View Slide

  90. ShoppingCartService
    • Add to shopping cart
    S
    ProductCatalogService
    • Product catalog
    maintenance
    • Search/show
    S
    CheckoutService
    • Checkout
    S
    ShipmentService
    • Shipment
    S
    CustomerMDService
    • Customer self care
    S
    Candidate
    Candidate
    Candidate
    Candidate
    Customer master data
    Shopping cart Buying order
    Inventory data / shipping order
    Product catalog

    View Slide

  91. ShoppingCartService
    • Add to shopping cart
    S
    ProductCatalogService
    • Product catalog
    maintenance
    • Search/show
    S
    CheckoutService
    • Checkout
    S
    ShipmentService
    • Shipment
    S
    CustomerMDService
    • Customer self care
    S
    Candidate
    Candidate
    Candidate
    Candidate
    Customer master data
    Shopping cart Buying order
    Inventory data / shipping order
    Product catalog
    Service candidates
    working on the
    same type of data
    (shopping cart is a
    preliminary order)

    View Slide

  92. Architectural reasoning
    • Some (sequential) cohesion and could work on same data ...
    • ... but unification is still not imperative
    • Need to ponder other aspects and balance trade-offs
    • Different representations for shopping cart and order needed?
    • UI part of the service?
    • How does payment interfere (not considered in the example)?
    à Here we assume that it is best to unite the services

    View Slide

  93. ProductCatalogService
    • Product catalog
    maintenance
    • Search/show
    S
    OrderCreationService
    • Add to shopping cart
    • Checkout
    S
    ShipmentService
    • Shipment
    S
    CustomerMDService
    • Customer self care
    S
    Candidate
    Candidate
    Customer master data
    Shopping cart / buying order
    Inventory data / shipping order
    Product catalog

    View Slide

  94. Additional reasoning
    • Buying order vs. shipping order
    • Less commonalities than shopping cart and buying order
    • Shipping order is only “ephemeral” entity
    • Different actors using them
    à Keep them separated (we need a signaling mechanism then)
    • Who updates items in stock?
    • No longer part of product catalog maintenance
    • Warehouse employee responsible (more reasonable anyway)
    à Add additional use case “Fill up inventory”

    View Slide

  95. CustomerMDService
    • Customer self care
    S
    Customer master data
    OrderCreationService
    • Add to shopping cart
    • Checkout
    S
    Shopping cart / buying order
    WarehouseService
    • Shipment
    • Fill up inventory
    S
    Inventory data / shipping order
    ProductCatalogService
    • Product catalog
    maintenance
    • Search/show
    S
    Product catalog

    View Slide

  96. Nice, but is this design any better?
    Again: How good is it in terms of remote communication?

    View Slide

  97. CheckoutService Payment Provider

    for payment method>

    pay

    calculate price

    View Slide

  98. WarehouseService Delivery Provider
    inform delivery provider

    information>
    initiate delivery>
    update items in stock

    View Slide

  99. Findings
    • All use cases are robust and fast
    Side note: It is not always as nice and simple as in this example

    View Slide

  100. Hmm, any trade-offs?

    View Slide

  101. 1st law of architectural work:
    Every decision has its price.
    No decision is for free.
    (Translation: No decision only has upsides. Every decision also has downsides.)

    View Slide

  102. 2nd law of architectural work:
    A decision can only be evaluated
    with respect to its context.
    (Translation: Decisions are not invariably “good” or “bad”, but only in a given context.)

    View Slide

  103. Trade-offs of the approach
    • Biggest concern: What about the data?
    • Data replication and reconciliation
    • Entity distribution (no single source of truth for an entity)
    • Question cannot be answered in general
    • Here we will evaluate it with respect to the given example
    • Plus some general considerations (but no general evaluation)

    View Slide

  104. *
    *
    *
    *
    Customer
    • name
    • address
    • payment methods
    E
    Order
    • customer
    • payment information
    • shipping information
    • order items
    • product
    • quantity
    E
    Product
    • name
    • description
    • image(s)
    • price
    • items in stock
    • packaging information
    (size, weight, special care)
    E
    This diagram is misleading!

    View Slide

  105. *
    *
    *
    *
    Customer
    • name
    • address
    • payment methods
    E
    Order
    • customer
    • payment information
    • shipping information
    • order items
    • product
    • quantity
    E
    Product
    • name
    • description
    • image(s)
    • price
    • items in stock
    • packaging information
    (size, weight, special care)
    E
    Only used as copy template
    Only relevant for search/show
    Only relevant for checkout
    Just an ID for
    business related
    referencing purposes
    Only relevant for checkout
    (including invoice address)
    Only relevant for shipment
    Only relevant for shipment
    (including delivery address)
    Different for checkout and shipment
    (only IDs and quantities needed)
    Immutable after completion
    (all data copied into order)

    View Slide

  106. CustomerMDService
    • Customer self care
    S
    Customer master data
    OrderCreationService
    • Add to shopping cart
    • Checkout
    S
    Shopping cart /
    buying order
    WarehouseService
    • Shipment
    • Fill up inventory
    S
    Inventory data / shipping order
    ProductCatalogService
    • Product catalog
    maintenance
    • Search/show
    S
    Product catalog
    Putting these use cases in a
    single service avoids the need
    for data replication
    3
    3
    3
    Putting these use cases in a
    single service avoids the need
    for data signaling
    4
    4
    Needs to signal data
    for shipment order
    (signaling mechanism required)
    2
    2
    2
    Needs to copy some
    product (and customer) data
    into the order
    (could be handled by the frontend)
    1
    1
    1
    1

    View Slide

  107. Findings
    • All use cases are robust and fast
    • Minimal need to transfer data between services
    • Solvable via frontend and standard data transfer solution
    (batch file, transfer table, message queue, ...)
    • No data replication and reconciliation solution needed
    Side note: It is not always as nice and simple as in this example

    View Slide

  108. Still, it is not always that nice and easy
    Translation: There are situations where two or more copies of the same data need to be kept in sync

    View Slide

  109. CustomerMDService
    • Customer self care
    S
    Customer master data
    OrderCreationService
    • Add to shopping cart
    • Checkout
    S
    Shopping cart /
    buying order
    WarehouseService
    • Shipment
    • Fill up inventory
    S
    Inventory data / shipping order
    ProductCatalogService
    • Product catalog
    maintenance
    • Search/show
    S
    Product catalog
    Might want to allow update
    of payment methods in the
    context of customer self care
    and checkout
    (requires one-way synchronization
    of master data after change)
    1
    1
    1
    Might want to allow adding
    items to shopping cart only if
    items are in stock
    (requires one-way synchronization
    of transactional data after change)
    2
    2
    2
    It might even hit us
    in our example

    View Slide

  110. How can we keep the data in sync?

    View Slide

  111. Options to keep data in sync
    • Shared database
    • Compromises original reasoning to use services
    • Distributed transactions (2-phase commit)
    • Tight coupling compromises service independence
    • Compromises availability and scalability
    • Eventual consistency
    • Sufficient for basically all functional requirements
    • Supports low coupling and high availability
    • Downside: Much harder programming model than ACID TX

    View Slide

  112. Options for eventual consistency
    • Batch pull
    • Consumer pulls data batch when ready to process data
    • Very robust approach (also suitable for legacy integration)
    • Data sync delay may be longer than acceptable
    • Batch bootstrapping & delta push
    • Initial state sync via batch pull, then push of delta updates
    • Often combined with event sourcing, CQRS, reactive, ...
    • Fast, robust (if done right) and still quite lightweight
    • Distributed log
    • Offers advantages of previous approach in one technology
    • Kafka currently is the best-known implementation
    • Still have a plan how to recover if the tool breaks

    View Slide

  113. And the single source of truth issue?

    View Slide

  114. Pondering single source of truth
    • Usually task for analytical data processing
    • Orthogonal, well-understood issue
    • Many solutions available
    • Sometimes needed in transactional systems (e.g., CRM)
    • Question if it is really a need or just a habit
    • Strive for eventual consistency
    • Go for event streams or distributed logs for fast updates

    View Slide

  115. Wrap-up

    View Slide

  116. Wrap-up
    • Think (at least) twice before distributing functionality
    • Strive for low coupling, support with high cohesion
    • Prefer horizontal decomposition in service design
    • Favor functional independence over reuse
    • The magic is in the behavior, not the data
    • Employ, e.g., use cases to find service boundaries
    • Prefer eventual consistency for data synchronization
    • Value the timeless wisdom
    • But update the instructions to the given current context

    View Slide

  117. View Slide

  118. Uwe Friedrichsen
    https://www.speakerdeck.com/ufried
    https://medium.com/@ufried
    @ufried

    View Slide