Getting (service) design right

This slide deck basically tries to address functional service design not only from a maintainability point of view, but also form an operational point of view. How to spread your business functionality across your service landscape to maximize availability and responsiveness?

It starts with the observation that virtually all system landscapes these days, including cloud-native, are distributed systems. Due to the failure modes of distributed systems every remote communication basically is a predetermined breaking point of your system.

After briefly discussing other options to increase availability and responsiveness the biggest part of the deck discusses the question how a good functional design can positively influence the aforementioned system properties.

Using a very simple eCommerce application as an example, first a typical design is shown and why it is suboptimal in terms of availability and responsiveness. After that, we briefly dive into classical CS papers and see what we can learn from them. We also learn that while the core insights of those papers are mostly timeless, their implementation instructions are not and that we need to update them to our current context.

Then we use the learnt ideas to derive a different functional decomposition approach starting with the system behavior and apply it to our example - which leads to a lot better system behavior in terms of availability and responsiveness. Finally we discuss the most important trade-offs of the different design approach - as every design decision of course also has its downsides ... ;)

As always, the voice track is missing. Still, I hope the slides on their own also offer you a few valuable ideas.

Transcript

1. Getting (service) design right
   or how timeless wisdom can lead the way
   Uwe Friedrichsen – codecentric AG – 2016-2019
   

   View Slide

2. Uwe Friedrichsen
   IT traveller.
   Dot Connector.
   Cartographer of uncharted territory.
   Keeper of timeless wisdom.
   CTO and Fellow at codecentric.
   https://www.speakerdeck.com/ufried
   https://medium.com/@ufried
   @ufried
   

   View Slide

3. The marvelous world of microservices
   

   View Slide

4. So, you are doing microservices ...
   

   View Slide

5. Yeah!
   

   View Slide

6. ... of course, putting them in Docker ...
   

   View Slide

7. Yeah!
   

   View Slide

8. ... and running them on Kubernetes
   

   View Slide

9. Yeah!
   

   View Slide

10. So, your are doing it the cloud-native way
    

    View Slide

11. “Cloud native computing uses an open source software stack
    to deploy applications as microservices,
    packaging each part into its own container,
    and dynamically orchestrating those containers
    to optimize resource utilization.”
    Source: Cloud Native Computing Foundation Homepage (https://www.cncf.io/)
    

    View Slide

12. Yeah!
    

    View Slide

13. Well, ever pondered the consequences?
    

    View Slide

14. Eh, what?
    

    View Slide

15. Why do
    you ask?
    

    View Slide

16. Consequences of cloud-native computing
    

    View Slide

17. Different processes at runtime
    Remote communication between services
    Distributed system
    Cloud-native computing
    Services packaged in containers
    

    View Slide

18. Distributed systems are not limited to cloud-native computing
    

    View Slide

19. (Almost) every system is a distributed system
    -- Chas Emerick
    http://www.infoq.com/presentations/problems-distributed-systems
    

    View Slide

20. The software you develop and maintain is most likely
    part of a (big) distributed system landscape
    

    View Slide

21. Consequences of distributed systems
    

    View Slide

22. Everything fails, all the time.
    -- Werner Vogels
    

    View Slide

23. Failures in distributed systems ...
    • Crash failure
    • Omission failure
    • Timing failure
    • Response failure
    • Byzantine failure
    

    View Slide

24. ... lead to a variety of effects
    • Lost messages
    • Incomplete messages
    • Duplicate messages
    • Distorted messages
    • Out-of-order message arrival
    • Partial, out-of-sync local memory
    • ...
    

    View Slide

25. These effects are based on the non-determinism
    introduced by remote communication
    due to distributed failure modes
    

    View Slide

26. Understand that remote communication points are
    predetermined breaking points of your application
    Accept that the effects will hit you at the application level
    

    View Slide

27. What can we do against it?
    

    View Slide

28. Typical measures
    • High-availability (HA) hardware/software
    • Only applicable for very small installations
    • Usually not available in cloud environments
    • Delegate failure handling to infrastructure level
    • Partial relief, will not solve all problems
    • Implement resilient software design patterns
    • Very important, still will not fix a bad design
    • Minimize number of remote communication points
    • Minimize problem surface by design
    

    View Slide

29. Reducing remote communication points
    

    View Slide

30. Reducing remote communication points
    Reduce the number of
    distributed runtime artifacts
    i.e., create coarser grained services
    Reduce the need to
    communicate across
    runtime artifact boundaries
    i.e., get the functional design right
    

    View Slide

31. Reducing remote communication points
    Reduce the number of
    distributed runtime artifacts
    i.e., create coarser grained services
    Reduce the need to
    communicate across
    runtime artifact boundaries
    i.e., get the functional design right
    

    View Slide

32. Remember?
    https://www.martinfowler.com/articles/
    distributed-objects-microservices.html
    

    View Slide

33. If you can get away with a monolith, just do it!
    You are still allowed to structure it well ... ;)
    

    View Slide

34. Still, your functional domain usually will be too big
    to put it all in a single monolith
    Thus, you will need to distribute your system landscape to a certain degree
    

    View Slide

35. Rule of thumb
    Always think (at least) twice before distributing functionality
    Only do it if you really need independent deployments
    Addendum
    You may also want to distribute your functionality if you have
    very disparate NFRs for different parts of your functionality
    (e.g., security *, availability, reliability, scalability **, portability,
    resource utilization)
    * forgotten most of the time
    ** less often needed than most people assume
    

    View Slide

36. Reducing remote communication points
    Reduce the number of
    distributed runtime artifacts
    i.e., create coarser grained services
    Reduce the need to
    communicate across
    runtime artifact boundaries
    i.e., get the functional design right
    

    View Slide

37. Case study
    (Very simple) eCommerce shop
    • Implements the core functionalities
    • Search & show
    • Add to shopping cart
    • Checkout
    • Shipment
    • Payments only touched as black box
    • No recommendations, etc.
    

    View Slide

38. The typical design approach ...
    a.k.a. “the counter-example”
    

    View Slide

39. Typical design approach
    Focus on avoiding redundancy and maximizing reuse
    1. Start with a comprehensive domain (actually: E/R) model
    

    View Slide

40. *
    *
    *
    *
    Customer
    • name
    • address
    • payment methods
    E
    Order
    • customer
    • payment information
    • shipping information
    • order items
    • product
    • quantity
    E
    Product
    • name
    • description
    • image(s)
    • price
    • items in stock
    • packaging information
    (size, weight, special care)
    E
    

    View Slide

41. Typical design approach
    Focus on avoiding redundancy and maximizing re-use
    1. Start with a comprehensive domain (actually: E/R) model
    2. Wrap entities with services
    

    View Slide

42. CustomerService
    S
    Customer OrderService
    S
    Order
    ProductService
    S
    Product
    

    View Slide

43. Typical design approach
    Focus on avoiding redundancy and maximizing re-use
    1. Start with a comprehensive domain (actually: E/R) model
    2. Wrap entities with services
    3. Spread functionality over services
    

    View Slide

44. ProductService
    OrderService
    CustomerService
    S
    Customer OrderService
    • Add to shopping cart
    S
    ProductService
    • Search/show
    S
    Order
    Product
    

    View Slide

45. Typical design approach
    Focus on avoiding redundancy and maximizing re-use
    1. Start with a comprehensive domain (actually: E/R) model
    2. Wrap entities with services
    3. Spread functionality over services
    4. Add “process services” for “complex use cases”
    • i.e., use cases that touch more than one data service
    

    View Slide

46. CustomerService
    S
    Customer OrderService
    • Add to shopping cart
    S
    ProductService
    • Search/show
    S
    Order
    Product
    CheckoutService
    • Checkout
    S
    ShipmentService
    • Shipment
    S
    

    View Slide

47. Typical design approach
    Focus on avoiding redundancy and maximizing re-use
    1. Start with a comprehensive domain (actually: E/R) model
    2. Wrap entities with services
    3. Spread functionality over services
    4. Add “process services” for “complex use cases”
    • i.e., use cases that touch more than one data service
    5. Add missing data maintenance use cases
    

    View Slide

48. CustomerService
    ProductService
    • Search/show
    CustomerService
    • Customer self care
    S
    Customer OrderService
    • Add to shopping cart
    S
    ProductService
    • Search/show
    • Product catalog
    maintenance
    S
    CheckoutService
    • Checkout
    S
    ShipmentService
    • Shipment
    S
    Order
    Product
    

    View Slide

49. This (familiar) design looks innocuous at first sight
    But how good is it in terms of remote communication?
    

    View Slide

50. CheckoutService OrderService ProductService CustomerService Payment Provider
    
    read order
    read price
    loop [order items]
    calculate price
    read payment methods
    for payment method>
    
    pay
    mark paid
    
    

    View Slide

51. ShipmentService OrderService ProductService CustomerService Delivery Provider
    
    read order
    read product and packaging information
    loop [order items]
    read delivery address
    information>
    initiate delivery>
    inform delivery provider
    mark dispatched
    update items in stock
    loop [order items]
    

    View Slide

52. Findings
    • Core business use cases are failure-prone and slow
    • Data maintenance use cases are robust and fast
    

    View Slide

53. Congratulations!
    You designed a system for a company
    that's core business purpose is
    to maintain data, not to make money!
    

    View Slide

54. Properties of the design
    • Focus on avoiding redundancy and maximizing reuse
    • Based on traditional OO design practices
    • Results in high coupling between services
    • Results in moderate cohesion inside services
    • Okay for CRUD applications
    • But then better use a generator, scaffolding framework, ...
    • Okay-ish for single-process applications
    • Tends to affect maintainability negatively
    • Not okay for distributed services
    • Big failure surface, bad response times
    

    View Slide

55. How can we do better?
    

    View Slide

56. Let’s do a bit of research ...
    

    View Slide

57. Structured design
    by W. P. Stevens, G. J. Myers and L. L. Constantine
    [Stev 1974]
    

    View Slide

58. “The fewer and simpler the connections between modules, the easier it is to
    understand each module without reference to other modules.
    Minimizing connections between modules also minimizes the paths
    along which changes and errors can propagate into other parts of the
    system, thus eliminating disastrous ‘ripple’ effects, where changes in one
    part cause errors in another, necessitating additional changes elsewhere,
    giving rise to new errors, etc.”
    [Ste 1974]
    

    View Slide

59. “Coupling is the measure of the strength of association
    established by a connection from one module to another.”
    [Ste 1974]
    

    View Slide

60. Coupling
    High
    Low
    Contributing factors
    Interface
    Complexity
    Type of
    Connection
    Type of
    Communication
    Simple,
    obvious
    Complicated,
    obscure
    To module
    by name
    (depending
    on interface)
    To internal
    elements
    (depending on
    implementation)
    Data
    (control flow handled
    by environment)
    Control
    (Explicit passing
    of control)
    Hybrid
    (Manipulation of internal
    control flow by parameters)
    [Ste 1974]
    

    View Slide

61. Realize that this paper was written at a very different time
    and in a very different context than we face today
    While the core concepts are timeless and still valid,
    we usually need to rethink the concrete instructions
    

    View Slide

62. Contributing factors
    Interface
    Complexity
    Type of
    Connection
    Type of
    Communication
    Simple,
    obvious
    Complicated,
    obscure
    To module
    by name
    (depending
    on interface)
    To internal
    elements
    (depending on
    implementation)
    Data
    (control flow handled
    by environment)
    Control
    (Explicit passing
    of control)
    Hybrid
    (Manipulation of internal
    control flow by parameters)
    * Ability of a service to complete its task without the other service being present
    Functional
    Independence *
    Independent
    (does not need other
    service to work)
    Fully dependent
    (does not work
    without other service)
    Partly dependent
    (graceful degradation
    of service)
    Coupling
    High
    Low
    

    View Slide

63. “Coupling is reduced when the relationships among elements not in the
    same module are minimized.
    There are two ways of achieving this – minimizing the relationships among
    modules and maximizing relationships among elements in the same
    module. In practice, both ways are used. [...]
    Binding is the measure of the cohesiveness of a module. The objective
    here is to reduce coupling by striving for high binding.”
    [Ste 1974]
    

    View Slide

64. On the criteria to be
    used in decomposing
    systems into modules
    by David L. Parnas
    [Par 1972]
    

    View Slide

65. Separation of concerns
    One concept/decision per module
    Information hiding
    Reveal as little as possible about
    internal implementation
    +
    Better changeability
    Changes are kept local
    Independent teams
    Teams can easier work
    independently on different
    modules
    Easier to comprehend
    Modules can be understood on
    their own easier
    

    View Slide

66. Research findings
    • High cohesion, low coupling leads the right way
    • Separation of Concerns and Information hiding
    support implementing them
    • Concrete paper instructions should not be followed blindly
    • Different context (single process, very limited hardware)
    • Would lead to nano or pico services à lots of remote calls
    • You need to rethink instructions in the current context
    • Required for all CS papers from a different time & context
    • Leads to concept of Functional Independence in this context
    • Reduces risk of “vertical decomposition” (i.e., layered design)
    

    View Slide

67. 
    Functionality
    vertical decomposition
    (layer design, composition)
    In practice, you typically
    use a combination
    of both approaches
    Core functional decomposition approaches
    horizontal decomposition
    (pillar design, segregation)
    

    View Slide

68. Vertical decomposition
    • Based on “uses” relation
    • Typical drivers are reuse and avoidance of redundancy
    • Creates strong coupling (high functional dependence)
    • Often useful pattern inside a process boundary
    • Due to deterministic communication behavior
    • Problematic across process boundaries
    à Should be avoided in service design
    

    View Slide

69. Horizontal decomposition
    • Based on functional segregation
    • Typical drivers are autonomy and independence
    • Creates low coupling (high functional independence)
    • Useful pattern across process boundaries
    • Can also be useful inside a process boundary
    • Less accidental "ripple" effects due to changes
    à Should be preferred in service design
    

    View Slide

70. Watch out!
    • Vertical decomposition is our default design approach
    • We’ve learned it in our CS education (divide and conquer, ...)
    • It’s emphasized in our daily work (DRY, reusability, ...)
    • Even our IDEs support it (“Extract method”)
    • It's everywhere! It's predominant!
    • It takes energy not to apply vertical decomposition
    • Most people never learned horizontal decomposition
    

    View Slide

71. How to learn horizontal decomposition?
    

    View Slide

72. Domain-Driven Design
    by Eric Evans
    [Eva 2004]
    

    View Slide

73. DDD to the rescue?
    • Naive application of building block patterns leads
    to the counter-example design we have seen before
    • Not useful in our context due to high coupling
    • “Service” pattern leads to process service working on entities
    • Anti-pattern in our context due to high coupling
    • “Conceptual contours” supports high cohesion
    • Yet, tends to be too fine grained for our context
    • “Bounded contexts” supports low coupling
    • Yet, tends to be too coarse grained for our context
    à Mixed emotions: Good, but not the expected panacea
    

    View Slide

74. And now?
    

    View Slide

75. For good service design, look at the behavior first, not the data
    

    View Slide

76. Case study
    (Very simple) eCommerce shop
    • Implements the core functionalities
    • Search & show
    • Add to shopping cart
    • Checkout
    • Shipment
    • Customer self-care
    • Product catalog maintenance
    • Payments only touched as black box
    • No recommendations, etc.
    

    View Slide

77. Core reasoning
    To reduce the number of remote calls needed for a given functionality,
    we need to spread the functionality between the services in a way
    that a single use case/user interaction less often needs to cross service
    boundaries.
    Therefore, we try to organize services around use cases/user interactions.
    

    View Slide

78. Search & Show
    Add to
    shopping cart
    Checkout
    Shipment
    Customer
    self-care
    Product catalog
    maintenance
    eCommerce shop
    Customer
    Back office
    employee
    Warehouse
    employee
    

    View Slide

79. Search & Show
    Add to
    shopping cart
    Checkout
    Shipment
    Customer
    self-care
    Product catalog
    maintenance
    eCommerce shop
    Customer
    Back office
    employee
    Warehouse
    employee
    Three different actors
    • Indicator for cohesion boundaries
    • (At least) three different UIs
    • Could be completely different architectures
    • Depending on user needs, usage patterns and other NFRs
    • As an architect this gives you additional options
    

    View Slide

80. Warehouse
    employee
    Search & Show
    Add to
    shopping cart
    Checkout
    Shipment
    Customer
    self-care
    Product catalog
    maintenance
    eCommerce shop
    Customer
    Back office
    employee
    Could be a mobile-first FE
    with service-oriented backend
    Could be a special warehouse device FE
    with a monolithic backend
    Could be a rich
    desktop app
    Could be a
    desktop browser
    first FE with a
    service-oriented
    backend
    

    View Slide

81. Search & Show
    Add to
    shopping cart
    Checkout
    Shipment
    Customer
    self-care
    Product catalog
    maintenance
    

    View Slide

82. Behavior-based design approach
    Focus on minimum cross-service communication
    inside a use case/user interaction
    1. Each use case/user interaction is a service candidate
    

    View Slide

83. ProductCatalogService
    • Product catalog
    maintenance
    S
    ShipmentService
    • Shipment
    S
    CustomerMDService *
    • Customer self care
    S
    Candidate
    Candidate
    Candidate
    ShoppingCartService
    • Add to shopping cart
    S
    CheckoutService
    • Checkout
    S
    SearchService
    • Search/show
    S
    Candidate
    Candidate
    Candidate
    * MD = Master Data
    

    View Slide

84. Behavior-based design approach
    Focus on minimum cross-service communication
    inside a use case/user interaction
    1. Each use case/user interaction is a service candidate
    2. Possibly split big use cases in multiple services
    • Only if really needed (e.g., multiple teams, disparate NFRs)
    • Look for functional clusters with low coupling between them
    

    View Slide

85. ShoppingCartService
    • Add to shopping cart
    S
    CheckoutService
    • Checkout
    S
    SearchService
    • Search/show
    S
    Candidate
    Candidate
    Candidate
    ProductCatalogService
    • Product catalog
    maintenance
    S
    ShipmentService
    • Shipment
    S
    CustomerMDService
    • Customer self care
    S
    Candidate
    Candidate
    Candidate
    Splitting up use cases in multiple services not needed in this example
    

    View Slide

86. Behavior-based design approach
    Focus on minimum cross-service communication
    inside a use case/user interaction
    1. Each use case/user interaction is a service candidate
    2. Possibly split big use cases in multiple services
    • Only if really needed (e.g., multiple teams, disparate NFRs)
    • Look for functional clusters with low coupling between them
    3. Try to group several use cases in a single service
    • Strive for a sweet spot in terms of an overall trade-off
    • Look for service candidates that operate on the same data
    

    View Slide

87. ShoppingCartService
    • Add to shopping cart
    S
    ProductCatalogService
    • Product catalog
    maintenance
    S
    CheckoutService
    • Checkout
    S
    ShipmentService
    • Shipment
    S
    SearchService
    • Search/show
    S
    CustomerMDService
    • Customer self care
    S
    Candidate
    Candidate
    Candidate
    Candidate
    Candidate
    Candidate
    Product catalog
    Customer master data
    Shopping cart Buying order
    Inventory data / shipping order Product catalog
    

    View Slide

88. ShoppingCartService
    • Add to shopping cart
    S
    ProductCatalogService
    • Product catalog
    maintenance
    S
    CheckoutService
    • Checkout
    S
    ShipmentService
    • Shipment
    S
    SearchService
    • Search/show
    S
    CustomerMDService
    • Customer self care
    S
    Candidate
    Candidate
    Candidate
    Candidate
    Candidate
    Candidate
    Product catalog
    Customer master data
    Shopping cart Buying order
    Inventory data / shipping order Product catalog
    Service candidates working on the same data
    

    View Slide

89. Architectural reasoning
    • Same data ...
    • ... but different actors
    • Option to work on a single product catalog database
    here outweighs different actors using a single service
    à Unite in one service
    (unless you decide to use a different architectural style
    for the back office employee application)
    

    View Slide

90. ShoppingCartService
    • Add to shopping cart
    S
    ProductCatalogService
    • Product catalog
    maintenance
    • Search/show
    S
    CheckoutService
    • Checkout
    S
    ShipmentService
    • Shipment
    S
    CustomerMDService
    • Customer self care
    S
    Candidate
    Candidate
    Candidate
    Candidate
    Customer master data
    Shopping cart Buying order
    Inventory data / shipping order
    Product catalog
    

    View Slide

91. ShoppingCartService
    • Add to shopping cart
    S
    ProductCatalogService
    • Product catalog
    maintenance
    • Search/show
    S
    CheckoutService
    • Checkout
    S
    ShipmentService
    • Shipment
    S
    CustomerMDService
    • Customer self care
    S
    Candidate
    Candidate
    Candidate
    Candidate
    Customer master data
    Shopping cart Buying order
    Inventory data / shipping order
    Product catalog
    Service candidates
    working on the
    same type of data
    (shopping cart is a
    preliminary order)
    

    View Slide

92. Architectural reasoning
    • Some (sequential) cohesion and could work on same data ...
    • ... but unification is still not imperative
    • Need to ponder other aspects and balance trade-offs
    • Different representations for shopping cart and order needed?
    • UI part of the service?
    • How does payment interfere (not considered in the example)?
    à Here we assume that it is best to unite the services
    

    View Slide

93. ProductCatalogService
    • Product catalog
    maintenance
    • Search/show
    S
    OrderCreationService
    • Add to shopping cart
    • Checkout
    S
    ShipmentService
    • Shipment
    S
    CustomerMDService
    • Customer self care
    S
    Candidate
    Candidate
    Customer master data
    Shopping cart / buying order
    Inventory data / shipping order
    Product catalog
    

    View Slide

94. Additional reasoning
    • Buying order vs. shipping order
    • Less commonalities than shopping cart and buying order
    • Shipping order is only “ephemeral” entity
    • Different actors using them
    à Keep them separated (we need a signaling mechanism then)
    • Who updates items in stock?
    • No longer part of product catalog maintenance
    • Warehouse employee responsible (more reasonable anyway)
    à Add additional use case “Fill up inventory”
    

    View Slide

95. CustomerMDService
    • Customer self care
    S
    Customer master data
    OrderCreationService
    • Add to shopping cart
    • Checkout
    S
    Shopping cart / buying order
    WarehouseService
    • Shipment
    • Fill up inventory
    S
    Inventory data / shipping order
    ProductCatalogService
    • Product catalog
    maintenance
    • Search/show
    S
    Product catalog
    

    View Slide

96. Nice, but is this design any better?
    Again: How good is it in terms of remote communication?
    

    View Slide

97. CheckoutService Payment Provider
    
    for payment method>
    
    pay
    
    calculate price
    

    View Slide

98. WarehouseService Delivery Provider
    inform delivery provider
    
    information>
    initiate delivery>
    update items in stock
    

    View Slide

99. Findings
    • All use cases are robust and fast
    Side note: It is not always as nice and simple as in this example
    

    View Slide

100. Hmm, any trade-offs?
     

     View Slide

101. 1st law of architectural work:
     Every decision has its price.
     No decision is for free.
     (Translation: No decision only has upsides. Every decision also has downsides.)
     

     View Slide

102. 2nd law of architectural work:
     A decision can only be evaluated
     with respect to its context.
     (Translation: Decisions are not invariably “good” or “bad”, but only in a given context.)
     

     View Slide

103. Trade-offs of the approach
     • Biggest concern: What about the data?
     • Data replication and reconciliation
     • Entity distribution (no single source of truth for an entity)
     • Question cannot be answered in general
     • Here we will evaluate it with respect to the given example
     • Plus some general considerations (but no general evaluation)
     

     View Slide

104. *
     *
     *
     *
     Customer
     • name
     • address
     • payment methods
     E
     Order
     • customer
     • payment information
     • shipping information
     • order items
     • product
     • quantity
     E
     Product
     • name
     • description
     • image(s)
     • price
     • items in stock
     • packaging information
     (size, weight, special care)
     E
     This diagram is misleading!
     

     View Slide

105. *
     *
     *
     *
     Customer
     • name
     • address
     • payment methods
     E
     Order
     • customer
     • payment information
     • shipping information
     • order items
     • product
     • quantity
     E
     Product
     • name
     • description
     • image(s)
     • price
     • items in stock
     • packaging information
     (size, weight, special care)
     E
     Only used as copy template
     Only relevant for search/show
     Only relevant for checkout
     Just an ID for
     business related
     referencing purposes
     Only relevant for checkout
     (including invoice address)
     Only relevant for shipment
     Only relevant for shipment
     (including delivery address)
     Different for checkout and shipment
     (only IDs and quantities needed)
     Immutable after completion
     (all data copied into order)
     

     View Slide

106. CustomerMDService
     • Customer self care
     S
     Customer master data
     OrderCreationService
     • Add to shopping cart
     • Checkout
     S
     Shopping cart /
     buying order
     WarehouseService
     • Shipment
     • Fill up inventory
     S
     Inventory data / shipping order
     ProductCatalogService
     • Product catalog
     maintenance
     • Search/show
     S
     Product catalog
     Putting these use cases in a
     single service avoids the need
     for data replication
     3
     3
     3
     Putting these use cases in a
     single service avoids the need
     for data signaling
     4
     4
     Needs to signal data
     for shipment order
     (signaling mechanism required)
     2
     2
     2
     Needs to copy some
     product (and customer) data
     into the order
     (could be handled by the frontend)
     1
     1
     1
     1
     

     View Slide

107. Findings
     • All use cases are robust and fast
     • Minimal need to transfer data between services
     • Solvable via frontend and standard data transfer solution
     (batch file, transfer table, message queue, ...)
     • No data replication and reconciliation solution needed
     Side note: It is not always as nice and simple as in this example
     

     View Slide

108. Still, it is not always that nice and easy
     Translation: There are situations where two or more copies of the same data need to be kept in sync
     

     View Slide

109. CustomerMDService
     • Customer self care
     S
     Customer master data
     OrderCreationService
     • Add to shopping cart
     • Checkout
     S
     Shopping cart /
     buying order
     WarehouseService
     • Shipment
     • Fill up inventory
     S
     Inventory data / shipping order
     ProductCatalogService
     • Product catalog
     maintenance
     • Search/show
     S
     Product catalog
     Might want to allow update
     of payment methods in the
     context of customer self care
     and checkout
     (requires one-way synchronization
     of master data after change)
     1
     1
     1
     Might want to allow adding
     items to shopping cart only if
     items are in stock
     (requires one-way synchronization
     of transactional data after change)
     2
     2
     2
     It might even hit us
     in our example
     

     View Slide

110. How can we keep the data in sync?
     

     View Slide

111. Options to keep data in sync
     • Shared database
     • Compromises original reasoning to use services
     • Distributed transactions (2-phase commit)
     • Tight coupling compromises service independence
     • Compromises availability and scalability
     • Eventual consistency
     • Sufficient for basically all functional requirements
     • Supports low coupling and high availability
     • Downside: Much harder programming model than ACID TX
     

     View Slide

112. Options for eventual consistency
     • Batch pull
     • Consumer pulls data batch when ready to process data
     • Very robust approach (also suitable for legacy integration)
     • Data sync delay may be longer than acceptable
     • Batch bootstrapping & delta push
     • Initial state sync via batch pull, then push of delta updates
     • Often combined with event sourcing, CQRS, reactive, ...
     • Fast, robust (if done right) and still quite lightweight
     • Distributed log
     • Offers advantages of previous approach in one technology
     • Kafka currently is the best-known implementation
     • Still have a plan how to recover if the tool breaks
     

     View Slide

113. And the single source of truth issue?
     

     View Slide

114. Pondering single source of truth
     • Usually task for analytical data processing
     • Orthogonal, well-understood issue
     • Many solutions available
     • Sometimes needed in transactional systems (e.g., CRM)
     • Question if it is really a need or just a habit
     • Strive for eventual consistency
     • Go for event streams or distributed logs for fast updates
     

     View Slide

115. Wrap-up
     

     View Slide

116. Wrap-up
     • Think (at least) twice before distributing functionality
     • Strive for low coupling, support with high cohesion
     • Prefer horizontal decomposition in service design
     • Favor functional independence over reuse
     • The magic is in the behavior, not the data
     • Employ, e.g., use cases to find service boundaries
     • Prefer eventual consistency for data synchronization
     • Value the timeless wisdom
     • But update the instructions to the given current context
     

     View Slide

117. View Slide

118. Uwe Friedrichsen
     https://www.speakerdeck.com/ufried
     https://medium.com/@ufried
     @ufried
     

     View Slide