Continuous Decision-Making for Evolving Microservices Architecture : Integrating Stakeholders, Viewpoints, and Domain-Driven Design

In today’s dynamic software landscape, effective decision-making for microservices architecture requires more than just technical proficiency; it demands a continuous, evolving approach that embraces stakeholder perspectives, adaptable architecture practices, and domain-driven design principles. This session explores how to make sustainable, informed decisions in implementing and refining a microservices architecture. We will delve into aligning with stakeholders through viewpoint-oriented collaboration, ensuring that architecture evolves continuously without compromising the system's integrity, and leveraging DDD to maintain a clear, bounded context. Participants will gain actionable insights into creating a responsive architecture that scales with changing requirements and drives long-term success.

About Speaker : Kim Kao

Currently, I am a Senior Solutions Architect at AWS Taiwan, focusing on the development of architect competencies, business requirements alignment, and the design and implementation of solutions. I am particularly committed to enhancing the value of IT professionals. Additionally, I am one of the founders of the Domain-Driven Design (DDD) Taiwan Community, dedicated to fostering socio-technical collaboration among technical professionals. My mission is to promote cross-group collaboration and co-creation in large-scale business system transformations. I advocate for enterprises to leverage Domain-Driven Design (DDD) and efficient cloud services to build tailored system solutions that address real business challenges.

Transcript

1. Continuous Decision-Making for Evolving Microservices Architecture Integrating Stakeholders, Viewpoints, and Domain-Driven Design Titansoft Tech Day 2024 Kim Kao Co-organizer of DDDesign Taiwan Sr. Developer Specialist Solutions Architect @ AWS
2. About Me
4. uncertainty.equals(dangerous) == true
5. Uncomfortable It’s okay to feel this way. @VaughnVernon
6. New products Scenario : Build up Next Generation platform Growth Feedback Customer experience Improvements Usage • Modular existing products • Cross channel selling • Parallel develop teams overseas • Launch new branding platform in 1 year • <2 seconds responding time in each client • Stable release & GA cycle (2w/1 release) • Team members grow up from 20 to 120+
7. Business IT Business leads to technology innovation Technology supports business growth Learnt from my first architect experience
8. 1 Know the legacy Monolith to microservices Keep Hands dirty Step in sb’s shoes to know the tough 2 Automate your bureaucracy Introduce Automatic Testing 3 Build it in, don’t bolt it on Involve in the operation, design the co-work model 4
9. Business Wants https://vaughnvernon.co/tag/event-storming/
10. But You Want https://vaughnvernon.co/tag/event-storming/
11. Changes to the architectural patterns
12. When the impact of change is small, release velocity can increase Monolith Does everything Microservices Do one thing
13. 100/200/300/1000? LOC Code matters?
14. Domain Driven Design Fit in • Standalone computing & persistence • Serve for specific business intentions • Business Capability in bounded Context • Embrace Rapidly Change • Automate operations at web scale • Engineering Decoupling Microservices stand for ...
15. Way to design Microservices
16. Start simple and only add complexity when it is called for Pete Loucks Core Modeling Practices Purist Compute Storage Network Monolith Model
17. (priority, market expectation) Domain Expert Matters
18. Strategies for Dealing with Legacy Systems • Bubble Context • Place your new functionality in a bubble and have repositories as an anti-corruption layer(ACL) toward the legacy code and data. • Autonomous Bubble • Start a new chapter OUTSIDE the legacy code with its own storage. Thus we need to synchronize ACL and similar information that you shares with the legacy system • Open Host Services in Published Language • Expose legacy assets through an open host service. Using an anti-corruption layer to convert the necessary information to the new system • Event Streams • The systems communicate trough events. The anti-corruption layer now publishes the event and monitor the state of the other system. The solution is similar to event sourcing. (priority, customer expectation)
19. Don’t know the explicit boundary
20. Effective way to crunch domain knowledge Ubiquitous Language Bounded Context Eric Evans Founder, Domain-Driven Design
21. Crunch domain knowledge Eric Evans Founder, Domain-Driven Design
23. PROBLEM SOLUTION
24. Way to deal with the complexity of a problem domain
25. User registered User authenticated Shipment fulfilled Goods selected Order placed Time Various business events
26. “All the key stakeholders in the same room with an unlimited modelling space, using stickies as domain events ” ~ Alberto Brandolini Founder, Event Storming
27. Traditional meeting
28. Leave space for brain storming
29. Immutable truth
30. (dealing with transaction, service lookup) Monolith Does everything Per Service Do only one thing Business matters Immutable facts - Order Created - Coupon applied - Account Registered Intention Business behavior - Create an Order - Apply Coupon - Register an Account Responsible for Capabilities - Order - Discount - Identity Management Accept & process Presentation Model Help to make decision Composite data type
31. (dealing with transaction, service lookup) Monolith Does everything Per Service Do only one thing
32. • Microservices candidate – Bounded Context • Per Bounded Context form up system Boundary • One Bounded Context may • Contains multiple co-related Aggregates • Or only one Aggregate with Specific business capability (dealing with transaction, service lookup)
34. Devices DB External Interfaces UI Web Controllers Use Cases Entities G ateways Presenters
35. QLDB RDS SNS, SQS, SMS Messaging UI S3 Static page ELB ingress Use Cases Entity (Aggregate) API G ateway Quicksight Blockchain Tim e DB Aggregate Dynamodb
36. (dealing with transaction, service lookup) • Are you ready to deal with M:N transaction compensation ? • Are you ready to embrace the rapidly change by contract ? Saga Patterns fit-in distributed transaction management
37. The rest of challenges
38. (by noun, organization, experience?) (CRM, ERP, Payment Gateway ...) (self employee, out sourcing, ISV)
39. • By business Capability • Form up boundary by Bounded Context • Clarify Sub Domains • Core sub domain – (most valuable) • General sub domain – (facilitate business) • Support sub domain – (support, infra) (by noun, organization, experience?)
40. (by noun, organization, experience?) A Team
41. • Resources allocation by value chain • Talents devote to build up core sub domain • Responsible for general sub domain • Out sourcing or ISV for support domain (self employee, out sourcing, ISV)
42. ISV/Package Support sub Domain Out Sourcing General Sub Domain Pay the most efforts on critical business component Talents developing code Core Sub Domain (self employee, out sourcing, ISV)
43. • Incrementally breakout dependencies • Cut-off Database Link • Do not allow cross schema access permissions • Define API contract only for data exchange • Considering to move out store procedure into application code • Leave the legacy system as a data container (CRM, ERP, Payment Gateway ...)
45. As an architect you design for the present, with an awareness of the past for a future which is essentially unknown Norman Foster
46. Current Challenges with architecture Focus on technology details rather than business context. Perception of architects are not delivering solutions or adding value. Inability of architectural practices to address the increase in speed of IT deliver and to adapt to modern delivery practices such as DevOps. Some architect’s discomfort with or lack of appropriate skills for migrating their IT environment to cloud-based platform.
47. Focus on technology details rather than business context Why the IPs are not enough to run my workloads in Kubernetes? Why the traffic just came across different subnets, and costs a lot ?
48. • Architect products; evolve from projects to products • Focus on qulity attributes, not on functional requirements • Delay design decisions until they are absolutely necessary • Architect for change – leverage the power of small • Architect for build, test, deploy, and operate • Model the organization of your teams after the design of the system you are working on 6 principles of Continuous architecture
49. Continuous architecture contains • context of the system • Key functional requirements that will impact the architecture • Quality attributes that drive the architecture • Making trade-off from the cycle
50. Difference between CA and traditional architecture approaches 1. End to end delivery 2. Not only in software architecture itself, but ask to have entire coverage 3. Way to avoid big-architecture-up-front syndrome
51. Making and governing architectural decisions Provide Guidelines Visibilities Developers Architects
52. Architectural decisions • Most of the time, architecture diagram is not readable or maintainable • Only authors realized the entire content • Key assests in the decisions should be recorded, try to use https://adr.github.io • 3 key points should be covered in adr • 1) clearly articulate all constraints related to a decision • 2) focus on quality attributes, not on functional requirements • 3) trade-off between the different opions and impact on quality attributes should be considered
53. Challenges of modernization
54. Goal Alignment Facilitating Problem Domain Resources alignment
55. Architecture viewpoints Concurrency viewpoint Deployment viewpoint & Operational viewpoint Information Viewpoint Development viewpoint Functional viewpoint Context Viewpoint • Function elements • Responsibilities • interactions • Function elements • Responsibilities • interactions • information flow between subsystems • Dependencies • Environment • Runtime Process • monitoring • Reacting mechanism • Runbook/ Playbook • Live Document • Glossary & Ubiquitous Language • Tier/Layer design guideline • Architecture & Tech Stack
56. Architecture integration
57. Integration choices • Remote procedure Call ◦ gRPC, MSDTC, EJB • Direct API Call ◦ Web API • SDK / Share libraries
58. The many facets of coupling Technology dependency: Java vs. C++ Location dependency: IP addresses, DNS Data format dependency: Binary, XML, JSON, ProtoBuf, Avro Data type dependency: int16, int32, string, UTF-8, null,empty Semantic dependency: Name, Middlename, ZIP Temporal dependency: sync, async Interaction style dependency: messaging, RPC, query-style Conversation dependency: pagination, caching, retries Source: EnterpriseIntegrationPatterns.com
59. The appropriate level of coupling depends on the level of control you have over the endpoints. Gregor Hohpe Enterprise Integration Patterns
60. You would like to be couple with lover, rather than having system highly coupled with un-maintainable codes…
61. Don’t let the code fool you • Considering the learning curve • Code first rarely lead to correct domain model • Pattern relationship lead to robustness design ( should Apply all or nothing) • Start with layering design • Apply Clean Architecture (optional) UI Layer Application Layer Domain Layer Infrastructure Layer Layer dependency
62. How to keep this dependency principal work well as expected? • Layered architecture and suitable for DDD tactical pattern implementation • Outer layers depends on inner layers • Good to keep Domain model Layer Unit test without specific tech/lib/framework impacted • Integration test from Application Services
63. https://www.archunit.org/use-cases Whenever code review is not enough …
64. Applying CA in Product design
65. Minimum Viable Architecture • Follow on current captured requirements • Just fit-in all of the feature needs • Proactively seeking for non-functional quality attributes • Base on the truth of the market, users • MVP should be throwable (* could be a burden for decision maker)
66. The art of Aggregate/System design
67. 67 Towards to Aggregate Specification Invariant Combination Reveal Domain knowledge Talk subjects in boundary Pattern used to encapsulate logics Can be modular, reuse & compose Collaboration Modeling Clarify each single one rule time by time Maintain consistency Enforced within an aggregate to ensure internal state is valid Maintaining consistency and correctness Responsibilities Inside of Aggregate Root; Single entry point Aggregate
68. Bounded Context Mapping
69. Enhance the design • Calculate the charge fee estimation by the end of month • The chargefee can be summarized by replaying booking events
70. Aggregate Canvas • Map artifacts to DDD Tactical design • Come out consensus for implementation • Kata; Deep dive the invariant constraints and Specifications 70
71. Arrange the artifacts
72. Invariant Constraints • To maintain internal consistency, which is often enforced through invariant constraints. • These constraints define business rules that must always hold true within the aggregate. • If an operation would violate these invariants, it should be rejected or trigger an error. public class Payment { private final String Id; private final BigDecimal amount; private boolean isPaid; // Constructor to create a new Payment public Payment(String OfficeId, BigDecimal amount) { // Ensure the amount is valid according to the invariant if (amount.compareTo(BigDecimal.ZERO) < 0) { throw new IllegalArgumentException("Amount must be greater than or equal to 0."); } }
73. Specifications • Encapsulate complex business rules and conditions to verify whether specific criteria are met • Allows for clearer aggregate logic by decoupling validation logic from the aggregate itself • Maintaining high cohesion and low coupling. public interface Specification<T> { boolean isSatisfiedBy(T t); } public class ApplicablePaymentMethodSpecification implements Specification<PaymentMethod> { @Override public boolean isSatisfiedBy(PaymentMethod method) { // Check if the method is applicable in current region return Region.getCurrent().accept(method); } }
74. Manipulate the legends iteratively
75. Map to execution runtime – Trade off
76. Incorporating quality attributes in architecture Key Quality Attributes Non-Functional Requirements Simplicity Simplify the implementation of communication among different domains (retail, loyalty points) Scalability Support high QPS/TPS during peak traffic Usability Maintain user experience as usual under high QPS/TPS Performance Near real-time loyalty points issuance and redemption
77. Aggregate Design Canvas • A modelling tool meant to be used as a part of design-level domain modelling activities. • An aggregate, a graph of objects that is a consistency boundary for our domain policies. Depending on the design of the aggregate we can either enforce them (make them invariant) or be forced to have corrective policies in place. • The canvas has a suggested order of working through it, that helps to iteratively discuss different aspects of the aggregate design.
79. Handling throughput Key metrics Average Maximum Command handling rate 2/d 100/d Total number of clients 10s 100s Concurrency conflict chance small medium (would be great to be clear once have data)
80. Size Key metrics Average Maximum Event growth rate 2/d 100/d Lifetime of a single instance 10s 100s Number of events persisted small medium (would be great to be clear once have data)
81. Everything fails, all the time.” Werner Vogels CTO, Amazon.com © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. “
82. Shared responsibility model for resilience Software deployment / management PR/FAQ, Design and Build Incident Management, COE (learnings, actions), Weekly operations meetings, Principal Engineers Engineering Culture: Clear scope of ownership
83. Monitor the health of your applications T H R E E P I L L A R S O F O B S E R V A B I L I T Y T O O L I N G • Metrics: • Numeric data • Measured at various time intervals • Such as SLIs • Logs: • Timestamped records of discrete events • Traces: • User’s journey across multiple applications & systems Observability Metrics Traces Logs
84. Value Driven in Microservices Team Partners Business Operation Implementat ion Key metrics Improvement Movement
85. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Are you ready to embrace microservices? Migration from Monolith to Microservices readiness https://bit.ly/2KVumDB
86. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Essential Capabilities behind microservices
87. Take Away Know Why/What/How • Do you really need Microservices? • Leverage Business Events to aggregate context and form up Service Boundary • There is no C4 to solve distributed persistence issue • State Machine to do Transaction Compensate (policy/long running process) • DDD is good to collaborate Business and Technology guys by speaking Ubiquitous Language • Crunch Problem then design solution
88. DDDesign TW Conference 2024 Thank You! 88