Continuous Decision-Making for Evolving Microservices Architecture : Integrating Stakeholders, Viewpoints, and Domain-Driven Design

Slide 1

Slide 1 text

Continuous Decision-Making for Evolving Microservices Architecture Integrating Stakeholders, Viewpoints, and Domain-Driven Design Titansoft Tech Day 2024 Kim Kao Co-organizer of DDDesign Taiwan Sr. Developer Specialist Solutions Architect @ AWS

Slide 2

Slide 2 text

About Me

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

uncertainty.equals(dangerous) == true

Slide 5

Slide 5 text

Uncomfortable It’s okay to feel this way. @VaughnVernon

Slide 6

Slide 6 text

New products Scenario : Build up Next Generation platform Growth Feedback Customer experience Improvements Usage • Modular existing products • Cross channel selling • Parallel develop teams overseas • Launch new branding platform in 1 year • <2 seconds responding time in each client • Stable release & GA cycle (2w/1 release) • Team members grow up from 20 to 120+

Slide 7

Slide 7 text

Business IT Business leads to technology innovation Technology supports business growth Learnt from my first architect experience

Slide 8

Slide 8 text

1 Know the legacy Monolith to microservices Keep Hands dirty Step in sb’s shoes to know the tough 2 Automate your bureaucracy Introduce Automatic Testing 3 Build it in, don’t bolt it on Involve in the operation, design the co-work model 4

Slide 9

Slide 9 text

Business Wants https://vaughnvernon.co/tag/event-storming/

Slide 10

Slide 10 text

But You Want https://vaughnvernon.co/tag/event-storming/

Slide 11

Slide 11 text

Changes to the architectural patterns

Slide 12

Slide 12 text

When the impact of change is small, release velocity can increase Monolith Does everything Microservices Do one thing

Slide 13

Slide 13 text

100/200/300/1000? LOC Code matters?

Slide 14

Slide 14 text

Domain Driven Design Fit in • Standalone computing & persistence • Serve for specific business intentions • Business Capability in bounded Context • Embrace Rapidly Change • Automate operations at web scale • Engineering Decoupling Microservices stand for ...

Slide 15

Slide 15 text

Way to design Microservices

Slide 16

Slide 16 text

Start simple and only add complexity when it is called for Pete Loucks Core Modeling Practices Purist Compute Storage Network Monolith Model

Slide 17

Slide 17 text

(priority, market expectation) Domain Expert Matters

Slide 18

Slide 18 text

Strategies for Dealing with Legacy Systems • Bubble Context • Place your new functionality in a bubble and have repositories as an anti-corruption layer(ACL) toward the legacy code and data. • Autonomous Bubble • Start a new chapter OUTSIDE the legacy code with its own storage. Thus we need to synchronize ACL and similar information that you shares with the legacy system • Open Host Services in Published Language • Expose legacy assets through an open host service. Using an anti-corruption layer to convert the necessary information to the new system • Event Streams • The systems communicate trough events. The anti-corruption layer now publishes the event and monitor the state of the other system. The solution is similar to event sourcing. (priority, customer expectation)

Slide 19

Slide 19 text

Don’t know the explicit boundary

Slide 20

Slide 20 text

Effective way to crunch domain knowledge Ubiquitous Language Bounded Context Eric Evans Founder, Domain-Driven Design

Slide 21

Slide 21 text

Crunch domain knowledge Eric Evans Founder, Domain-Driven Design

Slide 22

Slide 22 text

No content

Slide 23

Slide 23 text

PROBLEM SOLUTION

Slide 24

Slide 24 text

Way to deal with the complexity of a problem domain

Slide 25

Slide 25 text

User registered User authenticated Shipment fulfilled Goods selected Order placed Time Various business events

Slide 26

Slide 26 text

“All the key stakeholders in the same room with an unlimited modelling space, using stickies as domain events ” ~ Alberto Brandolini Founder, Event Storming

Slide 27

Slide 27 text

Traditional meeting

Slide 28

Slide 28 text

Leave space for brain storming

Slide 29

Slide 29 text

Immutable truth

Slide 30

Slide 30 text

(dealing with transaction, service lookup) Monolith Does everything Per Service Do only one thing Business matters Immutable facts - Order Created - Coupon applied - Account Registered Intention Business behavior - Create an Order - Apply Coupon - Register an Account Responsible for Capabilities - Order - Discount - Identity Management Accept & process Presentation Model Help to make decision Composite data type

Slide 31

Slide 31 text

(dealing with transaction, service lookup) Monolith Does everything Per Service Do only one thing

Slide 32

Slide 32 text

• Microservices candidate – Bounded Context • Per Bounded Context form up system Boundary • One Bounded Context may • Contains multiple co-related Aggregates • Or only one Aggregate with Specific business capability (dealing with transaction, service lookup)

Slide 33

Slide 33 text

No content

Slide 34

Slide 34 text

Devices DB External Interfaces UI Web Controllers Use Cases Entities G ateways Presenters

Slide 35

Slide 35 text

QLDB RDS SNS, SQS, SMS Messaging UI S3 Static page ELB ingress Use Cases Entity (Aggregate) API G ateway Quicksight Blockchain Tim e DB Aggregate Dynamodb

Slide 36

Slide 36 text

(dealing with transaction, service lookup) • Are you ready to deal with M:N transaction compensation ? • Are you ready to embrace the rapidly change by contract ? Saga Patterns fit-in distributed transaction management

Slide 37

Slide 37 text

The rest of challenges

Slide 38

Slide 38 text

(by noun, organization, experience?) (CRM, ERP, Payment Gateway ...) (self employee, out sourcing, ISV)

Slide 39

Slide 39 text

• By business Capability • Form up boundary by Bounded Context • Clarify Sub Domains • Core sub domain – (most valuable) • General sub domain – (facilitate business) • Support sub domain – (support, infra) (by noun, organization, experience?)

Slide 40

Slide 40 text

(by noun, organization, experience?) A Team

Slide 41

Slide 41 text

• Resources allocation by value chain • Talents devote to build up core sub domain • Responsible for general sub domain • Out sourcing or ISV for support domain (self employee, out sourcing, ISV)

Slide 42

Slide 42 text

ISV/Package Support sub Domain Out Sourcing General Sub Domain Pay the most efforts on critical business component Talents developing code Core Sub Domain (self employee, out sourcing, ISV)

Slide 43

Slide 43 text

• Incrementally breakout dependencies • Cut-off Database Link • Do not allow cross schema access permissions • Define API contract only for data exchange • Considering to move out store procedure into application code • Leave the legacy system as a data container (CRM, ERP, Payment Gateway ...)

Slide 44

Slide 44 text

No content

Slide 45

Slide 45 text

As an architect you design for the present, with an awareness of the past for a future which is essentially unknown Norman Foster

Slide 46

Slide 46 text

Current Challenges with architecture Focus on technology details rather than business context. Perception of architects are not delivering solutions or adding value. Inability of architectural practices to address the increase in speed of IT deliver and to adapt to modern delivery practices such as DevOps. Some architect’s discomfort with or lack of appropriate skills for migrating their IT environment to cloud-based platform.

Slide 47

Slide 47 text

Focus on technology details rather than business context Why the IPs are not enough to run my workloads in Kubernetes? Why the traffic just came across different subnets, and costs a lot ?

Slide 48

Slide 48 text

• Architect products; evolve from projects to products • Focus on qulity attributes, not on functional requirements • Delay design decisions until they are absolutely necessary • Architect for change – leverage the power of small • Architect for build, test, deploy, and operate • Model the organization of your teams after the design of the system you are working on 6 principles of Continuous architecture

Slide 49

Slide 49 text

Continuous architecture contains • context of the system • Key functional requirements that will impact the architecture • Quality attributes that drive the architecture • Making trade-off from the cycle

Slide 50

Slide 50 text

Difference between CA and traditional architecture approaches 1. End to end delivery 2. Not only in software architecture itself, but ask to have entire coverage 3. Way to avoid big-architecture-up-front syndrome

Slide 51

Slide 51 text

Making and governing architectural decisions Provide Guidelines Visibilities Developers Architects

Slide 52

Slide 52 text

Architectural decisions • Most of the time, architecture diagram is not readable or maintainable • Only authors realized the entire content • Key assests in the decisions should be recorded, try to use https://adr.github.io • 3 key points should be covered in adr • 1) clearly articulate all constraints related to a decision • 2) focus on quality attributes, not on functional requirements • 3) trade-off between the different opions and impact on quality attributes should be considered

Slide 53

Slide 53 text

Challenges of modernization

Slide 54

Slide 54 text

Goal Alignment Facilitating Problem Domain Resources alignment

Slide 55

Slide 55 text

Architecture viewpoints Concurrency viewpoint Deployment viewpoint & Operational viewpoint Information Viewpoint Development viewpoint Functional viewpoint Context Viewpoint • Function elements • Responsibilities • interactions • Function elements • Responsibilities • interactions • information flow between subsystems • Dependencies • Environment • Runtime Process • monitoring • Reacting mechanism • Runbook/ Playbook • Live Document • Glossary & Ubiquitous Language • Tier/Layer design guideline • Architecture & Tech Stack

Slide 56

Slide 56 text

Architecture integration

Slide 57

Slide 57 text

Integration choices ● Remote procedure Call ○ gRPC, MSDTC, EJB ● Direct API Call ○ Web API ● SDK / Share libraries

Slide 58

Slide 58 text

The many facets of coupling Technology dependency: Java vs. C++ Location dependency: IP addresses, DNS Data format dependency: Binary, XML, JSON, ProtoBuf, Avro Data type dependency: int16, int32, string, UTF-8, null,empty Semantic dependency: Name, Middlename, ZIP Temporal dependency: sync, async Interaction style dependency: messaging, RPC, query-style Conversation dependency: pagination, caching, retries Source: EnterpriseIntegrationPatterns.com

Slide 59

Slide 59 text

The appropriate level of coupling depends on the level of control you have over the endpoints. Gregor Hohpe Enterprise Integration Patterns

Slide 60

Slide 60 text

You would like to be couple with lover, rather than having system highly coupled with un-maintainable codes…

Slide 61

Slide 61 text

Don’t let the code fool you • Considering the learning curve • Code first rarely lead to correct domain model • Pattern relationship lead to robustness design ( should Apply all or nothing) • Start with layering design • Apply Clean Architecture (optional) UI Layer Application Layer Domain Layer Infrastructure Layer Layer dependency

Slide 62

Slide 62 text

How to keep this dependency principal work well as expected? • Layered architecture and suitable for DDD tactical pattern implementation • Outer layers depends on inner layers • Good to keep Domain model Layer Unit test without specific tech/lib/framework impacted • Integration test from Application Services

Slide 63

Slide 63 text

https://www.archunit.org/use-cases Whenever code review is not enough …

Slide 64

Slide 64 text

Applying CA in Product design

Slide 65

Slide 65 text

Minimum Viable Architecture • Follow on current captured requirements • Just fit-in all of the feature needs • Proactively seeking for non-functional quality attributes • Base on the truth of the market, users • MVP should be throwable (* could be a burden for decision maker)

Slide 66

Slide 66 text

The art of Aggregate/System design

Slide 67

Slide 67 text

67 Towards to Aggregate Specification Invariant Combination Reveal Domain knowledge Talk subjects in boundary Pattern used to encapsulate logics Can be modular, reuse & compose Collaboration Modeling Clarify each single one rule time by time Maintain consistency Enforced within an aggregate to ensure internal state is valid Maintaining consistency and correctness Responsibilities Inside of Aggregate Root; Single entry point Aggregate

Slide 68

Slide 68 text

Bounded Context Mapping

Slide 69

Slide 69 text

Enhance the design • Calculate the charge fee estimation by the end of month • The chargefee can be summarized by replaying booking events

Slide 70

Slide 70 text

Aggregate Canvas • Map artifacts to DDD Tactical design • Come out consensus for implementation • Kata; Deep dive the invariant constraints and Specifications 70

Slide 71

Slide 71 text

Arrange the artifacts

Slide 72

Slide 72 text

Invariant Constraints • To maintain internal consistency, which is often enforced through invariant constraints. • These constraints define business rules that must always hold true within the aggregate. • If an operation would violate these invariants, it should be rejected or trigger an error. public class Payment { private final String Id; private final BigDecimal amount; private boolean isPaid; // Constructor to create a new Payment public Payment(String OfficeId, BigDecimal amount) { // Ensure the amount is valid according to the invariant if (amount.compareTo(BigDecimal.ZERO) < 0) { throw new IllegalArgumentException("Amount must be greater than or equal to 0."); } }

Slide 73

Slide 73 text

Specifications • Encapsulate complex business rules and conditions to verify whether specific criteria are met • Allows for clearer aggregate logic by decoupling validation logic from the aggregate itself • Maintaining high cohesion and low coupling. public interface Specification { boolean isSatisfiedBy(T t); } public class ApplicablePaymentMethodSpecification implements Specification { @Override public boolean isSatisfiedBy(PaymentMethod method) { // Check if the method is applicable in current region return Region.getCurrent().accept(method); } }

Slide 74

Slide 74 text

Manipulate the legends iteratively

Slide 75

Slide 75 text

Map to execution runtime – Trade off

Slide 76

Slide 76 text

Incorporating quality attributes in architecture Key Quality Attributes Non-Functional Requirements Simplicity Simplify the implementation of communication among different domains (retail, loyalty points) Scalability Support high QPS/TPS during peak traffic Usability Maintain user experience as usual under high QPS/TPS Performance Near real-time loyalty points issuance and redemption

Slide 77

Slide 77 text

Aggregate Design Canvas • A modelling tool meant to be used as a part of design-level domain modelling activities. • An aggregate, a graph of objects that is a consistency boundary for our domain policies. Depending on the design of the aggregate we can either enforce them (make them invariant) or be forced to have corrective policies in place. • The canvas has a suggested order of working through it, that helps to iteratively discuss different aspects of the aggregate design.

Slide 78

Slide 78 text

No content

Slide 79

Slide 79 text

Handling throughput Key metrics Average Maximum Command handling rate 2/d 100/d Total number of clients 10s 100s Concurrency conflict chance small medium (would be great to be clear once have data)

Slide 80

Slide 80 text

Size Key metrics Average Maximum Event growth rate 2/d 100/d Lifetime of a single instance 10s 100s Number of events persisted small medium (would be great to be clear once have data)

Slide 81

Slide 81 text

Slide 82

Slide 82 text

Shared responsibility model for resilience Software deployment / management PR/FAQ, Design and Build Incident Management, COE (learnings, actions), Weekly operations meetings, Principal Engineers Engineering Culture: Clear scope of ownership

Slide 83

Slide 83 text

Monitor the health of your applications T H R E E P I L L A R S O F O B S E R V A B I L I T Y T O O L I N G • Metrics: • Numeric data • Measured at various time intervals • Such as SLIs • Logs: • Timestamped records of discrete events • Traces: • User’s journey across multiple applications & systems Observability Metrics Traces Logs

Slide 84

Slide 84 text

Value Driven in Microservices Team Partners Business Operation Implementat ion Key metrics Improvement Movement

Slide 85

Slide 85 text

Slide 86

Slide 86 text

Slide 87

Slide 87 text

Take Away Know Why/What/How • Do you really need Microservices? • Leverage Business Events to aggregate context and form up Service Boundary • There is no C4 to solve distributed persistence issue • State Machine to do Transaction Compensate (policy/long running process) • DDD is good to collaborate Business and Technology guys by speaking Ubiquitous Language • Crunch Problem then design solution

Slide 88

Slide 88 text

DDDesign TW Conference 2024 Thank You! 88