Architecting for Scale

@duffleit Architecting FOR SCALE @duﬄeit

@duffleit DAVID LEITNER Coding Architect 👋 david@squer.io 🌎 @duffleit

@duffleit 📦 👧 🧑 👧 🧑 👧 🔥 Safely and
sustainably reduce lead time to thank you. — Daniel Terhorst-North

@duffleit Simple System 👧 🧑

@duffleit Complicated System 👧 🧑 👧 🧑

@duffleit Complicated System 👧 🧑 👧 🧑 🧑 👧

@duffleit 👧 🧑 👧 🧑 🧑 👧 Program complexity grows
until it exceeds the capabilities of the programmer who must maintain it. — one of Lehman's laws

@duffleit Build Abstract Abstraction Abstraction Abstraction 👧 🧑 👧 🧑
🧑 👧

@duffleit Complicated System Sub Domains Sub Domains Sub Domains 👧
🧑 👧 🧑 🧑 👧

@duffleit Good modularization is the foundation of any well-designed software
systems.

@duffleit Complicated System Sub Domains Sub Domains Sub Domains 👧
🧑 👧 🧑 🧑 👧

@duffleit Distributed System Service A Service C Service B 👧
🧑 👧 🧑 🧑 👧

@duffleit Deployment Units Monolithic Distributed Modularisa3on Bad Well Big Ball
Of Mud

@duffleit “Sadly, architecture has been undervalued for so long that
many engineers regard life with a Big Ball of Mud as normal.“ Foote & Yoder

Of Mud Distributed MOnolith WelL-Structured MOdulith

@duffleit There is a multitude of reasons to go for
Distributed Systems. Modularisation is none of them.

@duffleit If you can’t build a well-structured monolith, what makes
you think microservices are the answer? Simon Brown

@duffleit Single Deployment Unit Individual Scaling Demand Technology Segmentation CO-Locating
👧 🧑 🧑 🧑 👧 👧 👩🦰 👧 👱

@duffleit Single Deployment Unit 👧 🧑 🧑 🧑 👧 👧
👩🦰 👧 👱

@duffleit Single Deployment Unit 👧 🧑 🧑 👧 👧 🧑
🧑 🧑 👧 👧 👩🦰 👧 👱 👧 🧑 🧑

@duﬄeit Single Deployment Unit 👧 🧑 🧑 👧 👧 🧑
🧑 🧑 👧 👧 👩🦰 👧 👱 👧 🧑 🧑 🔥

🧑 🧑 👧 👧 👩🦰 👧 👱 👧 🧑 🧑 🔥 🔥 🔥

@duffleit Independant Deployment Units 👧 🧑 🧑 👧 👧 🧑
🧑 🧑 👧 👧 👩🦰 👧 👱 👧 🧑 🧑

@duﬄeit Independant Deployment Units 👧 🧑 🧑 👧 👧 🧑
🧑 🧑 👧 👧 👩🦰 👧 👱 👧 🧑 🧑 Resilience 🔥

🧑 🧑 👧 👧 👩🦰 👧 👱 👧 🧑 🧑 📦

@duffleit Independant Deployment Units 👧 🧑 🧑 👧 👧 🧑
🧑 🧑 👧 👧 👩🦰 👧 👱 👧 🧑 🧑 📦 📦 📦 📦 📦 📦 Independent Deployability

@duﬄeit Independent Deployability Resilience Both of them only work on
top of proper modularization.

@duﬄeit Deployment Units Monolithic Distributed Modularisa3on Bad Well Big Ball
Of Mud WelL-Structured Modulith Distributed Monolith Distributed Self-Sufficient Services

@duffleit Deployment Units Monolithic Distributed Modularisation Bad Well Big Ball
Of Mud WelL-Structured Modulith Distributed Monolith Distributed Self-Sufficient Services Decomposition Fallacy

@duffleit Independent Deployability Resilience Both of them only work on
top of proper modularization.

@duffleit Independent Deployability Resilience Both of them will enable higher
team autonomy.

@duffleit Independent Deployability Resilience Both of them will introduce more
distribution.

@duffleit We go for distributed systems to enable independent, autonomous
teams.

Of Mud WelL-Structured Modulith Distributed Monolith Autonomous Service-Based SYSTEMS Cause of Failure Reliabilty

@duﬄeit Deployment Units Monolithic Distributed Modularisation Bad Well Big Ball
Of Mud WelL-Structured Modulith Distributed Monolith Autonomous Service-Based SYSTEMS Cause of Failure Reliabilty Availability

@duﬄeit What’s the diﬀerence between a method call within a
single deployment unit Deployment Unit Module A Module B moduleB.createUser() fun createUser() { /*...*/ }

single deployment unit, and a method call across multiple deployment units. Deployment Unit Deployment Unit Service A Service B restClient.user() fun createUser() { /*...*/ }

single deployment unit, and a method call across multiple deployment units. Everything.

@duﬄeit within a single deployment unit Deployment Unit Module A
Module B moduleB.createUser() fun createUser() { /*...*/ }

Module B moduleB.createUser() fun createUser() { /*...*/ } ✅

Module B moduleB.createUser() fun createUser() { /*...*/ } 🔥

@duﬄeit Deployment Unit Deployment Unit Service A Service B restClient.user()
fun createUser() { /*...*/ } within multiple deployment units

@duﬄeit Deployment Unit Deployment Unit Service A Service B restClient.user()
fun createUser() { /*...*/ } within multiple deployment units https://en.wikipedia.org/wiki/Fallacies_of_distributed_computing

@duﬄeit Deployment Units Monolithic Distributed Modularisation Bad Well Big Ball
Of Mud WelL-Structured Modulith Distributed Monolith Autonomous Service-Based SYSTEMS Cause of Failure Reliabilty Availability

@duﬄeit 👧 🧑 🧑 🔥 👧 🧑 🧑 🔴 👧
🧑 🧑 Loan Team Lending System

@duﬄeit 👧 🧑 🧑 🔥 👧 🧑 🧑 🔴 🔥
👧 🧑 🧑 🟢 Account Team Loan Team Lending System

@duﬄeit 👧 🧑 🧑 🔥 👧 🧑 🧑 👧 🧑
🧑 🔴 Loan Team Lending System Account Team

@duffleit We strive for a stable environment.

@duffleit There is no stable environment. In Complex Distributed System

@duffleit Resilience Architect for 🔥

@duffleit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 Payment
Service 🧑 Account Service

@duffleit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🔴
Payment Service 🧑 Account Service 🔥 Users[] 🔥

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🔴
Payment Service 🧑 Account Service Users[] 🔥

@duffleit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🔴
Payment Service 🧑 Account Service Users[] cache

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🔴
Payment Service 🧑 Account Service Users[] stream UserChanged

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🔴
Payment Service 🧑 Account Service readmodel stream Users[] UserChanged

@duffleit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🔴
Payment Service 🧑 Account Service readmodel stream Users[] UserChanged

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 Payment
Service 🧑 Account Service readmodel stream 🔥🔥 🔥 🔥 Users[]

Service 🧑 Account Service 🔥 🚀 Caching LBs CBs Marketing Service

Service 🧑 Account Service 🔥 🔥 🔥 🔥

@duﬄeit accept chaos as a constant in Distributed systems

@duﬄeit 👧 🧑 🧑 👧 🧑 🧑 👧 🧑 🧑
Gamedays 🤡 🤡 🤡

@duﬄeit A Services that can run in a stable environment
will run only there.

@duﬄeit A Services that can run in a chaotic environment
will run everywhere.

@duﬄeit Resilience Architect for 🔥

@duﬄeit Products Payments Users Accounts 👧 🧑 🧑 🧑 👧
👧 👩🦰 👧 👱 👧 🧑 🧑

@duffleit Products Payments Users Accounts 👧 🧑 🧑 🧑 👧
👧 👩🦰 👧 👱 👧 🧑 🧑 🔥

@duffleit Products Payments Users Accounts 👧 🧑 🧑 🧑 👧
👧 👩🦰 👧 👱 👧 🧑 🧑 🔥 💥

@duﬄeit Products Payments Users Accounts 👧 🧑 🧑 🧑 👧
👧 👩🦰 👧 👱 👧 🧑 🧑 Entity Service Antipattern Distributed Monolith

@duﬄeit Order Service Payments Service Users Delivery service 👧 🧑
🧑 🧑 👧 👧 👩🦰 👧 👱 👧 🧑 🧑 Services along Customer Journeys Self-Sufficient Services Warehouse Service

@duﬄeit Order Service Payments Service Users Delivery service 👧 🧑
🧑 🧑 👧 👧 👩🦰 👧 👱 👧 🧑 🧑 Services along Customer Journeys Self-Sufficient Services Warehouse Service 🔥 🔥 🔥 🟢

@duffleit The quality of a service-based system design can probably
be measured by the number of hops needed for the system's most business-critical customer journeys.

@duffleit On the Criteria To Be Used in Decomposing Systems
into Modules — D.L. Parnas

@duffleit Order Service Payments Service Delivery service 👧 🧑 🧑
🧑 👧 👧 👩🦰 👧 👱 👧 🧑 🧑 Warehouse Service USERS USERS USERS USERS

@duﬄeit Order Service Payments Service Delivery service 👧 🧑 🧑
🧑 👧 👧 👩🦰 👧 👱 👧 🧑 🧑 Warehouse Service USERS USERS USERS USERS User Service

🧑 👧 👧 👩🦰 👧 👱 👧 🧑 🧑 Warehouse Service USERS Event Bus USERS

🧑 👧 👧 👩🦰 👧 👱 👧 🧑 🧑 Warehouse Service USERS Event Bus USERS USERS

🧑 👧 👧 👩🦰 👧 👱 👧 🧑 🧑 Warehouse Service USERS Event Bus USERS USERS 🔥

@duﬄeit End-2-End Responsibilities. Architect for 🛶

🧑 👧 👧 👩🦰 👧 👱 👧 🧑 🧑 Warehouse Service 👑 👑 👑 👑

@duﬄeit Highly autonomous teams may lead to organizational silos and
localized optimization.

🧑 👧 👧 👩🦰 👧 👱 👧 🧑 🧑 Warehouse Service S3 Blob S3

🧑 👧 👧 👩🦰 👧 👱 👧 🧑 🧑 Warehouse Service S3 Blob S3 Global Effort

🧑 👧 👧 👩🦰 👧 👱 👧 🧑 🧑 Warehouse Service Storage Module Global Effort

🧑 👧 👧 👩🦰 👧 👱 👧 🧑 🧑 Warehouse Service Global Effort Storage Module

🧑 👧 👧 👩🦰 👧 👱 👧 🧑 🧑 Warehouse Service Platform Teams Storage Module RFCs 🚴Paved Path

🧑 👧 👧 👩🦰 👧 👱 👧 🧑 🧑 Warehouse Service Platform Teams Storage Module RFCs Dynamic Reteaming 🚴Paved Path

@duffleit System Harmony Architect for 🧘

@duffleit Domain

@duffleit Product Architecture

@duffleit Product Architecture Organisation Everything constantly changes in unpredictable ways;
that's the main constant in software. — Neal Ford

@duffleit 👧 🧑 🧑 Payment Serivce 👩🦰 👧 FrauD Logic

@duffleit User Fraud Data Readers Writers Readers Writers API FrauD
Service 👧 🧑 🧑 👩🦰 👧 Payment Serivce

@duffleit Payment Serivce FrauD Service User Fraud Data Readers Writers
Readers Writers API 👧 🧑 🧑

@duﬄeit Payment Serivce FrauD Service User Fraud Data Readers Writers
Readers Writers API 👧 🧑 🧑 Downtime ⏳

@duffleit Payment Serivce User Fraud Data Readers Writers Readers Writers
API Downtime ⏳

@duﬄeit Readers Writers Readers Writers API User Fraud Data 🔥
Payment Serivce

@duffleit User Fraud Data Readers Writers API 🔥 Readers Writers
Payment Serivce

@duﬄeit User Fraud Data Readers Writers API Readers Writers Payment
Serivce

@duﬄeit Event User Fraud Data Readers Writers Readers Writers API
User Fraud Data CDC Payment Serivce

@duffleit Event User Fraud Data Readers Readers Writers API User
Fraud Data Event Writers Payment Serivce

User Fraud Data Event 🔥 Payment Serivce

@duﬄeit Event User Fraud Data Readers Readers Writers API User
Fraud Data Event Writers 👷 Payment Serivce

User Fraud Data Event Payment Serivce

@duﬄeit Event User Fraud Data Readers Writers API User Fraud
Data Event Readers Writers Payment Serivce

@duffleit Payment Serivce CDC User Fraud Data Readers Writers API
User Fraud Data Event Readers Writers Canary Releases Zero Downtime Migration Dark Launching Scientist Pattern

@duﬄeit Change Anxiety Architect against 🦜

@duﬄeit So many thing to consider, Let’s sum up.

@duﬄeit Modularisation is key, and if done right, distribution on
top enables autonomous teams.

@duﬄeit Deployment Units Monolithic Distributed Reliabilty Availability

@duﬄeit Resilience Architect for 🔥 End-2-End Responsibilities Architect for 🛶
System Harmony Architect for 🧘 Change Anxiety Architect against 🦜

@duﬄeit Keep the lead time to thank you low EVENT
AT SCALE.

@duﬄeit DAVID LEITNER Coding Architect 👋 david@squer.io 🌎 @duﬄeit

Architecting for Scale

Architecting for Scale

More Decks by SQUER Solutions

Other Decks in Programming

Featured

Transcript