Architecting for Scale

@duffleit The Art of Architecting for Scale @duffleit

@duffleit DAVID LEITNER Coding Architect 👋 [email protected] 🌎 @duﬄeit

@duffleit 📦 👧 🧑 👧 🧑 👧 🔥 Safely and
sustainably reduce lead time to thank you. Daniel Terhorst-North Complex System

@duffleit Simple System

@duffleit Complicated System

@duffleit Complicated System Module A Module B Module C

@duffleit Complicated System Distributed Systems Service A Service B Service
C

@duffleit Deployment Units Monolithic Distributed Modularisa3on Bad Well Big Ball
Of Mud

@duffleit “Sadly, architecture has been undervalued for so long that
many engineers regard life with a Big Ball of Mud as normal.“ Foote & Yoder

Of Mud Distributed MOnolith WelL-Structured MOdulith

@duffleit There is a multitude of reasons to go for
Distributed Systems. Modularisation is none of them.

@duffleit If you can’t build a well-structured monolith, what makes
you think microservices are the answer? Simon Brown

@duffleit Module A Module B Module C Single Deployment Unit

@duffleit Service A Service B Serivice C

@duffleit Service A Service B Serivice C Individual Scaling Demand
10x Users Usual Traﬃc

Technology Segmentation

Technology Segmentation 🇩🇪 🇩🇪 🇺🇸 CO-Locating

@duﬄeit Service A Service B Serivice C Individual Scaling Demand
Technology Segmentation CO-Locating Technical Benefits

@duﬄeit Service A Service B Serivice C 👧 🧑 🧑
👧 🧑 🧑 👧 🧑 🧑 Ownership You build it, You Own it You build it, You Run it

@duﬄeit Single Deployment Unit MOdule A MOdule B MOdule C
👧 🧑 🧑 👧 🧑 🧑 👧 🧑 🧑 Ownership You build it, You Own it You build it, You Run it 🔥

@duffleit Single Deployment Unit MOdule A MOdule B MOdule C
👧 🧑 🧑 👧 🧑 🧑 👧 🧑 🧑 Ownership You build it, You Own it You build it, You Run it 🔥 🔥

@duffleit Service A Service B Serivice C 👧 🧑 🧑
👧 🧑 🧑 👧 🧑 🧑 Enable Teams to work autonomously

@duﬄeit Technical Benefits Team Autonomoy Scale Engineering CO-Locating Technology Segmentation
Individual Scaling Demand 80%

@duﬄeit We optimize for Autonomy.

@duﬄeit Responsibility The ops will hate it Zone The Devs
will Quit Zone Autonomy You build it you Own it You build it you run it You build it you care it

@duﬄeit We optimize for Responsibility. Therefore, teams strive for Autonomy.

@duﬄeit 👧 🧑 🧑 👧 🧑 🧑 👧 🧑 🧑
📦

@duﬄeit 📦 📦 👧 🧑 🧑 📦 👧 🧑 🧑
👧 🧑 🧑

@duﬄeit Deployment Units Monolithic Distributed Modularisa3on Bad Well Big Ball
Of Mud WelL-Structured Modulith Distributed Monolith Autonomous Service-Based SYSTEMS

Of Mud WelL-Structured Modulith Distributed Monolith Autonomous Service-Based SYSTEMS Decomposition Fallacy

@duﬄeit Deployment Units Monolithic Distributed Modularisation Bad Well Big Ball

Of Mud WelL-Structured Modulith Distributed Monolith Autonomous Service-Based SYSTEMS Change in Cause of Failure Reliabilty

Of Mud WelL-Structured Modulith Distributed Monolith Autonomous Service-Based SYSTEMS Change in Cause of Failure Reliabilty Availability

@duffleit What’s the diﬀerence between a method call within a
single deployment unit

@duffleit What’s the difference between a method call within a
single deployment unit Deployment Unit Module A Module B moduleB.createUser() fun createUser() { /*...*/ }

single deployment unit, and a method call across the network. Deployment Unit Deployment Unit Service A Service B restClient.user() fun createUser() { /*...*/ }

single deployment unit, and a method call across the network. Everything.

@duffleit 👧 🧑 🧑 🔥 👧 🧑 🧑 👧 🧑
🧑 🔴

@duﬄeit 👧 🧑 🧑 🔥 👧 🧑 🧑 👧 🧑
🧑 🔴 https://en.wikipedia.org/wiki/Fallacies_of_distributed_computing

@duﬄeit 📦 📦 👧 🧑 🧑 📦 👧 🧑 🧑
👧 🧑 🧑

@duﬄeit 📦 📦 👧 🧑 🧑 📦 👧 🧑 🧑
👧 🧑 🧑 🔥

@duﬄeit 📦 📦 👧 🧑 🧑 📦 👧 🧑 🧑
👧 🧑 🧑 🤯

@duﬄeit To enable teams to strive for Autonomy. We need
to Optimize for Availability.

@duﬄeit 👧 🧑 🧑 🔥 👧 🧑 🧑 🔴 Complex
Distributed System 👧 🧑 🧑 Loan Team Lending System

@duﬄeit 👧 🧑 🧑 🔥 👧 🧑 🧑 🔴 🔥
Complex Distributed System 👧 🧑 🧑 🟢 Account Team Loan Team Lending System

@duffleit 👧 🧑 🧑 🔥 👧 🧑 🧑 Complex Distributed
System 👧 🧑 🧑 🔴 Loan Team Lending System Account Team

@duﬄeit We strive for a stable environment.

@duﬄeit There is no stable environment.

@duﬄeit There is no stable environment. * The sooner we
accept this, the better.

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 Payment
Service 🧑 Account Service

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🔴
Payment Service 🧑 Account Service 🔥 Users[] 🔥

@duffleit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🔴
Payment Service 🧑 Account Service Users[] 🔥

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🔴
Payment Service 🧑 Account Service Users[] cache

@duffleit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🔴
Payment Service 🧑 Account Service Users[] stream UserChanged

@duffleit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🔴
Payment Service 🧑 Account Service readmodel stream Users[] UserChanged

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🔴
Payment Service 🧑 Account Service readmodel stream Users[] UserChanged

Service 🧑 Account Service readmodel stream 🔥🔥 🔥 🔥 Users[]

Service 🧑 Account Service 🔥 🚀 Caching LBs CBs

@duﬄeit Chaotic Environments Architect for 🔥

Service 🧑 Account Service 🔥 🔥 🔥 🔥

@duffleit we need to start dacining with the beast and
embrace Choas.

@duffleit 👧 🧑 🧑 👧 🧑 🧑 👧 🧑 🧑
Gamedays 🤡 🤡 🤡

@duﬄeit A Services that can run in a stable environment
can run only there.

@duﬄeit A Services that can run in a chaotic environment
can run everywhere.

@duﬄeit Chaotic Environments Architect for 🔥

@duﬄeit 👧 🧑 🧑 🔥 👧 🧑 👧 🧑 🧑
🔴 Payment Team 🧑 Account Team Execute Payment Payment Gateway Team

@duﬄeit 👧 🧑 🧑 👧 🧑 🧑 👧 🧑 Payment
Team 🧑 Account Team 🤯

@duffleit < 👧 🧑 🧑 SEPA Payment Team Account Team
CREDITCARD Payment Team < 👧 🧑 🧑 🧑 👧 🧑

@duffleit < 👧 🧑 🧑 End2End Responsibility SEPA Payment Team
Account Team CREDITCARD Payment Team < End2End Responsibility 👧 🧑 🧑 🧑 👧 🧑 😌

@duﬄeit < 👧 🧑 🧑 End2End Responsibility SEPA Payment Team
Account Team CREDITCARD Payment Team < End2End Responsibility 👧 🧑 🧑 🧑 👧 🧑 😌 We need to keep depenedencies in mind Accounts

@duffleit Customer Journey and Entity Data Mismatch Problem

@duffleit Customer Journey Entities User Accounts Transactions User Onboarding Payment
Execution Fraud Detection

@duﬄeit < 👧 🧑 🧑 End2End Responsibility Account Team <
End2End Responsibility 👧 🧑 🧑 🧑 👧 🧑 Accounts

End2End Responsibility 👧 🧑 🧑 🧑 👧 🧑 Accounts cache 🔥

End2End Responsibility 👧 🧑 🧑 🧑 👧 🧑 Accounts Account Changed readmodel 🔥

@duﬄeit End-2-End Responsibilities. Architect for 🛶

@duﬄeit End-to-End Autonomy ⚖ Dependencies Decrease Cognitive Load

@duﬄeit < SEPA Payment Team Account Team FRAUD CHECK <
👧 🧑 🧑 Check Payment Complicated Subsystem 👧 🧑 🧑 🧑 👧 🧑

@duffleit < SEPA Payment Team Account Team FRAUD CHECK <
👧 🧑 🧑 Check Payment Complicated Subsystem 👧 🧑 🧑 🧑 👧 🧑 💁🙋

👧 🧑 🧑 Check Payment Complicated Subsystem 👧 🧑 🧑 🧑 👧 🧑 Availability How many Nines? Nines Percentage Yearly Outage 1 Nine 90% 36,5 days 2 Nines 99% 3,65 days 3 Nines 99,9% 8,76 hours 4 Nines 99,99% 52,56 minutes 5 Nines 99,999% 5,26 minutes 6 Nines 99,9999% 31,5 seconds MTBF

@duffleit Can our stream aligned team still have a 99,99%
availability if our complex subsystem dependency has a 99,9% availability?

👧 🧑 🧑 Check Payment Complicated Subsystem 👧 🧑 🧑 🧑 👧 🧑 Availability How many Nines? Nines Percentage Yearly Outage 1 Nine 90% 36,5 days 2 Nines 99% 3,65 days 3 Nines 99,9% 8,76 hours 4 Nines 99,99% 52,56 minutes 5 Nines 99,999% 5,26 minutes 6 Nines 99,9999% 31,5 seconds rescheduler

👧 🧑 🧑 Check Payment Complicated Subsystem 👧 🧑 🧑 🧑 👧 🧑 Availability How many Nines? Nines Percentage Yearly Outage 1 Nine 90% 36,5 days 2 Nines 99% 3,65 days 3 Nines 99,9% 8,76 hours 4 Nines 99,99% 52,56 minutes 5 Nines 99,999% 5,26 minutes 6 Nines 99,9999% 31,5 seconds 🔄 rescheduler

@duffleit < Account Team FRAUD CHECK < 👧 🧑 🧑
Check Payment Complicated Subsystem 👧 🧑 🧑 Availability How many Nines? Nines Percentage Yearly Outage 1 Nine 90% 36,5 days 2 Nines 99% 3,65 days 3 Nines 99,9% 8,76 hours 4 Nines 99,99% 52,56 minutes 5 Nines 99,999% 5,26 minutes 6 Nines 99,9999% 31,5 seconds 🔄 rescheduler Platform Teams Developer Experience SLOs

@duffleit Pain = Strength x Distance x Volatility Vladik Khononov
Dependencies their three main aspects:

@duffleit Pain = Strength x Distance x Volatility Vladik Khononov
Dependencies

@duffleit Dependencies as a Service Architect for 9⃣

👧 🧑 🧑 Legacy System 👧 🧑 🧑 👧 🧑 🧑

👧 🧑 🧑 Legacy System 👧 🧑 🧑

@duffleit User Fraud Data Readers Writers Readers Writers

@duffleit User Fraud Data Readers Writers Readers Writers API

@duffleit User Fraud Data Readers Writers Readers Writers API Downtime
⏳

@duﬄeit Readers Writers Readers Writers API User Fraud Data 🔥

@duﬄeit User Fraud Data Readers Writers API 🔥 Readers Writers

@duﬄeit User Fraud Data Readers Writers API Readers Writers

@duﬄeit Event User Fraud Data Readers Writers Readers Writers API
User Fraud Data CDC

@duﬄeit Event User Fraud Data Readers Readers Writers API User
Fraud Data Event Writers

User Fraud Data Event 🔥

@duffleit Event User Fraud Data Readers Readers Writers API User
Fraud Data Event Writers 👷

User Fraud Data Event

@duﬄeit Event User Fraud Data Readers Writers API User Fraud
Data Event Readers Writers

@duffleit CDC User Fraud Data Readers Writers API User Fraud
Data Event Readers Writers Canary Releases Zero Downtime Migration Dark Launching Scientist Pattern

@duﬄeit Change Anxiety Architect against 🦜

@duﬄeit So many thing to consider, Let’s sum up.

@duﬄeit Modularisation is no reason to go for Distributed Systems.
Deployment Units Monolithic Distributed Reliabilty Availability Autonomy is.

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🧑

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🧑
Chaotic Environments Architect for 🔥

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🧑
Chaotic Environments Architect for 🔥 End-2-End Responsibilities Architect for 🛶

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🧑
Chaotic Environments Architect for 🔥 End-2-End Responsibilities Architect for 🛶 COMPLICATED Subsystem Teams Dependencies as a Service Architect for 9⃣

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🧑
Chaotic Environments Architect for 🔥 End-2-End Responsibilities Architect for 🛶 COMPLICATED Subsystem Teams Change Anxiety Architect against 🦜 Dependencies as a Service Architect for 9⃣

@duffleit 👧 🧑 🧑 👧 🧑 🧑 👧 🧑 🧑
🔥

@duffleit 📦 👧 🧑 🧑 📦 📦 keep the flow
of value high & the number of outages low. 👧 🧑 🧑 👧 🧑 🧑

@duﬄeit Keep the lead time to thank you low, even
at scale.

@duﬄeit DAVID LEITNER Principal Engineer 👋 [email protected] 🌎 @duffleit

Architecting for Scale

Architecting for Scale

More Decks by SQUER Solutions

Featured

Transcript