Architecting for Scale

@duffleit Architecting for Scale David Leitner

@duffleit DAVID LEITNER Principal Engineer 👋 [email protected] 🌎 @duﬄeit

@duffleit 📦 👧 🧑 👧 🧑 👧 🔥 Safely and
sustainably reduce lead time to thank you. Daniel Terhorst-North Complex System

@duffleit Simple System

@duffleit Complicated System

@duffleit Complicated System Module A Module B Module C

@duffleit Complicated System Distributed Systems Service A Service B Service
C

@duffleit Deployment Units Monolithic Distributed Modularisa3on Bad Well Big Ball
Of Mud

@duffleit “Sadly, architecture has been undervalued for so long that
many engineers regard life with a Big Ball of Mud as normal.“ Foote & Yoder

Of Mud Distributed MOnolith WelL-Structured MOdulith

@duffleit There is a multitude of reasons to go for
Distributed Systems. Modularisation is none of them.

@duffleit If you can’t build a well-structured monolith, what makes
you think microservices are the answer? Simon Brown

@duffleit Module A Module B Module C Single Deployment Unit

@duffleit Service A Service B Serivice C

@duffleit Service A Service B Serivice C Individual Scaling Demand
10x Users Usual Traﬃc

Technology Segmentation

Technology Segmentation 🇩🇪 🇩🇪 🇺🇸 CO-Locating

@duﬄeit Service A Service B Serivice C Individual Scaling Demand
Technology Segmentation CO-Locating Technical Benefits

@duﬄeit Service A Service B Serivice C 👧 🧑 🧑
👧 🧑 🧑 👧 🧑 🧑 Ownership You build it, You Own it You build it, You Run it

@duﬄeit Single Deployment Unit MOdule A MOdule B MOdule C
👧 🧑 🧑 👧 🧑 🧑 👧 🧑 🧑 Ownership You build it, You Own it You build it, You Run it 🔥

@duffleit Single Deployment Unit MOdule A MOdule B MOdule C
👧 🧑 🧑 👧 🧑 🧑 👧 🧑 🧑 Ownership You build it, You Own it You build it, You Run it 🔥 🔥

@duffleit Service A Service B Serivice C 👧 🧑 🧑
👧 🧑 🧑 👧 🧑 🧑 Enable Teams to work autonomously

@duﬄeit Technical Benefits Team Autonomoy Scale Engineering CO-Locating Technology Segmentation
Individual Scaling Demand 80%

@duﬄeit We optimize for Autonomy.

@duﬄeit Responsibility The ops will hate it Zone The Devs
will Quit Zone Autonomy You build it you Own it You build it you run it You build it you care it

@duﬄeit We optimize for Responsibility. Therefore, teams strive for Autonomy.

@duﬄeit 👧 🧑 🧑 👧 🧑 🧑 👧 🧑 🧑
📦

@duﬄeit 📦 📦 👧 🧑 🧑 📦 👧 🧑 🧑
👧 🧑 🧑

@duﬄeit Deployment Units Monolithic Distributed Modularisa3on Bad Well Big Ball
Of Mud WelL-Structured Modulith Distributed Monolith Autonomous Service-Based SYSTEMS

Of Mud WelL-Structured Modulith Distributed Monolith Autonomous Service-Based SYSTEMS Decomposition Fallacy

@duﬄeit Deployment Units Monolithic Distributed Modularisation Bad Well Big Ball

Of Mud WelL-Structured Modulith Distributed Monolith Autonomous Service-Based SYSTEMS Change in Cause of Failure Reliabilty

@duﬄeit Deployment Units Monolithic Distributed Modularisa3on Bad Well Big Ball
Of Mud WelL-Structured Modulith Distributed Monolith Autonomous Service-Based SYSTEMS Change in Cause of Failure Reliabilty Availability

@duffleit What’s the diﬀerence between a method call within a
single deployment unit

@duffleit What’s the difference between a method call within a
single deployment unit Deployment Unit Module A Module B moduleB.createUser() fun createUser() { /*...*/ }

single deployment unit, and a method call across the network. Deployment Unit Deployment Unit Service A Service B restClient.user() fun createUser() { /*...*/ }

single deployment unit, and a method call across the network. Everything.

@duffleit 👧 🧑 🧑 🔥 👧 🧑 🧑 👧 🧑
🧑 🔴

@duﬄeit 👧 🧑 🧑 🔥 👧 🧑 🧑 👧 🧑
🧑 🔴 https://en.wikipedia.org/wiki/Fallacies_of_distributed_computing

@duﬄeit 📦 📦 👧 🧑 🧑 📦 👧 🧑 🧑
👧 🧑 🧑

@duﬄeit 📦 📦 👧 🧑 🧑 📦 👧 🧑 🧑
👧 🧑 🧑 🔥

@duffleit To enable teams to strive for Autonomy. We need
to Optimize for Availability.

@duffleit 👧 🧑 🧑 🔥 👧 🧑 🧑 🔴 Complex
Distributed System 👧 🧑 🧑 Loan Team Lending System

@duﬄeit 👧 🧑 🧑 🔥 👧 🧑 🧑 🔴 🔥
Complex Distributed System 👧 🧑 🧑 🟢 Account Team Loan Team Lending System

@duﬄeit 👧 🧑 🧑 🔥 👧 🧑 🧑 Complex Distributed
System 👧 🧑 🧑 🔴 Loan Team Lending System Account Team

@duffleit We strive for a stable environment.

@duffleit There is no stable environment.

@duffleit There is no stable environment. * The sooner we
accept this, the better.

@duffleit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 Payment
Service 🧑 Account Service

@duffleit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🔴
Payment Service 🧑 Account Service 🔥 Users[] 🔥

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🔴
Payment Service 🧑 Account Service Users[] 🔥

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🔴
Payment Service 🧑 Account Service Users[] cache

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🔴
Payment Service 🧑 Account Service Users[] stream UserChanged

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🔴
Payment Service 🧑 Account Service readmodel stream Users[] UserChanged

@duffleit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🔴
Payment Service 🧑 Account Service readmodel stream Users[] UserChanged

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 Payment
Service 🧑 Account Service readmodel stream 🔥🔥 🔥 🔥 Users[]

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 Payment
Service 🧑 Account Service 🔥 🚀 Caching LBs CBs

@duffleit Chaotic Environments Architect for 🔥

@duffleit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 Payment
Service 🧑 Account Service 🔥 🔥 🔥 🔥

@duﬄeit we need to start dacining with the beast and
embrace Choas.

@duﬄeit 👧 🧑 🧑 👧 🧑 🧑 👧 🧑 🧑
Gamedays 🤡 🤡 🤡

@duﬄeit A Services that can run in a stable environment
can run only there.

@duﬄeit A Services that can run in a chaotic environment
can run everywhere.

@duﬄeit Chaotic Environments Architect for 🔥

@duﬄeit 👧 🧑 🧑 🔥 👧 🧑 👧 🧑 🧑
🔴 Payment Team 🧑 Account Team Execute Payment Payment Gateway Team

@duﬄeit 👧 🧑 🧑 👧 🧑 🧑 👧 🧑 Payment
Team 🧑 Account Team 🤯

@duﬄeit < 👧 🧑 🧑 SEPA Payment Team Account Team
CREDITCARD Payment Team < 👧 🧑 🧑 🧑 👧 🧑

@duﬄeit < 👧 🧑 🧑 End2End Responsibility SEPA Payment Team
Account Team CREDITCARD Payment Team < End2End Responsibility 👧 🧑 🧑 🧑 👧 🧑 😌

@duffleit < 👧 🧑 🧑 End2End Responsibility SEPA Payment Team
Account Team CREDITCARD Payment Team < End2End Responsibility 👧 🧑 🧑 🧑 👧 🧑 😌 We need to keep depenedencies in mind Accounts

@duﬄeit Customer Journey and Entity Data Mismatch Problem

@duﬄeit Customer Journey Entities User Accounts Transactions User Onboarding Payment
Execution Fraud Detection

@duﬄeit < 👧 🧑 🧑 End2End Responsibility Account Team <
End2End Responsibility 👧 🧑 🧑 🧑 👧 🧑 Accounts

@duffleit < 👧 🧑 🧑 End2End Responsibility Account Team <
End2End Responsibility 👧 🧑 🧑 🧑 👧 🧑 Accounts cache 🔥

@duffleit < 👧 🧑 🧑 End2End Responsibility Account Team <
End2End Responsibility 👧 🧑 🧑 🧑 👧 🧑 Accounts Account Changed readmodel 🔥

@duffleit End-2-End Responsibilities. Architect for 🛶

@duffleit End-to-End Autonomy ⚖ Dependencies Decrease Cognitive Load

@duffleit < SEPA Payment Team Account Team FRAUD CHECK <
👧 🧑 🧑 Check Payment Complicated Subsystem 👧 🧑 🧑 🧑 👧 🧑

@duﬄeit < SEPA Payment Team Account Team FRAUD CHECK <
👧 🧑 🧑 Check Payment Complicated Subsystem 👧 🧑 🧑 🧑 👧 🧑 💁🙋

👧 🧑 🧑 Check Payment Complicated Subsystem 👧 🧑 🧑 🧑 👧 🧑 Availability How many Nines? Nines Percentage Yearly Outage 1 Nine 90% 36,5 days 2 Nines 99% 3,65 days 3 Nines 99,9% 8,76 hours 4 Nines 99,99% 52,56 minutes 5 Nines 99,999% 5,26 minutes 6 Nines 99,9999% 31,5 seconds MTBF

@duffleit Can our stream aligned team still have a 99,99%
availability if our complex subsystem dependency has a 99,9% availability?

👧 🧑 🧑 Check Payment Complicated Subsystem 👧 🧑 🧑 🧑 👧 🧑 Availability How many Nines? Nines Percentage Yearly Outage 1 Nine 90% 36,5 days 2 Nines 99% 3,65 days 3 Nines 99,9% 8,76 hours 4 Nines 99,99% 52,56 minutes 5 Nines 99,999% 5,26 minutes 6 Nines 99,9999% 31,5 seconds rescheduler

👧 🧑 🧑 Check Payment Complicated Subsystem 👧 🧑 🧑 🧑 👧 🧑 Availability How many Nines? Nines Percentage Yearly Outage 1 Nine 90% 36,5 days 2 Nines 99% 3,65 days 3 Nines 99,9% 8,76 hours 4 Nines 99,99% 52,56 minutes 5 Nines 99,999% 5,26 minutes 6 Nines 99,9999% 31,5 seconds 🔄 rescheduler

@duffleit < Account Team FRAUD CHECK < 👧 🧑 🧑
Check Payment Complicated Subsystem 👧 🧑 🧑 Availability How many Nines? Nines Percentage Yearly Outage 1 Nine 90% 36,5 days 2 Nines 99% 3,65 days 3 Nines 99,9% 8,76 hours 4 Nines 99,99% 52,56 minutes 5 Nines 99,999% 5,26 minutes 6 Nines 99,9999% 31,5 seconds 🔄 rescheduler Platform Teams Developer Experience SLOs

@duffleit Pain = Strength x Distance x Volatility Vladik Khononov
Dependencies their three main aspects:

@duffleit Pain = Strength x Distance x Volatility Vladik Khononov
Dependencies

@duffleit Dependencies as a Service Architect for 9⃣

👧 🧑 🧑 Legacy System 👧 🧑 🧑 👧 🧑 🧑

👧 🧑 🧑 Legacy System 👧 🧑 🧑

@duffleit User Fraud Data Readers Writers Readers Writers

@duffleit User Fraud Data Readers Writers Readers Writers API

@duffleit User Fraud Data Readers Writers Readers Writers API Downtime
⏳

@duﬄeit Readers Writers Readers Writers API User Fraud Data 🔥

@duﬄeit User Fraud Data Readers Writers API 🔥 Readers Writers

@duﬄeit User Fraud Data Readers Writers API Readers Writers

@duﬄeit Event User Fraud Data Readers Writers Readers Writers API
User Fraud Data CDC

@duﬄeit Event User Fraud Data Readers Readers Writers API User
Fraud Data Event Writers

@duffleit Event User Fraud Data Readers Writers Readers Writers API
User Fraud Data Event 🔥

@duffleit Event User Fraud Data Readers Readers Writers API User
Fraud Data Event Writers 👷

@duﬄeit Event User Fraud Data Readers Writers Readers Writers API
User Fraud Data Event

@duffleit Event User Fraud Data Readers Writers API User Fraud
Data Event Readers Writers

@duﬄeit CDC User Fraud Data Readers Writers API User Fraud
Data Event Readers Writers Canary Releases Zero Downtime Migration Dark Launching Scientist Pattern

@duffleit Change Anxiety Architect against 🦜

@duffleit So many thing to consider, Let’s sum up.

@duffleit Modularisation is no reason to go for Distributed Systems.
Deployment Units Monolithic Distributed Reliabilty Availability Autonomy is.

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🧑

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🧑
Chaotic Environments Architect for 🔥

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🧑
Chaotic Environments Architect for 🔥 End-2-End Responsibilities Architect for 🛶

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🧑
Chaotic Environments Architect for 🔥 End-2-End Responsibilities Architect for 🛶 COMPLICATED Subsystem Teams Dependencies as a Service Architect for 9⃣

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🧑
Chaotic Environments Architect for 🔥 End-2-End Responsibilities Architect for 🛶 COMPLICATED Subsystem Teams Change Anxiety Architect against 🦜 Dependencies as a Service Architect for 9⃣

@duffleit 👧 🧑 🧑 👧 🧑 🧑 👧 🧑 🧑
🔥

@duﬄeit 📦 👧 🧑 🧑 📦 📦 keep the flow
of value high & the number of outages low. 👧 🧑 🧑 👧 🧑 🧑

@duﬄeit Keep the lead time to thank you low, even
at scale.

@duﬄeit DAVID LEITNER Principal Engineer 👋 [email protected] 🌎 @duffleit Slides

Architecting for Scale

Architecting for Scale

More Decks by SQUER Solutions

Other Decks in Technology

Featured

Transcript