Architecting for Scale - Speaker Deck

Slide 1

Slide 1 text

@duffleit The Art of Architecting for Scale @duffleit

Slide 2

Slide 2 text

@duffleit DAVID LEITNER Coding Architect 👋 [email protected] 🌎 @duﬄeit

Slide 3

Slide 3 text

@duffleit 📦 👧 🧑 👧 🧑 👧 🔥 Safely and sustainably reduce lead time to thank you. Daniel Terhorst-North Complex System

Slide 4

Slide 4 text

@duffleit Simple System

Slide 5

Slide 5 text

@duffleit Complicated System

Slide 6

Slide 6 text

@duffleit Complicated System

Slide 7

Slide 7 text

@duffleit Complicated System Module A Module B Module C

Slide 8

Slide 8 text

@duffleit Complicated System Distributed Systems Service A Service B Service C

Slide 9

Slide 9 text

@duffleit Deployment Units Monolithic Distributed Modularisa3on Bad Well Big Ball Of Mud

Slide 10

Slide 10 text

@duffleit “Sadly, architecture has been undervalued for so long that many engineers regard life with a Big Ball of Mud as normal.“ Foote & Yoder

Slide 11

Slide 11 text

@duffleit Deployment Units Monolithic Distributed Modularisa3on Bad Well Big Ball Of Mud Distributed MOnolith WelL-Structured MOdulith

Slide 12

Slide 12 text

@duffleit There is a multitude of reasons to go for Distributed Systems. Modularisation is none of them.

Slide 13

Slide 13 text

@duffleit If you can’t build a well-structured monolith, what makes you think microservices are the answer? Simon Brown

Slide 14

Slide 14 text

@duffleit Module A Module B Module C Single Deployment Unit

Slide 15

Slide 15 text

@duffleit Service A Service B Serivice C

Slide 16

Slide 16 text

@duffleit Service A Service B Serivice C Individual Scaling Demand 10x Users Usual Traﬃc

Slide 17

Slide 17 text

@duffleit Service A Service B Serivice C Individual Scaling Demand Technology Segmentation

Slide 18

Slide 18 text

@duffleit Service A Service B Serivice C Individual Scaling Demand Technology Segmentation 🇩🇪 🇩🇪 🇺🇸 CO-Locating

Slide 19

Slide 19 text

@duﬄeit Service A Service B Serivice C Individual Scaling Demand Technology Segmentation CO-Locating Technical Benefits

Slide 20

Slide 20 text

@duﬄeit Service A Service B Serivice C 👧 🧑 🧑 👧 🧑 🧑 👧 🧑 🧑 Ownership You build it, You Own it You build it, You Run it

Slide 21

Slide 21 text

@duﬄeit Single Deployment Unit MOdule A MOdule B MOdule C 👧 🧑 🧑 👧 🧑 🧑 👧 🧑 🧑 Ownership You build it, You Own it You build it, You Run it 🔥

Slide 22

Slide 22 text

@duffleit Single Deployment Unit MOdule A MOdule B MOdule C 👧 🧑 🧑 👧 🧑 🧑 👧 🧑 🧑 Ownership You build it, You Own it You build it, You Run it 🔥 🔥

Slide 23

Slide 23 text

@duffleit Service A Service B Serivice C 👧 🧑 🧑 👧 🧑 🧑 👧 🧑 🧑 Enable Teams to work autonomously

Slide 24

Slide 24 text

@duﬄeit Technical Benefits Team Autonomoy Scale Engineering CO-Locating Technology Segmentation Individual Scaling Demand 80%

Slide 25

Slide 25 text

@duﬄeit We optimize for Autonomy.

Slide 26

Slide 26 text

@duﬄeit Responsibility The ops will hate it Zone The Devs will Quit Zone Autonomy You build it you Own it You build it you run it You build it you care it

Slide 27

Slide 27 text

@duﬄeit Responsibility The ops will hate it Zone The Devs will Quit Zone Autonomy You build it you Own it You build it you run it You build it you care it

Slide 28

Slide 28 text

@duﬄeit We optimize for Responsibility. Therefore, teams strive for Autonomy.

Slide 29

Slide 29 text

@duﬄeit 👧 🧑 🧑 👧 🧑 🧑 👧 🧑 🧑 📦

Slide 30

Slide 30 text

@duﬄeit 📦 📦 👧 🧑 🧑 📦 👧 🧑 🧑 👧 🧑 🧑

Slide 31

Slide 31 text

@duﬄeit Deployment Units Monolithic Distributed Modularisa3on Bad Well Big Ball Of Mud WelL-Structured Modulith Distributed Monolith Autonomous Service-Based SYSTEMS

Slide 32

Slide 32 text

@duffleit Deployment Units Monolithic Distributed Modularisa3on Bad Well Big Ball Of Mud WelL-Structured Modulith Distributed Monolith Autonomous Service-Based SYSTEMS Decomposition Fallacy

Slide 33

Slide 33 text

@duﬄeit Deployment Units Monolithic Distributed Modularisation Bad Well Big Ball Of Mud WelL-Structured Modulith Distributed Monolith Autonomous Service-Based SYSTEMS Decomposition Fallacy

Slide 34

Slide 34 text

@duffleit Deployment Units Monolithic Distributed Modularisa3on Bad Well Big Ball Of Mud WelL-Structured Modulith Distributed Monolith Autonomous Service-Based SYSTEMS Decomposition Fallacy

Slide 35

Slide 35 text

@duﬄeit Deployment Units Monolithic Distributed Modularisation Bad Well Big Ball Of Mud WelL-Structured Modulith Distributed Monolith Autonomous Service-Based SYSTEMS Decomposition Fallacy

Slide 36

Slide 36 text

@duffleit Deployment Units Monolithic Distributed Modularisa3on Bad Well Big Ball Of Mud WelL-Structured Modulith Distributed Monolith Autonomous Service-Based SYSTEMS Decomposition Fallacy

Slide 37

Slide 37 text

@duﬄeit Deployment Units Monolithic Distributed Modularisation Bad Well Big Ball Of Mud WelL-Structured Modulith Distributed Monolith Autonomous Service-Based SYSTEMS Change in Cause of Failure Reliabilty

Slide 38

Slide 38 text

@duﬄeit Deployment Units Monolithic Distributed Modularisa3on Bad Well Big Ball Of Mud WelL-Structured Modulith Distributed Monolith Autonomous Service-Based SYSTEMS Change in Cause of Failure Reliabilty Availability

Slide 39

Slide 39 text

@duffleit What’s the diﬀerence between a method call within a single deployment unit

Slide 40

Slide 40 text

@duffleit What’s the difference between a method call within a single deployment unit Deployment Unit Module A Module B moduleB.createUser() fun createUser() { /*...*/ }

Slide 41

Slide 41 text

@duffleit What’s the diﬀerence between a method call within a single deployment unit, and a method call across the network. Deployment Unit Deployment Unit Service A Service B restClient.user() fun createUser() { /*...*/ }

Slide 42

Slide 42 text

@duffleit What’s the diﬀerence between a method call within a single deployment unit, and a method call across the network. Everything.

Slide 43

Slide 43 text

@duffleit 👧 🧑 🧑 🔥 👧 🧑 🧑 👧 🧑 🧑 🔴

Slide 44

Slide 44 text

@duﬄeit 👧 🧑 🧑 🔥 👧 🧑 🧑 👧 🧑 🧑 🔴 https://en.wikipedia.org/wiki/Fallacies_of_distributed_computing

Slide 45

Slide 45 text

@duﬄeit 📦 📦 👧 🧑 🧑 📦 👧 🧑 🧑 👧 🧑 🧑

Slide 46

Slide 46 text

@duﬄeit 📦 📦 👧 🧑 🧑 📦 👧 🧑 🧑 👧 🧑 🧑 🔥

Slide 47

Slide 47 text

@duﬄeit 📦 📦 👧 🧑 🧑 📦 👧 🧑 🧑 👧 🧑 🧑 🤯

Slide 48

Slide 48 text

@duﬄeit To enable teams to strive for Autonomy. We need to Optimize for Availability.

Slide 49

Slide 49 text

Slide 50

Slide 50 text

@duffleit Deployment Units Monolithic Distributed Modularisa3on Bad Well Big Ball Of Mud WelL-Structured Modulith Distributed Monolith Autonomous Service-Based SYSTEMS Change in Cause of Failure Reliabilty Availability

Slide 51

Slide 51 text

@duﬄeit 👧 🧑 🧑 🔥 👧 🧑 🧑 🔴 Complex Distributed System 👧 🧑 🧑 Loan Team Lending System

Slide 52

Slide 52 text

@duﬄeit 👧 🧑 🧑 🔥 👧 🧑 🧑 🔴 🔥 Complex Distributed System 👧 🧑 🧑 🟢 Account Team Loan Team Lending System

Slide 53

Slide 53 text

@duffleit 👧 🧑 🧑 🔥 👧 🧑 🧑 Complex Distributed System 👧 🧑 🧑 🔴 Loan Team Lending System Account Team

Slide 54

Slide 54 text

@duﬄeit We strive for a stable environment.

Slide 55

Slide 55 text

@duﬄeit There is no stable environment.

Slide 56

Slide 56 text

@duﬄeit There is no stable environment. * The sooner we accept this, the better.

Slide 57

Slide 57 text

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 Payment Service 🧑 Account Service

Slide 58

Slide 58 text

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🔴 Payment Service 🧑 Account Service 🔥 Users[] 🔥

Slide 59

Slide 59 text

@duffleit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🔴 Payment Service 🧑 Account Service Users[] 🔥

Slide 60

Slide 60 text

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🔴 Payment Service 🧑 Account Service Users[] cache

Slide 61

Slide 61 text

@duffleit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🔴 Payment Service 🧑 Account Service Users[] stream UserChanged

Slide 62

Slide 62 text

@duffleit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🔴 Payment Service 🧑 Account Service readmodel stream Users[] UserChanged

Slide 63

Slide 63 text

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🔴 Payment Service 🧑 Account Service readmodel stream Users[] UserChanged

Slide 64

Slide 64 text

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 Payment Service 🧑 Account Service readmodel stream 🔥🔥 🔥 🔥 Users[]

Slide 65

Slide 65 text

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 Payment Service 🧑 Account Service 🔥 🚀 Caching LBs CBs

Slide 66

Slide 66 text

@duﬄeit Chaotic Environments Architect for 🔥

Slide 67

Slide 67 text

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 Payment Service 🧑 Account Service 🔥 🔥 🔥 🔥

Slide 68

Slide 68 text

@duffleit we need to start dacining with the beast and embrace Choas.

Slide 69

Slide 69 text

@duffleit 👧 🧑 🧑 👧 🧑 🧑 👧 🧑 🧑 Gamedays 🤡 🤡 🤡

Slide 70

Slide 70 text

@duﬄeit A Services that can run in a stable environment can run only there.

Slide 71

Slide 71 text

@duﬄeit A Services that can run in a chaotic environment can run everywhere.

Slide 72

Slide 72 text

@duﬄeit Chaotic Environments Architect for 🔥

Slide 73

Slide 73 text

@duﬄeit 👧 🧑 🧑 🔥 👧 🧑 👧 🧑 🧑 🔴 Payment Team 🧑 Account Team Execute Payment Payment Gateway Team

Slide 74

Slide 74 text

@duﬄeit 👧 🧑 🧑 👧 🧑 🧑 👧 🧑 Payment Team 🧑 Account Team 🤯

Slide 75

Slide 75 text

@duffleit < 👧 🧑 🧑 SEPA Payment Team Account Team CREDITCARD Payment Team < 👧 🧑 🧑 🧑 👧 🧑

Slide 76

Slide 76 text

@duffleit < 👧 🧑 🧑 End2End Responsibility SEPA Payment Team Account Team CREDITCARD Payment Team < End2End Responsibility 👧 🧑 🧑 🧑 👧 🧑 😌

Slide 77

Slide 77 text

@duﬄeit < 👧 🧑 🧑 End2End Responsibility SEPA Payment Team Account Team CREDITCARD Payment Team < End2End Responsibility 👧 🧑 🧑 🧑 👧 🧑 😌 We need to keep depenedencies in mind Accounts

Slide 78

Slide 78 text

@duffleit Customer Journey and Entity Data Mismatch Problem

Slide 79

Slide 79 text

@duffleit Customer Journey Entities User Accounts Transactions User Onboarding Payment Execution Fraud Detection

Slide 80

Slide 80 text

@duﬄeit < 👧 🧑 🧑 End2End Responsibility Account Team < End2End Responsibility 👧 🧑 🧑 🧑 👧 🧑 Accounts

Slide 81

Slide 81 text

@duﬄeit < 👧 🧑 🧑 End2End Responsibility Account Team < End2End Responsibility 👧 🧑 🧑 🧑 👧 🧑 Accounts cache 🔥

Slide 82

Slide 82 text

@duﬄeit < 👧 🧑 🧑 End2End Responsibility Account Team < End2End Responsibility 👧 🧑 🧑 🧑 👧 🧑 Accounts Account Changed readmodel 🔥

Slide 83

Slide 83 text

@duﬄeit End-2-End Responsibilities. Architect for 🛶

Slide 84

Slide 84 text

@duﬄeit End-to-End Autonomy ⚖ Dependencies Decrease Cognitive Load

Slide 85

Slide 85 text

@duﬄeit < SEPA Payment Team Account Team FRAUD CHECK < 👧 🧑 🧑 Check Payment Complicated Subsystem 👧 🧑 🧑 🧑 👧 🧑

Slide 86

Slide 86 text

@duffleit < SEPA Payment Team Account Team FRAUD CHECK < 👧 🧑 🧑 Check Payment Complicated Subsystem 👧 🧑 🧑 🧑 👧 🧑 💁🙋

Slide 87

Slide 87 text

@duffleit < SEPA Payment Team Account Team FRAUD CHECK < 👧 🧑 🧑 Check Payment Complicated Subsystem 👧 🧑 🧑 🧑 👧 🧑 Availability How many Nines? Nines Percentage Yearly Outage 1 Nine 90% 36,5 days 2 Nines 99% 3,65 days 3 Nines 99,9% 8,76 hours 4 Nines 99,99% 52,56 minutes 5 Nines 99,999% 5,26 minutes 6 Nines 99,9999% 31,5 seconds MTBF

Slide 88

Slide 88 text

@duffleit Can our stream aligned team still have a 99,99% availability if our complex subsystem dependency has a 99,9% availability?

Slide 89

Slide 89 text

Slide 90

Slide 90 text

Slide 91

Slide 91 text

@duffleit < Account Team FRAUD CHECK < 👧 🧑 🧑 Check Payment Complicated Subsystem 👧 🧑 🧑 Availability How many Nines? Nines Percentage Yearly Outage 1 Nine 90% 36,5 days 2 Nines 99% 3,65 days 3 Nines 99,9% 8,76 hours 4 Nines 99,99% 52,56 minutes 5 Nines 99,999% 5,26 minutes 6 Nines 99,9999% 31,5 seconds 🔄 rescheduler Platform Teams Developer Experience SLOs

Slide 92

Slide 92 text

@duffleit Pain = Strength x Distance x Volatility Vladik Khononov Dependencies their three main aspects:

Slide 93

Slide 93 text

@duffleit Pain = Strength x Distance x Volatility Vladik Khononov Dependencies

Slide 94

Slide 94 text

@duffleit Dependencies as a Service Architect for 9⃣

Slide 95

Slide 95 text

@duffleit < SEPA Payment Team Account Team FRAUD CHECK < 👧 🧑 🧑 Legacy System 👧 🧑 🧑 👧 🧑 🧑

Slide 96

Slide 96 text

@duffleit < SEPA Payment Team Account Team FRAUD CHECK < 👧 🧑 🧑 Legacy System 👧 🧑 🧑

Slide 97

Slide 97 text

@duffleit User Fraud Data Readers Writers Readers Writers

Slide 98

Slide 98 text

@duffleit User Fraud Data Readers Writers Readers Writers API

Slide 99

Slide 99 text

@duffleit User Fraud Data Readers Writers Readers Writers API Downtime ⏳

Slide 100

Slide 100 text

@duffleit User Fraud Data Readers Writers Readers Writers API Downtime ⏳

Slide 101

Slide 101 text

@duﬄeit Readers Writers Readers Writers API User Fraud Data 🔥

Slide 102

Slide 102 text

@duﬄeit Readers Writers Readers Writers API User Fraud Data 🔥

Slide 103

Slide 103 text

@duﬄeit User Fraud Data Readers Writers API 🔥 Readers Writers

Slide 104

Slide 104 text

@duﬄeit User Fraud Data Readers Writers API Readers Writers

Slide 105

Slide 105 text

@duﬄeit Event User Fraud Data Readers Writers Readers Writers API User Fraud Data CDC

Slide 106

Slide 106 text

@duﬄeit Event User Fraud Data Readers Readers Writers API User Fraud Data Event Writers

Slide 107

Slide 107 text

@duﬄeit Event User Fraud Data Readers Writers Readers Writers API User Fraud Data Event 🔥

Slide 108

Slide 108 text

@duffleit Event User Fraud Data Readers Readers Writers API User Fraud Data Event Writers 👷

Slide 109

Slide 109 text

@duﬄeit Event User Fraud Data Readers Writers Readers Writers API User Fraud Data Event

Slide 110

Slide 110 text

@duﬄeit Event User Fraud Data Readers Writers API User Fraud Data Event Readers Writers

Slide 111

Slide 111 text

@duffleit CDC User Fraud Data Readers Writers API User Fraud Data Event Readers Writers Canary Releases Zero Downtime Migration Dark Launching Scientist Pattern

Slide 112

Slide 112 text

@duﬄeit Change Anxiety Architect against 🦜

Slide 113

Slide 113 text

@duﬄeit So many thing to consider, Let’s sum up.

Slide 114

Slide 114 text

@duﬄeit Modularisation is no reason to go for Distributed Systems. Deployment Units Monolithic Distributed Reliabilty Availability Autonomy is.

Slide 115

Slide 115 text

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🧑

Slide 116

Slide 116 text

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🧑 Chaotic Environments Architect for 🔥

Slide 117

Slide 117 text

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🧑 Chaotic Environments Architect for 🔥 End-2-End Responsibilities Architect for 🛶

Slide 118

Slide 118 text

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🧑 Chaotic Environments Architect for 🔥 End-2-End Responsibilities Architect for 🛶 COMPLICATED Subsystem Teams Dependencies as a Service Architect for 9⃣

Slide 119

Slide 119 text

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🧑 Chaotic Environments Architect for 🔥 End-2-End Responsibilities Architect for 🛶 COMPLICATED Subsystem Teams Change Anxiety Architect against 🦜 Dependencies as a Service Architect for 9⃣

Slide 120

Slide 120 text

@duffleit 👧 🧑 🧑 👧 🧑 🧑 👧 🧑 🧑 🔥

Slide 121

Slide 121 text

@duffleit 📦 👧 🧑 🧑 📦 📦 keep the flow of value high & the number of outages low. 👧 🧑 🧑 👧 🧑 🧑

Slide 122

Slide 122 text

@duﬄeit Keep the lead time to thank you low, even at scale.

Slide 123

Slide 123 text

@duﬄeit DAVID LEITNER Principal Engineer 👋 [email protected] 🌎 @duffleit