Cultivating Architecture in an Agile World

Slide 1

Slide 1 text

@duﬄeit Cultivating Architecture In an Agile World David Leitner

Slide 2

Slide 2 text

@duﬄeit DAVID LEITNER Coding Architect 👋 [email protected] 🌎 @duffleit

Slide 3

Slide 3 text

@duﬄeit 📦 👧 🧑 👧 🧑 👧 🔥 Safely and sustainably reduce lead time to thank you. Daniel Terhorst-North “Good Architecture”

Slide 4

Slide 4 text

@duﬄeit Simple System

Slide 5

Slide 5 text

@duﬄeit Complicated System

Slide 6

Slide 6 text

@duﬄeit Complicated System

Slide 7

Slide 7 text

@duﬄeit Complicated System Module A Module B Module C

Slide 8

Slide 8 text

@duﬄeit Complicated System Distributed Systems Service A Service B Service C

Slide 9

Slide 9 text

@duﬄeit Deployment Units Monolithic Distributed Modularisation Bad Well Big Ball Of Mud

Slide 10

Slide 10 text

@duﬄeit “Sadly, architecture has been undervalued for so long that many engineers regard life with a Big Ball of Mud as normal.“ Foote & Yoder

Slide 11

Slide 11 text

@duﬄeit Deployment Units Monolithic Distributed Modularisation Bad Well Big Ball Of Mud Distributed MOnolith WelL-Structured MOdulith

Slide 12

Slide 12 text

@duﬄeit There is a multitude of reasons to go for Distributed Systems. Modularisation is none of them.

Slide 13

Slide 13 text

@duﬄeit If you can’t build a well-structured monolith, what makes you think microservices are the answer? Simon Brown

Slide 14

Slide 14 text

@duﬄeit Module A Module B Module C Single Deployment Unit

Slide 15

Slide 15 text

@duﬄeit Service A Service B Serivice C

Slide 16

Slide 16 text

@duﬄeit Service A Service B Serivice C Individual Scaling Demand 10x Users Usual Traffic

Slide 17

Slide 17 text

@duﬄeit Service A Service B Serivice C Individual Scaling Demand Technology Segmentation

Slide 18

Slide 18 text

@duﬄeit Service A Service B Serivice C Individual Scaling Demand Technology Segmentation 🇩🇪 🇩🇪 🇺🇸 CO-Locating

Slide 19

Slide 19 text

@duffleit Service A Service B Serivice C Individual Scaling Demand Technology Segmentation CO-Locating Technical Benefits

Slide 20

Slide 20 text

@duffleit Service A Service B Serivice C 👧 🧑 🧑 👧 🧑 🧑 👧 🧑 🧑 Ownership You build it, You Own it You build it, You Run it

Slide 21

Slide 21 text

@duﬄeit Single Deployment Unit MOdule A MOdule B MOdule C 👧 🧑 🧑 👧 🧑 🧑 👧 🧑 🧑 Ownership You build it, You Own it You build it, You Run it 🔥

Slide 22

Slide 22 text

@duﬄeit Single Deployment Unit MOdule A MOdule B MOdule C 👧 🧑 🧑 👧 🧑 🧑 👧 🧑 🧑 Ownership You build it, You Own it You build it, You Run it 🔥 🔥

Slide 23

Slide 23 text

@duﬄeit Service A Service B Serivice C 👧 🧑 🧑 👧 🧑 🧑 👧 🧑 🧑 Enable Teams to work autonomously

Slide 24

Slide 24 text

@duffleit Technical Benefits Team Autonomoy Scale Engineering CO-Locating Technology Segmentation Individual Scaling Demand 80%

Slide 25

Slide 25 text

@duffleit We architect for autonomy. This is why we tend do go for Distributed Systems.

Slide 26

Slide 26 text

@duffleit Non-Functional Requirements Those requirements which make your software non-functional if you do not consider them.

Slide 27

Slide 27 text

@duffleit Scalability Availability Consistency Autonomy Your architecture is a tradeoff. …

Slide 28

Slide 28 text

@duffleit Business & Domain Organisation Technology

Slide 29

Slide 29 text

@duffleit a Threefold Architect needs to navigate the intersections of NFRs between all socio-technical aspects. Business & Domain Organisation Technology

Slide 30

Slide 30 text

@duffleit Scalability Availability Consistency Autonomy Your architecture is a tradeoff. …

Slide 31

Slide 31 text

@duffleit Scalability Availability Consistency Autonomy Your architecture is a tradeoﬀ. …

Slide 32

Slide 32 text

@duffleit Responsibility The ops will hate it Zone The Devs will Quit Zone Autonomy You build it you Own it You build it you run it You build it you care it

Slide 33

Slide 33 text

@duffleit Responsibility The ops will hate it Zone The Devs will Quit Zone Autonomy You build it you Own it You build it you run it You build it you care it

Slide 34

Slide 34 text

@duffleit Team Autonomy Decision Space Responsibility The ops will hate it Zone The Devs will Quit Zone Autonomy You build it you Own it You build it you run it You build it you care it Governance

Slide 35

Slide 35 text

@duﬄeit Team Autonomy Decision Space Responsibility The ops will hate it Zone The Devs will Quit Zone Autonomy You build it you Own it You build it you run it You build it you care it Governance

Slide 36

Slide 36 text

@duﬄeit Team Autonomy Decision Space Responsibility The ops will hate it Zone The Devs will Quit Zone Autonomy You build it you Own it You build it you run it You build it you care it Governance

Slide 37

Slide 37 text

@duffleit Team Autonomy Decision Space Responsibility The ops will hate it Zone The Devs will Quit Zone Autonomy You build it you Own it You build it you run it You build it you care it Governance Fitness Functions ADRs & RFCs Tech Radar Figures of Merit

Slide 38

Slide 38 text

Slide 39

Slide 39 text

@duffleit 👧 🧑 🧑 👧 🧑 🧑 👧 🧑 🧑 📦

Slide 40

Slide 40 text

@duffleit 📦 📦 👧 🧑 🧑 📦 👧 🧑 🧑 👧 🧑 🧑

Slide 41

Slide 41 text

@duffleit Deployment Units Monolithic Distributed Modularisa6on Bad Well Big Ball Of Mud WelL-Structured Modulith Distributed Monolith Autonomous Service-Based SYSTEMS

Slide 42

Slide 42 text

@duﬄeit Deployment Units Monolithic Distributed Modularisation Bad Well Big Ball Of Mud WelL-Structured Modulith Distributed Monolith Autonomous Service-Based SYSTEMS Change in Cause of Failure Reliabilty

Slide 43

Slide 43 text

@duffleit Deployment Units Monolithic Distributed Modularisa6on Bad Well Big Ball Of Mud WelL-Structured Modulith Distributed Monolith Autonomous Service-Based SYSTEMS Change in Cause of Failure Reliabilty Availability

Slide 44

Slide 44 text

@duffleit What’s the difference between a method call within a single deployment unit

Slide 45

Slide 45 text

@duffleit What’s the diﬀerence between a method call within a single deployment unit Deployment Unit Module A Module B moduleB.createUser() fun createUser() { /*...*/ }

Slide 46

Slide 46 text

@duffleit What’s the difference between a method call within a single deployment unit, and a method call across the network. Deployment Unit Deployment Unit Service A Service B restClient.user() fun createUser() { /*...*/ }

Slide 47

Slide 47 text

@duffleit What’s the difference between a method call within a single deployment unit, and a method call across the network. Everything.

Slide 48

Slide 48 text

@duffleit 👧 🧑 🧑 🔥 👧 🧑 🧑 👧 🧑 🧑 🔴

Slide 49

Slide 49 text

@duﬄeit 👧 🧑 🧑 🔥 👧 🧑 🧑 👧 🧑 🧑 🔴 https://en.wikipedia.org/wiki/Fallacies_of_distributed_computing

Slide 50

Slide 50 text

@duﬄeit 📦 📦 👧 🧑 🧑 📦 👧 🧑 🧑 👧 🧑 🧑

Slide 51

Slide 51 text

@duffleit 📦 📦 👧 🧑 🧑 📦 👧 🧑 🧑 👧 🧑 🧑 🔥

Slide 52

Slide 52 text

@duffleit 📦 📦 👧 🧑 🧑 📦 👧 🧑 🧑 👧 🧑 🧑 🤯

Slide 53

Slide 53 text

@duﬄeit To architect for Autonomy. We need to Optimize for Availability.

Slide 54

Slide 54 text

Slide 55

Slide 55 text

@duﬄeit Deployment Units Monolithic Distributed Modularisa6on Bad Well Big Ball Of Mud WelL-Structured Modulith Distributed Monolith Autonomous Service-Based SYSTEMS Change in Cause of Failure Reliabilty Availability

Slide 56

Slide 56 text

@duﬄeit 👧 🧑 🧑 🔥 👧 🧑 🧑 🔴 Complex Distributed System 👧 🧑 🧑 Loan Team Lending System

Slide 57

Slide 57 text

@duffleit 👧 🧑 🧑 🔥 👧 🧑 🧑 🔴 🔥 Complex Distributed System 👧 🧑 🧑 🟢 Account Team Loan Team Lending System

Slide 58

Slide 58 text

@duffleit 👧 🧑 🧑 🔥 👧 🧑 🧑 Complex Distributed System 👧 🧑 🧑 🔴 Loan Team Lending System Account Team

Slide 59

Slide 59 text

@duﬄeit We strive for a stable environment.

Slide 60

Slide 60 text

@duﬄeit There is no stable environment.

Slide 61

Slide 61 text

@duﬄeit There is no stable environment. * The sooner we accept this, the better.

Slide 62

Slide 62 text

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 Payment Service 🧑 Account Service

Slide 63

Slide 63 text

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🔴 Payment Service 🧑 Account Service 🔥 Users[] 🔥

Slide 64

Slide 64 text

@duffleit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🔴 Payment Service 🧑 Account Service Users[] 🔥

Slide 65

Slide 65 text

@duffleit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🔴 Payment Service 🧑 Account Service Users[] cache

Slide 66

Slide 66 text

@duffleit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🔴 Payment Service 🧑 Account Service Users[] stream UserChanged

Slide 67

Slide 67 text

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🔴 Payment Service 🧑 Account Service readmodel stream Users[] UserChanged

Slide 68

Slide 68 text

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🔴 Payment Service 🧑 Account Service readmodel stream Users[] UserChanged

Slide 69

Slide 69 text

@duffleit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 Payment Service 🧑 Account Service readmodel stream 🔥🔥 🔥 🔥 Users[]

Slide 70

Slide 70 text

@duffleit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 Payment Service 🧑 Account Service 🔥 🚀 Caching LBs CBs

Slide 71

Slide 71 text

@duﬄeit Chaotic Environments Architect for 🔥

Slide 72

Slide 72 text

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 Payment Service 🧑 Account Service 🔥 🔥 🔥 🔥

Slide 73

Slide 73 text

@duffleit we need to start dacining with the beast and embrace Choas.

Slide 74

Slide 74 text

@duffleit 👧 🧑 🧑 👧 🧑 🧑 👧 🧑 🧑 Gamedays 🤡 🤡 🤡

Slide 75

Slide 75 text

@duffleit A System that can run in a stable environment can run only there.

Slide 76

Slide 76 text

@duffleit A System that can run in a chaotic environment can run everywhere.

Slide 77

Slide 77 text

@duffleit Chaotic Environments Architect for 🔥

Slide 78

Slide 78 text

@duffleit 👧 🧑 🧑 🔥 👧 🧑 👧 🧑 🧑 🔴 Payment Team 🧑 Account Team Execute Payment Payment Gateway Team

Slide 79

Slide 79 text

@duffleit 👧 🧑 🧑 👧 🧑 🧑 👧 🧑 Payment Team 🧑 Account Team 🤯

Slide 80

Slide 80 text

@duffleit < 👧 🧑 🧑 SEPA Payment Team Account Team CREDITCARD Payment Team < 👧 🧑 🧑 🧑 👧 🧑

Slide 81

Slide 81 text

@duﬄeit < 👧 🧑 🧑 End2End Responsibility SEPA Payment Team Account Team CREDITCARD Payment Team < End2End Responsibility 👧 🧑 🧑 🧑 👧 🧑 😌

Slide 82

Slide 82 text

@duﬄeit < 👧 🧑 🧑 End2End Responsibility SEPA Payment Team Account Team CREDITCARD Payment Team < End2End Responsibility 👧 🧑 🧑 🧑 👧 🧑 😌 We need to keep depenedencies in mind Accounts

Slide 83

Slide 83 text

@duﬄeit Customer Journey and Entity Data Mismatch Problem

Slide 84

Slide 84 text

@duﬄeit Customer Journey Entities User Accounts Transactions User Onboarding Payment Execution Fraud Detection

Slide 85

Slide 85 text

@duffleit < 👧 🧑 🧑 End2End Responsibility Account Team < End2End Responsibility 👧 🧑 🧑 🧑 👧 🧑 Accounts

Slide 86

Slide 86 text

@duﬄeit < 👧 🧑 🧑 End2End Responsibility Account Team < End2End Responsibility 👧 🧑 🧑 🧑 👧 🧑 Accounts cache 🔥

Slide 87

Slide 87 text

@duﬄeit < 👧 🧑 🧑 End2End Responsibility Account Team < End2End Responsibility 👧 🧑 🧑 🧑 👧 🧑 Accounts Account Changed readmodel 🔥

Slide 88

Slide 88 text

@duﬄeit End-2-End Responsibilities. Architect for 🛶

Slide 89

Slide 89 text

@duﬄeit End-to-End Autonomy ⚖ Dependencies Decrease Cognitive Load

Slide 90

Slide 90 text

@duﬄeit < SEPA Payment Team Account Team FRAUD CHECK < 👧 🧑 🧑 Check Payment Complicated Subsystem 👧 🧑 🧑 🧑 👧 🧑

Slide 91

Slide 91 text

@duffleit < SEPA Payment Team Account Team FRAUD CHECK < 👧 🧑 🧑 Check Payment Complicated Subsystem 👧 🧑 🧑 🧑 👧 🧑 💁🙋

Slide 92

Slide 92 text

@duffleit < SEPA Payment Team Account Team FRAUD CHECK < 👧 🧑 🧑 Check Payment Complicated Subsystem 👧 🧑 🧑 🧑 👧 🧑 Availability How many Nines? Nines Percentage Yearly Outage 1 Nine 90% 36,5 days 2 Nines 99% 3,65 days 3 Nines 99,9% 8,76 hours 4 Nines 99,99% 52,56 minutes 5 Nines 99,999% 5,26 minutes 6 Nines 99,9999% 31,5 seconds MTBF

Slide 93

Slide 93 text

@duffleit Can our stream aligned team still have a 99,99% availability if our complex subsystem dependency has a 99,9% availability?

Slide 94

Slide 94 text

Slide 95

Slide 95 text

Slide 96

Slide 96 text

@duffleit < Account Team FRAUD CHECK < 👧 🧑 🧑 Check Payment Complicated Subsystem 👧 🧑 🧑 Availability How many Nines? Nines Percentage Yearly Outage 1 Nine 90% 36,5 days 2 Nines 99% 3,65 days 3 Nines 99,9% 8,76 hours 4 Nines 99,99% 52,56 minutes 5 Nines 99,999% 5,26 minutes 6 Nines 99,9999% 31,5 seconds 🔄 rescheduler Platform Teams Developer Experience SLOs

Slide 97

Slide 97 text

@duffleit Pain = Strength x Distance x Volatility Vladik Khononov Dependencies their three main aspects:

Slide 98

Slide 98 text

@duffleit Pain = Strength x Distance x Volatility Vladik Khononov Dependencies

Slide 99

Slide 99 text

@duffleit Dependencies as a Service Architect for 9⃣

Slide 100

Slide 100 text

@duffleit < SEPA Payment Team Account Team FRAUD CHECK < 👧 🧑 🧑 Legacy System 👧 🧑 🧑 👧 🧑 🧑

Slide 101

Slide 101 text

@duffleit < SEPA Payment Team Account Team FRAUD CHECK < 👧 🧑 🧑 Legacy System 👧 🧑 🧑

Slide 102

Slide 102 text

@duffleit User Fraud Data Readers Writers Readers Writers

Slide 103

Slide 103 text

@duffleit User Fraud Data Readers Writers Readers Writers API

Slide 104

Slide 104 text

@duffleit User Fraud Data Readers Writers Readers Writers API Downtime ⏳

Slide 105

Slide 105 text

@duffleit User Fraud Data Readers Writers Readers Writers API Downtime ⏳

Slide 106

Slide 106 text

@duffleit Readers Writers Readers Writers API User Fraud Data 🔥

Slide 107

Slide 107 text

@duffleit Readers Writers Readers Writers API User Fraud Data 🔥

Slide 108

Slide 108 text

@duffleit User Fraud Data Readers Writers API 🔥 Readers Writers

Slide 109

Slide 109 text

@duffleit User Fraud Data Readers Writers API Readers Writers

Slide 110

Slide 110 text

@duffleit Event User Fraud Data Readers Writers Readers Writers API User Fraud Data CDC

Slide 111

Slide 111 text

@duﬄeit Event User Fraud Data Readers Readers Writers API User Fraud Data Event Writers

Slide 112

Slide 112 text

@duﬄeit Event User Fraud Data Readers Writers Readers Writers API User Fraud Data Event 🔥

Slide 113

Slide 113 text

@duffleit Event User Fraud Data Readers Readers Writers API User Fraud Data Event Writers 👷

Slide 114

Slide 114 text

@duﬄeit Event User Fraud Data Readers Writers Readers Writers API User Fraud Data Event

Slide 115

Slide 115 text

@duﬄeit Event User Fraud Data Readers Writers API User Fraud Data Event Readers Writers

Slide 116

Slide 116 text

@duﬄeit CDC User Fraud Data Readers Writers API User Fraud Data Event Readers Writers Canary Releases Zero Downtime Migration Dark Launching A/B Testing Scientist Pattern

Slide 117

Slide 117 text

@duffleit

Slide 118

Slide 118 text

@duffleit

Slide 119

Slide 119 text

@duffleit Rollout when no disagreement within 24 hours and within 10 000 000 comparisons.

Slide 120

Slide 120 text

@duffleit CDC User Fraud Data Readers Writers API User Fraud Data Event Readers Writers Canary Releases Zero Downtime Migration Dark Launching A/B Testing Scientist Pattern

Slide 121

Slide 121 text

@duﬄeit Change Anxiety Architect against 🦜

Slide 122

Slide 122 text

@duﬄeit So many thing to consider, Let’s sum up.

Slide 123

Slide 123 text

@duﬄeit Business & Domain Organisation Technology Scalability Availability Consistency Autonomy …

Slide 124

Slide 124 text

@duﬄeit Architect for Autonomy. We usually need to

Slide 125

Slide 125 text

@duﬄeit Responsibility The ops will hate it Zone The Devs will Quit Zone Autonomy You build it you Own it You build it you run it You build it you care it Team Autonomy Decision Space Governance Fitness Functions ADRs & RFCs Tech Radar Figures of Merit Enablement

Slide 126

Slide 126 text

@duﬄeit Deployment Units Monolithic Distributed Reliabilty Availability

Slide 127

Slide 127 text

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🧑

Slide 128

Slide 128 text

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🧑 Chaotic Environments Architect for 🔥

Slide 129

Slide 129 text

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🧑 Chaotic Environments Architect for 🔥 End-2-End Responsibilities Architect for 🛶

Slide 130

Slide 130 text

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🧑 Chaotic Environments Architect for 🔥 End-2-End Responsibilities Architect for 🛶 COMPLICATED Subsystem Teams Dependencies as a Service Architect for 9⃣

Slide 131

Slide 131 text

@duﬄeit 👧 🧑 🧑 👧 🧑 👧 🧑 🧑 🧑 Chaotic Environments Architect for 🔥 End-2-End Responsibilities Architect for 🛶 COMPLICATED Subsystem Teams Change Anxiety Architect against 🦜 Dependencies as a Service Architect for 9⃣

Slide 132

Slide 132 text

@duffleit Safely and sustainably reduce lead time to thank you. Daniel Terhorst-North

Slide 133

Slide 133 text

@duffleit DAVID LEITNER Coding Architect 👋 [email protected] 🌎 @duﬄeit Slides