Continuous Delivery for Legacy Code (v3) 🇬🇧 @DevOpsCon Winter 2022

Slide 1

Slide 1 text

Continuous Delivery for Legacy Code 06.12.22 Richard Gross (he/him) IT-Tailor Archaeologist Auditor richargh.de/ speakerdeck.com/richargh @arghrich

Slide 2

Slide 2 text

Slides by @arghrich What are the properties of low- quality software? 2

Slide 3

Slide 3 text

Slides by @arghrich 3 Dave Farely, author of the best-selling book “Continuous Delivery” The Quality of a System is Defined by Our Ability to Change it! Dave Farley @davefarley77

Slide 4

Slide 4 text

Slides by @arghrich Bad (legacy) software is software where we no longer have any options 4

Slide 5

Slide 5 text

Slides by @arghrich A story of the worst software I have ever and you will never see. 5 And we still got to Continuous Delivery J

Slide 6

Slide 6 text

Slides by @arghrich Along comes a customer … 6 Audit Icons by Franziska Haaf binarykitten.de COTS „Apocalypto“ Please Audit Customer Me

Slide 7

Slide 7 text

Slides by @arghrich 4M LoC Delivery 7 Audit Excel Webservice Website Fat Client Jobs Database T-SQL JavaScript VBA C# Asp.NET + C# Classic Asp + VBScript WinForms + C# MIA 5M No Code delivered

Slide 8

Slide 8 text

Slides by @arghrich Copy-Based Version ControlTM with production server as origin 8 Audit File tree from actual project

Slide 9

Slide 9 text

Slides by @arghrich Cycle Time of 6 months 9 Audit weeks Dev Machine Cp from Prod Develop Compile Don‘t Test Release Cp to Prod 1 hour Bugfixing Return to Start 5 months 6 months

Slide 10

Slide 10 text

Slides by @arghrich „Interesting“ Deployment 15 years of Uncontrolled growth Horrible Cycle Time No Tests Conflated Logic, UI & SQL Too much to fix Not Versioned & Doesn‘t Compile Icon by Franziska Haaf binarykitten.de 10

Slide 11

Slide 11 text

Slides by @arghrich 11 Icons by Franziska Haaf binarykitten.de COTS „Apocalypto“ In-house „Fenix“ Buy I‘ll Customer Money

Slide 12

Slide 12 text

Slides by @arghrich 12 Icons by Franziska Haaf binarykitten.de In-house „Fenix“ Please Fix Me Customer

Slide 13

Slide 13 text

Oh Boy

Slide 14

Slide 14 text

Ken Mugrage @kmugrage Legacy Code is code that’s too scary to update and too profitable to delete. It‘s not CD if you can‘t deploy to production right now Dylan Beattie @dylanbeattie Slides by @arghrich 14

Slide 15

Slide 15 text

Slides by @arghrich Can we deploy this to production right now? Nope 15

Slide 16

Slide 16 text

16 Slides by @arghrich Not Versioned & Doesn‘t Compile Icon by Franziska Haaf binarykitten.de 16

Slide 17

Slide 17 text

• Code • Configuration • Deployment scripts • Infrastructure Provisioning • Alerting • Monitoring • Documentation • Onboarding.sh Scripts • … 17 Put everything into version control Slides by @arghrich 17 Git

Slide 18

Slide 18 text

Slides by @arghrich Establish a Pipeline 18 Commit Stage Compile Git

Slide 19

Slide 19 text

Slides by @arghrich Versioned & Automated Compile „Interesting“ Deployment 19

Slide 20

Slide 20 text

Slides by @arghrich Scripts to Provision Infrastructure Same but parameterized Scripts for TST, STG and PRD • Machine • Web Server • Application Server • Database • Email Server • FTP Server • Pipeline Server 20 Phoenix Infra: With one click we can (re)create any infrastructure Gives us

Slide 21

Slide 21 text

Slides by @arghrich Deploy via Pipeline to Test Env 21 Commit Stage Compile Git Deploy To TST (Cont.) STG (Click) Smoke Test Deploy

Slide 22

Slide 22 text

Slides by @arghrich Can we deploy this to production right now? We have a button to do it, but do we want to? 22

Slide 23

Slide 23 text

Automated Deployment Slides by @arghrich Versioned & Automated Compile 15 years of Uncontrolled growth

Slide 24

Slide 24 text

Slides by @arghrich Don’t “Fix everything at once, mindset”TM 24

Slide 25

Slide 25 text

Slides by @arghrich Do the “Hotspot Mindset”TM 25

Slide 26

Slide 26 text

Slides by @arghrich Finding Hotspots 26

Slide 27

Slide 27 text

Art by Zaufishan Slides by @arghrich 27

Slide 28

Slide 28 text

Visualize metrics with CodeCharta buildings SomeService.cs Lines of code Complexity number of authors Icons by Fontawesome Open-Source Tool CodeCharta: https://maibornwolff.github.io/codecharta/ Slides by @arghrich 28

Slide 29

Slide 29 text

Find potential hotspots Open-Source Tool CodeCharta: https://maibornwolff.github.io/codecharta/ Slides by @arghrich 29 Lot‘s of code, very complex, and everyone has to change it

Slide 30

Slide 30 text

Slides by @arghrich No tool to rule them all 30 Shamelessly putting CodeCharta first 1 Quality Views by Colin Breck Continuously Periodically Structure101 SonarQube CodeCharta CodeScene ArchUnit TSArch SonarArchitect Quality Views 1

Slide 31

Slide 31 text

Slides by @arghrich 31 1 By Matthias Verraes 2 Graphana Graphs + Push Gateway Photo by Pixabay from Pexels Count Fixed Windows 2 Three Post-it‘s To Refactor 1 Slides by @arghrich

Slide 32

Slide 32 text

Versioned & Automated Compile Automated Deployment We know what to focus on No Tests Conflated Logic, UI & SQL

Slide 33

Slide 33 text

Slides by @arghrich Start with tests for critical use cases and hotspots • ~10 E2E Tests For critical use cases • DB-Integration Tests For direct-db-access hotspots that you will refactor • Unit tests (should give 80% confidence) For refactored hotspots For no-direct-db-access hotspots For new code 33

Slide 34

Slide 34 text

Slides by @arghrich Categorization test1: Write Tests for code that you don’t understand 34 1 Also called approval or golden master test. 2 Put system in a known (database) state is also an input. 3 Log messages you add are also an output. vX - redesigned v1 - legacy Input for UI/API/DB 2 Output 3 Output 3 Continuous Compare

Slide 35

Slide 35 text

Slides by @arghrich Understand Coupling and Cohesion High Coupling Low Cohesion Low Coupling High Cohesion 35 Unknown source

Slide 36

Slide 36 text

Slides by @arghrich Uncouple your Code 36 Delete Obsolete (unreachable or unused) Code Categorization Test, then Refactor Code Sprout new tested Code Break Dependencies (Seams) Strangler Fig Unfixable code Feature Flag Legacy Code Write ArchUnit Tests

Slide 37

Slide 37 text

Slides by @arghrich Can we deploy this to production right now? Hmm, it’s somewhat tested but can we really be sure nothing breaks in production? 37

Slide 38

Slide 38 text

38 Automated Deployment Slides by @arghrich Versioned & Automated Compile Seperated Logic, UI & SQL In Hotspots Critical Use Case Tests Growing Unit Tests Unit tested new code We know what to focus on Horrible Cycle Time

Slide 39

Slide 39 text

Slides by @arghrich Do Monitor in Production 39

Slide 40

Slide 40 text

Slides by @arghrich RED Metrics 40 1 https://thenewstack.io/monitoring-microservices-red-method/ Community Dashboard https://community.grafana.com/t/how-can-i-configure-red-dashboard/69885 R -> Request Rate E -> Errors / second D -> Duration per request

Slide 41

Slide 41 text

Slides by @arghrich New Cycle Time, without bug-cycle 41 Slide adapted from Dave Farley Commit Stage Compile Unit Test Lint Security Docker Release Git Monitor Logs Events Metrics Alerts Automated Performance testing 20 min Deploy To TST (Cont.) STG (Click) Smoke Test <1 min Manual Testing 30 min Automated Acceptance Testing User Acceptance Test DB-Integration Test 20 min 5 min 60 min

Slide 42

Slide 42 text

Slides by @arghrich Can we deploy this to production right now? Yes. Depending on where we make the change, even within 30min. 43

Slide 43

Slide 43 text

Slides by @arghrich But there is still a problem… 44

Slide 44

Slide 44 text

Slides by @arghrich How did the previous developers get here? 45

Slide 45

Slide 45 text

Slides by @arghrich Healthy Software 46 Ideas Product Process Mindset Predicts

Slide 46

Slide 46 text

Slides by @arghrich GIGO 47 Ideas Product Process Mindset Predicts

Slide 47

Slide 47 text

Slides by @arghrich A Feature Factory helps no one 48 https://cutlefish.substack.com/p/tbm-4952-your-calendar-your-priorities

Slide 48

Slide 48 text

Slides by @arghrich 49 Henrik Kniberg @henrikkniberg Culture is the sum of everyone's attitude and actions, so model the behaviour you want to see! Observe behavior: • Who does the management promote? • Is there fear of failure? • Is quality appreciated or story points or …? • What is the delivery „process“? • Is there some kind of learning exchange?

Slide 49

Slide 49 text

Slides by @arghrich Make the desired behavior … 50 Obvious • Define Principles together • Shared purpose Attractive • Reduce (time) pressure • (Refactoring) score Atomic Habits 4 Laws of behavior change Easy • Tech-coach • Reduce friction Satisfying • Pair up • Never break the chain (twice)

Slide 50

Slide 50 text

Slides by @arghrich CD is a change in mindset 51

Slide 51

Slide 51 text

Slides by @arghrich CD = Working in a way that software is always in a releasable state It’s not about the tools, even though they are getting better all the time 52

Slide 52

Slide 52 text

Slides by @arghrich Thanks 53 IT-Tailor Richard Gross (he/him) Archaeologist Auditor richargh.de/ speakerdeck.com/richargh @arghrich Works at maibornwolff.de/ Tailored Software by People who Care. DE TN ES

Slide 53

Slide 53 text

Slides by @arghrich Backup 56

Slide 54

Slide 54 text

Slides by @arghrich 57

Slide 55

Slide 55 text

Slides by @arghrich 58

Slide 56

Slide 56 text

Slides by @arghrich 10 customer installations 59 Frobozz Co. Oceanic Airlines … Umbrella Corp Soylent Corp

Slide 57

Slide 57 text

Slides by @arghrich And also 10 separate codebases 60 … Oceanic Airlines Soylent Corp Frobozz Co. Umbrella Corp Similar C/P Copy/ Paste Feature Copy/Paste

Slide 58

Slide 58 text

Slides by @arghrich 10 codebases And 10 database schemata 61 Similar Similar