CD for Legacy Code

This is based on a true story.

My day job is software archeology. I find joy in recovering and analyzing code bones and culture as well as making the skeleton walk again. A short time ago, however, I was confronted with the most horrible code base I have ever seen. This talk is about how we managed to save it and achieve bi-weekly deployments with a high level of confidence.

Five million lines of code in multiple languages (Classic ASP, .NET, VBScript, VBA, JavaScript, T-SQL, PL-SQL) in one monolith. The business logic stretched from the UI (WebForms, Scripting, SQL Queries) down to the database (Stored Procedures), there was no test coverage and an enormous amount of hidden coupling. A version control system was not used, we had no test environment, deployments required developers to copy their local compilation to production and multiple customer installations are supported by uncommenting and commenting code. Together we will explore what to do.

https://www.the-architecture-gathering.de/programm/programm-details/1394/cd-for-legacy-code/

Transcript

1. Continuous Delivery for Legacy Code Slides by Richard Gross @arghrich 16.10.19 Audit Software Archeology Develop Many Icons by Franziska Haaf binarykitten.de
2. Dylan Beattie @dylanbeattie Legacy Code is code that’s too scary to update and too profitable to delete. Michael Feathers @mfeathers Legacy Code is code without tests. Slide by @arghrich 2
3. Legacy Code is code that’s too scary to update and too profitable to delete. Continuous Delivery If somebody thinks of a good idea, how do we [make it feasable to] deliver it to users as quickly as possibly? 3 Dylan Beattie @dylanbeattie Slide by @arghrich
4. Slide by @arghrich 4 Photo by Anne-sophie Parent from Pexels Spring
5. Slide by @arghrich 5 Spring: Audit Icons by Franziska Haaf binarykitten.de COTS „Apocalypto“ Please Audit
6. 4M LoC Delivery 6 Excel Webservice Website Fat Client Jobs Database Spring: Audit Slide by @arghrich T-SQL JavaScript VBA C# Asp.NET + C# Classic Asp + VBScript WinForms + C# MIA No Code delivered 5M 4M LoC Delivery
7. Replicated Databases 7 Excel Webservice Website Fat Client Jobs DB1 DB2 Spring: Audit Slide by @arghrich Replication Dependency
8. Slide by @arghrich 8 Icons by Franziska Haaf binarykitten.de Doesn‘t Compile BL & DB Everywhere Interesting Deployment 15 years old No Tests Every Duplication Spring: Audit
9. COTS „Apocalypto“ In-house „Fenix“ Slide by @arghrich 9 Spring: Audit Icons by Franziska Haaf binarykitten.de Buy I‘ll
10. Slide by @arghrich 10 Spring: Audit Icons by Franziska Haaf binarykitten.de In-house „Fenix“ Please Fix
11. Oh Boy
12. Slide by @arghrich 12 We could just …
13. Slide by @arghrich 13 Photo by Levi Damasceno from Pexels
14. The True Cost of Rewrites Features Releases over time Catch Up Missing Features Sub-Par Parity Enhancements Planned Rewrite Actual Features Undiscovered Scope Old App Adoption Slide Adaption by @arghrich Original Article by Doug Bradbury The True Cost of Rewrites
15. Slide by @arghrich 15 Early Summer Photo by Pixabay
16. Slide by @arghrich 16 How did they get here? Early Summer
17. GIGO 17 Ideas Product Process (Team) Spirit Early Summer Slide by @arghrich Predicts
18. GIGO 18 Ideas Product Process (Team) Spirit Early Summer Slide by @arghrich Predicts Fix
19. Slide by @arghrich 19 How did we not get here? Early Summer
20. We started a significant change in team and culture 20 Early Summer: Spirit Slide by @arghrich
21. 21 Henrik Kniberg @henrikkniberg Culture is the sum of everyone's attitude and actions, so model the behaviour you want to see! Observe behavior: • Who does the management promote? • Is there fear of failure? • What is the delivery „process“? • Is there some kind of learning exchange? • Is quality appreciated or story points? Early Summer: Spirit Slide by @arghrich
22. 4 Laws of behavior change Make it … Obvious • Define Principles together • Shared purpose Attractive • Reduce (time) pressure • (Refactoring) score Slide by @arghrich 22 Atomic Habits Early Summer: Spirit Easy • Tech-coach • Reduce friction Satisfying • Pair up • Never break the chain (twice)
23. We built a new Co-located Cross-functional Team with shared 23 DEV TEST OPS Key Accounter Early Summer: Spirit & principles purpose
24. Our Legacy CD Principles • Build quality in • Focus on current tree, not on the forest • Tackle fear head on • Focus on reducing batch size • Low-Risk releases are incremental • Decouple Deployment and Release • Computers perform repetitive tasks, people solve problems Slide Adaption by @arghrich Four Principles of Low-Risk Software Releases by Jez Humble CD Principles by Jez Humble Early Summer: Spirit 24 The Why
25. Slide by @arghrich 25 Early Summer: Spirit Principles provide adaptability
26. Slide by @arghrich 26 Summer Icons by Franziska Haaf binarykitten.de In-house „Fenix“ Please Cloud
27. Oh Boy!
28. Fix 28 Product Process (Team) Spirit Summer Slide by @arghrich Predicts
29. Local Env Source Repository Artifact Repository Commit Stage Acceptance Manual Test Env … Staging Env Prod Env Self-Service Automatic Our CD Pipe Feedback (Tag Artifact) Slide Adaption by @arghrich Dave Farley - Optimising Continuous Delivery
30. Our Branching Strategy • Don‘t Branch • Don‘t Branch • Don‘t Branch • Never Branch to Refactor •Branches indicate Ci =Continuous Isolation Slide by @arghrich 30 Strategy by Dave Farley Summer: Process
31. We versioned and tested Everything • Code • Config • Deployment scripts • Infrastructure scripts • Docs • … 31 Slide by @arghrich Summer: Process
32. They used File Version Anti-Practice 32 Slide by @arghrich File tree from actual project Summer: Process
33. Slide by @arghrich 33 Autumn Photo by Daniel Frank from Pexels
34. Fix 34 Product Process (Team) Spirit Autumn Slide by @arghrich
35. Slide by @arghrich 35 Where do we start? Autumn
36. Slide by @arghrich Art by Zaufishan 36 Autumn: Product
37. github.com/MaibornWolff/codecharta Slide by @arghrich 37 Shameless Autumn: Product
38. Visualize metrics with buildings Slide by @arghrich 38 AbstractService.java Lines of code Complexity number of authors Autumn: Product Icons by Fontawesome Very Shameless
39. Find potential hotspots Slide by @arghrich 39 Still Shameless Autumn: Product
40. Establish baseline with tools Then track progress • CodeCharta https://github.com/ MaibornWolff/codecharta • SonarQube https://www.sonarqube.org/ • Linter • Profiler • CodeMaat https://github.com/ adamtornhill/code-maat • Structure101 http://structure101.com/ Slide by @arghrich 40 Autumn: Product Baseline
41. Golden Master Baseline 41 vX - redesigned v1 - legacy Input for UI/API/DB Output Output Continuous Compare Slide by @arghrich Autumn: Product Baseline
42. Slide by @arghrich 42 Autumn Icons by Franziska Haaf binarykitten.de In-house „Fenix“ Add Feature
43. Slide by @arghrich 43 Photo by Bartek Wojtas from Pexels Autumn Fuuuu
44. Conquer the fear • Write categorization/function Tests • Safely verify it can handle production • Divide code into stable and unstable Slide by @arghrich 44 Autumn: Conquer the Fear
45. Slide by @arghrich 45 Photo by Emmet from Pexels Autumn: Conquer the Fear
46. Slide by @arghrich 46 Photo by Krizjohn Rosales from Pexels Autumn: Conquer the Fear Isolate Stable from Unstable Working Effectively with Legacy Code • Direct Read/Write à Facades • Classes à Interfaces + Dependency Injection • Fire&Forget calls à Events • Obsolete/duplicate tables à Table Views
47. Slide by @arghrich 47 Photo by Alex Fu from Pexels Autumn: Conquer the Fear Sprout quality code Working Effectively with Legacy Code Table View Interface Facade Event
48. Use Strangler Application for bigger features 48 Slide by @arghrich Autumn: Conquer the Fear Picture by Paul Hammant
49. Slide by @arghrich 49 Winter Photo by Riccardo Bresciani from Pexels
50. vX - redesigned Dark Launch to gain confidence Slide by @arghrich 50 Icons by Fontawesome Load Balancer v1 - legacy Continuous Compare Same number of: • 200? • 404? • 5xx? • … Winter: Launch confidence
51. Remove any obsolete code in Tech || Business 51 Slide by @arghrich Winter: Remove waste Don‘t fix what doesn‘t need fixing Slide by @arghrich Photo by Stas Knop from Pexels
52. Finding obsolete code • Add Analytics / Activity-Log for your dark pages • User surveys • IDE suggestions • grep FROM, JOIN, INSERT, UPDATE, DELETE and check if that code is reachable • Legacy Toggles • Forget to deploy parts of your application 52 Slide by @arghrich Winter: Remove waste
53. Slide by @arghrich 53 Next Spring: Summary Photo by Irina Iriser from Pexels
54. Legacy Delivery 54 Slide by @arghrich Next Spring: Summary Photo by pixabay from Pexels Before • 2 Weeks for a feature • 5 months until it works After • 4 Weeks for a feature • Then it works
55. Was it worth it? • Users had capitulated à then we started fixing bugs • More Stable then before • We couldn‘t have reimplemented this from scratch • à None can describe what „same as before“ means • à Even if, it would have taken more than a year • Seamless migration from COTS to in-house à min. feature freeze 55 Slide by @arghrich Next Spring: Summary
56. Slide by @arghrich 56 Would I do it again? Next Spring: Summary
57. Hell Yes We had • Autonomy • Mastery • Purpose 57 Slide by @arghrich Next Spring: Summary Model from „Drive“ by Daniel Pink
58. 58 Next Spring: Summary Slide by @arghrich (Team) Spirit • Conquer the fear • CD Principles • Build quality in • Shared purpose Product • Isolate stable from unstable • Golden Master • Strangler • Feathers Process • CD Pipeline • Infrastructure Automation
59. Ken Mugrage @kmugrage It‘s not CD if you can‘t deploy to production right now 59 Slide by @arghrich Next Spring: Summary
60. Done Questions, Remarks, Sympathy? J 30 Mio 410 Employees 9th year 23 Nations 300 Weeks Munich, Augsburg, Berlin, Frankfurt
61. Learn Domain with Reverse Object Mother • Start application with empty database and without domain model • Click through a UseCase • Analyse exceptions and errors „App needs at least an object A with this field“ • Expand Domain Modell with your finding • Create required state in the DB with your model • Document finding as categorization test • Repeat 61 Slide by @arghrich Autumn: Rake the forest
62. Reading • https://trello.com/b/Fdd876S8/continuous- delivery-checklist-template • I did not think I would show this slide 62 Slide by @arghrich