Upgrade to Pro — share decks privately, control downloads, hide ads and more …

CD for Legacy Code

October 16, 2019

CD for Legacy Code

This is based on a true story.

My day job is software archeology. I find joy in recovering and analyzing code bones and culture as well as making the skeleton walk again. A short time ago, however, I was confronted with the most horrible code base I have ever seen. This talk is about how we managed to save it and achieve bi-weekly deployments with a high level of confidence.

Five million lines of code in multiple languages (Classic ASP, .NET, VBScript, VBA, JavaScript, T-SQL, PL-SQL) in one monolith. The business logic stretched from the UI (WebForms, Scripting, SQL Queries) down to the database (Stored Procedures), there was no test coverage and an enormous amount of hidden coupling. A version control system was not used, we had no test environment, deployments required developers to copy their local compilation to production and multiple customer installations are supported by uncommenting and commenting code. Together we will explore what to do.



October 16, 2019

More Decks by Richard

Other Decks in Technology


  1. Continuous Delivery for Legacy Code Slides by Richard Gross @arghrich

    16.10.19 Audit Software Archeology Develop Many Icons by Franziska Haaf binarykitten.de
  2. Dylan Beattie @dylanbeattie Legacy Code is code that’s too scary

    to update and too profitable to delete. Michael Feathers @mfeathers Legacy Code is code without tests. Slide by @arghrich 2
  3. Legacy Code is code that’s too scary to update and

    too profitable to delete. Continuous Delivery If somebody thinks of a good idea, how do we [make it feasable to] deliver it to users as quickly as possibly? 3 Dylan Beattie @dylanbeattie Slide by @arghrich
  4. Slide by @arghrich 5 Spring: Audit Icons by Franziska Haaf

    binarykitten.de COTS „Apocalypto“ Please Audit
  5. 4M LoC Delivery 6 Excel Webservice Website Fat Client Jobs

    Database Spring: Audit Slide by @arghrich T-SQL JavaScript VBA C# Asp.NET + C# Classic Asp + VBScript WinForms + C# MIA No Code delivered 5M 4M LoC Delivery
  6. Replicated Databases 7 Excel Webservice Website Fat Client Jobs DB1

    DB2 Spring: Audit Slide by @arghrich Replication Dependency
  7. Slide by @arghrich 8 Icons by Franziska Haaf binarykitten.de Doesn‘t

    Compile BL & DB Everywhere Interesting Deployment 15 years old No Tests Every Duplication Spring: Audit
  8. COTS „Apocalypto“ In-house „Fenix“ Slide by @arghrich 9 Spring: Audit

    Icons by Franziska Haaf binarykitten.de Buy I‘ll
  9. Slide by @arghrich 10 Spring: Audit Icons by Franziska Haaf

    binarykitten.de In-house „Fenix“ Please Fix
  10. The True Cost of Rewrites Features Releases over time Catch

    Up Missing Features Sub-Par Parity Enhancements Planned Rewrite Actual Features Undiscovered Scope Old App Adoption Slide Adaption by @arghrich Original Article by Doug Bradbury The True Cost of Rewrites
  11. We started a significant change in team and culture 20

    Early Summer: Spirit Slide by @arghrich
  12. 21 Henrik Kniberg @henrikkniberg Culture is the sum of everyone's

    attitude and actions, so model the behaviour you want to see! Observe behavior: • Who does the management promote? • Is there fear of failure? • What is the delivery „process“? • Is there some kind of learning exchange? • Is quality appreciated or story points? Early Summer: Spirit Slide by @arghrich
  13. 4 Laws of behavior change Make it … Obvious •

    Define Principles together • Shared purpose Attractive • Reduce (time) pressure • (Refactoring) score Slide by @arghrich 22 Atomic Habits Early Summer: Spirit Easy • Tech-coach • Reduce friction Satisfying • Pair up • Never break the chain (twice)
  14. We built a new Co-located Cross-functional Team with shared 23

    DEV TEST OPS Key Accounter Early Summer: Spirit & principles purpose
  15. Our Legacy CD Principles • Build quality in • Focus

    on current tree, not on the forest • Tackle fear head on • Focus on reducing batch size • Low-Risk releases are incremental • Decouple Deployment and Release • Computers perform repetitive tasks, people solve problems Slide Adaption by @arghrich Four Principles of Low-Risk Software Releases by Jez Humble CD Principles by Jez Humble Early Summer: Spirit 24 The Why
  16. Local Env Source Repository Artifact Repository Commit Stage Acceptance Manual

    Test Env … Staging Env Prod Env Self-Service Automatic Our CD Pipe Feedback (Tag Artifact) Slide Adaption by @arghrich Dave Farley - Optimising Continuous Delivery
  17. Our Branching Strategy • Don‘t Branch • Don‘t Branch •

    Don‘t Branch • Never Branch to Refactor •Branches indicate Ci =Continuous Isolation Slide by @arghrich 30 Strategy by Dave Farley Summer: Process
  18. We versioned and tested Everything • Code • Config •

    Deployment scripts • Infrastructure scripts • Docs • … 31 Slide by @arghrich Summer: Process
  19. They used File Version Anti-Practice 32 Slide by @arghrich File

    tree from actual project Summer: Process
  20. Visualize metrics with buildings Slide by @arghrich 38 AbstractService.java Lines

    of code Complexity number of authors Autumn: Product Icons by Fontawesome Very Shameless
  21. Establish baseline with tools Then track progress • CodeCharta https://github.com/

    MaibornWolff/codecharta • SonarQube https://www.sonarqube.org/ • Linter • Profiler • CodeMaat https://github.com/ adamtornhill/code-maat • Structure101 http://structure101.com/ Slide by @arghrich 40 Autumn: Product Baseline
  22. Golden Master Baseline 41 vX - redesigned v1 - legacy

    Input for UI/API/DB Output Output Continuous Compare Slide by @arghrich Autumn: Product Baseline
  23. Conquer the fear • Write categorization/function Tests • Safely verify

    it can handle production • Divide code into stable and unstable Slide by @arghrich 44 Autumn: Conquer the Fear
  24. Slide by @arghrich 46 Photo by Krizjohn Rosales from Pexels

    Autumn: Conquer the Fear Isolate Stable from Unstable Working Effectively with Legacy Code • Direct Read/Write à Facades • Classes à Interfaces + Dependency Injection • Fire&Forget calls à Events • Obsolete/duplicate tables à Table Views
  25. Slide by @arghrich 47 Photo by Alex Fu from Pexels

    Autumn: Conquer the Fear Sprout quality code Working Effectively with Legacy Code Table View Interface Facade Event
  26. Use Strangler Application for bigger features 48 Slide by @arghrich

    Autumn: Conquer the Fear Picture by Paul Hammant
  27. vX - redesigned Dark Launch to gain confidence Slide by

    @arghrich 50 Icons by Fontawesome Load Balancer v1 - legacy Continuous Compare Same number of: • 200? • 404? • 5xx? • … Winter: Launch confidence
  28. Remove any obsolete code in Tech || Business 51 Slide

    by @arghrich Winter: Remove waste Don‘t fix what doesn‘t need fixing Slide by @arghrich Photo by Stas Knop from Pexels
  29. Finding obsolete code • Add Analytics / Activity-Log for your

    dark pages • User surveys • IDE suggestions • grep FROM, JOIN, INSERT, UPDATE, DELETE and check if that code is reachable • Legacy Toggles • Forget to deploy parts of your application 52 Slide by @arghrich Winter: Remove waste
  30. Legacy Delivery 54 Slide by @arghrich Next Spring: Summary Photo

    by pixabay from Pexels Before • 2 Weeks for a feature • 5 months until it works After • 4 Weeks for a feature • Then it works
  31. Was it worth it? • Users had capitulated à then

    we started fixing bugs • More Stable then before • We couldn‘t have reimplemented this from scratch • à None can describe what „same as before“ means • à Even if, it would have taken more than a year • Seamless migration from COTS to in-house à min. feature freeze 55 Slide by @arghrich Next Spring: Summary
  32. Hell Yes We had • Autonomy • Mastery • Purpose

    57 Slide by @arghrich Next Spring: Summary Model from „Drive“ by Daniel Pink
  33. 58 Next Spring: Summary Slide by @arghrich (Team) Spirit •

    Conquer the fear • CD Principles • Build quality in • Shared purpose Product • Isolate stable from unstable • Golden Master • Strangler • Feathers Process • CD Pipeline • Infrastructure Automation
  34. Ken Mugrage @kmugrage It‘s not CD if you can‘t deploy

    to production right now 59 Slide by @arghrich Next Spring: Summary
  35. Done Questions, Remarks, Sympathy? J 30 Mio 410 Employees 9th

    year 23 Nations 300 Weeks Munich, Augsburg, Berlin, Frankfurt
  36. Learn Domain with Reverse Object Mother • Start application with

    empty database and without domain model • Click through a UseCase • Analyse exceptions and errors „App needs at least an object A with this field“ • Expand Domain Modell with your finding • Create required state in the DB with your model • Document finding as categorization test • Repeat 61 Slide by @arghrich Autumn: Rake the forest