Upgrade to Pro — share decks privately, control downloads, hide ads and more …

CD for Legacy Code

Richard
October 16, 2019

CD for Legacy Code

This is based on a true story.

My day job is software archeology. I find joy in recovering and analyzing code bones and culture as well as making the skeleton walk again. A short time ago, however, I was confronted with the most horrible code base I have ever seen. This talk is about how we managed to save it and achieve bi-weekly deployments with a high level of confidence.

Five million lines of code in multiple languages (Classic ASP, .NET, VBScript, VBA, JavaScript, T-SQL, PL-SQL) in one monolith. The business logic stretched from the UI (WebForms, Scripting, SQL Queries) down to the database (Stored Procedures), there was no test coverage and an enormous amount of hidden coupling. A version control system was not used, we had no test environment, deployments required developers to copy their local compilation to production and multiple customer installations are supported by uncommenting and commenting code. Together we will explore what to do.

https://www.the-architecture-gathering.de/programm/programm-details/1394/cd-for-legacy-code/

Richard

October 16, 2019
Tweet

More Decks by Richard

Other Decks in Technology

Transcript

  1. Continuous Delivery
    for Legacy Code
    Slides by Richard Gross @arghrich
    16.10.19
    Audit
    Software
    Archeology
    Develop
    Many Icons by Franziska Haaf binarykitten.de

    View full-size slide

  2. Dylan Beattie
    @dylanbeattie
    Legacy Code is
    code that’s too
    scary to update
    and too profitable
    to delete.
    Michael Feathers
    @mfeathers
    Legacy
    Code is code
    without tests.
    Slide by @arghrich
    2

    View full-size slide

  3. Legacy Code is
    code that’s too
    scary to update
    and too profitable
    to delete.
    Continuous
    Delivery
    If somebody
    thinks of a good
    idea, how do we
    [make it feasable
    to] deliver it to
    users as quickly
    as possibly?
    3
    Dylan Beattie
    @dylanbeattie
    Slide by @arghrich

    View full-size slide

  4. Slide by @arghrich
    4
    Photo by Anne-sophie Parent from Pexels
    Spring

    View full-size slide

  5. Slide by @arghrich
    5 Spring: Audit
    Icons by Franziska Haaf binarykitten.de
    COTS
    „Apocalypto“
    Please Audit

    View full-size slide

  6. 4M LoC Delivery
    6
    Excel
    Webservice
    Website
    Fat Client
    Jobs
    Database
    Spring: Audit Slide by @arghrich
    T-SQL
    JavaScript
    VBA
    C#
    Asp.NET
    + C#
    Classic Asp
    + VBScript
    WinForms
    + C#
    MIA
    No Code
    delivered
    5M 4M LoC Delivery

    View full-size slide

  7. Replicated Databases
    7
    Excel
    Webservice
    Website
    Fat Client
    Jobs
    DB1 DB2
    Spring: Audit Slide by @arghrich
    Replication Dependency

    View full-size slide

  8. Slide by @arghrich
    8
    Icons by Franziska Haaf binarykitten.de
    Doesn‘t
    Compile
    BL & DB
    Everywhere
    Interesting
    Deployment
    15 years
    old
    No Tests
    Every
    Duplication
    Spring: Audit

    View full-size slide

  9. COTS
    „Apocalypto“
    In-house
    „Fenix“
    Slide by @arghrich
    9 Spring: Audit
    Icons by Franziska Haaf binarykitten.de
    Buy
    I‘ll

    View full-size slide

  10. Slide by @arghrich
    10 Spring: Audit
    Icons by Franziska Haaf binarykitten.de
    In-house
    „Fenix“
    Please Fix

    View full-size slide

  11. Slide by @arghrich
    12
    We could just …

    View full-size slide

  12. Slide by @arghrich
    13
    Photo by Levi Damasceno from Pexels

    View full-size slide

  13. The True Cost of Rewrites
    Features
    Releases over time
    Catch Up
    Missing
    Features
    Sub-Par Parity
    Enhancements
    Planned
    Rewrite
    Actual Features
    Undiscovered Scope
    Old App
    Adoption
    Slide Adaption by @arghrich
    Original Article by Doug Bradbury The True Cost of Rewrites

    View full-size slide

  14. Slide by @arghrich
    15 Early Summer
    Photo by Pixabay

    View full-size slide

  15. Slide by @arghrich
    16
    How did they get here?
    Early Summer

    View full-size slide

  16. GIGO
    17
    Ideas
    Product
    Process
    (Team)
    Spirit
    Early Summer Slide by @arghrich
    Predicts

    View full-size slide

  17. GIGO
    18
    Ideas
    Product
    Process
    (Team)
    Spirit
    Early Summer Slide by @arghrich
    Predicts
    Fix

    View full-size slide

  18. Slide by @arghrich
    19
    How did we not get here?
    Early Summer

    View full-size slide

  19. We started a
    significant
    change
    in team and culture
    20 Early Summer: Spirit Slide by @arghrich

    View full-size slide

  20. 21
    Henrik Kniberg
    @henrikkniberg
    Culture is the sum of everyone's
    attitude and actions, so model the
    behaviour you want to see!
    Observe behavior:
    • Who does the management promote?
    • Is there fear of failure?
    • What is the delivery „process“?
    • Is there some kind of learning exchange?
    • Is quality appreciated or story points?
    Early Summer: Spirit Slide by @arghrich

    View full-size slide

  21. 4 Laws of behavior change
    Make it …
    Obvious
    • Define Principles
    together
    • Shared purpose
    Attractive
    • Reduce (time)
    pressure
    • (Refactoring) score
    Slide by @arghrich
    22
    Atomic Habits
    Early Summer: Spirit
    Easy
    • Tech-coach
    • Reduce friction
    Satisfying
    • Pair up
    • Never break the chain
    (twice)

    View full-size slide

  22. We built a new Co-located
    Cross-functional Team
    with
    shared
    23
    DEV TEST OPS
    Key Accounter
    Early Summer: Spirit
    & principles
    purpose

    View full-size slide

  23. Our Legacy CD Principles
    • Build quality in
    • Focus on current tree, not on the forest
    • Tackle fear head on
    • Focus on reducing batch size
    • Low-Risk releases are incremental
    • Decouple Deployment and Release
    • Computers perform repetitive tasks, people solve
    problems
    Slide Adaption by @arghrich
    Four Principles of Low-Risk Software Releases by Jez Humble
    CD Principles by Jez Humble
    Early Summer: Spirit
    24
    The Why

    View full-size slide

  24. Slide by @arghrich
    25 Early Summer: Spirit
    Principles provide adaptability

    View full-size slide

  25. Slide by @arghrich
    26 Summer
    Icons by Franziska Haaf binarykitten.de
    In-house
    „Fenix“
    Please Cloud

    View full-size slide

  26. Fix
    28
    Product
    Process
    (Team)
    Spirit
    Summer Slide by @arghrich
    Predicts

    View full-size slide

  27. Local
    Env Source
    Repository
    Artifact
    Repository
    Commit
    Stage
    Acceptance
    Manual
    Test
    Env

    Staging
    Env
    Prod
    Env
    Self-Service
    Automatic
    Our CD Pipe
    Feedback (Tag Artifact) Slide Adaption by @arghrich
    Dave Farley - Optimising Continuous Delivery

    View full-size slide

  28. Our Branching Strategy
    • Don‘t Branch
    • Don‘t Branch
    • Don‘t Branch
    • Never Branch to Refactor
    •Branches indicate
    Ci
    =Continuous Isolation
    Slide by @arghrich
    30
    Strategy by Dave Farley
    Summer: Process

    View full-size slide

  29. We versioned and tested
    Everything
    • Code
    • Config
    • Deployment scripts
    • Infrastructure scripts
    • Docs
    • …
    31 Slide by @arghrich
    Summer: Process

    View full-size slide

  30. They used File Version
    Anti-Practice
    32 Slide by @arghrich
    File tree from actual project
    Summer: Process

    View full-size slide

  31. Slide by @arghrich
    33 Autumn
    Photo by Daniel Frank from Pexels

    View full-size slide

  32. Fix
    34
    Product
    Process
    (Team)
    Spirit
    Autumn Slide by @arghrich

    View full-size slide

  33. Slide by @arghrich
    35
    Where do we start?
    Autumn

    View full-size slide

  34. Slide by @arghrich
    Art by Zaufishan
    36 Autumn: Product

    View full-size slide

  35. github.com/MaibornWolff/codecharta
    Slide by @arghrich
    37
    Shameless
    Autumn: Product

    View full-size slide

  36. Visualize metrics with buildings
    Slide by @arghrich
    38
    AbstractService.java
    Lines of code
    Complexity
    number of authors
    Autumn: Product
    Icons by Fontawesome
    Very Shameless

    View full-size slide

  37. Find potential hotspots
    Slide by @arghrich
    39
    Still Shameless
    Autumn: Product

    View full-size slide

  38. Establish baseline with tools
    Then track progress
    • CodeCharta
    https://github.com/
    MaibornWolff/codecharta
    • SonarQube
    https://www.sonarqube.org/
    • Linter
    • Profiler
    • CodeMaat
    https://github.com/
    adamtornhill/code-maat
    • Structure101
    http://structure101.com/
    Slide by @arghrich
    40 Autumn: Product Baseline

    View full-size slide

  39. Golden Master Baseline
    41
    vX - redesigned
    v1 - legacy
    Input for UI/API/DB
    Output
    Output
    Continuous
    Compare
    Slide by @arghrich
    Autumn: Product Baseline

    View full-size slide

  40. Slide by @arghrich
    42 Autumn
    Icons by Franziska Haaf binarykitten.de
    In-house
    „Fenix“
    Add Feature

    View full-size slide

  41. Slide by @arghrich
    43
    Photo by Bartek Wojtas from Pexels
    Autumn
    Fuuuu

    View full-size slide

  42. Conquer the fear
    • Write categorization/function Tests
    • Safely verify it can handle production
    • Divide code into stable and unstable
    Slide by @arghrich
    44 Autumn: Conquer the Fear

    View full-size slide

  43. Slide by @arghrich
    45
    Photo by Emmet from Pexels
    Autumn: Conquer the Fear

    View full-size slide

  44. Slide by @arghrich
    46
    Photo by Krizjohn Rosales from Pexels
    Autumn: Conquer the Fear
    Isolate
    Stable from Unstable
    Working
    Effectively with
    Legacy Code
    • Direct Read/Write à Facades
    • Classes à Interfaces + Dependency Injection
    • Fire&Forget calls à Events
    • Obsolete/duplicate tables à Table Views

    View full-size slide

  45. Slide by @arghrich
    47
    Photo by Alex Fu from Pexels
    Autumn: Conquer the Fear
    Sprout quality code
    Working
    Effectively with
    Legacy Code
    Table View
    Interface Facade Event

    View full-size slide

  46. Use Strangler Application
    for bigger features
    48 Slide by @arghrich
    Autumn: Conquer the Fear
    Picture by Paul Hammant

    View full-size slide

  47. Slide by @arghrich
    49 Winter
    Photo by Riccardo Bresciani from Pexels

    View full-size slide

  48. vX - redesigned
    Dark Launch to gain confidence
    Slide by @arghrich
    50
    Icons by Fontawesome
    Load Balancer
    v1 - legacy
    Continuous
    Compare
    Same number of:
    • 200?
    • 404?
    • 5xx?
    • …
    Winter: Launch confidence

    View full-size slide

  49. Remove
    any
    obsolete code
    in
    Tech || Business
    51 Slide by @arghrich
    Winter: Remove waste
    Don‘t fix what doesn‘t need fixing
    Slide by @arghrich
    Photo by Stas Knop from Pexels

    View full-size slide

  50. Finding obsolete code
    • Add Analytics / Activity-Log for your dark pages
    • User surveys
    • IDE suggestions
    • grep FROM, JOIN, INSERT, UPDATE, DELETE
    and check if that code is reachable
    • Legacy Toggles
    • Forget to deploy parts of your application
    52 Slide by @arghrich
    Winter: Remove waste

    View full-size slide

  51. Slide by @arghrich
    53 Next Spring: Summary
    Photo by Irina Iriser from Pexels

    View full-size slide

  52. Legacy Delivery
    54 Slide by @arghrich
    Next Spring: Summary
    Photo by pixabay from Pexels
    Before
    • 2 Weeks for a feature
    • 5 months until it works
    After
    • 4 Weeks for a feature
    • Then it works

    View full-size slide

  53. Was it worth it?
    • Users had capitulated
    à then we started fixing bugs
    • More Stable then before
    • We couldn‘t have reimplemented this from scratch
    • à None can describe what „same as before“ means
    • à Even if, it would have taken more than a year
    • Seamless migration from COTS to in-house
    à min. feature freeze
    55 Slide by @arghrich
    Next Spring: Summary

    View full-size slide

  54. Slide by @arghrich
    56
    Would I do it again?
    Next Spring: Summary

    View full-size slide

  55. Hell Yes
    We had
    • Autonomy
    • Mastery
    • Purpose
    57 Slide by @arghrich
    Next Spring: Summary
    Model from „Drive“ by Daniel Pink

    View full-size slide

  56. 58 Next Spring: Summary Slide by @arghrich
    (Team)
    Spirit
    • Conquer the
    fear
    • CD Principles
    • Build quality in
    • Shared
    purpose
    Product
    • Isolate stable
    from unstable
    • Golden Master
    • Strangler
    • Feathers
    Process
    • CD Pipeline
    • Infrastructure
    Automation

    View full-size slide

  57. Ken Mugrage
    @kmugrage
    It‘s not CD if you can‘t deploy to
    production right now
    59 Slide by @arghrich
    Next Spring: Summary

    View full-size slide

  58. Done
    Questions, Remarks, Sympathy? J
    30 Mio
    410
    Employees 9th year
    23
    Nations
    300
    Weeks
    Munich, Augsburg,
    Berlin, Frankfurt

    View full-size slide

  59. Learn Domain with
    Reverse Object Mother
    • Start application with empty database and
    without domain model
    • Click through a UseCase
    • Analyse exceptions and errors
    „App needs at least an object A with this field“
    • Expand Domain Modell with your finding
    • Create required state in the DB with your model
    • Document finding as categorization test
    • Repeat
    61 Slide by @arghrich
    Autumn: Rake the forest

    View full-size slide

  60. Reading
    • https://trello.com/b/Fdd876S8/continuous-
    delivery-checklist-template
    • I did not think I would show this slide
    62 Slide by @arghrich

    View full-size slide