Technical Debt: Maintaining Software Long Term

Software often becomes less maintainable the longer development teams work on it. The metaphor "technical debt" has been established for this. But technical debt can happen "just like that" and it doesn't always make sense to eliminate it. That's what this talk is about - and about the foundations of the metaphor, how it helps when communicating with managers, why the metaphor is actually not very well chosen, and of course how to deal with technical debt in a sensible way.

Transcript

1. Technical Debt: Maintaining Systems in the Long Run Eberhard Wolff Head of Architecture https://swaglab.rocks/ https://ewolff.com/
2. Why This Talk? •Handling technical debt is important to succeed in software development. •Many misunderstandings •Goal: Show decisions that can be made about technical debt.
3. External Quality •What users see •Performance, availability •Important for architecture! •Quality scenarios External quality
4. Internal Quality •What developers see •Code quality •Code structure •Tests •Might be related to external quality Internal quality External quality
5. What is Technical Debt? •Not just developers need to reason about internal quality. •Technical debt: metaphor to explain internal quality …and possible decisions about it.
6. Technical Debt Cost of additional rework caused by choosing an easy (limited) solution now instead of using a better approach that would take longer. https://en.wikipedia.org/wiki/Technical_debt Conscious decision Debt can be avoided
7. Technical Debt As with monetary debt, if technical debt is not repaid, it can accumulate "interest", making it harder to implement changes. https://en.wikipedia.org/wiki/Technical_debt It can be fully repaid.
8. Technical Debt Technical Debt Productivity
9. The Problem with Metaphors •Metaphors explain something in different terms. •“Technical debt is like monetary debt for code.” •But the original point sometimes gets lost in translation.
10. Origins of the Metaphor
11. Origins of the Metaphor •Ward Cunningham •Inventor of the Wiki •Worked on Wy Cash •Technical debt: metaphor for finance experts.
12. What Ward Cunningham Meant •Team learns new facts about the domain. •Facts should be represented properly. •So: refactor •If you don’t refactor, you’ll be in debt . https://wiki.c2.com/?WardExplainsDebtMetapWhat https://www.youtube.com/watch?v=pqeJFYwnkjE
13. What Ward Cunningham Meant •Technical debt = lack of understanding. •No conscious decision •Can’t be avoided •Technical debt is meant to be repaid immediately. •Can debt be fully repaid?
14. Fowler’s Technical Debt Quadrants
15. Technical Debt Quadrants Reckless Prudent Deliberate “We don’t have time for design.” “We must ship now and deal with the consequences (later)” Inadvertent “What’s layering?” “Now we know how we should have done it!”
16. Quadrants & Decisions •Inadvertent: Technical debt without a conscious decision. •Repaying is not so important for Fowler. •Reckless: Probably the #1 reason for debt.
17. Quadrants & Avoiding Debt •Deliberate decisions “Ship now – deal with consequences later” •Might make sense •No design •Stupid (and probably obviously stupid)
18. Avoiding Debt •Sometimes, debt is OK. •Avoid recklessness! •Do sophisticated technical approaches help? •Not sure •Technical debt: a cultural problem?
19. The Problem with Technical Debt
20. Technical Debt Just Happens •Often no conscious decision •Technology update •Now: simpler way to do something. •Developer leaves company. •Nobody understand the code.
21. Technical Debt Just Happens If technical debt is not caused by conscious decision … how can it help to make conscious decisions?
22. What does 78d mean? How is it relevant? Is there a debt-free system? Should debt be reduced?
23. No Debt-free Systems •There are different opinions about “ideal” i.e. debt- free. •My experience: In every system, there are numerous potential improvements.
24. No Debt-free Systems •I have yet to see a system that couldn’t be optimized. •The question is usually what to fix first. •A complete inventory of technical debt doesn’t help.
25. Technical Debt is Meaningless •Technical debt = changing code is hard. •Irrelevant for code that is not changed
26. Technical Debt is Meaningless •Diagrams and code might be ugly. •Architecture should be led by business decision… not aesthetics.
27. Technical Debt & Standstill? Technical Productivity
28. Technical Debt & Standstill •Some debt is fine •Too much technical debt: Standstill •What now? •Rewrite? •Massive, hard to solve problem
29. Technical Debt & Standstill •Too much technical debt: bankruptcy •How much is too much? •Debt invisible •A dangerous game •Does the metaphor communicate a possible sudden bankruptcy?
30. Conclusion: Problems •Just happens •No debt-free systems •Meaningless - unless code is changed. •Does not facilitate decisions. •Risk of a standstill
31. All Systems have Technical Debt
32. Technical Debt Does Not Facilitate Decisions!
33. Quite a Few Systems Have HUGE Technical Debt 😱
34. Different Way to Reason about Quality?
35. Quality Investments
36. Quality Investments •Investment should have a pay-off •I.e. repay technical debt where is matters.
37. Quality Investments •Financial metaphor •Managers should be used to think and decide about investments. •More than a metaphor: Investing development effort = money
38. Quality Investments •Potentially unlimited •No optimum
39. Quality Investments: Pay Off •Should have a pay-off •i.e. higher productivity •No investments in code that does not matter! •No investments that are too large to pay off!
40. Quality Investments: Pay Off •No precise estimates •But rather rough ballpark numbers •Helps to justify changes •Helps to evaluate ideas
41. Applications
42. Investments & Domains Business Differentiation Model Complexity Core Domain Supporting Subdomain Generic Subdomain Buy? Cost Speed
43. Investment: Core Domain •How many more stories can we implement? •What would the business value be? •Business value might be used to justify stories. •How much do we save on maintenance?
44. Investment: Core Domain •Stories were worth €10 Mio in business value. •Assume we gain 5% productivity. •We could spend €0.5 Mio on those improvements •Management compatible reasoning •Successful in the real world
45. Investment: Supporting Subdomain •How much do we save on maintenance? •Do we gain business value? •E.g. by better support for core domains? •Investment might not be justifiable.
46. Some Types of Investments
47. Refactoring
48. Refactoring •Restructure code without changing its functionality. •Extract method •Rename method / class •…
49. Refactoring •Not one large change •But ongoing activity Refactor-Test Implement-Test Refactor-Test •Done and decided by developers
50. Refactoring Investment Productivity
51. Refactoring •Can be introduced by code retreats, pair programming, mob programming … •Done and decided by developers •Really?
52. Refactoring •Refactoring, tests etc. are a best practice. •I don’t want to explain it to non-tech people. •I don’t want to justify it to non-tech people. •Business people should not have to justify business decisions to me. •Why had Ward Cunningham to explain it?
53. Refactoring •Probably not done if stress level is high. •So others can still influence it. •Can lead to bigger changes •Refactor towards a bigger goal… each time a part of the system is changed. •Larger changes: different approach?
54. Rewrite / Rearchitect
55. Rewrite / Rearchitect •Change to a new architecture •I.e. Domain-driven Design •Often starting point for a consulting gig. •Benefits are obviously.
56. Rewrite / Rearchitect •Goal is clear: New, great architecture! •But how to get there? •Sometimes completely unclear
57. Rewrite / Rearchitect •Estimate effort to redo architecture •Estimate productivity increase •Estimates are common in software engineering.
58. Rewrite / Rearchitect Investment Productivity
59. Rewrite / Rearchitect •Huge investment •Can take several years •Team will keep current productivity for quite some time. •Project might not implement any features. •Risk
60. Rewrite / Rearchitect •Investment only make sense if code is changed. •Is all of the system relevant enough?
61. We want to change the architecture entirely! Let’s estimate the time Can you really invest 7 year for a big bang rewrite? Estimation based on gut feeling About 7 years
62. Rewrite / Rearchitect •Smart decision? •Needs management buy-in •Estimate total effort roughly •Probably stay away from it •Architecture improvement has to take current status into account!
63. Incremental Investments
64. Incremental Investments •Do several bigger investments •Steps towards a new architecture? •But: Each must pay-off •E.g. modernize a part that is changed frequently that is changed anyways
65. Incremental Investments Investment Productivity
66. Incremental Investments •Still big investment •But: pay-off rather short term •Improvements can be stopped •Risk smaller
67. Incremental Investments •Hard to come up with individual steps that each pay off. •No hard science, just estimates and guidelines. •Is not improving the situation even an option?
68. Remember: Standstill! Technical Debt Productivity
69. Who Decides about Investments?
70. Good Craftsmanship •Unit tests •Refactoring •Developers decide about it. •Do managers even notice these details?
71. Team Budget •Set aside time to work on quality •Management might not care about sustainable pace. •Then the budget assures that something happens. •But: might not support business goals •i.e. investment in the wrong places.
72. Management Decision •Can prioritize business goals •Ship now vs. invest in sustainable pace •But: Might have a bias to SHIP NOW! •Might be the only way to do larger investments.
73. Conclusion
74. Conclusion Technical Debt •Metaphor for internal quality •Debt can lead to a standstill. •Avoid recklessness
75. Conclusion Quality Investments •Alternative •Supports decision process •Refactoring •Rewrite / Rearchitect •Incremental Investments
76. Links EN •Article about Quality Investments (with Felix Müller) https://www.infoq.com/articles/no-more-technical-debt •Podcast Technical Debt with Sven Johann https://www.se- radio.net/2015/04/episode-224-sven-johann-and- eberhard-wolff-on-technical-debt/ •Managing Technical Debt with Sven Johann https://www.infoq.com/articles/managing-technical- debt/
77. Links DE • Podcast https://www.heise.de/developer/artikel/Episode-73-Technische- Schulden-4771190.html • Blog „Technische Schulden entstehen einfach so“ https://www.heise.de/developer/artikel/Technische-Schulden-entstehen- einfach-so-3969279.html • Artikel über Qualitätsinvestitonen mit Felix Müller – DE https://www.sigs- datacom.de/fachzeitschriften/objektspektrum/archiv/artikelansicht/artikel- titel/qualitaetsinvestitionen-statt-technischer-schuldenwarum-wir-eine- neue-metapher-benoetigen.html • Artikel Umgang mit technischen Schulden https://jaxenter.de/der-umgang- mit-technischen-schulden-2548
78. DE Software Architektur im Stream •Technische Schulden https://software- architektur.tv/2021/02/05/folge37.html •André Neubauer - CTO = Chief Technical Debt Owner? https://software-architektur.tv/2021/01/15/folge35.html •Qualitätsszenarien https://software- architektur.tv/2021/07/16/folge67.html •Patterns zu u.a. Refactoring https://software- architektur.tv/2020/12/18/folge033.html
79. 60-Minuten-Consulting •Online •Inkl. Bericht •99€ https://swaglab.rocks/60-min-consulting/