Technical Debt: Do Not Underestimate The Danger

This is the slides of my talk delivered in several meetups about the "technical debt" concept.

Transcript

1. Do not underestimate the danger technical Lemİ Orhan ERGİN Principal Software Engineer @ Sony @lemiorhan agilistanbul.com debt @lemiorhan
2. Lemİ Orhan Ergİn Principal Software Engineer at Sony has worked in Tüsside, BYM, GittiGidiyor/eBay and Sony as lead developer, team leader, technical coordinator and scrum master got CSM certificate from Jim Coplien year as Scrum Master sprints in 4 years as team member and scrum master experienced in agile transformation and building agile culture in teams & organizations 2001 2013 2009 1 56 agile CSM, PSM1
4. financial debt and the metaphor
5. expensive DEBTsin every second of our life We incur Living is
6. taxesgas fuel water phone electricity rent credit cards cable tv tuition TOLLS mortgage school payments transportation Holidays INTERNET 3g/4G municipal services shopping communication food health entombment Parking
7. Financial debt is not always a bad thing. It lets us buy what we want or what we need and keep us live at some level.
8. Financial debt is dangerous if the incurred interest and the debt itself are not payed
9. Financial debt is a silent killer
10. Financial debt hides problems and leads to other problems
11. technical debt is a metaphor developed by Ward Cunningham in 1992 to communicate problems due to“developing not in the right way” with non-technical stakeholders. met·a·phor noun \ˈme-tə-ˌfȯr also -fər\ a word or phrase for one thing that is used to refer to another thing in order to show or suggest that they are similar
12. deficit Enron financing paying the principal debt Interest on the debt bankruptcy borrowing money Intertest on the loan Inflation financial burden very similar technical debt is to financial debt
14. the cost
15. technical debt costs money, time and effort to stakeholders, developers, companies, even to economies
16. most companies have to spend 80% of their software development budget maintaining code Aaron Erickson, Informit.com “ ”
17. U.S. Air Force pulls plug on ERP project after blowing through $1 billion chris kanaracus, CIO.com “ ”
18. we don’t know how much technical debt is really costing us Jim bird, Agile.dzone.com “ ”
19. Technical debt is like smoking addiction. Once you start hacking your code, the code asks for more. you never know what that will cost in the future. Lemİ Orhan ERGİN, Agilistanbul.com “ ”
20. High amount of Technical Debt is #1 impediment to teams being agile Dan Rawsthorne “ ”
21. Sufficient amount of messy code may bring whole Engineering department to a stand-still Sven Johann & Eberhard Wolff, Infoq “ ”
23. the description
24. 2 ways of doing things
25. clean and smart way takes longer to implement but make changes easier in the future
26. quick and dirty way get your features sooner, but make the future changes very hard
27. why should the sponsors of a project accept higher costs ? Question:
28. why should the sponsors spend money on features that don’t deliver business value ? Question:
29. Living with technical debt might work perfectly for customers if it delivers the desired business value Answer:
30. but this will lead an uncontrollable codebase, extremely specialized developers and to an inflexible product
31. Shipping firsttime code is like going into debt
32. a little debt speeds development as it is paid back with a rewrite
33. It’s a question of Pay me now or Pay me later
35. good or bad
36. ... if you have strategic design decisions. It allow rapid delivery to elicit quick feedback and correct design. However, clean code is required to pay back debt therefore the code sould be refactored ıncurring technical debt is a good idea
37. ... if principal amount and interest is smaller than the yield of investment. Sometimes, you don't have to pay back the debt, it is pointless if the code won't be touched again ıncurring technical debt is a good idea
38. ... because having messy code and cutting quality slows you down in reality. And you must generate more mess to keep up. Do you ask permission to do your job correctly???? ıncurring technical debt is a bad idea
40. extend the metaphor
41. similarities with financial debt Every minute spent on not-quite-write code counts as interest on the debt Interest Image by Martin Schoeller from “Identical: Portraits of Twins” book
42. similarities with financial debt Refactoring is like paying off the principal debt paying the principal debt Image by Martin Schoeller from “Identical: Portraits of Twins” book
43. similarities with financial debt Neglecting the design is like borrowing money borrowing money Image by Martin Schoeller from “Identical: Portraits of Twins” book
44. similarities with financial debt Bankruptcy refers to a situation of overwhelming debt interest, whereby progress is halted and a complete rewrite is necessary bankruptcy Image by Martin Schoeller from “Identical: Portraits of Twins” book
45. similarities with financial debt It occurs when the current level of technology is old enough to lose compatibility with the industry technical Inflation Image by Martin Schoeller from “Identical: Portraits of Twins” book
46. similarities with financial debt It represents technical frauds undermine ROIs Enron Financing Image by Martin Schoeller from “Identical: Portraits of Twins” book
47. similarities with financial debt Throw away prototypes, retired products and complete failures on will-not-be-used components does not have to be repaid amnesty Image by Martin Schoeller from “Identical: Portraits of Twins” book
49. the Symptoms
50. the loss of productivity We cannot measure productivity, that's why we can't really see real the true effect of technical debt and losing motivation and morale
51. Increase in testing If the team does the same tests again and again on every release or testing takes too much time, the debt seems to be in critical level. the death by manual testing
52. Postponed stories in releases Postponed releases It becomes harder to catch the deadlines. An increase on the postponed releases rate is an indicator. being not ready to go
53. Code duplication Code duplication leads to update anomalies, purpose masking and issues on understanding code copy & Paste programming
54. Low code coverage 100% coverage is the asymptotic target. Below 75% indicates serious problems. not knowing how sW behaves
55. Increase in bugs The increase in number of bugs directly shows the decrease of quality and too much support cases
56. heavy stress on appoaching deadlineS The team is under heavy stree at the end of releases or when the deadlines are approaching
57. being scared of changing anything Software is so brittle that developers are scared of changing anything in the code not to break. Adding new features starts to cost more and more.
58. Evil hacks wrong design It’s hard to detect the debt in advance, but any hacks and workarounds could cause issues in the future
59. wrong choice of technology Not the most mature, not the newest, not the simplest... Choosing the cheapest-to-adapt is the key to pay the debt.
60. unreadable code Developers spend 80% of development time for reading code. You cannot build if you cannot understand. hard to understand what happens
61. decreased velocity Decreased team velocity is a good indicator for possible impediments that slows the team down even if nothing has changed
62. Using old libraries Softwares age and cause technical debt by time. Maintenance costs are the main expenditure item. and technologies
63. Reports of Sonar Sonar Reports Softwares age and cause technical debt by time. Maintenance costs are the main expenditure item. with technical debt plugin
64. bad smells The indicator of weaknesses in design that may be slowing down development or increasing the risk of bugs or failures in the future “Only he knows can change this part” “Lets copy & paste this code” “If I touch, everything will break” “Too many TODOs or FIXMEs in the code”
65. 36 classic mistakes Making mistakes is inevitable. Only experienced software developers realize when they're making mistakes. outlined in McConnell's Rapid Development http://www.construx.com/10x_Software_Development/Classic_Mistakes_Updated
67. the types by scope
68. unavoidable debt Usually unpredictable and unpreventable and the team has no fault on that Due to legal issues, we have to rewrite some of the components “ ”
69. Strategic debt Long term debt, usually incurred proactively I want to be the first in the market “ ”
70. tactical debt It’s different that strategic by the reactive manner for the short term We don't have time to implement in the right way, just a hack. We'll fix later. “ ”
71. Incremental debt Hundreds or thousands of small shortcuts, like credit card debt We have to do quick hacks and dirty solutions to catch the deadline “ ”
72. INADVERTENT (naive) debt It occurs due to irresponsible behavior or immature practices on the part of the people involved We have to build the software product with inexperienced newbies “ ”
74. the types by content
75. Design & architectural Debt Shortcuts and shortcommings Long and detailed upfront designs Sub-optimal solutions
76. code quality Debt Short time between failures Severe defect count Every hack, work around, bad piece of code builds Unnecessary code duplication and complexity
77. testing Debt Missing automated tests Too much time spending for regression testing
78. Knowledge distribution and documentation debt Only few people knows the system When all key people left the organization, the debt becomes extremely high
79. Environmental Debt Problems in development related processes Issues in hardware, infrastructure, supporting apps Having too many manual operational tasks
80. Monetary cost Developer’s time is expensive No developer enjoys to work on brittle and complicated code That leads turnovers and it is one of the real economic costs of technical debt
82. the stakeholders
83. customers Annoyed by bugs, missing features, slow lead times and expensive costs
84. helpdesk Additional costs
85. operations team Frequent patches and fixes
86. management Annoyed due to bugs, delays, costs and security issues
87. developers No one wants to develop bad code No one wants to take over bad work of others
89. the strategic design
90. A system can't have the same high level of quality throughout the system strategic design The team can choose which parts will have high quality or kept as low quality Consumer understands quick & dirty solutions lead to debt Understanding Strategic Design leads to better decisions from stakeholders
92. fixing the debt
93. Merciless refactoring
94. fast automation
95. share knowledge knowledge decays fast
96. slow down to go fast
97. clean code principles clean, readable and simple code
98. clear definition of done showing us how to avoid incurring debt
99. key engineering practices Pair Programming Test Driven Development Continuous Integration 10 min Builds Refactoring Automated unit tests etc. these should always directly associated with a requirement
100. test driven development Pre-release defect density decreased 40-90% relative to similar products that do not use TDD Initial development time increased 15-35% after TDD
101. fail fast Peer Review Code Review Don't leave all testing to the last phase
102. Limit work-in-progress
103. fix constantly
104. Monitor your debt Code coverage: Monitor trends, not points Cyclomatic complexity: Number of branches in code Coupling: Interconnectedness of systems
105. clean constantly
106. fix root causes
107. Follow the Boy Scout Rule quality is your responsibility
108. never make intentional mess
109. Never ask permission to do your job correctly
111. The Payment strategies From Frank Buschmann
112. debt repayment Refactor or replace code
113. debt conversion Replace the current solution with a "good but not perfect" solution
114. just pay the interest Live with the code Refactoring is more expensive than the work with bad code End-of-life software or will-be-retired software
116. The Payment Approaches
117. Minimal Iteration Debt Payment Half a day in a week
118. Technical Backlog Debt is made visible and clear for everbody Cost per task is trackable No mixture between technical and feature tasks adv. disadv. 2 backlogs and hard to prioritize Customers may not understand the real benefits of a tech task Very expensive changes must always a business reason
119. buffer tasks for refactoring 10% of the team's time
120. clean-up releases
122. The measurment
123. time investment is actually worth it it is important to decide that
124. even impossible to predict The effect of bad code quality for future requirements is difficult
125. The delta from ideal Errors & warnings Code duplication Code coverage Gut feeling in large apps, the measure is basically useless
126. Story Probing The assumption is that technical debt increases the estimate
127. formulas to guess the debt There are many formulas proposed and used
128. Estimate the cost of a failure and then multiply by the probability it will occur Technical debt is the cost of implementing the required redundancy
130. the conclusion
131. you got the red pill Understanding technical debt will let you, your team, the business and the stakeholders make better decisions
132. Technical Debt http://www.infoq.com/articles/managing-technical-debt http://theagileexecutive.com/category/technical-debt/ http://martinfowler.com/bliki/TechnicalDebt.html http://www.slideshare.net/garyshort/technical-debt-2985889 http://www.slideshare.net/woodyp/technical-debt-7436402 http://www.slideshare.net/jhlittle/technical-debt-6896714 http://www.slideshare.net/RobMyers64/technical-debt-10124606 http://www.slideshare.net/RedgateSoftware/measuring-technical-debt 1 Billion Dolar Fail http://www.cio.com/article/721628/Air_Force_scraps_massive_ERP_project_after_racking_up_1_billion_in_costs Evluation with Sonar http://www.sonarqube.org/evaluate-your-technical-debt-with-sonar/ 36 Classic Mistakes http://www.stevemcconnell.com/rdenum.htm http://www.codinghorror.com/blog/2007/06/escaping-from-gilligans-island.html Calculating the ocost http://www.ontechnicaldebt.com/blog/time-to-start-estimating-technical-debt/ http://www.infoq.com/news/2010/03/monetizing-technical-debt http://blog.acrowire.com/cloud-computing/failing-to-plan-is-planning-to-fail/
133. http://www.flickr.com/photos/guilleavalos/2139208615 http://www.flickr.com/photos/maclufus/5571639487 http://www.flickr.com/photos/grantmac/2282389818 http://www.flickr.com/photos/freefoto/2539768604 http://farm5.staticflickr.com/4043/5129934527_552d08a0e4_o.jpg http://www.flickr.com/photos/ethermoon/4045176015 http://upload.wikimedia.org/wikipedia/commons/2/2a/Dirty_dishes.jpg http://fc07.deviantart.net/fs41/f/2009/030/9/6/The_Good__The_Bad_and_The_Ugly_by_ROMAragorn.jpg http://ngm.nationalgeographic.com/2012/01/twins/portraits/img/01-johanna-eva-gill.jpg http://www.flickr.com/photos/epsos/6749663099 http://darkroom.baltimoresun.com/wp-content/uploads/2012/12/REU-POY-243.jpg http://www.flickr.com/photos/defenceimages/7021362271 http://www.flickr.com/photos/widget8/4121151605 http://www.flickr.com/photos/cafemama/206657382 http://www.flickr.com/photos/staflo/6015670061 http://www.flickr.com/photos/crashmaster/3192341451 http://www.propcboost.com/wp-content/uploads/2013/05/Blue-screen-error.jpg http://www.flickr.com/photos/stuant63/2240432052 http://www.flickr.com/photos/didmyself/7685629372 http://www.flickr.com/photos/untickalock/41536377 http://www.flickr.com/photos/fxtc/8105411995 http://tocea.com/wp-content/uploads/Tocea-Scertify-TechDebt-Overview.jpg http://www.flickr.com/photos/cristiano_betta/2757049946 http://www.flickr.com/photos/mhusiak/3216291202 http://www.flickr.com/photos/eneas/9541686914 http://www.flickr.com/photos/amalthya/84364820 http://www.flickr.com/photos/erik-n/1550380661 http://www.algoafm.co.za/img/uploads/Website/cat%20scared.jpg
134. Lemİ orhan ergİn lemiorhan@agilistanbul.com @lemiorhan @lemiorhan agilistanbul.com @lemiorhan LINKEDIN TWITTER SLIDESHARE BLOG Principal Software Engineer @ Sony Founder & Author @ agilistanbul.com flyingtomoon.com