The Past, Present and Future of Technical Debt

Transcript

1. The Past, Present and Future of Technical Debt Learning from the past to prepare for the future Technical Debt 2018 Gothenburg, May 2018 Eoin Woods Endava @eoinwoodz
2. Background Eoin Woods • Endava’s CTO, based in London • 10 years in product dev - Bull, Sybase, InterTrust • 10 years in capital markets applications - UBS and BGI • Software engineer, then architect, now CTO • Author, editor, speaker, community guy I’ve had a lot of technical debt
3. What is Technical Debt?
4. Defining Technical Debt • “Not quite right code which we postpone making it right” – Ward Cunningham • “The obligation that a software organization incurs when it chooses a design or construction approach that's expedient in the short term but that increases complexity and is more costly in the long term” – Steve McConnell • “Compromises made in the implementation of a piece of software that are likely to limit how it can be used or changed later” – Eoin Woods
5. (Some) Types of Technical Debt Intra-Module Inter-Module Testing User Experience Data Operation Platform + Functional
6. Why Do We Care? • Reduces speed of delivery • Reduces options for change • Reduces morale in the team • Reduces reliability • Increases cost of change • Increases cost of operation REDUCES OUR RETURN ON INVESTMENT
7. TECHNICAL DEBT IS OFTEN WHAT PREVENTS US MEETING A FUTURE NEED IN A TIMELY AND COST EFFECTIVE MANNER
8. Like Death and Taxes
9. Technical Debt Through the Ages
10. 5 Ages of Software Systems Intelligent Connected (2020s) Internet is the System (2010s) Internet Connected (2000s) Distributed Monoliths (1990s) Monolithic (1980s)
11. Monolithic Technical Debt • Procedure tangles • GOTO spaghetti • Data incoherence Advances • Structured Programming • Early metrics • Early analysis tools Structuring of computer programs
12. Distributed Monoliths Technical Debt • Tiering confusion • all in the UI, all in the DB, … • Spaghetti database • Monster data models Advances • Emergence of the metaphor • Refactoring • Commercial tools The rise of client/server and architectural style
13. Internet Connected Technical Debt • Unsuitable platforms • Unsustainable operation • One-man deployment “pipeline” • Quality property hacks Advances • Software architecture • Patterns and platforms for large-scale systems • Open source tools Mass market scaling and explosion in software architecture
14. Internet as the System Technical Debt • External service dependencies • Microservice choreography • Data replication & duplication Advances • API descriptions • Runtime tracing & visualisation • Containers & platforms Everything is a service, architecture dynamic & packaged
15. New Sources of Technical Debt • Architecture hidden in the network (e.g. microservices) • Digital channels => constant functional change • Shift to unstructured and replicated data • Rapidity and self-service of cloud platforms • New types of code (“X as Code”) • Rapid tech stack evolution (e.g. framework churn)
16. Current Technical Debt in Practice • Ubiquitous term • General focus on code, sometimes tests • Data, platform, operational debt often not recognised • Some teams actively manage the debt • Many just talk about it • “Everyone” has a mountain of it • No widely adopted reasoning frameworks • Looming “new debt” problems don’t get much attention
17. On the Future
18. Intelligent Connected
19. Intelligent Connected Technical Debt • Unknown connected devices • Data needed for “intelligence” • ML models Advances (Needed) • Better IoT security & mgmt.? • More on data & algorithms? • Self-explaining AI? Data, algorithms, “intelligence”, emergent architecture
20. To Conclude
21. Conclusions • Our past can point to the future • Monolithic complexity led to structure and metrics • Distributed Monoliths brought a metaphor & refactoring • Internet Connected systems needed architecture & triggered open source tooling • Each era has tried to manage the debt it finds itself with … often with limited impact
22. Conclusions • Internet as the System brings its own problems too: • External dependencies (services) • Microservice dependencies • Data replication & duplication • New types of code and platform • Response is largely project-by-project • containers? runtime tracing? api doc techniques? ... • Technical Debt practice often lags the need by years!
23. Conclusions • What about Intelligent Connected systems? - We’re not great at ”data debt” but these systems depend on it - AI debt still not understood but will become a standard feature - Devices bring lots of debt but we struggle with today’s challenge • We’ve got some work to do … - We can see what is coming - This time could research and practice predict the debt and think about managing it … before we have a lot of it?
24. THANK YOU Eoin Woods Endava eoin.woods@endava.com @eoinwoodz
25. Picture Credits • Erik Pitti – S360, slide 11 (Flickr) • Tech Republic – S370, slide 11 • Tim Dorr – server rack, slide 12 (Flickr) • JasonTheGreat – Sun server room, slide 13 (Flickr) • Intel Free Press – Intel server room, slide 14, 15, 16 (Flickr) • Geralt – IoT background, slide 18, 19