The Past, Present and Future of Technical Debt Learning from the past to prepare for the future Technical Debt 2018 Gothenburg, May 2018 Eoin Woods Endava @eoinwoodz

Background Eoin Woods • Endava’s CTO, based in London • 10 years in product dev - Bull, Sybase, InterTrust • 10 years in capital markets applications - UBS and BGI • Software engineer, then architect, now CTO • Author, editor, speaker, community guy I’ve had a lot of technical debt

What is Technical Debt?

Defining Technical Debt • “Not quite right code which we postpone making it right” – Ward Cunningham • “The obligation that a software organization incurs when it chooses a design or construction approach that's expedient in the short term but that increases complexity and is more costly in the long term” – Steve McConnell • “Compromises made in the implementation of a piece of software that are likely to limit how it can be used or changed later” – Eoin Woods

(Some) Types of Technical Debt Intra-Module Inter-Module Testing User Experience Data Operation Platform + Functional

Why Do We Care? • Reduces speed of delivery • Reduces options for change • Reduces morale in the team • Reduces reliability • Increases cost of change • Increases cost of operation REDUCES OUR RETURN ON INVESTMENT

TECHNICAL DEBT IS OFTEN WHAT PREVENTS US MEETING A FUTURE NEED IN A TIMELY AND COST EFFECTIVE MANNER

Like Death and Taxes

Technical Debt Through the Ages

5 Ages of Software Systems Intelligent Connected (2020s) Internet is the System (2010s) Internet Connected (2000s) Distributed Monoliths (1990s) Monolithic (1980s)

Monolithic Technical Debt • Procedure tangles • GOTO spaghetti • Data incoherence Advances • Structured Programming • Early metrics • Early analysis tools Structuring of computer programs

Distributed Monoliths Technical Debt • Tiering confusion • all in the UI, all in the DB, … • Spaghetti database • Monster data models Advances • Emergence of the metaphor • Refactoring • Commercial tools The rise of client/server and architectural style

Internet Connected Technical Debt • Unsuitable platforms • Unsustainable operation • One-man deployment “pipeline” • Quality property hacks Advances • Software architecture • Patterns and platforms for large-scale systems • Open source tools Mass market scaling and explosion in software architecture

Internet as the System Technical Debt • External service dependencies • Microservice choreography • Data replication & duplication Advances • API descriptions • Runtime tracing & visualisation • Containers & platforms Everything is a service, architecture dynamic & packaged

New Sources of Technical Debt • Architecture hidden in the network (e.g. microservices) • Digital channels => constant functional change • Shift to unstructured and replicated data • Rapidity and self-service of cloud platforms • New types of code (“X as Code”) • Rapid tech stack evolution (e.g. framework churn)

Current Technical Debt in Practice • Ubiquitous term • General focus on code, sometimes tests • Data, platform, operational debt often not recognised • Some teams actively manage the debt • Many just talk about it • “Everyone” has a mountain of it • No widely adopted reasoning frameworks • Looming “new debt” problems don’t get much attention

On the Future

Intelligent Connected

Intelligent Connected Technical Debt • Unknown connected devices • Data needed for “intelligence” • ML models Advances (Needed) • Better IoT security & mgmt.? • More on data & algorithms? • Self-explaining AI? Data, algorithms, “intelligence”, emergent architecture

To Conclude

Conclusions • Our past can point to the future • Monolithic complexity led to structure and metrics • Distributed Monoliths brought a metaphor & refactoring • Internet Connected systems needed architecture & triggered open source tooling • Each era has tried to manage the debt it finds itself with … often with limited impact

Conclusions • Internet as the System brings its own problems too: • External dependencies (services) • Microservice dependencies • Data replication & duplication • New types of code and platform • Response is largely project-by-project • containers? runtime tracing? api doc techniques? ... • Technical Debt practice often lags the need by years!

Conclusions • What about Intelligent Connected systems? - We’re not great at ”data debt” but these systems depend on it - AI debt still not understood but will become a standard feature - Devices bring lots of debt but we struggle with today’s challenge • We’ve got some work to do … - We can see what is coming - This time could research and practice predict the debt and think about managing it … before we have a lot of it?

THANK YOU Eoin Woods Endava [email protected] @eoinwoodz

Picture Credits • Erik Pitti – S360, slide 11 (Flickr) • Tech Republic – S370, slide 11 • Tim Dorr – server rack, slide 12 (Flickr) • JasonTheGreat – Sun server room, slide 13 (Flickr) • Intel Free Press – Intel server room, slide 14, 15, 16 (Flickr) • Geralt – IoT background, slide 18, 19