The Past, Present and Future of Technical Debt

Transcript

1. The Past, Present and Future
   of Technical Debt
   Learning from the past to prepare for the future
   Technical Debt 2018
   Gothenburg, May 2018
   Eoin Woods
   Endava
   @eoinwoodz
   

   View Slide

2. Background
   Eoin Woods
   • Endava’s CTO, based in London
   • 10 years in product dev - Bull, Sybase, InterTrust
   • 10 years in capital markets applications - UBS and BGI
   • Software engineer, then architect, now CTO
   • Author, editor, speaker, community guy
   I’ve had a lot of technical debt
   

   View Slide

3. What is Technical Debt?
   

   View Slide

4. Defining Technical Debt
   • “Not quite right code which we postpone making it right”
   – Ward Cunningham
   • “The obligation that a software organization incurs when it
   chooses a design or construction approach that's expedient in
   the short term but that increases complexity and is more
   costly in the long term”
   – Steve McConnell
   • “Compromises made in the implementation of a piece of
   software that are likely to limit how it can be used or changed
   later”
   – Eoin Woods
   

   View Slide

5. (Some) Types of Technical Debt
   Intra-Module Inter-Module
   Testing
   User
   Experience
   Data
   Operation
   Platform
   + Functional
   

   View Slide

6. Why Do We Care?
   • Reduces speed of delivery
   • Reduces options for change
   • Reduces morale in the team
   • Reduces reliability
   • Increases cost of change
   • Increases cost of operation
   REDUCES OUR RETURN ON INVESTMENT
   

   View Slide

7. TECHNICAL DEBT IS OFTEN WHAT PREVENTS
   US MEETING A FUTURE NEED IN A
   TIMELY AND COST EFFECTIVE MANNER
   

   View Slide

8. Like Death and Taxes
   

   View Slide

9. Technical Debt Through the Ages
   

   View Slide

10. 5 Ages of Software Systems
    Intelligent
    Connected
    (2020s)
    Internet
    is the System
    (2010s)
    Internet
    Connected
    (2000s)
    Distributed
    Monoliths
    (1990s)
    Monolithic
    (1980s)
    

    View Slide

11. Monolithic
    Technical Debt
    • Procedure tangles
    • GOTO spaghetti
    • Data incoherence
    Advances
    • Structured Programming
    • Early metrics
    • Early analysis tools
    Structuring of computer programs
    

    View Slide

12. Distributed Monoliths
    Technical Debt
    • Tiering confusion
    • all in the UI, all in the DB, …
    • Spaghetti database
    • Monster data models
    Advances
    • Emergence of the metaphor
    • Refactoring
    • Commercial tools
    The rise of client/server and architectural style
    

    View Slide

13. Internet Connected
    Technical Debt
    • Unsuitable platforms
    • Unsustainable operation
    • One-man deployment
    “pipeline”
    • Quality property hacks
    Advances
    • Software architecture
    • Patterns and platforms for
    large-scale systems
    • Open source tools
    Mass market scaling and explosion in software architecture
    

    View Slide

14. Internet as the System
    Technical Debt
    • External service dependencies
    • Microservice choreography
    • Data replication & duplication
    Advances
    • API descriptions
    • Runtime tracing & visualisation
    • Containers & platforms
    Everything is a service, architecture dynamic & packaged
    

    View Slide

15. New Sources of Technical Debt
    • Architecture hidden in the network (e.g. microservices)
    • Digital channels => constant functional change
    • Shift to unstructured and replicated data
    • Rapidity and self-service of cloud platforms
    • New types of code (“X as Code”)
    • Rapid tech stack evolution (e.g. framework churn)
    

    View Slide

16. Current Technical Debt in Practice
    • Ubiquitous term
    • General focus on code, sometimes tests
    • Data, platform, operational debt often not recognised
    • Some teams actively manage the debt
    • Many just talk about it
    • “Everyone” has a mountain of it
    • No widely adopted reasoning frameworks
    • Looming “new debt” problems don’t get much attention
    

    View Slide

17. On the Future
    

    View Slide

18. Intelligent Connected
    

    View Slide

19. Intelligent Connected
    Technical Debt
    • Unknown connected devices
    • Data needed for “intelligence”
    • ML models
    Advances (Needed)
    • Better IoT security & mgmt.?
    • More on data & algorithms?
    • Self-explaining AI?
    Data, algorithms, “intelligence”, emergent architecture
    

    View Slide

20. To Conclude
    

    View Slide

21. Conclusions
    • Our past can point to the future
    • Monolithic complexity led to structure and metrics
    • Distributed Monoliths brought a metaphor & refactoring
    • Internet Connected systems needed architecture &
    triggered open source tooling
    • Each era has tried to manage the debt it finds itself with …
    often with limited impact
    

    View Slide

22. Conclusions
    • Internet as the System brings its own problems too:
    • External dependencies (services)
    • Microservice dependencies
    • Data replication & duplication
    • New types of code and platform
    • Response is largely project-by-project
    • containers? runtime tracing? api doc techniques? ...
    • Technical Debt practice often lags the need by years!
    

    View Slide

23. Conclusions
    • What about Intelligent Connected systems?
    - We’re not great at ”data debt” but these systems depend on it
    - AI debt still not understood but will become a standard feature
    - Devices bring lots of debt but we struggle with today’s challenge
    • We’ve got some work to do …
    - We can see what is coming
    - This time could research and practice predict the debt and think
    about managing it … before we have a lot of it?
    

    View Slide

24. THANK YOU
    Eoin Woods
    Endava
    eoin.woods@endava.com
    @eoinwoodz
    

    View Slide

25. Picture Credits
    • Erik Pitti – S360, slide 11 (Flickr)
    • Tech Republic – S370, slide 11
    • Tim Dorr – server rack, slide 12 (Flickr)
    • JasonTheGreat – Sun server room, slide 13 (Flickr)
    • Intel Free Press – Intel server room, slide 14, 15, 16 (Flickr)
    • Geralt – IoT background, slide 18, 19
    

    View Slide