Save 37% off PRO during our Black Friday Sale! »

Averting Tragedy on the Architectural Commons (ScanDev 2013)

Averting Tragedy on the Architectural Commons (ScanDev 2013)

Much has been made of the need to establish software architectures to provide the firm foundations on which successful products are constructed, but if a product is successful over the long term its architecture will not only need to evolve, but must be actively defended against malignant forces. The fact that software architectures tend to outlive the tenure of developers, architects, management teams and even companies makes the maintenance of software architectures over the long term crucial for the ability of products to deliver ongoing value. What happens to the architecture of established systems in an environment where concurrent feature delivery projects make architectural ‘adjustments’ for their own ends? How can erosion of this shared architecture be managed across multiple teams? How can projects acting in their own best interests avoid over-exploiting the common architectural resource on which they all depend? Should ownership of the architecture be distributed and shared, or centralised and tightly controlled? Perhaps most importantly, can we detect when the architecture has been violated? This talk explores how shared resources in other fields are managed for the common good, and draws analogies and lessons which can be applied to the shared ‘resource’ of a software architecture. Examples garnered from over twelve years working with greenfield and legacy software systems illustrate how to diagnose, understand the causes of, and address the erosion of application architectures so that products can flourish and be productive for future generations.

4be361182fa13cf39c00ec69c1cb9e30?s=128

Robert Smallshire

March 05, 2013
Tweet

Transcript

  1. Avoiding Tragedy on the Architectural Commons 1 Robert Smallshire Roxar

    Software Solutions, Oslo @robsmallshire Tuesday, 5 March, 13
  2. Systems and their architectures are long lived Lifetimes in the

    software industry 2 Developers Windows XP Applications CEOs Lines of code FTSE100 Classes Modules 0 15 30 45 60 58 37 22 13 6.8 6.2 4.7 3.1 Sources: W3schools, Software Lifetime and its Evolution Process over Generations, CEO Succession Practices: 2012 Edition, Investors Chronicle, Half-lives of software related entities The number of years over which half the entities are replaced Tuesday, 5 March, 13
  3. 3 “Farming looks mighty easy when your plow is a

    pencil and you're a thousand miles from the corn field.” Dwight Eisenhower Tuesday, 5 March, 13
  4. 4 Tuesday, 5 March, 13

  5. 5 Sustainable Development Provision meets ongoing needs whilst preserving the

    supporting environment. Tuesday, 5 March, 13
  6. 6 Tuesday, 5 March, 13

  7. 7 The tragedy of the commons What is the tragedy

    and how does it relate to software architecture? The causes and effects of architectural erosion How to recognise and detect compromised architectures. Defending the architecture Managing the architecture and its stakeholders. 1 2 3 Tuesday, 5 March, 13
  8. Garrett Hardin 8 “Resources held for common use by all

    will ultimately be destroyed.” Hardin, G. (1968) The Tragedy of the Commons. Science 162 1243-1248 “There is no technical solution.” Tuesday, 5 March, 13
  9. 9 Tuesday, 5 March, 13

  10. 10 Tuesday, 5 March, 13

  11. Potential and actual tragedies The Commons 11 Global Climate Fisheries

    Groundwater Grazing Habitat Forests Radio Spectrum Atmosphere The Internet Roads Parks Museums Silence Software Systems Welfare State Tuesday, 5 March, 13
  12. 12 The tragedy of the commons What is the tragedy

    and how does it relate to software architecture? The causes and effects of architectural erosion How to recognise and detect compromised architectures. Defending the architecture Managing the architecture and its stakeholders. 1 2 3 Tuesday, 5 March, 13
  13. What takes us from this ... How does this happen?

    13 Tuesday, 5 March, 13
  14. ... to this? How does this happen? 14 Tuesday, 5

    March, 13
  15. The tragedy unfolds inexorably, one dependency at a time. How

    does this happen? 15 using App.Foo.Bar import App.Foo.Bar from App.Foo import Bar #include <app/foo/bar.h> require “app/foo/bar.h” Tuesday, 5 March, 13
  16. has second order effect on outcomes Organization 16 Command and

    Control Self Organising Tuesday, 5 March, 13
  17. Large numbers of smart people - does smartness scale? 17

    Organization Tuesday, 5 March, 13
  18. Smartness is not an additive quantity. 18 Organization Tuesday, 5

    March, 13
  19. 100 1000 10000 100000 1000 10000 100000 1000000 10000000 Productivity

    (Lines of Code / Year) Total Lines of Code Use published productivity data to forward model code size. Modelling team and code evolution 19 Sources: COCOMO II At any given system size we can predict a distribution for developer productivity. Tuesday, 5 March, 13
  20. 20 3 years 54 kLoC Overall Employees : 4 Team

    Size : 2 LoC : 54 k Author present : 60% Tuesday, 5 March, 13
  21. 21 3 years 152 kLoC Cumulative team size : 11

    ± 2 @ 1σ Team Size : 7 LoC : 157 k ± 23 k @ 1σ Author present : 70% ± 14% @ 1σ Tuesday, 5 March, 13
  22. 22 20 years 1.8 MLoC Cumulative team size : 114

    ± 9 @ 1σ Team Size : 21 LoC : 1.8 M ± 0.08 M @ 1σ Author present : 19% ± 4% @ 1σ Tuesday, 5 March, 13
  23. Most authors of your product quit way back when. Who

    can you still talk to? 23 days Proportion of code written by current colleagues. 20% after 20 years Tuesday, 5 March, 13
  24. Step back and take in the big picture Detecting compromised

    architectures 24 Tuesday, 5 March, 13
  25. 25 Tuesday, 5 March, 13

  26. How is complexity distributed in your system? Mapping complexity 26

    Tuesday, 5 March, 13
  27. How are defects distributed in your system? Mapping quality 27

    Tuesday, 5 March, 13
  28. Do you understand your actual dependencies? 28 Dependency Structure Tuesday,

    5 March, 13
  29. 29 Tuesday, 5 March, 13

  30. 30 Tuesday, 5 March, 13

  31. Reap all the immediate benefit but share the deferred cost

    31 Introduced dependencies Tuesday, 5 March, 13
  32. 32 Tuesday, 5 March, 13

  33. 33 “any situation in which one person makes the decision

    about how much risk to take, while someone else bears the cost if things go badly” Paul Krugman, Economist Tuesday, 5 March, 13
  34. 34 The tragedy of the commons What is the tragedy

    and how does it relate to software architecture? The causes and effects of architectural erosion How to recognize and detect compromised architectures. Defending the architecture Managing the architecture and its stakeholders. 1 2 3 Tuesday, 5 March, 13
  35. We have met the enemy and he is us. Tragedy

    on the Architectural Commons 35 Short Tenure Developer turnover outpaces change of the software system. Moral Hazard Developers acting rationally for local gain aren’t exposed to the deferred and global cost of their actions. The Tragedy The common environment of the software architecture erodes, making ongoing development more frustrating and costly. Productivity falls. 1 2 3 Tragic Cycle Tuesday, 5 March, 13
  36. Often used in combination Solutions to the tragedy 36 Education

    Encourage ‘enlightened self-interest’. Foster cooperation through the architecture. Privatisation Create a ‘core’ team who own the architecture. Does this confuse architecture with infrastructure? Taxation Tax feature projects by requiring them to contribute beyond their narrow project scope. Regulation Give clear ownership to a central authority. Introduce regulating software. Tuesday, 5 March, 13
  37. Education 37 ‣ Enlightened self interest • “do well by

    doing good” ‣ Deferred gratification • sacrifice short-term interests in favour of long-term interests Tuesday, 5 March, 13
  38. The code is the design? “most complete, unambiguous, and irreducible

    expression of a design” 38 The code is the most, Maybe. But only the de facto design, not the intended design. Tuesday, 5 March, 13
  39. Privatisation 39 ‣ Transfer ownership of the architecture to a

    single organisation • ownership obligates stewardship • acts on behalf of feature teams, who must rent or purchase services • does this confuse architecture with infrastructure? Tuesday, 5 March, 13
  40. 40 Creating a core team who owns the architecture usually

    results in another feature team. It’s hard to maintain stakeholder support for infrastructure or design artefacts. Tuesday, 5 March, 13
  41. Taxation 41 ‣ Introduce transaction costs • attach an immediate

    cost to potentially detrimental changes • changes reflect their long-term, shared cost • discourage frivolous or merely convenient dependencies Tuesday, 5 March, 13
  42. 42 Not all changes should be treated equally Tuesday, 5

    March, 13
  43. 43 Not all changes should be treated equally Have new

    dependencies require review from a wider constituency Tuesday, 5 March, 13
  44. Regulation 44 ‣ Central authority • architects are easy to

    ignore • architects can’t be omniscient • architects should encode design rules in such a way that they can’t be ignored Tuesday, 5 March, 13
  45. 45 Encode design constraints into automated systems: • vigilant build

    systems • commit triggers • custom static analysis tools Tuesday, 5 March, 13
  46. 46 The microeconomics of change By understanding the incentives which

    drive architectural erosion we can avoid tragic decline. Tuesday, 5 March, 13
  47. Dependencies and complexity are too cheap in the short term

    Confront the moral hazard 47 Short Tenure Developer turnover outpaces change of the software system. Moral Hazard Developers acting rationally for local gain aren’t exposed to the deferred cost of their actions, of technical debt. The Tragedy The common environment of the system erodes, making ongoing development more frustrating and costly. Productivity falls. 1 2 3 Tragic Cycle Tuesday, 5 March, 13
  48. 48 Tuesday, 5 March, 13

  49. Thank you Questions? 49 Robert Smallshire Roxar Software Solutions, Oslo

    @robsmallshire starting in May Tuesday, 5 March, 13