Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Averting Tragedy on the Architectural Commons (JavaZone 2012)

Averting Tragedy on the Architectural Commons (JavaZone 2012)

Much has been made of the need to establish software architectures to provide the firm foundations on which successful products are constructed, but if a product is successful over the long term its architecture will not only need to evolve, but must be actively defended against malignant forces. The fact that software architectures tend to outlive the tenure of developers, architects, management teams and even companies makes the maintenance of software architectures over the long term crucial for the ability of products to deliver ongoing value.

What happens to the architecture of established systems in an environment where concurrent feature delivery projects make architectural ‘adjustments’ for their own ends? How can erosion of this shared architecture be managed across multiple teams? How can projects acting in their own best interests avoid over-exploiting the common architectural resource on which they all depend? Should ownership of the architecture be distributed and shared, or centralised and tightly controlled? Perhaps most importantly, can we detect when the architecture has been violated?

This talk explores how shared resources in other fields are managed for the common good, and draws analogies and lessons which can be applied to the shared ‘resource’ of a software architecture. Examples garnered from over twelve years working with greenfield and legacy software systems illustrate how to diagnose, understand the causes of, and address the erosion of application architectures so that products can flourish and be productive for future generations.

Robert Smallshire

September 13, 2012
Tweet

More Decks by Robert Smallshire

Other Decks in Programming

Transcript

  1. Avoiding Tragedy
    on the Architectural Commons
    1
    Robert Smallshire
    Roxar Software Solutions, Oslo
    @robsmallshire
    Thursday, 13 September, 12

    View Slide

  2. Systems and their architectures are long lived
    Lifetimes in the software industry
    2
    Developers
    Windows XP
    Applications
    CEOs
    Lines of code
    FTSE100
    Classes
    Modules
    0 15 30 45 60
    58
    37
    22
    13
    6.8
    6.2
    4.7
    3.1
    Sources: W3schools, Software Lifetime and its Evolution Process over Generations, CEO Succession Practices: 2012 Edition, Investors Chronicle,
    Half-lives of software related entities
    The number of years over which half the entities are replaced
    Thursday, 13 September, 12

    View Slide

  3. 3
    Thursday, 13 September, 12

    View Slide

  4. 3
    “Farming looks mighty easy
    when your plow is a pencil and
    you're a thousand miles from
    the corn field.” Dwight Eisenhower
    Thursday, 13 September, 12

    View Slide

  5. 4
    Thursday, 13 September, 12

    View Slide

  6. 5
    Sustainable Development
    Provision meets ongoing needs whilst preserving the supporting
    environment.
    Thursday, 13 September, 12

    View Slide

  7. 6
    Thursday, 13 September, 12

    View Slide

  8. 7
    The tragedy of the commons
    What is the tragedy and how does it relate to software architecture?
    The causes and effects of architectural erosion
    How to recognise and detect compromised architectures.
    Defending the architecture
    Managing the architecture and its stakeholders.
    1
    2
    3
    Thursday, 13 September, 12

    View Slide

  9. Garrett Hardin
    8
    “Resources held for common use by all will
    ultimately be destroyed.”
    Hardin, G. (1968) The Tragedy of the Commons. Science 162 1243-1248
    Thursday, 13 September, 12

    View Slide

  10. Garrett Hardin
    8
    “Resources held for common use by all will
    ultimately be destroyed.”
    Hardin, G. (1968) The Tragedy of the Commons. Science 162 1243-1248
    “There is no technical solution.”
    Thursday, 13 September, 12

    View Slide

  11. 9
    Thursday, 13 September, 12

    View Slide

  12. 9
    Thursday, 13 September, 12

    View Slide

  13. 9
    Thursday, 13 September, 12

    View Slide

  14. 9
    Thursday, 13 September, 12

    View Slide

  15. 9
    Thursday, 13 September, 12

    View Slide

  16. 9
    Thursday, 13 September, 12

    View Slide

  17. 9
    Thursday, 13 September, 12

    View Slide

  18. 10
    Thursday, 13 September, 12

    View Slide

  19. 11
    Thursday, 13 September, 12

    View Slide

  20. 11
    4      047 880
    visitors to Yosemite National Park in 2010.
    Thursday, 13 September, 12

    View Slide

  21. 12
    Thursday, 13 September, 12

    View Slide

  22. 13
    Thursday, 13 September, 12

    View Slide

  23. 14
    Thursday, 13 September, 12

    View Slide

  24. 15
    Collapse of Newfoundland Cod Stocks
    Thursday, 13 September, 12

    View Slide

  25. Potential and actual tragedies
    The Commons
    16
    Global
    Climate
    Fisheries
    Groundwater
    Grazing
    Habitat
    Forests
    Radio
    Spectrum
    Atmosphere
    The
    Internet
    Roads
    Parks
    Museums
    Silence
    Welfare
    State
    Thursday, 13 September, 12

    View Slide

  26. Potential and actual tragedies
    The Commons
    16
    Global
    Climate
    Fisheries
    Groundwater
    Grazing
    Habitat
    Forests
    Radio
    Spectrum
    Atmosphere
    The
    Internet
    Roads
    Parks
    Museums
    Silence
    Software
    Architectures
    Welfare
    State
    Thursday, 13 September, 12

    View Slide

  27. 17
    The tragedy of the commons
    What is the tragedy and how does it relate to software architecture?
    The causes and effects of architectural erosion
    How to recognise and detect compromised architectures.
    Defending the architecture
    Managing the architecture and its stakeholders.
    1
    2
    3
    Thursday, 13 September, 12

    View Slide

  28. What takes us from this ...
    How does this happen?
    18
    Thursday, 13 September, 12

    View Slide

  29. ... to this?
    How does this happen?
    19
    Thursday, 13 September, 12

    View Slide

  30. The tragedy unfolds inexorably, one dependency at a time.
    How does this happen?
    20
    Thursday, 13 September, 12

    View Slide

  31. The tragedy unfolds inexorably, one dependency at a time.
    How does this happen?
    20
    using App.Foo.Bar
    import App.Foo.Bar
    from App.Foo import Bar
    #include
    require “app/foo/bar.h”
    Thursday, 13 September, 12

    View Slide

  32. has second order effect on outcomes
    Organization
    21
    Command
    and Control
    Self
    Organising
    Thursday, 13 September, 12

    View Slide

  33. Large numbers of smart people - does smartness scale?
    22
    Organization
    Thursday, 13 September, 12

    View Slide

  34. Large numbers of smart people - does smartness scale?
    22
    Organization
    Thursday, 13 September, 12

    View Slide

  35. Smartness is not an additive quantity.
    23
    Organization
    Thursday, 13 September, 12

    View Slide

  36. Reap all the immediate benefit but share the deferred cost
    24
    Introduced dependencies
    Thursday, 13 September, 12

    View Slide

  37. Reap all the immediate benefit but share the deferred cost
    24
    Introduced dependencies
    Thursday, 13 September, 12

    View Slide

  38. 25
    Thursday, 13 September, 12

    View Slide

  39. 26
    “any situation in which one person makes the
    decision about how much risk to take, while
    someone else bears the cost if things go badly”
    Paul Krugman, Economist
    Thursday, 13 September, 12

    View Slide

  40. Step back and take in
    the big picture
    Detecting compromised architectures
    27
    Thursday, 13 September, 12

    View Slide

  41. 28
    Thursday, 13 September, 12

    View Slide

  42. 28
    Thursday, 13 September, 12

    View Slide

  43. How is complexity distributed in your system?
    Mapping complexity
    29
    Thursday, 13 September, 12

    View Slide

  44. How are defects distributed in your system?
    Mapping quality
    30
    Thursday, 13 September, 12

    View Slide

  45. How are defects distributed in your system?
    Mapping quality
    30
    Thursday, 13 September, 12

    View Slide

  46. Do you understand your actual dependencies?
    31
    Dependency Structure
    Thursday, 13 September, 12

    View Slide

  47. 32
    Thursday, 13 September, 12

    View Slide

  48. 33
    Thursday, 13 September, 12

    View Slide

  49. 34
    The tragedy of the commons
    What is the tragedy and how does it relate to software architecture?
    The causes and effects of architectural erosion
    How to recognize and detect compromised architectures.
    Defending the architecture
    Managing the architecture and its stakeholders.
    1
    2
    3
    Thursday, 13 September, 12

    View Slide

  50. We have met the enemy and he is us.
    Tragedy on the Architectural Commons
    35
    Short Tenure
    Developer turnover outpaces
    change of the software system.
    Moral Hazard
    Developers acting rationally for
    local gain aren’t exposed to the
    deferred and global cost of their
    actions.
    The Tragedy
    The common environment of the
    software architecture erodes, making
    ongoing development more frustrating
    and costly. Productivity falls.
    1
    2
    3
    Tragic
    Cycle
    Thursday, 13 September, 12

    View Slide

  51. Often used in combination
    Solutions to the tragedy
    36
    Education Encourage ‘enlightened self-interest’. Foster cooperation through
    the architecture.
    Privatisation Create a ‘core’ team who own the architecture. Does this confuse
    architecture with infrastructure?
    Taxation Tax feature projects by requiring them to contribute beyond their
    narrow project scope.
    Regulation Give clear ownership to a central authority. Introduce regulating
    software.
    Thursday, 13 September, 12

    View Slide

  52. Education
    37
    ‣ Enlightened self interest
    • “do well by doing good”
    ‣ Deferred gratification
    • sacrifice short-term interests in favour of
    long-term interests
    Thursday, 13 September, 12

    View Slide

  53. The code is the design?
    “most complete, unambiguous, and
    irreducible expression of a design”
    38
    The code is the most,
    Maybe. But only the de facto design,
    not the intended design.
    Thursday, 13 September, 12

    View Slide

  54. Privatisation
    39
    ‣ Transfer ownership of the architecture
    to a single organisation
    • ownership obligates stewardship
    • acts on behalf of feature teams, who must
    rent or purchase services
    • does this confuse architecture with
    infrastructure?
    Thursday, 13 September, 12

    View Slide

  55. 40
    Creating a core team who owns
    the architecture usually results
    in another feature team.
    It’s hard to maintain stakeholder
    support for infrastructure or
    design artefacts.
    Thursday, 13 September, 12

    View Slide

  56. 40
    Creating a core team who owns
    the architecture usually results
    in another feature team.
    It’s hard to maintain stakeholder
    support for infrastructure or
    design artefacts.
    Thursday, 13 September, 12

    View Slide

  57. Taxation
    41
    ‣ Introduce transaction costs
    • attach an immediate cost to potentially
    detrimental changes
    • changes reflect their long-term, shared cost
    • discourage frivolous or merely convenient
    dependencies
    Thursday, 13 September, 12

    View Slide

  58. 42
    Not all changes should be
    treated equally
    Thursday, 13 September, 12

    View Slide

  59. 42
    Not all changes should be
    treated equally
    Thursday, 13 September, 12

    View Slide

  60. 42
    Not all changes should be
    treated equally
    Thursday, 13 September, 12

    View Slide

  61. 43
    Not all changes should be
    treated equally
    Have new dependencies require
    review from a wider constituency
    Thursday, 13 September, 12

    View Slide

  62. Regulation
    44
    ‣ Central authority
    • architects are easy to ignore
    • architects can’t be omniscient
    • architects should to encode design rules
    in such a way that they can’t be ignored
    Thursday, 13 September, 12

    View Slide

  63. 45
    Encode design constraints into
    automated systems:
    • build systems
    • commit triggers
    • custom static analysis tools
    Thursday, 13 September, 12

    View Slide

  64. 46
    The microeconomics of change
    By understanding the incentives which drive architectural erosion
    we can avoid tragic decline.
    Thursday, 13 September, 12

    View Slide

  65. Dependencies and complexity are too cheap in the short term
    Confront the moral hazard
    47
    Short Tenure
    Developer turnover outpaces
    change of the software system.
    Moral Hazard
    Developers acting rationally for
    local gain aren’t exposed to the
    deferred cost of their actions, of
    technical debt.
    The Tragedy
    The common environment of the system
    erodes, making ongoing development
    more frustrating and costly. Productivity
    falls.
    1
    2
    3
    Tragic
    Cycle
    Thursday, 13 September, 12

    View Slide

  66. To avert the tragedy
    48
    Hire smart people Be vigilant
    Dependencies should immediately
    reflect their lifetime costs
    Automate regulations and enforcement
    and organise them any which way you like. At large scales and over the
    long term team structures and processes are second order effects.
    use wide-angle tools to monitor system-scale dependencies and
    qualities. Follow-up at code level. Avoid a ‘the code is the design’
    mentality.
    dedicate your effort to encoding and automation of local rules and
    enforcement. Special cases aren’t special enough.
    more closely by having new dependencies trigger design or code
    reviews to bring some of their cost forwards in time.
    Thursday, 13 September, 12

    View Slide

  67. 49
    Thursday, 13 September, 12

    View Slide

  68. 50
    Robert Smallshire
    Roxar Software Solutions, Oslo
    @robsmallshire
    Thursday, 13 September, 12

    View Slide

  69. Thank you
    Questions?
    50
    Robert Smallshire
    Roxar Software Solutions, Oslo
    @robsmallshire
    Thursday, 13 September, 12

    View Slide