The reusability fallacy

This slide deck discusses reusability, a topic that is sort of a "holy grail" in software architecture. Unfortunately, reusability often is discussed in the wrong places and for the wrong reasons, especially if it is used to sell a new architectural paradigm ("via reuse, the initial investment will amortize very soon and it will pay for itself").

I start with the origins of the topic in the early 1970s (and the different conditions how software was developed back then). Then I discuss and debunk the most widespread fallacies that are typically used to press for reusability.

With that knowledge I present a few advices (not dogmas!) how to deal better with reusability - when not to use it, what to do instead of those places, and: When to use it!

Reusability has a huge value in software development and we need to make sure to harvest it. But pressing for it in the wrong places for the wrong reasons only causes problems and harm. We need to avoid that and the intention of the talk is to have better arguments if you encounter such a situation.

This is the unabridged slide deck containing all slides of the talk. For conferences, I usually leave out several slides to meet the session length constraints.

I also accompanied this slide deck with a blog series that can be found at https://ufried.com/blog/reusability_fallacy_1/. The series consists of 4 posts, covers the contents of this talk and provides a few more ideas and pointers.

Edit: Uploaded an updated version of the slide deck on Jun-30-2023

Transcript

1. The reusability fallacy
   Killing a beloved dogma
   Uwe Friedrichsen – codecentric AG – 2008-2023
   

   View Slide

2. Uwe Friedrichsen
   Works @ codecentric
   https://twitter.com/ufried
   https://www.speakerdeck.com/ufried
   https://ufried.com/blog/reusability_fallacy_1/
   

   View Slide

3. The “holy grail” of architecture
   

   View Slide

4. The promise (the dream)
   • Create software module once
   • Reuse as often as you like at no reproduction cost
   • “Invest once, harvest forever”
   • Big efficiency lever
   • Huge cost saver
   

   View Slide

5. The origins
   

   View Slide

6. Structured design
   by W. P. Stevens, G. J. Myers and L. L. Constantine
   [Ste 1974]
   

   View Slide

7. “Considerations and techniques are proposed that reduce the complexity of
   programs by dividing them into functional modules.
   This can make it possible to create complex systems from simple,
   independent, reusable modules. […] And, as the module library grows,
   increasingly sophisticated programs can be implemented using less and
   less new code.”
   [Ste 1974]
   

   View Slide

8. The origins
   • Reusability discussion started ~1970
   • Most companies and projects started at scratch
   • Barely more than 3GL compiler and linker available
   • No big standard libraries bundled with language
   • No big ecosystems available around language
   • Even most basic functionalities needed to be developed
   • Reusability could increase productivity significantly
   

   View Slide

9. Reusability today (the fallacy)
   • Became an end in itself
   • “Reusability is needed. Period!”
   • Demanded by managers to improve cost efficiency
   • “I want reusability to improve the IT cost ratio”
   • Used by architects to sell new architectural paradigms
   • “Required investment will amortize soon due to reuse”
   à Never worked this way. Never will work this way.
   

   View Slide

10. Still, reusability is widely popular.
    Thus, where is the misconception?
    

    View Slide

11. Actually, it is a mix of misconceptions
    

    View Slide

12. Misconception #1
    Industrial production analogies
    

    View Slide

13. “Let us apply the reusability approaches from
    house building or car manufacturing”
    -- The unknown “industrialization of IT” advocate
    

    View Slide

14. Reuse ≈ Standardization
    

    View Slide

15. Standardization
    • Standardization helps building at reduced price
    • Same part can be reused in many products
    • Saves production costs over time
    • Design efforts only once
    • Production process design and set up only once
    • Required tooling created and installed only once
    • Cost savings are realized in production
    • Spreads design and set up costs over parts produced
    

    View Slide

16. Software development has nothing in common
    with producing a physical good
    

    View Slide

17. The products we build are not source code,
    but executable programs
    

    View Slide

18. Our assembly lines are so efficient for more than 50 years
    that most people forgot they even exist
    We call them “compiler” *
    * or CI/CD pipelines in the settings of today
    

    View Slide

19. Software development is about designing the product
    The design is only complete after the last line of code is written *
    * Building the product from the design (source code) is merely pushing a button
    

    View Slide

20. We cannot make software production more cost-efficient
    by building more parts from the same design
    

    View Slide

21. Our production process is already unbelievably cheap *
    and we can produce multiple copies from a design at zero cost
    We call it the “copy command”
    * compared to the production process of any physical goods
    

    View Slide

22. All the metaphors comparing software development to
    house building or car assembly are just plain wrong
    

    View Slide

23. Conclusion
    Potential cost or efficiency improvements
    of reusability are limited to design time
    

    View Slide

24. New (industrially minted) idea
    ”The more reusable parts we use in our design process
    (a.k.a. software development),
    the more time, costs and effort we can save”
    

    View Slide

25. No silver bullet
    by Frederick P. Brooks, Jr.
    [Bro 1986]
    

    View Slide

26. “Software entities are more complex for their size than perhaps any other
    human construct, because no two parts are alike (at least above the
    statement level). If they are, we make the two similar parts into one, a
    subroutine, open or closed.
    In this respect software systems differ profoundly from computers, buildings,
    or automobiles, where repeated elements abound. [...]
    Likewise, a scaling-up of a software entity is not merely a repetition of the
    same elements in larger size; it is necessarily an increase in the number of
    different elements.”
    [Bro 1986]
    

    View Slide

27. Reusing the same part repeatedly does not work
    

    View Slide

28. Reuse is limited to lower-level functionalities supporting
    higher-level functionalities, still resulting in a
    significant increase of different parts
    

    View Slide

29. Misconception #2
    Reusability comes for free
    

    View Slide

30. “Each reusable part saves me a lot of money!”
    -- The unknown decision maker
    

    View Slide

31. Well, ever considered the cost of creating a reusable asset?
    

    View Slide

32. The tar pit
    by Frederick P. Brooks, Jr.
    (taken from the “The mythical man-month”)
    [Bro 1995]
    

    View Slide

33. “One occasionally reads newspaper accounts of how two programmers in a
    remodeled garage have built an important program that surpasses the best
    efforts of large teams. And every programmer is prepared to believe such
    tales, for he knows that he could build any program much faster than the 1000
    statements/year reported for industrial teams.
    Why then have not all industrial programming teams been replaced by
    dedicated garage duos? One must look at what is being produced.”
    [Bro 1995]
    

    View Slide

34. 1x
    Program
    The original module,
    suitable for the context
    it was created for
    3x
    Programming
    Product
    A generalized module,
    suitable for multiple
    contexts
    A module, ready to be
    used in an ecosystem
    of interacting modules
    A (re-)usable module,
    that provides a general
    solution for a problem
    3x
    Programming
    System
    9x
    Programming
    Systems
    Product
    

    View Slide

35. 1x
    Program
    9x
    Programming
    Systems
    Product
    Completeness of functionality
    • Handling of edge cases
    • Hardening implementation
    • Thorough functional testing
    • Design Documentation
    Completeness of accessibility
    • Precise interface definition
    • Clear behavioral contract
    • Thorough integration (or alike) testing
    • API Documentation
    

    View Slide

36. Reusability rarely pays!
    Creating a reusable asset is a lot harder and costs a lot more than most people think
    

    View Slide

37. Shirking the costs
    • Asset not implemented for being reused
    • Each “reusing” party needs to
    • Understand the functionality of the asset
    • Understand if it covers the given use case
    • Modify the asset or adopt the using parts
    • Test the resulting functionality
    • …
    • Increases costs and impedes reliability
    • Often leads to avoiding the asset and reimplementing it
    

    View Slide

38. Conclusion
    Creating a reusable asset costs 9 times as much
    as creating a regular project asset.
    You only start saving money the 10th time you reuse the asset.
    

    View Slide

39. Misconception #3
    Reuse requires new architectural paradigms
    

    View Slide

40. “To foster reusability, we need to
    introduce ”
    -- The unknown architect
    

    View Slide

41. The long version of the reasoning
    IT person (typically an architect) to business sponsor:
    “We definitely need to invest in implementing architectural paradigm>. I know this does not create any
    business value now, but it allows us to create reusable
    !
    This way, we will have to pay a lot less for future projects,
    they will be a lot faster and we will be a lot more flexible.
    In short: the investment will pay for itself very soon.”
    Never worked this way, never will work this way!
    

    View Slide

42. Everything we need for decoupling and reusability
    is already in place for more than 60 years
    We call it “modularization” *
    * being available ubiquitously in IT, new paradigms are not needed to foster reuse
    

    View Slide

43. If you were not able to foster reuse with the means you have,
    what makes you think a new architectural paradigm will?
    

    View Slide

44. Reusability suitability Poor
    Great
    Fine-grained
    Stable
    Simple
    Universal
    Technical
    Coarse-grained
    Volatile
    Complicated
    Specific
    Business
    Reusability promoters
    Reusability inhibitors
    Collection
    classes
    Technical
    framework
    Embedded
    HTTP server
    Currency
    Account &
    Transaction
    Ledger
    Order
    handling
    Available through
    programming language
    ecosystems, OSS
    solutions and alike
    Subject of
    “reusability via new
    paradigm”
    discussions
    Could be provided
    via internal
    company-specific
    libraries or alike
    

    View Slide

45. All assets providing good reusability suitability properties
    already exist. Just download (or buy) and use them.
    

    View Slide

46. The assets worth being made reusable
    inside a company scope are usually ignored.
    

    View Slide

47. The new architectural paradigms usually target an area
    where aiming for reusability is not worth the effort
    

    View Slide

48. Conclusion
    You do not need to introduce a new architectural paradigm
    to foster reusability
    

    View Slide

49. Misconception #4
    Distributed paradigms as reusability enabler
    

    View Slide

50. “To foster reusability, we need to introduce
    ”
    -- The unknown architect
    

    View Slide

51. The long version again …
    IT person (typically an architect) to business sponsor:
    “We definitely need to invest in implementing distributed architectural paradigm>. I know this does not
    create any business value now, but it allows us to create
    reusable !
    This way, we will have to pay a lot less for future projects,
    they will be a lot faster and we will be a lot more flexible.
    In short: the investment will pay for itself very soon.”
    Now distributed hell breaks loose …
    

    View Slide

52. Distributed architectural paradigms
    • DCE
    • CORBA
    • EJB / DCOM
    • SOA
    • Microservices
    

    View Slide

53. Distributed systems in a nutshell
    

    View Slide

54. Everything fails, all the time.
    -- Werner Vogels
    

    View Slide

55. If X then Y
    What we learned in our IT education
    If X then maybe Y
    This changes
    everything!
    What we need for distributed systems
    We are good at this (due to how our brains work)
    Inside process thinking
    Reasoning about
    deterministic behavior
    Designing a complicated system
    We are poor at that (due to how our brains work)
    Reasoning about
    non-deterministic behavior
    Across process thinking
    Designing a complex system
    

    View Slide

56. Yet, we usually use deterministic thinking
    to reason about distributed systems
    

    View Slide

57. Failures in distributed systems ...
    • Crash failure
    • Omission failure
    • Timing failure
    • Response failure
    • Byzantine failure
    

    View Slide

58. ... lead to a variety of effects …
    • Lost messages
    • Incomplete messages
    • Duplicate messages
    • Distorted messages
    • Delayed messages
    • Out-of-order message arrival
    • Partial, out-of-sync local memory
    • ...
    

    View Slide

59. ... turning seemingly simple issues into very hard ones
    Time & Ordering
    Leslie Lamport
    "Time, clocks, and the
    ordering of events in
    distributed systems"
    Consensus
    Leslie Lamport
    ”The part-time
    parliament”
    (Paxos)
    CAP
    Eric A. Brewer
    "Towards robust
    distributed systems"
    Faulty processes
    Leslie Lamport,
    Robert Shostak,
    Marshall Pease
    "The Byzantine
    generals problem"
    Consensus
    Michael J. Fischer,
    Nancy A. Lynch,
    Michael S. Paterson
    "Impossibility of
    distributed consensus
    with one faulty
    process” (FLP)
    Impossibility
    Nancy A. Lynch
    ”A hundred
    impossibility proofs
    for distributed
    computing"
    

    View Slide

60. These effects are based on the non-determinism
    introduced by remote communication
    due to distributed failure modes
    

    View Slide

61. Understand that remote communication points are
    predetermined breaking points of your application
    

    View Slide

62. To minimize the non-deterministic effects of distributed systems
    you need to minimize coupling between distributed parts
    

    View Slide

63. The coupling dilemma
    • Reusability is tightest form of coupling possible
    • If a reused part fails, reusing parts fail, too
    • Availability requires avoiding cascading failures
    • If a part fails, the failure must not spread to other parts
    • Reusability means cascading failures by definition
    • Due to the tight coupling between reused and reusing part
    • Technical measures cannot avoid it
    • The tight coupling is at the functional level
    

    View Slide

64. Conclusion
    Aiming for reusability in a distributed system compromises
    system availability and response times by design
    

    View Slide

65. The value of reuse unfolds inside a process boundary
    Across process boundaries it should be avoided as it maximizes coupling
    

    View Slide

66. Misconception #5
    Use means reuse
    

    View Slide

67. “But every time I call a remote service,
    I reuse that service”
    -- The unknown software engineer
    

    View Slide

68. Well, it depends …
    

    View Slide

69. Reusable asset Usable asset
    Asset
    Problem
    Solution
    Composition
    • Asset is part of the solution
    • Solution cannot work without
    the asset
    Collaboration
    • Asset is not part of the solution
    • Solution can work without the
    asset (if designed right)
    Asset
    Problem
    Solution
    

    View Slide

70. Usability is typically a result of horizontal decomposition,
    reusability a result of vertical decomposition
    

    View Slide

71. 
    Functionality
    vertical decomposition
    (layer design, composition)
    In practice, you typically
    use a combination
    of both approaches
    Core functional decomposition approaches
    horizontal decomposition
    (pillar design, segregation)
    

    View Slide

72. Vertical decomposition
    • Based on functional decomposition
    • Typical drivers are reuse and avoidance of redundancy
    • Creates strong coupling (high functional dependence)
    • Often useful pattern inside a process boundary
    • Due to deterministic communication behavior
    • Problematic across process boundaries
    à Should be avoided in distributed design
    

    View Slide

73. Horizontal decomposition
    • Based on functional segregation
    • Typical drivers are autonomy and independence
    • Creates low coupling (high functional independence)
    • Useful pattern across process boundaries
    • Can also be useful inside a process boundary
    • Less accidental "ripple" effects due to changes
    à Should be preferred in distributed design
    

    View Slide

74. Watch out!
    • Vertical decomposition is our default design approach
    • We’ve learned it in our CS education (divide and conquer, ...)
    • It’s emphasized in our daily work (DRY, reusability, ...)
    • Even our IDEs support it (“Extract method”)
    • It's everywhere! It's predominant!
    • It takes energy not to apply vertical decomposition
    • Most people never learned horizontal decomposition
    

    View Slide

75. While the distinction between “use” and “reuse” seems to be
    irrelevant at first sight, mastering it is essential in practice
    

    View Slide

76. Conclusion
    Aim for usability across process boundaries
    (collaborating, functionally independent parts) *
    * and learn the associated design approaches
    (see, e.g. https://speakerdeck.com/ufried/getting-service-design-right)
    

    View Slide

77. Doing it better
    Recommendations
    

    View Slide

78. When not to go for reusability
    

    View Slide

79. Advice #1
    Do not use reusability to justify the introduction
    of a new architectural paradigm
    

    View Slide

80. Reusability as business case
    • Neither works nor makes sense
    • Modularization is all you need, exists for more than 60 years
    • Usually leads to bad designs
    • Many tight couplings
    • Especially in conjunction with distributed paradigms
    • Bad for all people affected
    • Leads to cost savings pressure (the amortization promise)
    • Does not work as promised
    • Increased pressure – reinforces problems
    

    View Slide

81. Advice #2
    Be wary of reusability if the problem you try to solve
    does not exhibit good reusability suitability properties
    

    View Slide

82. What to do instead
    

    View Slide

83. Advice #3
    At a business case level, better argue with benefits like
    "increased business agility" or
    "better technology obsolescence management” *
    * especially if you want to introduce a a distributed modularization paradigm like microservices
    

    View Slide

84. Corollary to advice #3
    Do not try to introduce a distributed modularization paradigm
    like microservices if cost reduction is your sole driver
    

    View Slide

85. Advice #4
    In distributed systems prefer "usability" (collaboration)
    over "reusability" (composition)
    

    View Slide

86. Avoid dogmatism
    

    View Slide

87. Corollary to advices #2 and #4
    Design advices are heuristics, not a dogma
    They are not a replacement for thinking
    

    View Slide

88. Going for reusability
    

    View Slide

89. Advice #5
    Try to use existing reusable assets to increase your productivity
    Do not try to reinvent the wheel
    

    View Slide

90. Advice #6
    If you solve a problem that has good reusability suitability properties
    and is needed in many places, consider creating a reusable asset *
    * if you can afford to pay the extra costs
    

    View Slide

91. Bonus advice
    Think about reusability in other areas of software engineering,
    e.g., on the level of ideas
    

    View Slide

92. Moving on
    

    View Slide

93. This was not an argument against reusability
    

    View Slide

94. This was an argument against using reusability
    in the wrong places and for the wrong reasons
    

    View Slide

95. Use reusability to continuously improve productivity
    

    View Slide

96. Problem
    3 GL development
    base level
    Self-development
    base level
    New reusable
    assets
    created
    Reusability improves
    productivity and
    the scope of solvable
    problems over time
    Problem
    3 GL development
    base level
    Self-development
    base level
    Recurring
    solution parts
    emerge
    Problem
    3 GL development
    base level
    Reusable
    assets
    Self-developed
    solution part
    Self-development
    base level
    3 GL development
    base level
    Self-development
    base level
    Less self-
    development
    needed
    Problem
    Self-developed
    solution part
    Reusable
    assets
    

    View Slide

97. Wrap-up
    

    View Slide

98. Wrap-up
    • Do not use reusability as sales argument for new paradigms
    • All you need for reusability is modularization
    • Avoid reusability across process boundaries
    • Learn the difference between “use” and “reuse”
    • Be willing to pay the price for creating reusable assets
    • But first consider reusability suitability
    • Use reusability as productivity driver
    • Reuse has a huge value used in the right places
    

    View Slide

99. References
    [Bro 1986] Frederick P. Brooks, jr., “No silver bullet”, from H.-J.
    Kugler (ed.), "Information Processing 1986: World Congress
    Proceedings (IFIP congress series, vol 10)", 1986
    [Bro 1995] Frederick P. Brooks, jr., “The tar pit”, from “The
    mythical man-month”, anniversary edition 1995
    [Ste 1974] W. P. Stevens, G. J. Myers, and L. L. Constantine,
    “Structured design”, IBM Systems Journal, Vol 13, No. 2, 1974
    

    View Slide

100. Uwe Friedrichsen
     Works @ codecentric
     https://twitter.com/ufried
     https://www.speakerdeck.com/ufried
     https://ufried.com/blog/reusability_fallacy_1/
     

     View Slide