Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Architecture,
Centralization, Autonomy

Stefan Tilkov
September 16, 2019

Architecture,
Centralization, Autonomy

A presentation about the pros and cons of autonomous organization in software projects, its effects on architecture, the reasons it sometimes fails, and thoughts on what to do about it.

Stefan Tilkov

September 16, 2019
Tweet

More Decks by Stefan Tilkov

Other Decks in Technology

Transcript

  1. Architecture,

    Centralization,
    Autonomy
    Stefan Tilkov

    [email protected]
    @stilkov
    Software Architecture Summit, 2019

    View full-size slide

  2. @stilkov
    (Software) Architecture Definitions
    A system’s elements, their
    relationships, and the rules
    and principles that govern
    their design and evolution Whatever the architect
    considers important enough
    to merit their attention
    Decisions that you want
    to be correct because they
    are costly to change

    View full-size slide

  3. @stilkov
    Order
    Management
    Production
    Planning
    Billing Production Fulfillment
    Domain architecture

    View full-size slide

  4. @stilkov
    Macro (technical) architecture

    View full-size slide

  5. @stilkov
    JRuby C#
    Scala
    Groovy

    Java
    Clojure

    View full-size slide

  6. @stilkov
    RDBMS
    NoSQL
    K/V
    RDBMS RDBMS/DWH
    NoSQL

    DocDB

    View full-size slide

  7. @stilkov
    RDBMS
    NoSQL
    K/V
    RDBMS RDBMS/DWH
    NoSQL

    DocDB
    Micro architecture

    View full-size slide

  8. @stilkov
    Pattern: Autonomous Cells
    Stakeholder
    Stakeholder
    Stakeholder
    Biz
    Dev
    Ops
    Biz
    Dev
    Ops
    Biz
    Dev
    Ops

    View full-size slide

  9. @stilkov
    Pattern: Autonomous Cells
    Stakeholder
    Stakeholder
    Stakeholder
    Biz
    Dev
    Ops
    Biz
    Dev
    Ops
    Biz
    Dev
    Ops

    View full-size slide

  10. @stilkov
    Why you should centralize everything

    View full-size slide

  11. @stilkov
    Why you should centralize nothing at all

    View full-size slide

  12. @stilkov
    Why autonomous teams rule

    View full-size slide

  13. @stilkov
    Why autonomous teams fail

    View full-size slide

  14. @stilkov
    If your goal is to support autonomous teams,
    architecture is an essential ingredient

    View full-size slide

  15. @stilkov
    Just as it is gravely wrong to take from individuals
    what they can accomplish by their own initiative
    and industry and give it to the community, so also
    it is an injustice and at the same time a grave evil
    and disturbance of right order to assign to a
    greater and higher association what lesser and
    subordinate organizations can do. […]

    The supreme authority of the State ought,
    therefore, to let subordinate groups handle
    matters and concerns of lesser importance, which
    would otherwise dissipate its efforts greatly.
    Thereby the State will more freely, powerfully, and
    effectively do all those things that belong to it
    alone because it alone can do them: directing,
    watching, urging, restraining, as occasion requires
    and necessity demands.
    Subsidiarity
    Pope Pius XI, Encyclical Quadragesimo anno, 1931
    Autonomy
    Centralization

    View full-size slide

  16. Pattern: Regulated Market

    View full-size slide

  17. @stilkov
    Context:
    • …
    Observation(s):
    • …
    Lesson(s) learned:
    • …

    View full-size slide

  18. @stilkov
    Context:
    • E-Commerce/Online shop (Retail)
    • 100-120 developers, ~10 teams
    Observation(s):
    • Lack of front-end expertise led to central UI/design team,
    bottleneck for development, deployment, operations, evolution
    Lesson(s) learned:
    • Local optimization needs can trigger centralization
    • Full stack teams require full stack capabilities

    View full-size slide

  19. @stilkov
    A general lack of specific skills, combined with a
    select few who have it, will sabotage any
    attempt at decentralizing anything requiring it

    View full-size slide

  20. @stilkov
    Context:
    • E-Commerce/Online shop (Retail)
    • 100-120 developers, ~10 teams
    Observation(s):
    • Extremely inefficient UI integration runtime due to lack of
    standardization
    • Vast differences in API style, formats, documentation
    Lesson(s) learned:
    • Complete lack of guidance creates unproductive diversity

    View full-size slide

  21. @stilkov
    You cannot decide to not have an architecture;
    if you don’t actively create it, be prepared to
    deal with the one that emerges

    View full-size slide

  22. @stilkov
    There’s a fine line between diversity (that adds
    value) and chaos (that doesn’t)

    View full-size slide

  23. @stilkov
    Context:
    • Insurance customer portal
    • 10-15 developers, 1 team
    Observation(s):
    • Potential for independent decisions in separated systems
    (almost) never exploited
    • Engineering effort spent on coordination
    Lesson(s) learned:
    • Premature modularization can lead to increased effort without
    matching benefits

    View full-size slide

  24. https://en.wikipedia.org/wiki/Amdahl's_law#/media/File:AmdahlsLaw.svg

    View full-size slide

  25. @stilkov
    Amdahl’s law for teams
    • Threshold set by non-parallelizable part of work
    • Adding more teams will not help you if you’ve reached
    the threshold

    View full-size slide

  26. @stilkov
    Law of diminishing returns
    • Coordination effort increases with # of people/teams
    • Returns from re-use possibly far outweighed by extra
    effort

    View full-size slide

  27. @stilkov
    Context:
    • E-Commerce/Online shop (Retail)
    • 100-120 developers, ~10 teams
    Observation(s):
    • Common standard micro architecture at start of project
    • Gradual increase in degrees of freedom
    • Increase in actual diversity of tools, languages, architecture
    Lesson(s) learned:
    • Increased maturity allows for less dogma/fewer rules

    View full-size slide

  28. @stilkov
    Start with a common internal (micro)
    architecture, but allow for separate evolution
    according to specific needs

    View full-size slide

  29. @stilkov
    Pattern: Marketing-based Governance

    View full-size slide

  30. @stilkov
    Context:
    • Global logistics company
    • m projects, n teams
    Observation(s):
    • Inside-out development of rich, multi-faceted, highly functional
    platform, sophisticated tool support for developing platform
    applications
    • Teams resist perceived proprietary, complex, useless platform
    • Ultimate decommissioning of platform after MM€ investment
    Lesson(s) learned:
    • Platform development as high risk activity

    View full-size slide

  31. @stilkov
    It’s difficult to get a man to
    understand something when his
    salary depends on his not
    understanding it.
    Change Resistance
    Upton Sinclair, 1934

    View full-size slide

  32. @stilkov
    Sunk Cost Fallacy

    View full-size slide

  33. @stilkov
    Eating your own dog food is an excellent idea.

    If you’re a dog.

    View full-size slide

  34. @stilkov
    Context:
    • Company-wide digitization effort
    • 150-300 developers, 10-15 teams
    Observation(s):
    • Common standard platform and team to support other teams
    • Standardized CI/CD pipeline & runtime platform
    • Severe inefficiencies due to one-size-fits-all platform (esp. DB)
    • Continuous fighting between teams and platform engineering
    Lesson(s) learned:
    • Platform teams can take on a significant life of their own

    View full-size slide

  35. @stilkov
    Closed organizational systems will do everything
    they can to maintain themselves

    View full-size slide

  36. @stilkov
    Closing your system to external influences is a
    great way to ensure it will suck, eventually

    View full-size slide

  37. @stilkov
    Context:
    • E-Commerce marketplace
    • 25-75 developers, 5-10 teams
    Observation(s):
    • Strategic decision to outsource platform to external party
    (public cloud provider)
    • 100% “all-in” strategy (no worries about vendor lock-in)
    Lesson(s) learned:
    • Significantly decreased emotional attachment to platform
    • Underestimated need for platform expertise

    View full-size slide

  38. @stilkov
    Don’t fall in love with your own tools or libraries,
    maintain a strictly professional relationship

    View full-size slide

  39. @stilkov
    Dreyfus model of skill acquisition
    Novice Advanced
    Beginner
    Competence Proficient Expert
    Recollection
    Non-
    Situational
    Situational Situational Situational Situational
    Recognition Decomposed Decomposed Holistic Holistic Holistic
    Decision Analytical Analytical Analytical Intuitive Intuitive
    Awareness Monitoring Monitoring Monitoring Monitoring Absorbed
    Quality
    Stage

    View full-size slide

  40. @stilkov
    The more experienced you are at (active and
    passive) architectural governance, the less you
    can do of it

    View full-size slide

  41. @stilkov
    Growing architectural maturity means less
    guidance and rules are needed

    View full-size slide

  42. @stilkov
    1.
    Autonomy is the goal
    (unless you waste effort without benefit)

    View full-size slide

  43. @stilkov
    2.
    Control is tempting
    (unless you’re the one being controlled)

    View full-size slide

  44. @stilkov
    3.
    Letting go is the hardest part
    (unless everyone sees benefits)

    View full-size slide

  45. @stilkov
    4.
    Decentralization must be managed
    (to the degree that’s needed to keep it)

    View full-size slide

  46. @stilkov
    5.
    Standardization helps
    (if it’s only mandatory as an exception)

    View full-size slide

  47. Stefan Tilkov
    @stilkov

    [email protected]
    Phone: +49 170 471 2625
    innoQ Deutschland GmbH
    Krischerstr. 100
    40789 Monheim am Rhein
    Germany
    Phone: +49 2173 3366-0
    innoQ Schweiz GmbH
    Gewerbestr. 11
    CH-6330 Cham
    Switzerland
    Phone: +41 41 743 0116
    www.innoq.com
    Ohlauer Straße 43
    10999 Berlin
    Germany
    Phone: +49 2173 3366-0
    Ludwigstr. 180E
    63067 Offenbach
    Germany
    Phone: +49 2173 3366-0
    Kreuzstraße 16

    80331 München
    Germany
    Phone: +49 2173 3366-0
    @stilkov
    That’s all I have.

    Thanks for listening!

    View full-size slide

  48. @stilkov
    www.innoq.com
    OFFICES
    Monheim
    Berlin
    Offenbach
    Munich
    Hamburg
    Zurich
    FACTS
    ~150 employees
    Privately owned
    Vendor-independent
    SERVICES
    Strategy & technology consulting
    Digital business models
    Software architecture & development
    Digital platforms & infrastructures
    Knowledge transfer, coaching & trainings
    CLIENTS
    Finance
    Telecommunications
    Logistics
    E-commerce
    Fortune 500
    SMBs
    Startups

    View full-size slide