Tweet
Tweet

Slide 1

Slide 1 text

Fighting Software Entropy December 2022 Stewart Gleadow (Exec Mgr, Engineering) Alison Rosewarne (Exec Mgr, Architecture) Image credit: http://www.thenegativepsychologist.com/

Slide 2

Slide 2 text

Alison Rosewarne Stewart Gleadow Hi! 👋 We’ve been at this awhile…

Slide 3

Slide 3 text

The life of successful software

Slide 4

Slide 4 text

TTL (Time To Legacy) The life of successful software

Slide 5

Slide 5 text

Physics Lesson: What is entropy? Entropy, as described in the 2nd Law of Thermodynamics • The measure of disorder of a system. Characteristics of entropy: • Systems inherently move to disorder. • The more disorder that is present, the less energy is available to do work. Rudolf Clausius

Slide 6

Slide 6 text

Source: Aaron (unsplash.com) Structured systems have lower entropy Photo by Aaron on Unsplash

Slide 7

Slide 7 text

YOW! Lesson: What is software entropy? Entropy, as described by Stew and Alison • The measure of disorder in software. Characteristics of software entropy: • Software inherently moves to disorder. • The more disorder that is present, the less energy is available to do work. Stewart Gleadow Alison Rosewarne Photo by Markus Spiske on Unsplash

Slide 8

Slide 8 text

Structured software has lower entropy "...high-performing teams were more likely to have loosely coupled architectures.” Source: State of DevOps Report, 2017. It’s our software architecture that defines the modularity and interaction between systems.

Slide 9

Slide 9 text

Why does software decay? Jaye Haych (unsplash.com) Compromises are made Shared understanding is lost Dependencies date Approaches change

Slide 10

Slide 10 text

Confirmed by IBM researchers Meir M Lehman & Laszlo Belady [Some of] The laws of software evolution Continuing Change Increasing Complexity Declining Quality Continuing Growth

Slide 11

Slide 11 text

“With most software systems, it becomes harder to add new features over time.” Martin Fowler, 2019 Software entropy affects internal quality Clean system Time to market Crufty system Time to market

Slide 12

Slide 12 text

Do you agree with the following statement? “I would like to tear up all of our organization’s core systems.” Why should you care about deteriorating structure and quality? Source: Digital Decoupling: U.S. Federal Survey Results, Accenture, 2018. 81% 17%

Slide 13

Slide 13 text

2005 2010 2015 2020 More software increases entropy Shifted to an internal platform strategy Adopting microservices Company really started scaling Number of systems

Slide 14

Slide 14 text

Varying technologies increases entropy

Slide 15

Slide 15 text

Decreasing entropy is possible … in open systems Isolated System Matter exchange Open System Matter exchange Energy exchange

Slide 16

Slide 16 text

How can we fight software entropy? 1. Define the structure 2. Add the energy Photo by Jaime Spaniol on Unsplash

Slide 17

Slide 17 text

Architecture practice Engineering excellence Continuous improvement Adopting platforms Photo by Daiga Ellaby on Unsplash

Slide 18

Slide 18 text

arch-i-tec-ture n the design and structure of a computer system Photo by Heather McKean on Unsplash

Slide 19

Slide 19 text

Principle Name What – a short explanation Why – articulates the value In Practice • Examples of how this shows up • Aiming to simplify adoption

Slide 20

Slide 20 text

Constant cleaning is part of work well done. Incremental improvement, no matter how small, adds up. This is a healthy habit. • Add or improve documentation. • Fix static analysis violations. • Increase test coverage. • Tune monitoring. Keep it clean

Slide 21

Slide 21 text

Our architectural principles Adopt the platform Make it approachable Keep it clean Deploy continuously Build security in Manage your data Map the business domain

Slide 22

Slide 22 text

Other mechanisms to improve architecture Jaye Haych (unsplash.com) Technology strategy Sensible defaults Decision making frameworks Knowledge management

Slide 23

Slide 23 text

“People who have no choice are generally unhappy. But people with too many choices are almost as unhappy as those who have no choice at all.” - Ellen Ullman

Slide 24

Slide 24 text

Support freedom from choice with an internal tech radar.

Slide 25

Slide 25 text

No content

Slide 26

Slide 26 text

We also need to add energy to our architecture Photo by Joshua Gaunt on Unsplash

Slide 27

Slide 27 text

Architects from across our business work together with dedicated capacity Chair (EM, Arch) Sponsor (CTO) Other reps Group B Member (Architect) Team Team Group C Member (Architect) Team Team Group A Member (Architect) Team Team Group F Member (Architect) Team Team Group E Member (Architect) Team Team Group D Member (Architect) Team Team

Slide 28

Slide 28 text

Architectural alignment decreases software entropy Principles increase consistency of decisions Freedom from choice reduces cognitive load Dedicated time is required to drive outcomes Key takeaways Photo by Rudolf-Peter Bakker on Unsplash

Slide 29

Slide 29 text

Architecture practice Engineering excellence Continuous improvement Adopting platforms Photo by Daiga Ellaby on Unsplash

Slide 30

Slide 30 text

What even is good software? Photo by Kelli McClintock on Unsplash

Slide 31

Slide 31 text

First, we started with opinions

Slide 32

Slide 32 text

Radiating information Visualisation was powerful Changing the language from tech debt to system health was important

Slide 33

Slide 33 text

Remove ambiguity to be really clear about the state you are after Photo by Phil Hearing on Unsplash

Slide 34

Slide 34 text

Development Can I confidently and safely make changes? Operations Can I support the system and confirm acceptable production behaviour? Architecture Is the design extensible and fit for purpose? Photo by Maxim Abramov on Unsplash

Slide 35

Slide 35 text

(START HERE) (START HERE) (START HERE) DEVELOPMENT OPERATIONS ARCHITECTURE Adding structure to our expectations

Slide 36

Slide 36 text

Supported Language (START HERE) Containerised Up-to-date libraries Continuous Integration System Recovery Plan Security Controls Actionable Alerts Adequate Logging Support documentation Infrastructure As Code Clear Data Ownership Documented Architecture Well-defined Interface (START HERE) (START HERE) DEVELOPMENT OPERATIONS ARCHITECTURE

Slide 37

Slide 37 text

Endorsed Language Multiple Contributors Multiple Deployments Continuous Deployment Static Analysis Supported Language (START HERE) Containerised Up-to-date libraries Continuous Integration System Recovery Plan Security Controls Actionable Alerts Adequate Logging Support documentation Infrastructure As Code Cost Optimised System Recovery Tested SLIs/SLOs Clear Data Ownership Documented Architecture Well-defined Interface Platform Aligned Documented Decisions Single Responsibility (START HERE) (START HERE) DEVELOPMENT OPERATIONS ARCHITECTURE

Slide 38

Slide 38 text

Don’t forget to add a little energy… Photo by Ussama Azam on Unsplash

Slide 39

Slide 39 text

Rating workflow Rate systems g3 .treasuremap-data.json PR Data publication Data visualisation TreasureMap Raw data Insights Team

Slide 40

Slide 40 text

All anyone asks for is a chance to work with pride. - W. Edwards Deming “All anyone asks for is a chance to work with pride.” - W. Edwards Deming

Slide 41

Slide 41 text

Agree what engineering excellence means to you Radiate information and build shared understanding Use objective measurements as a baseline for improvement Build a culture of engineering excellence Key takeaways Photo by Rudolf-Peter Bakker on Unsplash

Slide 42

Slide 42 text

Architecture practice Engineering excellence Continuous improvement Adopting platforms Photo by Daiga Ellaby on Unsplash

Slide 43

Slide 43 text

0 years 5 years 10 years The passage of time increases entropy Number of systems Most of our software is 1 to 7 years old

Slide 44

Slide 44 text

“The deal with engineering goes like this. Product management takes 20% of the capacity right off the table and gives this to engineering to spend as they see fit” “If you’re in really bad shape today, you might need to make this 30% or even more” Marty Cagan, 2007

Slide 45

Slide 45 text

Custodianship: putting the energy in 1. Reducing risk 2. Paying down tech debt 3. Lowering total cost of ownership Rudolf Clausius 20% - Custodianship

Slide 46

Slide 46 text

Adding structure to custodianship

Slide 47

Slide 47 text

Analysis of health data guides custodianship Provide automation pathway

Slide 48

Slide 48 text

Sample outcome of SHIP work Provide automation pathway

Slide 49

Slide 49 text

Investing in automation is highly recommended Renovate Automated dependency updates Buildkite Scheduled CI/CD builds

Slide 50

Slide 50 text

Company level OKR: Improve System Health by 40% Provide automation pathway

Slide 51

Slide 51 text

Provide automation pathway Systems Health Improvement Plan introduced

Slide 52

Slide 52 text

There is a constant minimum investment required to fight software entropy We default to 20% of a team’s capacity and plan this carefully Automation allows you fight entropy with much less ongoing effort Broadcast system health to get buy in at all levels Key takeaways Photo by Rudolf-Peter Bakker on Unsplash

Slide 53

Slide 53 text

Architecture practice Engineering excellence Continuous improvement Adopting platforms Photo by Daiga Ellaby on Unsplash

Slide 54

Slide 54 text

How do we make it easier to fight software entropy?

Slide 55

Slide 55 text

1. A mindset shift from local to company-wide tech thinking. 2. Approaching tech platforms as long-lived products.

Slide 56

Slide 56 text

Data Identity Deploy Design Visualisation Finance Security

Slide 57

Slide 57 text

2000 2005 2010 2015 2020 Our platform strategy at work

Slide 58

Slide 58 text

Photo by Mourizal Zativa on Unsplash The less software you have to maintain, the lower the entropy.

Slide 59

Slide 59 text

Our web platform, Argonaut http://realestate.com.au/?mfeTag=dev:yow-demo

Slide 60

Slide 60 text

Jaye Haych (unsplash.com) 35% less code 90% lower custodianship 75% less developer effort 50% time to market Do our platforms help us to reduce the total entropy in our systems? 90% lower custodianship

Slide 61

Slide 61 text

Programming Language Multiple Contributors Multiple Deployments Continuous Deployment Linting Programming Language … Containerised Libraries & Frameworks Continuous Integration … System Recovery Plan Security Controls Actionable Alerts Adequate Logging Support 1pager Infrastructure As Code Cost Optimised Disaster Recovery Tested SLIs/SLOs … Clear Data Ownership Documented Architecture Well-defined Interface Platform Aligned Documented Decisions Single Responsibility Platforms make maintaining healthy software easier

Slide 62

Slide 62 text

A platform strategy helps the long-term fight against software entropy Platforms reduce the amount of software you write Platforms reduce the amount of software you maintain Platforms increase overall consistency and structure Key takeaways Photo by Rudolf-Peter Bakker on Unsplash

Slide 63

Slide 63 text

Architecture practice Engineering excellence Continuous improvement Adopting platforms Photo by Daiga Ellaby on Unsplash

Slide 64

Slide 64 text

Software entropy is a very real and present issue in our industry. Ignore it at your peril. REA fights software entropy with: • Architectural alignment • A culture of engineering excellence • Dedicated capacity for teams and architects • A platform strategy reducing bespoke software By getting the structures in place and the right energy investment, you too can fight software entropy. Key takeaways Photo by Rudolf-Peter Bakker on Unsplash

Slide 65

Slide 65 text

Fighting Software Entropy December 2022 Stewart Gleadow (Exec Mgr, Engineering) Alison Rosewarne (Exec Mgr, Architecture) Image credit: http://www.thenegativepsychologist.com/