Slide 1

Slide 1 text

Deferring the Last Responsible Moment Eoin Woods, Endava Join the conversation on Twitter: @SoftArchConf #SoftwareArchitect2015 20151013.2

Slide 2

Slide 2 text

2 2 Introductions Eoin Woods • CTO at Endava • Career has spanned products and applications • Architecture and software engineering • Bull, Sybase, InterTrust • BGI (Barclays) and UBS • Endava • Constantly fighting tendency to make decisions too early!

Slide 3

Slide 3 text

3 3 Content Introducing the Last Responsible Moment Introducing Real Options Balancing Early and Late Decision Making Tactics to Allow Decisions to be Delayed Summary

Slide 4

Slide 4 text

Introducing the Last Responsible Moment

Slide 5

Slide 5 text

5 5 Roots in Lean Thinking Lean working systematically avoids waste • Initially applied to manufacturing • Honda and Toyota applied it to product development • In the 1990s “Lean Construction Institute” formed • Applied to software from early 2000s • Mary and Tom Poppendieck • Fits well with many of Agile’s principles

Slide 6

Slide 6 text

6 6 Key Principles for Lean Working Key principles 1. Eliminate waste 2. Amplify learning 3. Decide as late as possible 4. Deliver as fast as possible 5. Empower the team 6. Build quality in 7. See the whole

Slide 7

Slide 7 text

7 7 The Last Responsible Moment A term coined by Lean Construction authors in ~2000 • LCI’s white paper series by Howell, Ballard & Zabell A single conceptual design will normally be selected before the end of [the lean design] phase because the last responsible moment for making that decision will have usually passed. Design decisions will be deferred until the last responsible moment if doing so offers an opportunity to increase customer value.

Slide 8

Slide 8 text

8 8 The Last Responsible Moment Kenneth Rubin’s definition for software development • Essential Scrum, 2012 A strategy of not making a premature decision but instead delaying commitment and keeping important and irreversible decisions open until the cost of not making a decision becomes greater than the cost of making a decision .

Slide 9

Slide 9 text

9 9 Difficulties with the Last Responsible Moment The problem is … simple to say, difficult to use • Actually spotting “the moment” is difficult • Very easy to realise it passed some time ago • How do you know when the cost of deferring is greater than the cost of committing? • Usually a period rather than a “moment” For example, you want to decide which JS UI framework to use … when is the last moment for this decision?

Slide 10

Slide 10 text

Introducing Real Options

Slide 11

Slide 11 text

11 11 Real Options An extended model for LRM is “Real Options” • Based on the idea of a financial option • Defined by Chris Matts and Olav Maassen in ~2007 A real option – “never commit without knowing why” • An option but not an obligation for a future decision • Has a “value” • Has an “expiry” (date or other condition) • Has a “cost to buy” and a “cost to exercise” Introduction from Matts and Maassen: http://www.infoq.com/articles/real-options-enhance-agility

Slide 12

Slide 12 text

12 12 Defining a Real Option The constituent of a Real Option are: • Value – what benefit will exercising this option have for the organisation? Could be time, money, opportunity, … • Expiry – what conditions mean that the option can no longer be exercised? A project date or when something happens (e.g. remaining budget falls to a point) • Cost to Buy – what do you need to do in order to have the option? PoCs? Research? Analysis? Design change? • Cost to Exercise – once you commit to this option, what is the cost to achieve it? Development work? Changes in Operations? New testing? Additional analysis? Licenses?

Slide 13

Slide 13 text

13 13 An Example of a Real Option Suppose we need an option on a database … The Option than a To use a document oriented database (rather relational database) The Value Reduced development cost (~15%) if database model matches problem domain The Expiry Time of first production release less time taken to replace DB access layer & deploy chosen database Purchase Cost A DB access layer (mocked) plus research to prove that both dbs could be deployed Exercise Cost db Write DB access layer, design operational environment, deploy db

Slide 14

Slide 14 text

14 14 Example of a Real Option (ii) Time Cost/ Value R0 DB Deploy Time Data Access Layer Build Time Value Cost Expiry Benefit T0

Slide 15

Slide 15 text

15 15 Using Real Options To create and manage a Real Option 1. Identify your options for a decision 2. Identify the conditions defining the last decision point • e.g. decision point = deadline - implementation time • e.g. decision point = resource0 < Value1 3. Keep learning and looking for options until decision point 4. Move back decision point where possible 5. When decision point arrives, act as quickly as possible • Real Options ≠ procrastination! (More on moving back the decision point in the next section.)

Slide 16

Slide 16 text

16 16 How Real Options Help with Decision Making The value of Real Options • Real Options overcome natural tendency to decide early • Makes a “no decision” position clear • Requires precision about when a decision is needed • Overcomes aversion to uncertainty • Real Options encourage thinking about alternatives • Real Options force clarity in the options available • Real Options encourage gathering information Combination of factors create likelihood of better decisions

Slide 17

Slide 17 text

Balancing Early and Late Decision Making

Slide 18

Slide 18 text

18 18 Balancing Late vs Early Decisions Not every decision can (or should) be delayed • Like any idea deferring decisions can be over used • Little benefit in deferring decisions with low impact • Decisions that can be changed at low cost • Decisions that have little tangible impact on the outcome • Decisions that aren’t going to affect customer value • Making no decisions early will just stall progress • Many people feel reassured when a decision is made • “any decision is better than no decision” • Real Options helps decision making to be more transparent • Lack of a decision may block others from progressing

Slide 19

Slide 19 text

19 Early Decisions • Harder to validate due to less information • Have a tendency to optimise unwisely • Have a tendency to be wasteful (YAGNI) • Often look decisive • Can be (falsely?) reassuring Late Decisions • Maximise information available for decision • Avoid premature optimisation problems • Can block others from progressing • Can overwhelm decision making if too many • Can cause worry about lack of direction Late vs Early Decisions Remember need for communication, clarity and reasoning when making late decisions to avoid looking indecisive or unsettling people – psychological factors

Slide 20

Slide 20 text

Tactics to Defer Decisions

Slide 21

Slide 21 text

21 21 Tactics to Defer Decisions (Re)Organise for Change • Run projects that can absorb change as late as possible Architect for Change • Design systems that can be changed regularly and reliably Design for Change • Implement software that limits the impact of change

Slide 22

Slide 22 text

22 22 (Re)Organise for Change Organise project to absorb change and allow late decisions • Domain Analysis • Know the domain to identify the options for late decisions • Early Sharing of Information • Don’t wait for perfection before validating • Minimum Viable Product • Build the smallest thing possible to gain more knowledge • Set Based Design • Back enough options to allow late selection of the solution

Slide 23

Slide 23 text

23 Benefit • Good domain knowledge identifies alternatives • Domain analysis leads to new options • Collaboration across business and developers Cost • Significant time and effort to develop • Requires specialisation Domain Analysis www.uscis.gov

Slide 24

Slide 24 text

24 Benefit • Allows collaboration to identify options • Early feedback to spot problems early • Clarity on direction Cost • Time for communication Share Information Early

Slide 25

Slide 25 text

25 25 Minimum Viable Product Don’t build anything that isn’t essential • A MVP is the product level version of “do the simplest thing that could possibly work” • The simplest set of features that could meet the requirement • Allows for rapid delivery, cost effective feedback loop and affordable mistakes to be made, so facilitates learning • The key is to decide what the minimum really is

Slide 26

Slide 26 text

26 Benefit • Smallest number of decisions made each time • Options stay open until feature needed Cost • Effort to fit approach into many organisations • Difficult to “sell” without a defined end-state Minimum Viable Product cloudmanic.com

Slide 27

Slide 27 text

27 27 Set Based Design Why choose before the end? • Set based design develops a set of solutions for each problem where the best solution isn’t clear at the start • e.g. a “safe” solution, a “likely” solution & an “innovative” solution • As time progresses stop work on solutions that don’t work well or where their LRM has passed • Overall project assembled from “winning” solutions

Slide 28

Slide 28 text

28 Benefit • Hedges options right through project • Latest decision possible Cost • Duplicated effort • Integration difficult if multiple choices • Can be confusing without careful management Set Based Design (ii)

Slide 29

Slide 29 text

29 29 Architect for Change Design a system that can be changed regularly, reliably • Separate Concerns • Limit items to be changed • Avoid Duplication (DRY) • Limit impact of change • Dependency Management • Know the impact of change • Variation Points • Manage a set of options through the lifecycle • Testing and Continuous Delivery • Enable reliable and rapid change

Slide 30

Slide 30 text

30 Benefit • Avoid duplication • Change isolation • Simpler components • Adds options for reuse & extension Cost • Design effort • Refactoring Separate Concerns (Cohesion)

Slide 31

Slide 31 text

31 Benefit • Avoids wasted effort • Easier change • Efficiency Cost • Effort • Refactoring impact Avoid Duplication (“Don’t Repeat Yourself”) deviq.com/don-t-repeat-yourself

Slide 32

Slide 32 text

32 Benefit • Find change blackspots • Avoid high coupling • Structural insight Cost • Time & prioritisation • Refactoring • Tools Dependency Management (Coupling)

Slide 33

Slide 33 text

33 33 Variation Points Variation Points build options into design thinking • An idea from product line engineering • Design software as a set of assets designed for use in different situations • Explicit variation points in the (functional) design • Makes existence, constraints and cost of options clear • Promotes variability to be a feature of the system • Document & test the variation points for product owner

Slide 34

Slide 34 text

34 34 Variation Points (ii) Example in a payment processing system • Identify explicit options for variation of: • transaction authorisation mechanism • transaction message transport • security mechanisms • Product owner can now decide how and when to invest in these options • Variation points are enablers of a set of Real Options • Key is explicit design and management of the variation point set

Slide 35

Slide 35 text

35 Benefit • First class options in the design • Excellent visibility of options for PO Cost • Up front design effort • Implementation effort • Requires a little clairvoyance (i.e. domain knowledge) Variation Points (iii)

Slide 36

Slide 36 text

36 Benefit • Confident change • Reliable change • Rapid delivery of changes to production Cost • Implementation cost • Initial effort • Customer commitment Test Automation and Continuous Delivery derberg.github.io

Slide 37

Slide 37 text

37 37 Design for Change Design a system that can be changed regularly, reliably • Interfaces and Abstractions • Encourage substitutability • Parameterisation • Create logic that expects to delegate responsibility for details • Encapsulate Variation • Limit the impact of changing a decision

Slide 38

Slide 38 text

38 Benefit • Delay implementation decision point • Substitutability (LSP) allows future options Cost • Complexity • Design effort • Mocking effort • LCD risks Oracle Adapter MSSQL Adapter «interface» Database Adapter Interfaces and Abstractions

Slide 39

Slide 39 text

39 Benefit • Delays implementation decision • Allows build and test of parameterised modules • Allows decision to be changed repeatedly Cost • Complexity • Design effort Parameterisation setScoreCalculator(ScoreCalculator sc) Credit Score Reporting Customer List Rule Based Calculator Model Based Calculator OneR Calculator Bayesian Calculator

Slide 40

Slide 40 text

40 Benefit • Localised change impact • Predictable and reliable change • Avoiding duplication Cost • Vigilance • Refactoring as knowledge improves • Structural complexity (?) Modularise and Encapsulate Variation Payroll Manager «interface» Hourly Rate Calculator Hourly Paid Hourly Rate Calculator Salary Based Hourly Rate Calculator

Slide 41

Slide 41 text

Example

Slide 42

Slide 42 text

42 42 Delaying the Last Responsible Moment An enterprise system for compliance reporting • Receives all the business transactions for the company • Analyses, sorts, classifies the business activity • Contains a configurable rule set for report generation • Sends real time report feeds to some regulators • Generates document reports for other regulators • Provides controls such as “4 eyes” checks on dispatch • Needed “immediately” due to regulatory inspection

Slide 43

Slide 43 text

43 43 Regulatory Reporting System Project organisation to allow late change? Architecture for delaying change? Design to defer the last responsible moment? Regulatory Reporting System Reporting Analysts Supervisors Business Transactions Regulator Activity Reports

Slide 44

Slide 44 text

44 44 Regulatory Reporting System Project organisation ideas: • What is a minimum viable reporting system? • What will the regulator put up with? (Domain knowledge) • Which regulator first? (Domain knowledge) • Develop a simple file generator with manual processes alongside a fully automated system!

Slide 45

Slide 45 text

45 45 Regulatory Reporting System Architecture ideas: • What variation points will we need? • Report type (data included, data items, ….) • Regulator specific calculations for report items (e.g. NPV) • Transport for dispatch of reports • Continuous delivery to improve system every couple of days – allows better negotiation with the regulators • Automated testing of reports to avoid manual test cycle delays • Stress cohesion and low coupling to reuse and replace pieces of the process as the real requirements emerge

Slide 46

Slide 46 text

46 46 Regulatory Reporting System Design ideas: • Encapsulate all of the variation points we identified to allow rapid safe extension • Parameterise the report generation process to allow regulator specific calculations to be “injected”

Slide 47

Slide 47 text

Summary

Slide 48

Slide 48 text

48 48 Summary Decisions need information so making them late helps • The last responsible moment is a difficult concept • Hard to measure and use • Real Options add some important concepts to the idea • Value, cost, expiry • It is often possible to move the “expiry moment” • Organise for Change • Architect for Change • Design for Change

Slide 49

Slide 49 text

49 49 Summary (ii) Moving the moment an option expires • Organise for Change • Domain Analysis, Share information early, MVP, Set-based design • Architect for Change • Separate Concerns, Avoid duplication, Manage dependencies, Variation points, Testing and continuous delivery • Design for Change • Interfaces and abstractions, Parameterisation, Encapsulate variation The tactics aren’t novel but using them to delay commitment may be

Slide 50

Slide 50 text

50 50 Resources http://www.infoq.com/articles/real-options-enhance-agility http://www.agilecoach.net

Slide 51

Slide 51 text

51 Thank you QUALITY. PRODUCTIVITY. INNOVATION. Eoin Woods Endava [email protected] +44 207 367 1000 en_ewoods