Gillian Julius - Crafting An Agile Supplier Contract

How do you design a contract that promotes learning? In an outsourced software development project, the structure of the contract between client and vendor is the fundamental basis for all team interaction and inescapably shapes the nature of collaboration throughout the life of the project. A well-crafted contract means the difference between an on-going battle and an adaptive partnership true to the spirit of Agile. This session will examine the dynamics created by several typical contract structures and discuss an alternate approach to fixed-price piloted at General Electric.

Transcript

1. Crafting An Agile Supplier Contract September 21, 2012 gillian.julius@gmail.com Gillian Julius
2. About me gillian.julius@gmail.com • Computer science • GE Transportation – 8 years Project Manager, financial modeling and analytics for locomotive service • 3 years promoting Agile @ GE • Fan of systems thinking, cooperation and exchange as basis for progress, health & happiness
3. Topics 1) Systems perspective 2) Our experiment
4. Thinking about systems & learning
5. Customer Supplier What happens in a typical* contract when we learn? * fixed price, fixed scope
6. Customer Supplier What happens in a typical* contract when we learn? • New insights on requirements • Realization of risks • Incorrect assumptions • True complexity and effort What do we learn? * fixed price, fixed scope
7. Customer Supplier What happens in a typical* contract when we learn? • New insights on requirements • Realization of risks • Incorrect assumptions • True complexity and effort • Assigning blame & responsibility • Lack of incentive to help resolve issues • Effort spent estimating, negotiating, managing changes – process overhead • Compromise to stay within project commitments – cost, schedule, scope, quality What do we learn? How do we respond? * fixed price, fixed scope
8. Customer Supplier What happens in a typical* contract when we learn? What do we learn? How do we respond? * fixed price, fixed scope Ever wonder why?
9. Opposing incentives discourage learning Customer Supplier Customer wants… Supplier wants… Define scope late Define scope early Change scope Freeze scope Avoid change requests Increase change requests Push risk to supplier, low cost Protect against contingencies, maximize margin Invest minimal resource to support development & issues Customer to provide adequate resource where needed but is powerless to demand this Push responsibility of good engineering to supplier, low maintenance costs Incur technical debt to meet cost and schedule constraints – push maintenance cost to customer Get most value for least cost Get most price for least effort Success measured in terms of business value delivery Success measured as conformance to specification
10. Try again (failure demand!) Wrong Require- ments Wrong Require- ments Organizational effects
11. Deflation & Margin Pressure Deflation & Margin Pressure Try again (failure demand!) Protect margin Pad estimates Reduce transparency Wrong Require- ments Wrong Require- ments Deflation pressure Less experienced people Less convenient locations Share resources / form silos Competition / swap teams Organizational effects
12. Deflation & Margin Pressure Deflation & Margin Pressure Overhead Overhead Try again (failure demand!) Protect margin Pad estimates Reduce transparency Wrong Require- ments Wrong Require- ments Deflation pressure Less experienced people Less convenient locations Share resources / form silos Competition / swap teams More controls Fix / manage cost & schedule Control scope changes Incentives / metrics around conformance to plan More management overhead Incentive to hide risk Warning: Explosive! Organizational effects
13. Deflation & Margin Pressure Deflation & Margin Pressure Overhead Overhead Quality Quality Try again (failure demand!) Protect margin Pad estimates Reduce transparency Long-term Deterioration Quality as release valve High cost of defects Unnecessary complexity Technical debt Wrong Require- ments Wrong Require- ments Deflation pressure Less experienced people Less convenient locations Share resources / form silos Competition / swap teams More controls Fix / manage cost & schedule Control scope changes Incentives / metrics around conformance to plan More management overhead Incentive to hide risk Warning: Explosive! Organizational effects
14. Deflation & Margin Pressure Deflation & Margin Pressure Overhead Overhead Quality Quality Try again (failure demand!) Protect margin Pad estimates Reduce transparency Long-term Deterioration Quality as release valve High cost of defects Unnecessary complexity Technical debt Wrong Require- ments Wrong Require- ments Deflation pressure Less experienced people Less convenient locations Share resources / form silos Competition / swap teams Learning! More retraining More management overhead More quality issues / troubleshooting More technical debt More contention Delayed feedback More controls Fix / manage cost & schedule Control scope changes Incentives / metrics around conformance to plan More management overhead Incentive to hide risk Warning: Explosive! Organizational effects
15. Deflation & Margin Pressure Deflation & Margin Pressure Overhead Overhead Quality Quality Try again (failure demand!) Protect margin Pad estimates Reduce transparency Long-term Deterioration Quality as release valve High cost of defects Unnecessary complexity Technical debt Wrong Require- ments Wrong Require- ments Deflation pressure Less experienced people Less convenient locations Share resources / form silos Competition / swap teams Learning! More retraining More management overhead More quality issues / troubleshooting More technical debt More contention Delayed feedback More controls Fix / manage cost & schedule Control scope changes Incentives / metrics around conformance to plan More management overhead Incentive to hide risk Warning: Explosive! Organizational effects
16. Our hypothesis
17. How do we promote learning? Embrace scope change Global objective Shared risk / reward Insurance policy Need a fundamental mechanism to enable natural change, not a change control process
18. How do we promote learning? Embrace scope change Global objective Shared risk / reward Insurance policy Incentive alignment key to preventing contention Fairness basis for relationship Need a fundamental mechanism to enable natural change, not a change control process
19. How do we promote learning? Embrace scope change Global objective Shared risk / reward Insurance policy Incentive alignment key to preventing contention Fairness basis for relationship Need a fundamental mechanism to enable natural change, not a change control process Avoid sub-optimizations Closed loop adaptation to issues
20. How do we promote learning? Embrace scope change Global objective Shared risk / reward Insurance policy Incentive alignment key to preventing contention Fairness basis for relationship Need a fundamental mechanism to enable natural change, not a change control process Avoid sub-optimizations Closed loop adaptation to issues Changing economics: Lower the cost of mistakes Make action less expensive than inaction
21. Our approach – target scope 1) Set target # of scope points 2) Track real velocity of team 3) Adjust target by 50% of variance
22. Our approach – target scope 1) Set target # of scope points 2) Track real velocity of team 3) Adjust target by 50% of variance Cost Schedule fixed fixed with some flexibility Scope flexible Shared risk / reward
23. How we calibrated Identify initial backlog of stories Group according to similarity of effort
24. How we calibrated 1 2 3 5 8 13 20+ 1 2 2 2 3 3 5 5 5 5 5 8 13 13 Assign natural (Fibonacci) values Recalibrate against this throughout project
25. How we calibrated 1 2 3 5 8 13 20+ 1 2 2 2 3 3 5 5 5 5 5 8 13 13 Sample set defined in detail
26. Ground rules • Customer and supplier each get 50% vote on point weight estimates • In case of irreconcilable disagreement, go with higher estimate • Customer gets to decide when to adjust target • First 2 sprints data points ignored • Customer gets to decide what to work on and final acceptance • Definition of done can change – this weighs in on velocity and adjustment
27. 0 10 20 30 40 50 60 70 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 How our project played out Phase I – Velocity ran high; ended 2 weeks ahead of schedule Phase II – Velocity ran on target / slightly under – no adjustment Phase I Target Phase II Target Phase I Adjustment Actual Trend
28. Behaviours we observed • Pulling together to resolve issues • Quick agreement to move forward • Habit of multitasking & avoiding obstacles still difficult to overcome – but contract promoted good behaviour • Accelerated domain learning • Spirit of experimentation • Creativity to find smaller, simpler, and less risky solutions – willingness to invest more time in finding the best solution
29. Things to watch out for • Inflation / deflation – good idea to keep recalibrating your “currency” against large population of stories • Changes to Definition of Done, unexpected architectural stories – recalibrate against contract intentions & business objectives – don’t let a number become your goal • Natural tendency to “eliminate excess” – don’t let suppliers reassign team members unless exceeding target • Minimize WIP – contract won’t force you to do this (but will reward you for doing so)
30. Evidence of disruption Deflation & Margin Pressure Deflation & Margin Pressure Overhead Overhead Quality Quality Learning! Wrong Require- ments Wrong Require- ments
31. Deflation & Margin Pressure Deflation & Margin Pressure Overhead Overhead Quality Quality Learning! Wrong Require- ments Wrong Require- ments 70% of delivered scope different from initial backlog Evidence of disruption
32. Deflation & Margin Pressure Deflation & Margin Pressure Overhead Overhead Quality Quality Learning! Wrong Require- ments Wrong Require- ments Cost per month: ↑ 53% Cost per feature: ↑ 16% Cost per USEFUL feature: ↓ 37% 70% of delivered scope different from initial backlog Evidence of disruption
33. Deflation & Margin Pressure Deflation & Margin Pressure Overhead Overhead Quality Quality Learning! Wrong Require- ments Wrong Require- ments Cost per month: ↑ 53% Cost per feature: ↑ 16% Cost per USEFUL feature: ↓ 37% No change control processes Very limited task / project tracking Very limited estimation 25% of sprints “missed commitments” 70% of delivered scope different from initial backlog Evidence of disruption All business objectives met On budget 8/9 releases on time and with committed scope Overall project 2 weeks early Avg delivery cycle Ɵme ↓ 72%
34. Deflation & Margin Pressure Deflation & Margin Pressure Overhead Overhead Quality Quality Learning! Wrong Require- ments Wrong Require- ments Cost per month: ↑ 53% Cost per feature: ↑ 16% Cost per USEFUL feature: ↓ 37% PRD defects: ↓ 94% DEV defects: ↓ 86% Avg defect queue: ↓ 92% Avg defect cycle time: ↓ 84% No change control processes Very limited task / project tracking Very limited estimation 25% of sprints “missed commitments” 70% of delivered scope different from initial backlog Evidence of disruption All business objectives met On budget 8/9 releases on time and with committed scope Overall project 2 weeks early Avg delivery cycle Ɵme ↓ 72%
35. Deflation & Margin Pressure Deflation & Margin Pressure Overhead Overhead Quality Quality Learning! Wrong Require- ments Wrong Require- ments Cost per month: ↑ 53% Cost per feature: ↑ 16% Cost per USEFUL feature: ↓ 37% PRD defects: ↓ 94% DEV defects: ↓ 86% Avg defect queue: ↓ 92% Avg defect cycle time: ↓ 84% No change control processes Very limited task / project tracking Very limited estimation 25% of sprints “missed commitments” 70% of delivered scope different from initial backlog Evidence of disruption ? All business objectives met On budget 8/9 releases on time and with committed scope Overall project 2 weeks early Avg delivery cycle Ɵme ↓ 72%
36. Highly used Used infrequently or incompletely Rarely or never used Fixed Price-Fixed Scope 18 months, 100% cost Prototyping, phased releases Target-Scope 9 months, 58% cost Agile / Scrum Evidence of learning Feature waste ↓78%
37. Evidence of learning Highly used Used infrequently or incompletely Rarely or never used No improvement over time Initial scope well defined Uncharted territory begins Ideas kept improving Feedback at end Feedback throughout Fixed Price-Fixed Scope 18 months, 100% cost Prototyping, phased releases Target-Scope 9 months, 58% cost Agile / Scrum
38. Deflation & Margin Pressure Deflation & Margin Pressure Overhead Overhead Quality Quality Learning! Wrong Require- ments Wrong Require- ments Cost per month: ↑ 53% Cost per feature: ↑ 16% Cost per USEFUL feature: ↓ 37% PRD defects: ↓ 94% DEV defects: ↓ 86% Avg defect queue: ↓ 92% Avg defect cycle time: ↓ 84% No change control processes Very limited task / project tracking Very limited estimation 25% of sprints “missed commitments” 70% of delivered scope different from initial backlog Evidence of disruption ? All business objectives met On budget 8/9 releases on time and with committed scope Overall project 2 weeks early Avg delivery cycle Ɵme ↓ 72%
39. Thank you!
40. Questions?