Mary Poppendieck: The Aware Organization

We now have a pretty good idea of what Just-in-Time means in software development. With Continuous Delivery moving to the mainstream, rapid flow of value through the development process is becoming routine. However, as software systems get larger and more complex, we may lose sight of what Jidoka has to offer. At the Lean IT Summit 2014, Mary Poppendieck explained what Jidoka, or situational awareness, means for groups developing large software systems.

Transcript

1. l e a n software development www.poppendieck.com Mary Poppendieck mary@poppendieck.com mary@poppendieck.com The Aware Organization Mindfulness
2. l e a n Toyota Production System November 14 Copyright©2014 Poppendieck.LLC 2 Thoroughly Remove Waste Just in Time • Make only what, when and the amount needed • Downstream process takes from upstream Jidoka • Processes detect errors and stop on their own • Built in human intelligence Flow
3. l e a n Theory of Constraints Every system has a bottleneck. Value cannot flow through the system any faster than it flows through that bottleneck. So the best way to improve the system flow is to improve the rate at which value flows through the bottleneck. November 14 Copyright©2014 Poppendieck.LLC 3
4. l e a n Constraint #1 The Integration Problem % of Release Cycle Spent “Hardening” November 14 Copyright©2014 Poppendieck.LLC 4 Typical: 30% Sometimes: 50% Release Cycle Top Companies: What is a Release Cycle?
5. l e a n Constraint #2 Cost of Complexity November 14 Copyright©2014 Poppendieck.LLC 5 Knowing What to Build Cost Time Features / Functions Used in a Typical System Standish Group Study Reported at XP2002 by Jim Johnson, Chairman Always 7% Often 13% Sometimes 16% Rarely 19% Never 45% Rarely / Never Used: 64% Often / Always Used: 20%
6. l e a n Toyota Production System November 14 Copyright©2014 Poppendieck.LLC 6 Thoroughly Remove Waste Just in Time • Make only what, when and the amount needed • Downstream process takes from upstream Jidoka • Processes detect errors and stop on their own • Built in human intelligence Flow ???
7. l e a n What Makes a Winning Team? November 14 Copyright©2014 Poppendieck.LLC 7
8. l e a n The Military Model Understand Command Intent Two Levels Up Command Intent: A concise expression of the purpose of the campaign, the desired results, and the expected team progress toward achieving the desired end state. Maintain Situational Awareness One Level Up 1. Collaborative Planning 2. Situational awareness of progress of other platoons 3. Adapt to make sure the company reaches the end state November 14 8 Copyright©2014 Poppendieck.LLC
9. l e a n The Company Model High Reliability Organizations  Have more than their fair share of unexpected events  Have less than their fair share of accidents  Firefighters  Nuclear Power Plants  Power Grid Dispatching Centers  Hospital Emergency Rooms  Air Traffic Control  Aircraft Carriers Common Characteristic Mindfulness November 14 Copyright©2014 Poppendieck.LLC 9 Managing the Unexpected: Assuring High Performance in an Age of Complexity by Karl E. Weick and Kathleen M. Sutcliffe, 2001
10. l e a n Mindfulness Sensitivity to Operations Reluctance to Simplify Preoccupation with Failure Commitment to Resilience Deference to Expertise November 14 Copyright©2014 Poppendieck.LLC 10 Mindfulness Exceptionally Safe, Reliable, World Class Organizations
11. l e a n Toyota Production System November 14 Copyright©2014 Poppendieck.LLC 11 Thoroughly Remove Waste Just in Time • Make only what, when and the amount needed • Downstream process takes from upstream Jidoka • Processes detect errors and stop on their own • Built in human intelligence Flow Mindfulness / Situational Awareness
12. l e a n Mindfulness Sensitivity to Operations Reluctance to Simplify Preoccupation with Failure Commitment to Resilience Deference to Expertise November 14 Copyright©2014 Poppendieck.LLC 12 Mindfulness Exceptionally Safe, Reliable, World Class Organizations
13. l e a n Understand How the Work Works November 14 13 Copyright©2014 Poppendieck.LLC The Biggest Barrier to Continuous Delivery The Wrong Reason to Automate Tests
14. l e a n Visualization: Essential for Scaling November 14 14 Copyright©2014 Poppendieck.LLC
15. l e a n A Company in Hamburg November 14 Copyright©2014 Poppendieck.LLC 15
16. l e a n Mindfulness Sensitivity to Operations Reluctance to Simplify Preoccupation with Failure Commitment to Resilience Deference to Expertise November 14 Copyright©2014 Poppendieck.LLC 16 Mindfulness Exceptionally Safe, Reliable, World Class Organizations
17. l e a n Previous Generation: Software Development Life Cycle November 14 Copyright©2014 Poppendieck.LLC 17
18. l e a n Current Generation: Agile / Lean Processes November 14 18 Copyright©2014 Poppendieck.LLC Discover Dev/Unit Test Deploy orem ipsum dolor sit amet, co nse ctetur orem ipsum dolor sit amet, co nse ctetur orem ipsum dolor sit amet, co nse ctetur orem ipsum dolor sit amet, co nse ctetur SIT 5 6 3 FLOW Avg cycle time: days 12 orem ipsum dolor sit amet, co nse ctetur orem ipsum dolor sit amet, co nse ctetur orem ipsum dolor sit amet, co nse ctetur Next 3 orem ipsum dolor sit amet, co nse ctetur orem ipsum dolor sit amet, co nse ctetur Weekly Dev Ready SIT Ready Ready to Develop: 1. Wireframes 2. Story Tests 3. Less than 3 days work Ready for system test 1. Story test passed & in regression harness 2. No false negatives 3. Proper environment Ready to Design: 1. Vendor time committed 2. …….. Ready to Deploy: 1. Regression passed 2. ……..
19. l e a n Next Generation: Software Engineering November 14 Copyright©2014 Poppendieck.LLC 19  Acceptance test driven development process  Tight collaboration between business and delivery teams  Cross-functional teams include QA and operations  Automated build, testing, db migration, and deployment  Incremental development on mainline with continuous integration  Software always production ready  Releases tied to business needs, not operational constraints
20. l e a n A Company in Durban November 14 Copyright©2014 Poppendieck.LLC 20
21. l e a n Mindfulness Sensitivity to Operations Reluctance to Simplify Preoccupation with Failure Commitment to Resilience Deference to Expertise November 14 Copyright©2014 Poppendieck.LLC 21 Mindfulness Exceptionally Safe, Reliable, World Class Organizations
22. l e a n Preoccupation With Failure Resilience Engineering Learning to Embrace Failure November 14 Copyright©2013 Poppendieck.LLC 22 ACM Queue Sep 2012 GameDay Chaos Monkey
23. l e a n One Thing We Know for Sure For Complex Systems This does not work November 14 23 Copyright©2014 Poppendieck.LLC
24. l e a n For Complex Systems This works One Thing We Know for Sure November 14 24 Copyright©2014 Poppendieck.LLC
25. l e a n Mindfulness Sensitivity to Operations Reluctance to Simplify Preoccupation with Failure Commitment to Resilience Deference to Expertise November 14 Copyright©2014 Poppendieck.LLC 25 Mindfulness Exceptionally Safe, Reliable, World Class Organizations
26. l e a n The team dropped everything to help and then stayed as long as it took. Steven Brill November 14 26 Copyright©2014 Poppendieck.LLC Todd Park Chief Technology Officer Jeff Zients Chief Bureaucracy Troubleshooter Gabriel Burt Led the technology group for Obama Campaign Analytics Ryan Panchadsaram Presidential Innovation Fellow Mike Abbott Led Twitter’s technology group Jini Kim Google alumni I loved every minute of it … I believe in getting people health care. I am so proud of this. …this is what we do. And this job had special meaning. Paul Smith Obama Campaign technology alumni Mikey Dickerson Google reliability engineer
27. l e a n Resilience: Learn from Mistakes Process Problems Product Features November 14 27 Copyright©2014 Poppendieck.LLC
28. l e a n Mindfulness Sensitivity to Operations Reluctance to Simplify Preoccupation with Failure Commitment to Resilience Deference to Expertise November 14 Copyright©2014 Poppendieck.LLC 28 Mindfulness Exceptionally Safe, Reliable, World Class Organizations
29. l e a n Who is Responsible for Delivering Value? Control development with the critical-few results that define value. Make sure those results are business results, not technical results. Most of our so-called functional requirements are not actually requirements. They are designs to meet unarticulated, higher-level, critical requirements. Give the technical team the freedom to discover how to deliver those results. The worst scenario I can imagine is when we allow real customers, users, and our own salespeople to dictate ‘functions and features’ to the developers, carefully disguised as ‘customer requirements’. Maybe conveyed by our product owners. If you go slightly below the surface of these false ‘requirements’ you will find that they are not really requirements. They are really bad amateur design for the ‘real’ requirements. Let the technical team engineer technical solutions to meet the quantified requirements. This gets the right job (design) done by the right people (engineers) towards the right requirements (higher level views of the qualities of the application). November 14 Copyright©2014 Poppendieck.LLC 29 From: Value-Driven Development Principles and Values by Tom Gilb, Agile Record, July 2010 Tom Gilb Published 1988 19th Printing
30. l e a n An Engineering Organization Many years of successful software development without detailed specifications, a backlog of stories, or a long list of features and functions. Requirements for a Process Control System 1. The new equipment has to make good product. 2. It should contain the latest technology. 3. It MUST be on time. 4. Operators must find it convenient to use. 5. It will be maintained by the plant engineer. Everything else is design! November 14 Copyright©2013 Poppendieck.LLC 30
31. l e a n Another Thing We Know for Sure Delivery Organizations Do Not Work Very Well Problem Solving Organizations Work Better November 14 31 Copyright©2014 Poppendieck.LLC
32. l e a n Case Study: Government Case: British National Health Service, Electronic Patient Records, 2002 – 2011, £10bn. In Response: Gov.UK. Governance Principles:  Don’t slow down delivery  Decisions when they’re needed, at the right level  Do it with the right people  Go see for yourself  Only do it if it adds value  Trust and verify November 14 Copyright©2014 Poppendieck.LLC 32 4-8 weeks 6-8 weeks A few months, then iterate See: https://www.gov.uk/transformation Delivery team charter:  service vision  quantifiable goals [impacts]  key performance indicators [metrics] that show how they will meet user needs
33. l e a n Impact-driven Development November 14 Copyright©2014 Poppendieck.LLC 33 1. Start with WHY – Purpose, Problem 2. Understand the desired impact: a. Who cares about the impact of potential solutions? b. How will these people measure the impact of outcomes? c. What changes can create outcomes that move the metrics – in the right direction – enough to matter? 3. Prove that the impact is being achieved: a. Experiment: Prototype the most promising changes. b. Implement a change only if its impact is validated. c. Iterate rapidly until the desired impact is achieved. Work Backward from Impact Tom & Kai Gilb
34. l e a n software development www.poppendieck.com Mary Poppendieck mary@poppendieck.com mary@poppendieck.com Questions? More Information: www.poppendieck.com