Growing a Culture of Experimentation

Many organizations claim to have adopted agile when in fact engineering teams must still follow orders on what to build and how to build it. Thus, we crush the ability of our people to improve their work, preventing them from learning from users or from each other. In this talk, Jez will show how a test-driven approach to both product development and process improvement can enable even large, regulated organizations to harness the creativity of their people. Discover how a combination of architectural and cultural approaches enables experimentation at all levels of your organization, supported by research and examples from current practice.

Transcript

1. @jezhumble
   oop 6 february 2018
   growing a culture of experimentation
   

   View Slide

2. @jezhumble
   start small (teams and scope) and deliver often
   culture is measurable and it can be changed… slowly
   change culture by changing behavior and doing things differently
   leaders must make it cheap, quick and safe to run experiments
   practice making mistakes so it becomes normal
   too long: didn’t listen;
   

   View Slide

3. what is culture?
   “A pattern of shared tacit assumptions that was learned
   by a group as it solved its problems of external
   adaptation and internal integration, that has worked
   well enough to be considered valid and, therefore, to
   be taught to new members as the correct way to
   perceive, think, and feel in relation to those problems.”
   — Edgar Schein, The Corporate Culture Survival Guide
   

   View Slide

4. software delivery as a competitive advantage
   “Firms with high-performing IT
   organizations were twice as likely to
   exceed their profitability, market share
   and productivity goals.”
   http://bit.ly/2014-devops-report
   

   View Slide

5. software delivery as a competitive advantage
   high performers were more than twice as likely to
   achieve or exceed the following objectives:
   • Quantity of products or services
   • Operating efficiency
   • Customer satisfaction
   • Quality of products or services provided
   • Achieving organizational and mission goals
   • Measures that demonstrate to external parties
   whether or not the organization is achieving
   intended results
   http://bit.ly/2017-devops-report
   

   View Slide

6. @jezhumble
   time to restore service
   lead time for changes
   release frequency
   change fail rate
   it performance
   http://bit.ly/2014-devops-report
   

   View Slide

7. View Slide

8. high trust culture
   Westrum, “A Typology of Organizational Cultures” | http://bmj.co/1BRGh5q
   how organizations process information
   

   View Slide

9. effective teams
   https://rework.withgoogle.com/blog/five-keys-to-a-successful-google-team/
   

   View Slide

10. capabilities that drive high performance
    Accelerate: The Science of Lean Software and DevOps, Forsgren, Humble and Kim 2018
    

    View Slide

11. lean product development
    • working in small batches & use of MVPs
    • visibility into the flow of work across the value stream
    • seeking—and acting on—feedback from customers
    • giving teams the authority to create & change specs
    

    View Slide

12. @jezhumble
    culture of experimentation
    1. in the small
    

    View Slide

13. impact mapping
    Gojko Adzic, Impact Mapping
    

    View Slide

14. hypothesis-driven delivery
    We believe that
    [building this feature]
    [for these people]
    will achieve [this outcome].
    We will know we are successful when we see
    [this signal from the market].
    Jeff Gothelf “Better product definition with Lean UX and Design” http://bit.ly/TylT6A
    

    View Slide

15. experiments
    Different types of user research, courtesy of Janice Fraser
    

    View Slide

16. Jon Jenkins, “Velocity Culture, The Unmet Challenge in Ops” | http://bit.ly/1vJo1Ya
    

    View Slide

17. continuous delivery
    

    View Slide

18. “Evaluating well-designed and executed experiments that
    were designed to improve a key metric, only about 1/3 were
    successful at improving the key metric!”
    do less
    “Online Experimentation at Microsoft”, Kohavi et al http://stanford.io/130uW6X
    

    View Slide

19. @jezhumble
    culture of experimentation
    2. in the large
    

    View Slide

20. the production line
    http://www.flickr.com/photos/toyotauk/4711057997/
    

    View Slide

21. methodologies
    Certainly the thieves may be able to follow the design plans and produce
    a loom. But we are modifying and improving our looms every day. So by
    the time the thieves have produced a loom from the plans they stole, we
    will have already advanced well beyond that point.
    And because they do not have the expertise gained from the failures it took to
    produce the original, they will waste a great deal more time than us as
    they move to improve their loom. We need not be concerned about what
    happened. We need only continue as always, making our improvements.
    Kiichiro Toyoda, quoted in Toyota Kata, p40 (Rother)
    

    View Slide

22. how to change culture?
    http://www.thisamericanlife.org/radio-archives/episode/403/nummi
    http://sloanreview.mit.edu/article/how-to-change-a-culture-lessons-from-nummi/
    “What changed the culture was giving
    employees the means by which they
    could successfully do their jobs. It was
    communicating clearly to employees
    what their jobs were and providing
    the training and tools to enable them
    to perform those jobs successfully.”
    —John Shook
    

    View Slide

23. View Slide

24. @jezhumble
    What obstacles are preventing you from reaching it? which one are you
    addressing now?
    What is the target condition? (The challenge)
    What is the actual condition now?
    When can we go and see what we learned from taking that step?
    What is your next step? (Start of PDCA cycle)
    improvement kata
    

    View Slide

25. @jezhumble
    hp futuresmart
    

    View Slide

26. hp laserjet firmware
    2008
    ~5% - innovation capacity
    15% - manual testing
    25% - product support
    25% - porting code
    20% - detailed planning
    10% - code integration
    Costs
    Full manual regression: 6 wks
    Builds / day: 1-2
    Commit to trunk: 1 week
    Cycle times
    

    View Slide

27. @jezhumble
    implement continuous integration
    reduce hardware variation
    create a single package
    create a simulator
    implement comprehensive test automation
    futuresmart rearchitecture
    

    View Slide

28. @jezhumble
    target conditions
    

    View Slide

29. @jezhumble
    1 month program level plan for 400-person org fits on 1 piece of paper
    teams can use any process / methodology they like
    turning objectives into features / stories happens within teams
    leaders don’t tell people what to do or how to do it
    same heuristic for process improvement and product development
    observations
    

    View Slide

30. https://www.slideshare.net/reed2001/culture-1798664/94-Highly_Aligned_Loosely_Coupled_Highly
    

    View Slide

31. @jezhumble
    deliver and create fast feedback loops
    Mike Bracken | @MTBracken | http://mikebracken.com/blog/the-strategy-is-delivery-again/
    

    View Slide

32. capabilities that drive high performance
    Accelerate: The Science of Lean Software and DevOps, Forsgren, Humble and Kim 2018
    

    View Slide

33. “I think building this culture is the key to innovation. Creativity must
    flow from everywhere. Whether you are a summer intern or the CTO,
    any good idea must be able to seek an objective test, preferably a
    test that exposes the idea to real customers. Everyone must be able
    to experiment, learn, and iterate.”
    innovation culture
    http://glinden.blogspot.com/2006/04/early-amazon-shopping-cart.html
    

    View Slide

34. thank you!
    © 2018 Jez Humble & Associates LLC
    https://continuousdelivery.com/ | https://devops-research.com/
    To receive the following:
    • 30% off my new video course: creating high performance organizations
    • 50% off my CD video training, interviews with Eric Ries, and more
    • A copy of this presentation
    • A 100 page excerpt from Lean Enterprise
    • An excerpt from The DevOps Handbook
    • A 20m preview of my Continuous Delivery video workshop
    Just pick up your phone and send an email
    To: jezhumble@sendyourslides.com
    Subject: devops
    

    View Slide