Prague DevOps - Scaling Devops

DevOps is commonly known as the union of people, process, and products and a way to enable continuous delivery of value to customers. Many teams are experimenting with these principles at scale — but how do you go from a handful of people to scaling it to the whole company?
In this talk, we’ll explore four critical components for bringing DevOps practices to your wider organization: customer listening, teams, quality, and deployment.

Transcript

1. Four points you should
   consider for scaling DevOps
   Christopher MANEU
   Azure Engineer & Advocate @ Microsoft
   @cmaneu
   

   View Slide

2. DevOps @ Microsoft
   107k
   engineers
   in 1ES
   4.6m
   builds per
   month
   82000
   deployments per day
   28k
   work items
   created per day
   2.4m
   git commits per
   month
   25k
   employees
   contributing to
   OSS
   372k
   pull requests per
   month
   

   View Slide

3. “DevOps is
   development
   and operations
   collaboration”
   “DevOps is
   treating your
   infrastructure
   as code”
   “DevOps
   is using
   automation”
   “Kanban
   for Ops?”
   “DevOps
   is feature
   switches”
   “DevOps
   is small
   deployments”
   

   View Slide

4. What is DevOps?
   DevOps is the union of
   people, process, and tools
   to enable continuous
   delivery of value to our
   customers.
   

   View Slide

5. @cmaneu
   Scaling DevOps
   Scaling Teams
   Scaling Customer Listening
   Scaling Quality
   Scaling Deployments
   

   View Slide

6. View Slide

7. @cmaneu
   Being agile in a corporate world
   Autonomy is crucial for a team to be efficient.
   But how to achieve organizational objectives if all teams are
   autonomous?
   

   View Slide

8. Alignment
   Autonomy
   “Let’s try to give our teams three
   things…. Autonomy, Mastery, Purpose”
   

   View Slide

9. View Slide

10. View Slide

11. View Slide

12. View Slide

13. UI
    API
    Data
    

    View Slide

14. UI
    API
    Data
    

    View Slide

15. View Slide

16. View Slide

17. View Slide

18. Listen to our customers
    

    View Slide

19. @cmaneu
    Our definition of done
    

    View Slide

20. @cmaneu
    Our definition of done
    

    View Slide

21. @cmaneu
    What to measure
    • Acquisition
    • Engagement
    • Satisfaction
    • Churn
    • Feature Usage
    Usage
    • Time to Detect
    • Time to Communicate
    • Time to Mitigate
    • Customer Impact
    • Incident Prevention Items
    • Aging Live Site Problems
    • SLA per Customer
    • Customer Support Metrics
    Live Site Health
    • Time to Build
    • Time to Self Test
    • Time to Deploy
    • Time to Learn
    Velocity
    

    View Slide

22. @cmaneu
    In-app Feedback
    

    View Slide

23. @cmaneu
    In-app Feedback
    

    View Slide

24. @cmaneu
    Microfeedback
    

    View Slide

25. @cmaneu
    Get this into development process
    

    View Slide

26. @cmaneu
    Gif based deployment
    

    View Slide

27. View Slide

28. @cmaneu
    Not everything is about code reviews/pull requests
    15%
    of the code
    reviews revelas
    a bug
    50%
    of the code review
    contains long-term
    maintenability
    feedback
    https://docs.microsoft.com/en-us/azure/devops/learn/devops-at-microsoft/boosting-
    code-reviews-useful-comments?wt.mc_id=dzone-article-chmaneu
    

    View Slide

29. Code Test & Stabilize Code Test & Stabilize
    Code
    Complete
    Planning
    

    View Slide

30. https://docs.microsoft.com/en-us/azure/devops/learn/devops-at-microsoft/shift-left-make-testing-fast-reliable
    

    View Slide

31. engineers on
    your team
    # 5 ?
    x =
    

    View Slide

32. Rule: If your bug count exceeds your bug cap… stop working
    on new features until you’re back under the cap.
    5 50
    x =
    10
    

    View Slide

33. View Slide

34. How to get feedback from deployments
    

    View Slide

35. Science experiments
    

    View Slide

36. Understand your blast radius
    

    View Slide

37. …and control it with rings
    

    View Slide

38. …and control it with feature flags
    

    View Slide

39. Comparing rings & feature flags
    DEPLOYMENT RING FEATURE FLAG
    Progressive exposure Yes Yes
    A/B Testing All users within ring All or selected users
    Cost
    Production environment
    maintenance
    Feature Flag database
    and code maintenance
    Primary use
    Manage impact "blast
    radius"
    Show or hide features in
    a release
    Blast radius - Canaries 0-9 canary users
    0, all, or specific canary
    users
    Blast radius - Early
    Adopters
    10-100 early adopter
    users
    0, all, or specific early
    adopter users
    Blast radius - Users 10000+ of users 0, all, or specific users
    

    View Slide

40. Q&A
    Christopher MANEU
    Azure Engineer & Advocate @ Microsoft R&D
    @cmaneu
    

    View Slide

41. Thanks !
    Christopher MANEU
    Azure Engineer & Advocate @ Microsoft R&D
    @cmaneu
    

    View Slide

42. Image credits
    DonWells -
    https://en.wikipedia.org/wiki/File:
    XP-feedback.gif CC BY-SA 3.0
    

    View Slide