From 100 to 1000+ deployments a day [CodeMotion Edition]

The slides from this mornings session about what it takes to TRULY go faster as a team/department/group of developers.

Transcript

1. From 100 to 1000+
   deployments a day
   Pat Hermens
   Amsterdam | April 2-3, 2019
   

   View Slide

2. From 100 to 1,000+ deployments
   What does it take to get to the next level?
   

   View Slide

3. Pat Hermens
   Development Manager
   Coding for 15+ years
   Father & husband
   Rotterdam, Netherlands
   @phermens
   pat@coolblue.nl
   

   View Slide

4. View Slide

5. Why 1,000+ deployments/day?
   

   View Slide

6. 6
   @phermens
   ¯\_(ツ)_/¯
   

   View Slide

7. 7
   @phermens
   “It depends”
   

   View Slide

8. View Slide

9. View Slide

10. View Slide

11. View Slide

12. View Slide

13. View Slide

14. View Slide

15. 100 / 1,000 / 10,000; so what?
    

    View Slide

16. http://www.cs.utexas.edu/users/ewd/ewd02xx/ewd249.pdf
    

    View Slide

17. … Apparently we are too much trained to disregard
    differences in scale, to treat them as “gradual differences that
    are not essential.”
    We tell ourselves that what we can do once, we can also do
    twice and by induction, we fool ourselves into believing that
    we can do it as many times as needed, but this is just not true!
    — Edsger Dijkstra, 1969
    

    View Slide

18. View Slide

19. Prerequisites...
    

    View Slide

20. View Slide

21. 21
    @phermens
    The “faster to master” checklist
    Developing
    ⬜ Coding guidelines
    ⬜ Code reviews
    ⬜ Code analysis
    ⬜ Short-lived branches
    ⬜ Feature toggles
    Monitoring
    ⬜ Defined key metrics
    ⬜ Measure – create dashboards
    ⬜ Proactivity – add alerting (& act!)
    Testing
    ⬜ Unit tests
    ⬜ Module tests
    ⬜ Integration tests
    ⬜ End-to-end tests
    ⬜ Test automation
    Deploying
    ⬜ Continuous Integration
    ⬜ Continuous Delivery
    ⬜ Infrastructure as Code
    

    View Slide

22. 22
    @phermens
    Developing
    ☑ Coding guidelines
    ☑ Code reviews
    ⬜ Code analysis
    ☑ Short-lived branches
    ⬜ Feature toggles
    Monitoring
    ☑ Defined key metrics
    ⬜ Measure – create dashboards
    ⬜ Proactivity – add alerting (& act!)
    Testing
    ☑ Unit tests
    ⬜ Module tests
    ⬜ Integration tests
    ⬜ End-to-end tests
    ☑ Test automation
    Deploying
    ☑ Continuous Integration
    ⬜ Continuous Delivery
    ⬜ Infrastructure as Code
    The “faster to master” checklist
    

    View Slide

23. 23
    @phermens
    Developing
    ☑ Coding guidelines
    ☑ Code reviews
    ⬜ Code analysis
    ☑ Short-lived branches
    ☑ Feature toggles
    Monitoring
    ☑ Defined key metrics
    ☑ Measure – create dashboards
    ⬜ Proactivity – add alerting (& act!)
    Testing
    ☑ Unit tests
    ☑ Module tests
    ⬜ Integration tests
    ☑ End-to-end tests
    ☑ Test automation
    Deploying
    ☑ Continuous Integration
    ☑ Continuous Delivery
    ⬜ Infrastructure as Code
    The “faster to master” checklist
    

    View Slide

24. 24
    @phermens
    Developing
    ☑ Coding guidelines
    ☑ Code reviews
    ☑ Code analysis
    ☑ Short-lived branches
    ☑ Feature toggles
    Monitoring
    ☑ Defined key metrics
    ☑ Measure – create dashboards
    ☑ Proactivity – add alerting (& act!)
    Testing
    ☑ Unit tests
    ☑ Module tests
    ☑ Integration tests
    ☑ End-to-end tests
    ☑ Test automation
    Deploying
    ☑ Continuous Integration
    ☑ Continuous Delivery
    ☑ Infrastructure as Code
    The “faster to master” checklist
    

    View Slide

25. Story time!
    

    View Slide

26. Responsibility Autonomy
    Failure
    Ownership
    

    View Slide

27. Responsibility Autonomy
    Failure
    Ownership
    

    View Slide

28. Responsibility
    

    View Slide

29. 29
    @phermens
    Hosting &
    Deployment
    “DevOps”
    

    View Slide

30. 30
    @phermens
    Hosting &
    Deployment
    “DevOps”
    Team
    A
    Team
    D
    Team
    B
    Team
    C
    

    View Slide

31. 31
    @phermens
    Hosting &
    Deployment
    “DevOps”
    Team
    A
    Team
    D
    Team
    F
    Team
    B
    Team
    C
    Team
    E
    

    View Slide

32. 32
    @phermens
    Hosting &
    Deployment
    “DevOps”
    Team
    A
    Team
    D
    Team
    F
    Team
    B
    Team
    C
    Team
    E
    

    View Slide

33. 33
    @phermens
    Hosting &
    Deployment
    “DevOps”
    Team
    H
    Team
    A
    Team
    D
    Team
    G
    Team
    E
    Team
    J
    Team
    F
    Team
    B
    Team
    I
    Team
    C
    

    View Slide

34. 34
    @phermens
    Hosting &
    Deployment
    “DevOps”
    Team
    H
    Team
    A
    Team
    D
    Team
    G
    Team
    E
    Team
    J
    Team
    F
    Team
    B
    Team
    I
    Team
    C
    

    View Slide

35. 35
    @phermens
    Hosting &
    Deployment
    “DevOps”
    Team
    H
    Team
    A
    Team
    D
    Team
    G
    Team
    E
    Team
    J
    Team
    F
    Team
    B
    Team
    I
    Team
    C
    

    View Slide

36. 36
    @phermens
    Hosting &
    Deployment
    “DevOps”
    Team
    H
    Team
    A
    Team
    D
    Team
    G
    Team
    E
    Team
    J
    Team
    F
    Team
    B
    Team
    I
    Team
    C
    

    View Slide

37. 37
    @phermens
    Hosting &
    Deployment
    “DevOps”
    Team
    H
    Team
    A
    Team
    D
    Team
    G
    Team
    E
    Team
    J
    Team
    F
    Team
    B
    Team
    I
    Team
    C
    

    View Slide

38. 38
    @phermens
    Team
    H
    Team
    A
    Team
    C
    Team
    G
    Team
    E
    Team
    J
    Team
    F
    Team
    B
    Team
    I
    Team
    C
    Team
    D
    Team
    L
    Deploy-
    ment
    

    View Slide

39. 39
    @phermens
    

    View Slide

40. Responsibility Autonomy
    Failure
    Ownership
    

    View Slide

41. Responsibility Autonomy
    Failure
    Ownership
    

    View Slide

42. Autonomy
    

    View Slide

43. Autonomy
    

    View Slide

44. 44
    @phermens
    Serilog + Splunk + DataDog
    

    View Slide

45. View Slide

46. View Slide

47. View Slide

48. View Slide

49. View Slide

50. Autonomy
    

    View Slide

51. View Slide

52. 52
    @phermens
    

    View Slide

53. 53
    @phermens
    

    View Slide

54. 54
    @phermens
    

    View Slide

55. 55
    @phermens
    

    View Slide

56. 56
    @phermens
    

    View Slide

57. 57
    @phermens
    

    View Slide

58. 58
    @phermens
    UDP
    

    View Slide

59. 59
    @phermens
    UDP
    

    View Slide

60. 60
    @phermens
    UDP
    

    View Slide

61. 61
    @phermens
    UDP
    

    View Slide

62. 62
    @phermens
    UDP
    

    View Slide

63. 63
    @phermens
    

    View Slide

64. 64
    @phermens
    TCP
    

    View Slide

65. 65
    @phermens
    TCP
    

    View Slide

66. 66
    @phermens
    

    View Slide

67. 67
    @phermens
    

    View Slide

68. 68
    @phermens
    

    View Slide

69. 69
    @phermens
    Hosting &
    Deployment
    “DevOps”
    Team
    H
    Team
    A
    Team
    D
    Team
    G
    Team
    E
    Team
    J
    Team
    F
    Team
    B
    Team
    I
    Team
    C
    

    View Slide

70. 70
    @phermens
    

    View Slide

71. 71
    @phermens
    

    View Slide

72. 72
    @phermens
    

    View Slide

73. 73
    @phermens
    

    View Slide

74. 74
    @phermens
    

    View Slide

75. Responsibility Autonomy
    Failure
    Ownership
    

    View Slide

76. Responsibility Autonomy
    Failure
    Ownership
    

    View Slide

77. 77
    @phermens
    Ownership
    

    View Slide

78. 78
    @phermens
    Build System
    

    View Slide

79. 79
    @phermens
    

    View Slide

80. 80
    @phermens
    

    View Slide

81. 81
    @phermens
    Development team tooling choices
    

    View Slide

82. 82
    @phermens
    Infrastructure team tooling choices
    

    View Slide

83. 83
    @phermens
    

    View Slide

84. 84
    @phermens
    What’s next?
    

    View Slide

85. 85
    @phermens
    ?
    What’s next?
    

    View Slide

86. 86
    @phermens
    ¯\_(ツ)_/¯
    

    View Slide

87. 87
    @phermens
    

    View Slide

88. Responsibility Autonomy
    Failure
    Ownership
    

    View Slide

89. Responsibility Autonomy
    Failure
    Ownership
    

    View Slide

90. 90
    @phermens
    Fail early, fail often,
    but always fail forward.
    

    View Slide

91. 91
    @phermens
    Failure
    

    View Slide

92. View Slide

93. View Slide

94. View Slide

95. View Slide

96. View Slide

97. View Slide

98. 98
    @phermens
    [REDACTED]
    [REDACTED]
    

    View Slide

99. View Slide

100. View Slide

101. View Slide

102. View Slide

103. View Slide

104. View Slide

105. View Slide

106. View Slide

107. View Slide

108. View Slide

109. View Slide

110. View Slide

111. View Slide

112. View Slide

113. View Slide

114. View Slide

115. 115
     @phermens
     

     View Slide

116. 116
     @phermens
     

     View Slide

117. View Slide

118. 118
     @phermens
     #war-room
     

     View Slide

119. Responsibility Autonomy
     Failure
     Ownership
     

     View Slide

120. Responsibility Autonomy
     Failure
     Ownership
     

     View Slide

121. 121
     @phermens
     The “faster to master” checklist
     Developing
     ⬜ Coding guidelines
     ⬜ Code reviews
     ⬜ Code analysis
     ⬜ Short-lived branches
     ⬜ Feature toggles
     Monitoring
     ⬜ Defined key metrics
     ⬜ Measure – create dashboards
     ⬜ Proactivity – add alerting (& act!)
     Testing
     ⬜ Unit tests
     ⬜ Module tests
     ⬜ Integration tests
     ⬜ End-to-end tests
     ⬜ Test automation
     Deploying
     ⬜ Continuous Integration
     ⬜ Continuous Delivery
     ⬜ Infrastructure as Code
     

     View Slide

122. 122
     @phermens
     The “100 to 1,000” checklist
     Responsibility
     ⬜ Can I control my deployments?
     ⬜ Have I removed all bottlenecks?
     ⬜ Do I have good support systems?
     ⬜ Am I collaborating with my peers?
     Autonomy
     ⬜ Freedom to innovate/improve?
     ⬜ Am I in control and independent?
     ⬜ Can I find out what went wrong?
     ⬜ Do I have the skills to fix it?
     Ownership
     ⬜ Agreed interfaces (or contracts?)
     ⬜ Out-of-the-box solutions?
     ⬜ Tooling is 100% automated?
     ⬜ Local build == “Production” build?
     Failure
     ⬜ Monitoring = Alerting = Action
     ⬜ Take pride in improving
     ⬜ Small deliverables, quickly
     ⬜ Non-blocking deployments
     

     View Slide

123. Questions?
     

     View Slide

124. Checklists!
     

     View Slide

125. 125
     @phermens
     The “faster to master” checklist
     Developing
     ⬜ Coding guidelines
     ⬜ Code reviews
     ⬜ Code analysis
     ⬜ Short-lived branches
     ⬜ Feature toggles
     Monitoring
     ⬜ Defined key metrics
     ⬜ Measure – create dashboards
     ⬜ Proactivity – add alerting (& act!)
     Testing
     ⬜ Unit tests
     ⬜ Module tests
     ⬜ Integration tests
     ⬜ End-to-end tests
     ⬜ Test automation
     Deploying
     ⬜ Continuous Integration
     ⬜ Continuous Delivery
     ⬜ Infrastructure as Code
     

     View Slide

126. 126
     @phermens
     The “100 to 1,000” checklist
     Responsibility
     ⬜ Can I control my deployments?
     ⬜ Have I removed all bottlenecks?
     ⬜ Do I have good support systems?
     ⬜ Am I collaborating with my peers?
     Autonomy
     ⬜ Freedom to innovate/improve?
     ⬜ Am I in control and independent?
     ⬜ Can I find out what went wrong?
     ⬜ Do I have the skills to fix it?
     Ownership
     ⬜ Agreed interfaces or contracts?
     ⬜ Out-of-the-box solutions?
     ⬜ Tooling is 100% automated?
     ⬜ Local build == “Production” build?
     Failure
     ⬜ Monitoring = Alerting = Action
     ⬜ Take pride in improving
     ⬜ Small deliverables, quickly
     ⬜ Non-blocking deployments
     

     View Slide

127. https://sddconf.com/brands/sdd/library/DOLLARD-Debugging.pdf
     

     View Slide

128. 128
     @phermens
     Thanks!
     @phermens hermens.com.au
     pat@coolblue.nl careersatcoolblue.com
     

     View Slide