From 100 to 1000+ deployments a day [CodeMotion Edition]

The slides from this mornings session about what it takes to TRULY go faster as a team/department/group of developers.

Transcript

1. From 100 to 1000+ deployments a day Pat Hermens Amsterdam | April 2-3, 2019
2. From 100 to 1,000+ deployments What does it take to get to the next level?
3. Pat Hermens Development Manager Coding for 15+ years Father & husband Rotterdam, Netherlands @phermens pat@coolblue.nl
5. Why 1,000+ deployments/day?
6. 6 @phermens ¯\_(ツ)_/¯
7. 7 @phermens “It depends”
15. 100 / 1,000 / 10,000; so what?
16. http://www.cs.utexas.edu/users/ewd/ewd02xx/ewd249.pdf
17. … Apparently we are too much trained to disregard differences in scale, to treat them as “gradual differences that are not essential.” We tell ourselves that what we can do once, we can also do twice and by induction, we fool ourselves into believing that we can do it as many times as needed, but this is just not true! — Edsger Dijkstra, 1969
19. Prerequisites...
21. 21 @phermens The “faster to master” checklist Developing ⬜ Coding guidelines ⬜ Code reviews ⬜ Code analysis ⬜ Short-lived branches ⬜ Feature toggles Monitoring ⬜ Defined key metrics ⬜ Measure – create dashboards ⬜ Proactivity – add alerting (& act!) Testing ⬜ Unit tests ⬜ Module tests ⬜ Integration tests ⬜ End-to-end tests ⬜ Test automation Deploying ⬜ Continuous Integration ⬜ Continuous Delivery ⬜ Infrastructure as Code
22. 22 @phermens Developing ☑ Coding guidelines ☑ Code reviews ⬜ Code analysis ☑ Short-lived branches ⬜ Feature toggles Monitoring ☑ Defined key metrics ⬜ Measure – create dashboards ⬜ Proactivity – add alerting (& act!) Testing ☑ Unit tests ⬜ Module tests ⬜ Integration tests ⬜ End-to-end tests ☑ Test automation Deploying ☑ Continuous Integration ⬜ Continuous Delivery ⬜ Infrastructure as Code The “faster to master” checklist
23. 23 @phermens Developing ☑ Coding guidelines ☑ Code reviews ⬜ Code analysis ☑ Short-lived branches ☑ Feature toggles Monitoring ☑ Defined key metrics ☑ Measure – create dashboards ⬜ Proactivity – add alerting (& act!) Testing ☑ Unit tests ☑ Module tests ⬜ Integration tests ☑ End-to-end tests ☑ Test automation Deploying ☑ Continuous Integration ☑ Continuous Delivery ⬜ Infrastructure as Code The “faster to master” checklist
24. 24 @phermens Developing ☑ Coding guidelines ☑ Code reviews ☑ Code analysis ☑ Short-lived branches ☑ Feature toggles Monitoring ☑ Defined key metrics ☑ Measure – create dashboards ☑ Proactivity – add alerting (& act!) Testing ☑ Unit tests ☑ Module tests ☑ Integration tests ☑ End-to-end tests ☑ Test automation Deploying ☑ Continuous Integration ☑ Continuous Delivery ☑ Infrastructure as Code The “faster to master” checklist
25. Story time!
26. Responsibility Autonomy Failure Ownership
27. Responsibility Autonomy Failure Ownership
28. Responsibility
29. 29 @phermens Hosting & Deployment “DevOps”
30. 30 @phermens Hosting & Deployment “DevOps” Team A Team D Team B Team C
31. 31 @phermens Hosting & Deployment “DevOps” Team A Team D Team F Team B Team C Team E
32. 32 @phermens Hosting & Deployment “DevOps” Team A Team D Team F Team B Team C Team E
33. 33 @phermens Hosting & Deployment “DevOps” Team H Team A Team D Team G Team E Team J Team F Team B Team I Team C
34. 34 @phermens Hosting & Deployment “DevOps” Team H Team A Team D Team G Team E Team J Team F Team B Team I Team C
35. 35 @phermens Hosting & Deployment “DevOps” Team H Team A Team D Team G Team E Team J Team F Team B Team I Team C
36. 36 @phermens Hosting & Deployment “DevOps” Team H Team A Team D Team G Team E Team J Team F Team B Team I Team C
37. 37 @phermens Hosting & Deployment “DevOps” Team H Team A Team D Team G Team E Team J Team F Team B Team I Team C
38. 38 @phermens Team H Team A Team C Team G Team E Team J Team F Team B Team I Team C Team D Team L Deploy- ment
39. 39 @phermens
40. Responsibility Autonomy Failure Ownership
41. Responsibility Autonomy Failure Ownership
42. Autonomy
43. Autonomy
44. 44 @phermens Serilog + Splunk + DataDog
50. Autonomy
52. 52 @phermens
53. 53 @phermens
54. 54 @phermens
55. 55 @phermens
56. 56 @phermens
57. 57 @phermens
58. 58 @phermens UDP
59. 59 @phermens UDP
60. 60 @phermens UDP
61. 61 @phermens UDP
62. 62 @phermens UDP
63. 63 @phermens
64. 64 @phermens TCP
65. 65 @phermens TCP
66. 66 @phermens
67. 67 @phermens
68. 68 @phermens
69. 69 @phermens Hosting & Deployment “DevOps” Team H Team A Team D Team G Team E Team J Team F Team B Team I Team C
70. 70 @phermens
71. 71 @phermens
72. 72 @phermens
73. 73 @phermens
74. 74 @phermens
75. Responsibility Autonomy Failure Ownership
76. Responsibility Autonomy Failure Ownership
77. 77 @phermens Ownership
78. 78 @phermens Build System
79. 79 @phermens
80. 80 @phermens
81. 81 @phermens Development team tooling choices
82. 82 @phermens Infrastructure team tooling choices
83. 83 @phermens
84. 84 @phermens What’s next?
85. 85 @phermens ? What’s next?
86. 86 @phermens ¯\_(ツ)_/¯
87. 87 @phermens
88. Responsibility Autonomy Failure Ownership
89. Responsibility Autonomy Failure Ownership
90. 90 @phermens Fail early, fail often, but always fail forward.
91. 91 @phermens Failure
98. 98 @phermens [REDACTED] [REDACTED]
115. 115 @phermens
116. 116 @phermens
118. 118 @phermens #war-room
119. Responsibility Autonomy Failure Ownership
120. Responsibility Autonomy Failure Ownership
121. 121 @phermens The “faster to master” checklist Developing ⬜ Coding guidelines ⬜ Code reviews ⬜ Code analysis ⬜ Short-lived branches ⬜ Feature toggles Monitoring ⬜ Defined key metrics ⬜ Measure – create dashboards ⬜ Proactivity – add alerting (& act!) Testing ⬜ Unit tests ⬜ Module tests ⬜ Integration tests ⬜ End-to-end tests ⬜ Test automation Deploying ⬜ Continuous Integration ⬜ Continuous Delivery ⬜ Infrastructure as Code
122. 122 @phermens The “100 to 1,000” checklist Responsibility ⬜ Can I control my deployments? ⬜ Have I removed all bottlenecks? ⬜ Do I have good support systems? ⬜ Am I collaborating with my peers? Autonomy ⬜ Freedom to innovate/improve? ⬜ Am I in control and independent? ⬜ Can I find out what went wrong? ⬜ Do I have the skills to fix it? Ownership ⬜ Agreed interfaces (or contracts?) ⬜ Out-of-the-box solutions? ⬜ Tooling is 100% automated? ⬜ Local build == “Production” build? Failure ⬜ Monitoring = Alerting = Action ⬜ Take pride in improving ⬜ Small deliverables, quickly ⬜ Non-blocking deployments
123. Questions?
124. Checklists!
125. 125 @phermens The “faster to master” checklist Developing ⬜ Coding guidelines ⬜ Code reviews ⬜ Code analysis ⬜ Short-lived branches ⬜ Feature toggles Monitoring ⬜ Defined key metrics ⬜ Measure – create dashboards ⬜ Proactivity – add alerting (& act!) Testing ⬜ Unit tests ⬜ Module tests ⬜ Integration tests ⬜ End-to-end tests ⬜ Test automation Deploying ⬜ Continuous Integration ⬜ Continuous Delivery ⬜ Infrastructure as Code
126. 126 @phermens The “100 to 1,000” checklist Responsibility ⬜ Can I control my deployments? ⬜ Have I removed all bottlenecks? ⬜ Do I have good support systems? ⬜ Am I collaborating with my peers? Autonomy ⬜ Freedom to innovate/improve? ⬜ Am I in control and independent? ⬜ Can I find out what went wrong? ⬜ Do I have the skills to fix it? Ownership ⬜ Agreed interfaces or contracts? ⬜ Out-of-the-box solutions? ⬜ Tooling is 100% automated? ⬜ Local build == “Production” build? Failure ⬜ Monitoring = Alerting = Action ⬜ Take pride in improving ⬜ Small deliverables, quickly ⬜ Non-blocking deployments
127. https://sddconf.com/brands/sdd/library/DOLLARD-Debugging.pdf
128. 128 @phermens Thanks! @phermens hermens.com.au pat@coolblue.nl careersatcoolblue.com