Evolving Continuous Delivery

This is a story of guiding a team's CD adoption and maturity, from a chaotic start, to being awesome.

This is a journey from a troublesome, struggling start of chaotic manual deployments, merge hell, regular production roll backs and lost code, to deliver a single commit to master automatically and reliably, under an hour, many times a day.

We’ll look at a few simple practices that worked, to introduce CD practices such as monitoring, build automation and test automation gradually, instead of going all in at first.

You’ll start to see that it isn’t daunting at all, and start delivering with joy.

Related post: http://www.hibri.net//2016/06/18/continuous-delivery-rags-to-riches/

Transcript

1. Evolving Continuous Delivery From Rags to Riches Hibri Marzook @hibri
2. @hibri http://www.hibri.net
3. Hibri Marzook, February 2017 @hibri Fintech Post startup phase Successful product in the market ~ 30 in engineering ~ 200 employees Context
4. Hibri Marzook, February 2017 @hibri First impressions Public facing API, running in the Cloud No automation Inconsistent environments Context switching. Not finishing work in progress Old, stale branches merged into active branch Manual deploys
5. Hibri Marzook, February 2017 @hibri
6. Hibri Marzook, February 2017 @hibri What to fix first? Branching strategy? Lack of automation? Lack of technical practices? TDD? Lack of CI? Process? Scrum, Kanban ceremonies?
7. Hibri Marzook, February 2017 @hibri A focus on delivery is needed for continuous improvement
8. Hibri Marzook, February 2017 @hibri Gain situational awareness Photo Credit - Hugh Trainer via Flickr
9. Hibri Marzook, February 2017 @hibri Gaining situational awareness Install an application monitoring tool. New Relic Zero friction install Alerting out of the box PagerDuty integration
10. Hibri Marzook, February 2017 @hibri We could be the first responders How much traffic? How many users are affected? Usage trends? Error rates? Can this scale?
11. Hibri Marzook, February 2017 @hibri Shared understanding of the system Draw environment map Draw components of the system Ensure that all team members have the same understanding Keep maps visible
12. Hibri Marzook, February 2017 @hibri Better situational awareness Photo Credit - Steve Jurveston via Flickr
13. Hibri Marzook, February 2017 @hibri
14. Hibri Marzook, February 2017 @hibri Expose the realities of the system instead of relying on assumptions
15. Hibri Marzook, February 2017 @hibri In a legacy distributed application, there is a disconnect between how the system behaves in production and how the creators think it “should” behave.
16. Hibri Marzook, February 2017 @hibri Create visibility
17. Hibri Marzook, February 2017 @hibri Agile ≠ JIRA
18. Hibri Marzook, February 2017 @hibri Creating visibility Made work visible with a simple card wall Pull from the right attitude Use JIRA only for tracking Created transparency about the work we are doing
19. Hibri Marzook, February 2017 @hibri Limit work in progress Did one thing at a time Got into the habit of finishing one thing Whole team focussed on delivery Only then did we do more
20. Hibri Marzook, February 2017 @hibri When you don’t have anything else, put everyone together
21. Hibri Marzook, February 2017 @hibri Put everyone together Everyone we worked with sat in the same area Introduced Slack Devs are always around to fix an issue Practice remote working early
22. Hibri Marzook, February 2017 @hibri
23. Hibri Marzook, February 2017 @hibri Use slack time to learn and improve
24. Hibri Marzook, February 2017 @hibri How did we create slack? WIP Limits Production issues are quantified, don’t need to jump on every issue Have data to push back on management pressure Can ignore some problems
25. Hibri Marzook, February 2017 @hibri Long term change happens during slack time
26. Hibri Marzook, February 2017 @hibri Before CI Couldn’t reliably build the system. “Works on my machine” Key man dependencies to do a release. Build time dependencies have to be fixed manually.
27. Hibri Marzook, February 2017 @hibri
28. Hibri Marzook, February 2017 @hibri When the team spends more time talking about branching and merging, you have the wrong strategy
29. Hibri Marzook, February 2017 @hibri Introducing CI Built the system reliably and repeatedly on a CI server Removed the key man dependency Used this build as the source for manual deploys No ATDD yet Manual regression testing
30. Hibri Marzook, February 2017 @hibri Getting branching under control
31. Hibri Marzook, February 2017 @hibri
32. Hibri Marzook, February 2017 @hibri Amplify the good things
33. Hibri Marzook, February 2017 @hibri Adding automated testing Made the tests work in a local dev environment Made the same tests run in other environments Kept adding tests for all new work and fixes Refactored tests continuously
34. Hibri Marzook, February 2017 @hibri Adding automated deploys Deployed to test environments on every commit Used Azure’s “slots” feature for blue/green TeamCity + Octopus Deploy Smoke tests after every deploy
35. Hibri Marzook, February 2017 @hibri
36. Hibri Marzook, February 2017 @hibri The ‘A’ word
37. Hibri Marzook, February 2017 @hibri Talk and discuss the values Focus on the impact Explain cause and effect Introduce and repeat the principles Agile manifesto. Left over right
38. Hibri Marzook, February 2017 @hibri https://landing.google.com/sre/book/chapters/part3.html#fig_part-practices_reliability-hierarchy
39. Hibri Marzook, February 2017 @hibri Evolutionary approach Deal with current constraints Sometimes it’s OK to do things manually Every change should make sense and help the team a little bit more Team should not have to wait for all the changes to gain value
40. Hibri Marzook, February 2017 @hibri Have a primal push to get your software into the hands of users
41. Hibri Marzook, February 2017 @hibri Thanks! Questions? @hibri hibri@hibri.net