Evolving Continuous Delivery

80574e8bd66d0806b027fa03dc16dbe0?s=47 Hibri Marzook
February 27, 2017

Evolving Continuous Delivery

This is a story of guiding a team's CD adoption and maturity, from a chaotic start, to being awesome.

This is a journey from a troublesome, struggling start of chaotic manual deployments, merge hell, regular production roll backs and lost code, to deliver a single commit to master automatically and reliably, under an hour, many times a day.

We’ll look at a few simple practices that worked, to introduce CD practices such as monitoring, build automation and test automation gradually, instead of going all in at first.

You’ll start to see that it isn’t daunting at all, and start delivering with joy.

Related post: http://www.hibri.net//2016/06/18/continuous-delivery-rags-to-riches/

80574e8bd66d0806b027fa03dc16dbe0?s=128

Hibri Marzook

February 27, 2017
Tweet

Transcript

  1. Evolving Continuous Delivery From Rags to Riches Hibri Marzook @hibri

  2. @hibri http://www.hibri.net

  3. Hibri Marzook, February 2017 @hibri Fintech Post startup phase Successful

    product in the market ~ 30 in engineering ~ 200 employees Context
  4. Hibri Marzook, February 2017 @hibri First impressions Public facing API,

    running in the Cloud No automation Inconsistent environments Context switching. Not finishing work in progress Old, stale branches merged into active branch Manual deploys
  5. Hibri Marzook, February 2017 @hibri

  6. Hibri Marzook, February 2017 @hibri What to fix first? Branching

    strategy? Lack of automation? Lack of technical practices? TDD? Lack of CI? Process? Scrum, Kanban ceremonies?
  7. Hibri Marzook, February 2017 @hibri A focus on delivery is

    needed for continuous improvement
  8. Hibri Marzook, February 2017 @hibri Gain situational awareness Photo Credit

    - Hugh Trainer via Flickr
  9. Hibri Marzook, February 2017 @hibri Gaining situational awareness Install an

    application monitoring tool. New Relic Zero friction install Alerting out of the box PagerDuty integration
  10. Hibri Marzook, February 2017 @hibri We could be the first

    responders How much traffic? How many users are affected? Usage trends? Error rates? Can this scale?
  11. Hibri Marzook, February 2017 @hibri Shared understanding of the system

    Draw environment map Draw components of the system Ensure that all team members have the same understanding Keep maps visible
  12. Hibri Marzook, February 2017 @hibri Better situational awareness Photo Credit

    - Steve Jurveston via Flickr
  13. Hibri Marzook, February 2017 @hibri

  14. Hibri Marzook, February 2017 @hibri Expose the realities of the

    system instead of relying on assumptions
  15. Hibri Marzook, February 2017 @hibri In a legacy distributed application,

    there is a disconnect between how the system behaves in production and how the creators think it “should” behave.
  16. Hibri Marzook, February 2017 @hibri Create visibility

  17. Hibri Marzook, February 2017 @hibri Agile ≠ JIRA

  18. Hibri Marzook, February 2017 @hibri Creating visibility Made work visible

    with a simple card wall Pull from the right attitude Use JIRA only for tracking Created transparency about the work we are doing
  19. Hibri Marzook, February 2017 @hibri Limit work in progress Did

    one thing at a time Got into the habit of finishing one thing Whole team focussed on delivery Only then did we do more
  20. Hibri Marzook, February 2017 @hibri When you don’t have anything

    else, put everyone together
  21. Hibri Marzook, February 2017 @hibri Put everyone together Everyone we

    worked with sat in the same area Introduced Slack Devs are always around to fix an issue Practice remote working early
  22. Hibri Marzook, February 2017 @hibri

  23. Hibri Marzook, February 2017 @hibri Use slack time to learn

    and improve
  24. Hibri Marzook, February 2017 @hibri How did we create slack?

    WIP Limits Production issues are quantified, don’t need to jump on every issue Have data to push back on management pressure Can ignore some problems
  25. Hibri Marzook, February 2017 @hibri Long term change happens during

    slack time
  26. Hibri Marzook, February 2017 @hibri Before CI Couldn’t reliably build

    the system. “Works on my machine” Key man dependencies to do a release. Build time dependencies have to be fixed manually.
  27. Hibri Marzook, February 2017 @hibri

  28. Hibri Marzook, February 2017 @hibri When the team spends more

    time talking about branching and merging, you have the wrong strategy
  29. Hibri Marzook, February 2017 @hibri Introducing CI Built the system

    reliably and repeatedly on a CI server Removed the key man dependency Used this build as the source for manual deploys No ATDD yet Manual regression testing
  30. Hibri Marzook, February 2017 @hibri Getting branching under control

  31. Hibri Marzook, February 2017 @hibri

  32. Hibri Marzook, February 2017 @hibri Amplify the good things

  33. Hibri Marzook, February 2017 @hibri Adding automated testing Made the

    tests work in a local dev environment Made the same tests run in other environments Kept adding tests for all new work and fixes Refactored tests continuously
  34. Hibri Marzook, February 2017 @hibri Adding automated deploys Deployed to

    test environments on every commit Used Azure’s “slots” feature for blue/green TeamCity + Octopus Deploy Smoke tests after every deploy
  35. Hibri Marzook, February 2017 @hibri

  36. Hibri Marzook, February 2017 @hibri The ‘A’ word

  37. Hibri Marzook, February 2017 @hibri Talk and discuss the values

    Focus on the impact Explain cause and effect Introduce and repeat the principles Agile manifesto. Left over right
  38. Hibri Marzook, February 2017 @hibri https://landing.google.com/sre/book/chapters/part3.html#fig_part-practices_reliability-hierarchy

  39. Hibri Marzook, February 2017 @hibri Evolutionary approach Deal with current

    constraints Sometimes it’s OK to do things manually Every change should make sense and help the team a little bit more Team should not have to wait for all the changes to gain value
  40. Hibri Marzook, February 2017 @hibri Have a primal push to

    get your software into the hands of users
  41. Hibri Marzook, February 2017 @hibri Thanks! Questions? @hibri hibri@hibri.net