Evolving Continuous Delivery

80574e8bd66d0806b027fa03dc16dbe0?s=47 Hibri Marzook
February 27, 2017

Evolving Continuous Delivery

This is a story of guiding a team's CD adoption and maturity, from a chaotic start, to being awesome.

This is a journey from a troublesome, struggling start of chaotic manual deployments, merge hell, regular production roll backs and lost code, to deliver a single commit to master automatically and reliably, under an hour, many times a day.

We’ll look at a few simple practices that worked, to introduce CD practices such as monitoring, build automation and test automation gradually, instead of going all in at first.

You’ll start to see that it isn’t daunting at all, and start delivering with joy.

Related post: http://www.hibri.net//2016/06/18/continuous-delivery-rags-to-riches/

80574e8bd66d0806b027fa03dc16dbe0?s=128

Hibri Marzook

February 27, 2017
Tweet

Transcript

  1. 3.

    Hibri Marzook, February 2017 @hibri Fintech Post startup phase Successful

    product in the market ~ 30 in engineering ~ 200 employees Context
  2. 4.

    Hibri Marzook, February 2017 @hibri First impressions Public facing API,

    running in the Cloud No automation Inconsistent environments Context switching. Not finishing work in progress Old, stale branches merged into active branch Manual deploys
  3. 6.

    Hibri Marzook, February 2017 @hibri What to fix first? Branching

    strategy? Lack of automation? Lack of technical practices? TDD? Lack of CI? Process? Scrum, Kanban ceremonies?
  4. 7.
  5. 9.

    Hibri Marzook, February 2017 @hibri Gaining situational awareness Install an

    application monitoring tool. New Relic Zero friction install Alerting out of the box PagerDuty integration
  6. 10.

    Hibri Marzook, February 2017 @hibri We could be the first

    responders How much traffic? How many users are affected? Usage trends? Error rates? Can this scale?
  7. 11.

    Hibri Marzook, February 2017 @hibri Shared understanding of the system

    Draw environment map Draw components of the system Ensure that all team members have the same understanding Keep maps visible
  8. 14.

    Hibri Marzook, February 2017 @hibri Expose the realities of the

    system instead of relying on assumptions
  9. 15.

    Hibri Marzook, February 2017 @hibri In a legacy distributed application,

    there is a disconnect between how the system behaves in production and how the creators think it “should” behave.
  10. 18.

    Hibri Marzook, February 2017 @hibri Creating visibility Made work visible

    with a simple card wall Pull from the right attitude Use JIRA only for tracking Created transparency about the work we are doing
  11. 19.

    Hibri Marzook, February 2017 @hibri Limit work in progress Did

    one thing at a time Got into the habit of finishing one thing Whole team focussed on delivery Only then did we do more
  12. 21.

    Hibri Marzook, February 2017 @hibri Put everyone together Everyone we

    worked with sat in the same area Introduced Slack Devs are always around to fix an issue Practice remote working early
  13. 24.

    Hibri Marzook, February 2017 @hibri How did we create slack?

    WIP Limits Production issues are quantified, don’t need to jump on every issue Have data to push back on management pressure Can ignore some problems
  14. 26.

    Hibri Marzook, February 2017 @hibri Before CI Couldn’t reliably build

    the system. “Works on my machine” Key man dependencies to do a release. Build time dependencies have to be fixed manually.
  15. 28.

    Hibri Marzook, February 2017 @hibri When the team spends more

    time talking about branching and merging, you have the wrong strategy
  16. 29.

    Hibri Marzook, February 2017 @hibri Introducing CI Built the system

    reliably and repeatedly on a CI server Removed the key man dependency Used this build as the source for manual deploys No ATDD yet Manual regression testing
  17. 33.

    Hibri Marzook, February 2017 @hibri Adding automated testing Made the

    tests work in a local dev environment Made the same tests run in other environments Kept adding tests for all new work and fixes Refactored tests continuously
  18. 34.

    Hibri Marzook, February 2017 @hibri Adding automated deploys Deployed to

    test environments on every commit Used Azure’s “slots” feature for blue/green TeamCity + Octopus Deploy Smoke tests after every deploy
  19. 37.

    Hibri Marzook, February 2017 @hibri Talk and discuss the values

    Focus on the impact Explain cause and effect Introduce and repeat the principles Agile manifesto. Left over right
  20. 39.

    Hibri Marzook, February 2017 @hibri Evolutionary approach Deal with current

    constraints Sometimes it’s OK to do things manually Every change should make sense and help the team a little bit more Team should not have to wait for all the changes to gain value
  21. 40.

    Hibri Marzook, February 2017 @hibri Have a primal push to

    get your software into the hands of users