Continuous Delivery For the Rest of Us

Continuous Delivery For the Rest of Us

Companies like Netflix, Google and Amazon are leading the way with Continuous Delivery. Most of us don’t work in a company like them. We don’t have a simian army. We don’t have a canary testing process. Working out where to start can feel overwhelming.

This talk provides simple tips and tricks for improving delivery without investing lots of time up front creating complex deployment frameworks.

Culture and process is as important as tools and automation. This talk gives real-world examples of introducing the principles and practices of Continuous Delivery to existing teams, and some of the difficulties encountered along the way.

68d40c5cc1496c2abc1f862dd148a036?s=128

Lisa van Gelder

May 20, 2015
Tweet

Transcript

  1. 3.

    Continuous Delivery Is not: • Continuous deployment • Automating all

    the things Is: • Removing the bottlenecks that stop you delivering
  2. 4.
  3. 5.
  4. 6.
  5. 7.
  6. 8.
  7. 9.
  8. 10.
  9. 11.
  10. 12.
  11. 13.
  12. 14.
  13. 16.
  14. 17.

    Continuous Delivery • Measure the total cycle time • Reduce

    your cycle time • Improve your reaction time
  15. 20.

    Benefits of smaller releases • Code only has value in

    production - get it there quicker! • Less co-ordination required • Easier to test • Easier to see if it caused issues in production • Easier to rollback
  16. 21.
  17. 23.

    Blockers • Releases cause problems for users • Ops team

    don’t have time to do more releases • Ops/dev don’t have time to support more releases • QA don’t have time to test new features • Takes too long to get a green build
  18. 25.

    The cms that made journalists stop work for 20 minutes…

    • db changes tied to code changes • too much state in session
  19. 26.

    Releases cause performance problems • play back logs • soak

    test • dark launch • performance test as soon as you can
  20. 30.
  21. 31.
  22. 32.

    Ops/dev don’t have time to watch releases • Release interrupts

    normal work • Team on standby in case of issues
  23. 33.
  24. 34.
  25. 35.

    Rollbacks • The risk is much higher when rollback isn’t

    possible • Rollbacks should be normal operation, not a failure • Users don’t care about your new feature if the site is down 1. It should be possible to rollback 2. It should be quick to rollback
  26. 36.
  27. 37.

    A release should be a non- event • Done in

    working hours • Easy to monitor • Easy to rollback
  28. 39.
  29. 40.

    Cross-functional issues • release waiting on qa sign off •

    release waiting on product manager sign off • developers waiting on designs • ux waiting on developers • front-end developers waiting on back-end
  30. 41.
  31. 42.

    Cross functional teams • don’t start story if all resources

    aren’t available • blockers should block • when work is held up - can someone else perform that function?
  32. 43.

    Takes too long to get a green build • Flaky

    tests • Slow-running tests • Merge hell
  33. 44.

    Flaky tests • The tests that cry wolf • Isolate

    them • Fix them or delete them
  34. 45.

    Slow running tests • More than 5 minutes is slow

    • Waste of developer time • Interrupt flow to fix • People deploy without waiting for tests • Frequent broken build
  35. 46.

    Slow running tests • separate unit tests from acceptance tests

    • limit the amount of acceptance tests • mock dependencies - limit calls to db • run tests in parallel
  36. 47.

    Merge hell • Continuous Integration is more often than you

    think! • Don’t have long lived feature branches • Check in to master at least once a day • Feature switches • Branch by abstraction
  37. 48.
  38. 51.

    • Have a performance test environment that is a scaled-

    down replica of production • Automate log collection, make sure tests reflect current traffic patterns. • Use your application-specific metrics • Define acceptable ranges for your application
  39. 52.

    Automate release process What criteria do humans use to evaluate

    a successful release? Use your application-specific metrics and acceptable ranges
  40. 55.

    Suggestions for further reading • Continuous Delivery by Jez Humble

    & David Farley • The Phoenix Project by Gene Kim, Kevin Behr & George Spafford • The Goal by Eliyahu M. Goldratt & Jeff Cox • Lean Software Development by Mary Poppendieck & Tom Poppendieck • Release It! by Michael T. Nygard
  41. 56.

    Image Credits: https://www.flickr.com/photos/raster/ http://commons.wikimedia.org/wiki/User:Ozhiker NASA/JPL/Cornell University, Maas Digital LLC -

    http:// photojournal.jpl.nasa.gov/catalog/PIA04413 https://www.flickr.com/photos/godutchbaby/ http://graphite.wikidot.com/screen-shots http://www.theguardian.com/info/developer-blog/2012/oct/04/winning- the-metrics-battle