Small batch deployments - 2019-05-17

Small batch deployments - 2019-05-17

#ContinuousDelivery
#Deployments
#DevOps

99d4c0ed3c8027ea1c37b53d6441d042?s=128

sullis

May 17, 2019
Tweet

Transcript

  1. Small Batch Deployments Sean Sullivan May 17, 2019

  2. Principles of Continuous Delivery continuousdelivery.com

  3. 1986: Fred Brooks “No Silver Bullet” paper Growing software organically,

    adding more and more function to systems as they are run, used, and tested. 

  4. 1999: Kent Beck * first edition, published October 1999 We

    need to control the development of software by making many small adjustments, not by making a few large adjustments, kind of like driving a car.
  5. 1999: Kent Beck * first edition, published October 1999 Big

    changes made all at once just don't work. Any problem is solved with a series of the smallest changes that make a difference.
  6. 2003: Poppendieck * book published May 2003 Remember the rule

    of small batches: If you integrate changes in small batches, it will be infinitely easier to detect and fix problems.
  7. 2004: Kent Beck * second edition, published November 2004 Software

    development has long delivered value in big chunks. Many teams make the problem worse by tending to respond to stress by making the chunks of value bigger, from deploying software less frequently to integrating less often.
  8. 2004: Kent Beck * second edition, published November 2004 The

    more things are deferred, the larger the chunk, the higher the risk. In contrast, the principle of flow suggests that for improvement, deploy smaller increments of value ever more frequently.
  9. 2004: Kent Beck * second edition, published November 2004 It's

    always tempting to make big changes in big steps. Momentous change taken all at once is dangerous.
  10. 2004: Kent Beck * second edition, published November 2004 Baby

    steps acknowledge that the overhead of small steps is much less than when a team wastefully recoils from aborted big changes.
  11. 2004: Kent Beck * second edition, published November 2004 Baby

    steps are expressed in practices like test-first programming, which proceeds one test at a time, and continuous integration, which integrates and tests a few hours' worth of changes at a time.
  12. 2004: Kent Beck * second edition, published November 2004 Big

    deployments have a high risk and high human and economic costs.
  13. 2004: Kent Beck * second edition, published November 2004 Daily

    Deployment Put new software into production every night. Any gap between what is on a programmer's desk and what is in production is a risk.
  14. 2004: Kent Beck * second edition, published November 2004 The

    deployment tools must be automated, including the ability to roll out incrementally and roll back in case of failure.
  15. 2004: Kent Beck * second edition, published November 2004 There

    are many barriers to deploying frequently. Some are psychological or social, like a deployment process so stressful that people don't want to go through it twice as often.
  16. 2004: Kent Beck * second edition, published November 2004 Whatever

    the barrier, working to remove it and then letting more frequent deployment come as a natural consequence will help you improve development.
  17. 2006: Poppendieck * book published September 2006 Work in small

    batches Reduce project size. Shorten release cycles. Stabilize. Repeat.
  18. 2007: Michael Nygard * first edition, published March 2007 One

    solution — the easy but harmful one — is to slow down the release calendar. Like going to the dentist less frequently because it hurts, this response to the problem can only exacerbate the issue.
  19. 2007: Michael Nygard * first edition, published March 2007 The

    right response is to reduce the effort needed, remove people from the process, and make the whole thing more automated and standardized.
  20. http://www.startuplessonslearned.com/2009/02/work-in-small-batches.html software should be designed, written, and deployed in small

    batches 2009: Eric Ries * February 2009
  21. 2009: Flickr * Velocity Conference, June 2009 https://www.youtube.com/watch?v=LdOe18KhtT4

  22. 2009: Flickr * Velocity Conference, June 2009 https://www.youtube.com/watch?v=LdOe18KhtT4

  23. https://omniti.com/seeds/online-application-deployment-reducing-risk.html 2010: Theo Schlossnagle * published March 2010 micro-releases: releases

    must never get too large instead amortize risk with small, controlled releases
  24. 2010: Facebook * Velocity Conference, June 2010 a culture of

    making very frequent small changes if you've only changed one thing at a time, it is really easy to figure out what broke the site https://www.youtube.com/watch?v=nEmJ_5UHs1g Robert Johnson
  25. 2010: Etsy * Velocity Conference, June 2010 https://www.kitchensoap.com/2010/06/24/ops-meta-metrics-velocity-2010-slides/ John Allspaw

  26. 2010: Etsy * Velocity Conference, June 2010 https://www.kitchensoap.com/2010/06/24/ops-meta-metrics-velocity-2010-slides/ John Allspaw

  27. * book published August 2010 2010: Continuous Delivery Imagine that

    you released frequently, so the delta between what is currently in production and the new release is small. If that were true, the risk of release would be greatly diminished
  28. incremental deployment 2012: gilt.com deploy early, deploy often * Portland

    Oregon, January 2012
  29. Work in small batches. With a small batch size, you

    are reducing the amount of complexity that has to be dealt with at any one time by the people working on the batch. Break down large releases into small units of deployment 2012: Damon Edwards http://dev2ops.org/2012/03/devops-lessons-from-lean-small-batches-improve-flow/ * published March 2012
  30. each release is a rollback point small incremental releases actually

    help reduce risk 2012: Sam Newman * GeeCon, May 2012
  31. https://www.joelonsoftware.com/2012/07/09/software-inventory/ * published July 2012 2012: Joel Spolsky If you’re

    already shipping monthly, figure out how to ship weekly. Keep pushing the bar on more and more frequent deployment of smaller and smaller changes.
  32. 2012: PuppetConf https://twitter.com/RealGeneKim/status/251457175128530944 * tweet, September 2012

  33. How long would it take your organization to deploy a

    change that involved just one single line of code? Do you do this on a repeatable, reliable basis? * October 2012 2012: Jez Humble https://www.youtube.com/watch?v=skLJuksCRTw
  34. 2012: Etsy https://www.youtube.com/watch?v=JR-ccCTmMKY * GOTO Aarhus conference, October 2012

  35. 2012: Don Reinertsen https://www.youtube.com/watch?v=cAMyGVSj9m4 * December 2012

  36. 2012: Don Reinertsen https://www.youtube.com/watch?v=cAMyGVSj9m4 * December 2012

  37. And deploying smaller sets of features more frequently reduces the

    number of changes per deployment, which is an inherent benefit of continuous delivery and helps mitigate risk by making it easier to identify and triage problems if things go south during a deployment. https://medium.com/netflix-techblog/deploying-the-netflix-api-79b6176cc3f0 2013: Netflix * published August 2013
  38. 2013: gilt.com continuous delivery is critical Every release is small.

    You know what's changed in production. If it doesn't work, rollback is simple * Surge conference, September 2013 https://www.youtube.com/watch?v=8-6azNVq2X0
  39. We love to deploy new code incrementally hundreds of times

    a day. And there's good reason for that: it's safer overall. Incremental deploys are easier to understand and fix than one gigantic deploy once a year. https://zachholman.com/talk/move-fast-break-nothing/ 2014: Github * published October 2014
  40. https://codeascraft.com/2015/02/20/re-introducing-deployinator-now-as-a-gem/ 2015: Etsy We deploy code about 40 times per

    day. This allows us to push smaller changes we are confident about and experiment at a fast rate. Jayson Paul * published February 2015
  41. in practice, large-impact, high- risk deployments end up happening infrequently

    due to understandable fear unfortunately, this means that our changes build up between releases 2015: Building Microservices * book published February 2015
  42. the bigger the delta between releases, the higher the risk

    that we’ll get something wrong! * book published February 2015 2015: Building Microservices
  43. throughout the book, I promote the need to make small,

    incremental changes one of the key drivers is to understand the impact of each alteration * book published February 2015 2015: Building Microservices
  44. 2015: Lyft * DockerCon, June 2015 https://www.youtube.com/watch?v=iC2T3gJsB0g Matthew Leventi

  45. 2015: gilt.com * QCon London, July 2015 simpler and easier

    deployments and rollbacks Yoni Goldberg teams can decide when they want to deploy deploy multiple times per day
  46. 2015: gilt.com * QCon London, July 2015 database schema changes

    are required to be incremental Yoni Goldberg database schema changes are released separately from service code
  47. 2016: Instagram * SREcon, April 2016 Michael Gorven https://www.youtube.com/watch?v=XNJ30Vy5xSg

  48. 2016: Instagram * SREcon, April 2016 Michael Gorven https://www.youtube.com/watch?v=XNJ30Vy5xSg

  49. 2016: Instagram * SREcon, April 2016 Michael Gorven https://www.youtube.com/watch?v=XNJ30Vy5xSg

  50. Simple releases are generally better than complicated releases. 2016: Google

    * book published April 2016
  51. It is much easier to measure and understand the impact

    of a single change rather than a batch of changes released simultaneously. 2016: Google * book published April 2016
  52. If the release is performed in smaller batches, we can

    move faster with more confidence because each code change can be understood in isolation in the larger system. 2016: Google * book published April 2016
  53. 2016: Uber * published June 2016 https://eng.uber.com/micro-deploy/

  54. 2016: Uber * published June 2016 https://eng.uber.com/micro-deploy/ Engineers write code,

    review it, test it, and put it into production the same day. Each week in 2016, thousands of engineers push to prod several thousand service builds
  55. * book published June 2016 2016: Microservice Architecture reducing the

    size of the deployment unit a notion we call “batch size reduction” reducing the size or scope of the problem
  56. emphasis on small batch sizes 2016: DevOps Handbook the principle

    of small batch sizes also applies to code reviews * book published October 2016
  57. make regular, small, incremental changes 2016: AWS re:Invent * Las

    Vegas - December 1, 2016 Tim DiLauro AWS Solutions Architect https://www.youtube.com/watch?v=8e2zCHx-K3M which allows you to ultimately move faster
  58. 2017: Uber * SRECon, March 2017 Sebastian Yates https://www.youtube.com/watch?v=4A4SLk1PRvg

  59. 2017: Uber * SRECon, March 2017 Sebastian Yates https://www.youtube.com/watch?v=4A4SLk1PRvg [service

    owners are] able to deploy at any schedule they like they can do 40 deploys a day
  60. the heart of doing evolutionary architecture is to make small

    changes * book published September 2017 2017: Building Evolutionary Architectures
  61. an architecture that allows small, incremental changes is easier to

    evolve because developers have a smaller scope of change 2017: Building Evolutionary Architectures * book published September 2017
  62. with improved engineering practices comes decreased risk 2017: Building Evolutionary

    Architectures * book published September 2017
  63. Melanie Cebula 2017: Airbnb * FutureStack New York, September 2017

    https://www.youtube.com/watch?v=N1BWMW9NEQc
  64. * second edition, published January 2018 designing for deployment gives

    you the ability to make large changes in small steps 2018: Release It!
  65. * second edition, published January 2018 to be successful, your

    software will be deployed early and often smaller, easier deployments mean you can make big changes over a series of small steps 2018: Release It!
  66. 2018: Accelerate * book published March 2018

  67. * tweet, April 2018 https://twitter.com/bridgetkromhout/status/983739341074845697 2018: Bridget Kromhout

  68. 2018: Michele Titolo Microservice deployments should be small. If you

    have small services, you are probably making small changes. * QCon New York, June 2018
  69. 2018: Michele Titolo One of the benefits of small changes

    is that you can release frequently * QCon New York, June 2018
  70. 2018: Flow Commerce * QCon New York, June 2018 Mike

    Bryzek
  71. 2018: Flow Commerce * QCon New York, June 2018 Mike

    Bryzek
  72. 2018: Charity Majors https://charity.wtf/2018/08/19/shipping-software-should-not-be-scary/ * published August 2018 suffice it

    to say that we now know that smaller and more frequent changes are much safer than larger and less frequent changes ship early, ship often, ship smaller change sets
  73. 2019: Financial Times * QCon London, March 2019 https://www.infoq.com/presentations/microservices-financial-times Sarah

    Wells
  74. 2019: Jez Humble * tweet, March 2019 https://twitter.com/jezhumble/status/1106297185119797248

  75. 2019: Starbucks * tweet, April 10 2019 https://twitter.com/davidbrunelle/status/1116169369879363584

  76. 2019: Starbucks * tweet, April 10 2019 https://twitter.com/davidbrunelle/status/1116169370911141888

  77. 2019: Charity Majors * tweet, April 15 2019 https://twitter.com/mipsytipsy/status/1117858831294320640

  78. 2019: Charity Majors * published May 1, 2019 https://charity.wtf/2019/05/01/friday-deploy-freezes-are-exactly-like-murdering-puppies/ Fear

    of deploys is the ultimate technical debt Do not batch up multiple people’s commits into a deploy Good teams ship often
  79. 2019: Adrian Cockcroft * CraftConf, May 9 2019 https://www.youtube.com/watch?v=rnygCAvVBj8

  80. 2019: Adrian Cockcroft * CraftConf, May 9 2019 https://www.youtube.com/watch?v=rnygCAvVBj8

  81. 2019: Adrian Cockcroft * CraftConf, May 9 2019 https://www.youtube.com/watch?v=rnygCAvVBj8

  82. 2019: Adrian Cockcroft * CraftConf, May 9 2019 https://www.youtube.com/watch?v=rnygCAvVBj8

  83. 2019: Adrian Cockcroft * CraftConf, May 9 2019 https://www.youtube.com/watch?v=rnygCAvVBj8 Number

    of meetings per deploy: drive to zero You don’t have time to have 10 meetings a day for the 10 times you deploy today.
  84. 2019: Adrian Cockcroft * CraftConf, May 9 2019 https://www.youtube.com/watch?v=rnygCAvVBj8 If

    you have a competitor that is updating 10 times a day and you are updating every 2 weeks, then you will fall behind.
  85. 2019: Twilio * CraftConf, May 10 2019 https://www.youtube.com/watch?v=BIkDeTEdbSo Justin Kitagawa

  86. Make It So small batches of work frequent deployments

  87. None