Small batch deployments - 2019-05-17

#ContinuousDelivery
#Deployments
#DevOps

Transcript

1. Small Batch Deployments Sean Sullivan May 17, 2019
2. Principles of Continuous Delivery continuousdelivery.com
3. 1986: Fred Brooks “No Silver Bullet” paper Growing software organically, adding more and more function to systems as they are run, used, and tested. 

4. 1999: Kent Beck * first edition, published October 1999 We need to control the development of software by making many small adjustments, not by making a few large adjustments, kind of like driving a car.
5. 1999: Kent Beck * first edition, published October 1999 Big changes made all at once just don't work. Any problem is solved with a series of the smallest changes that make a difference.
6. 2003: Poppendieck * book published May 2003 Remember the rule of small batches: If you integrate changes in small batches, it will be infinitely easier to detect and fix problems.
7. 2004: Kent Beck * second edition, published November 2004 Software development has long delivered value in big chunks. Many teams make the problem worse by tending to respond to stress by making the chunks of value bigger, from deploying software less frequently to integrating less often.
8. 2004: Kent Beck * second edition, published November 2004 The more things are deferred, the larger the chunk, the higher the risk. In contrast, the principle of flow suggests that for improvement, deploy smaller increments of value ever more frequently.
9. 2004: Kent Beck * second edition, published November 2004 It's always tempting to make big changes in big steps. Momentous change taken all at once is dangerous.
10. 2004: Kent Beck * second edition, published November 2004 Baby steps acknowledge that the overhead of small steps is much less than when a team wastefully recoils from aborted big changes.
11. 2004: Kent Beck * second edition, published November 2004 Baby steps are expressed in practices like test-first programming, which proceeds one test at a time, and continuous integration, which integrates and tests a few hours' worth of changes at a time.
12. 2004: Kent Beck * second edition, published November 2004 Big deployments have a high risk and high human and economic costs.
13. 2004: Kent Beck * second edition, published November 2004 Daily Deployment Put new software into production every night. Any gap between what is on a programmer's desk and what is in production is a risk.
14. 2004: Kent Beck * second edition, published November 2004 The deployment tools must be automated, including the ability to roll out incrementally and roll back in case of failure.
15. 2004: Kent Beck * second edition, published November 2004 There are many barriers to deploying frequently. Some are psychological or social, like a deployment process so stressful that people don't want to go through it twice as often.
16. 2004: Kent Beck * second edition, published November 2004 Whatever the barrier, working to remove it and then letting more frequent deployment come as a natural consequence will help you improve development.
17. 2006: Poppendieck * book published September 2006 Work in small batches Reduce project size. Shorten release cycles. Stabilize. Repeat.
18. 2007: Michael Nygard * first edition, published March 2007 One solution — the easy but harmful one — is to slow down the release calendar. Like going to the dentist less frequently because it hurts, this response to the problem can only exacerbate the issue.
19. 2007: Michael Nygard * first edition, published March 2007 The right response is to reduce the effort needed, remove people from the process, and make the whole thing more automated and standardized.
20. http://www.startuplessonslearned.com/2009/02/work-in-small-batches.html software should be designed, written, and deployed in small batches 2009: Eric Ries * February 2009
21. 2009: Flickr * Velocity Conference, June 2009 https://www.youtube.com/watch?v=LdOe18KhtT4
22. 2009: Flickr * Velocity Conference, June 2009 https://www.youtube.com/watch?v=LdOe18KhtT4
23. https://omniti.com/seeds/online-application-deployment-reducing-risk.html 2010: Theo Schlossnagle * published March 2010 micro-releases: releases must never get too large instead amortize risk with small, controlled releases
24. 2010: Facebook * Velocity Conference, June 2010 a culture of making very frequent small changes if you've only changed one thing at a time, it is really easy to figure out what broke the site https://www.youtube.com/watch?v=nEmJ_5UHs1g Robert Johnson
25. 2010: Etsy * Velocity Conference, June 2010 https://www.kitchensoap.com/2010/06/24/ops-meta-metrics-velocity-2010-slides/ John Allspaw
26. 2010: Etsy * Velocity Conference, June 2010 https://www.kitchensoap.com/2010/06/24/ops-meta-metrics-velocity-2010-slides/ John Allspaw
27. * book published August 2010 2010: Continuous Delivery Imagine that you released frequently, so the delta between what is currently in production and the new release is small. If that were true, the risk of release would be greatly diminished
28. incremental deployment 2012: gilt.com deploy early, deploy often * Portland Oregon, January 2012
29. Work in small batches. With a small batch size, you are reducing the amount of complexity that has to be dealt with at any one time by the people working on the batch. Break down large releases into small units of deployment 2012: Damon Edwards http://dev2ops.org/2012/03/devops-lessons-from-lean-small-batches-improve-flow/ * published March 2012
30. each release is a rollback point small incremental releases actually help reduce risk 2012: Sam Newman * GeeCon, May 2012
31. https://www.joelonsoftware.com/2012/07/09/software-inventory/ * published July 2012 2012: Joel Spolsky If you’re already shipping monthly, figure out how to ship weekly. Keep pushing the bar on more and more frequent deployment of smaller and smaller changes.
32. 2012: PuppetConf https://twitter.com/RealGeneKim/status/251457175128530944 * tweet, September 2012
33. How long would it take your organization to deploy a change that involved just one single line of code? Do you do this on a repeatable, reliable basis? * October 2012 2012: Jez Humble https://www.youtube.com/watch?v=skLJuksCRTw
34. 2012: Etsy https://www.youtube.com/watch?v=JR-ccCTmMKY * GOTO Aarhus conference, October 2012
35. 2012: Don Reinertsen https://www.youtube.com/watch?v=cAMyGVSj9m4 * December 2012
36. 2012: Don Reinertsen https://www.youtube.com/watch?v=cAMyGVSj9m4 * December 2012
37. And deploying smaller sets of features more frequently reduces the number of changes per deployment, which is an inherent benefit of continuous delivery and helps mitigate risk by making it easier to identify and triage problems if things go south during a deployment. https://medium.com/netflix-techblog/deploying-the-netflix-api-79b6176cc3f0 2013: Netflix * published August 2013
38. 2013: gilt.com continuous delivery is critical Every release is small. You know what's changed in production. If it doesn't work, rollback is simple * Surge conference, September 2013 https://www.youtube.com/watch?v=8-6azNVq2X0
39. We love to deploy new code incrementally hundreds of times a day. And there's good reason for that: it's safer overall. Incremental deploys are easier to understand and fix than one gigantic deploy once a year. https://zachholman.com/talk/move-fast-break-nothing/ 2014: Github * published October 2014
40. https://codeascraft.com/2015/02/20/re-introducing-deployinator-now-as-a-gem/ 2015: Etsy We deploy code about 40 times per day. This allows us to push smaller changes we are confident about and experiment at a fast rate. Jayson Paul * published February 2015
41. in practice, large-impact, high- risk deployments end up happening infrequently due to understandable fear unfortunately, this means that our changes build up between releases 2015: Building Microservices * book published February 2015
42. the bigger the delta between releases, the higher the risk that we’ll get something wrong! * book published February 2015 2015: Building Microservices
43. throughout the book, I promote the need to make small, incremental changes one of the key drivers is to understand the impact of each alteration * book published February 2015 2015: Building Microservices
44. 2015: Lyft * DockerCon, June 2015 https://www.youtube.com/watch?v=iC2T3gJsB0g Matthew Leventi
45. 2015: gilt.com * QCon London, July 2015 simpler and easier deployments and rollbacks Yoni Goldberg teams can decide when they want to deploy deploy multiple times per day
46. 2015: gilt.com * QCon London, July 2015 database schema changes are required to be incremental Yoni Goldberg database schema changes are released separately from service code
47. 2016: Instagram * SREcon, April 2016 Michael Gorven https://www.youtube.com/watch?v=XNJ30Vy5xSg
48. 2016: Instagram * SREcon, April 2016 Michael Gorven https://www.youtube.com/watch?v=XNJ30Vy5xSg
49. 2016: Instagram * SREcon, April 2016 Michael Gorven https://www.youtube.com/watch?v=XNJ30Vy5xSg
50. Simple releases are generally better than complicated releases. 2016: Google * book published April 2016
51. It is much easier to measure and understand the impact of a single change rather than a batch of changes released simultaneously. 2016: Google * book published April 2016
52. If the release is performed in smaller batches, we can move faster with more confidence because each code change can be understood in isolation in the larger system. 2016: Google * book published April 2016
53. 2016: Uber * published June 2016 https://eng.uber.com/micro-deploy/
54. 2016: Uber * published June 2016 https://eng.uber.com/micro-deploy/ Engineers write code, review it, test it, and put it into production the same day. Each week in 2016, thousands of engineers push to prod several thousand service builds
55. * book published June 2016 2016: Microservice Architecture reducing the size of the deployment unit a notion we call “batch size reduction” reducing the size or scope of the problem
56. emphasis on small batch sizes 2016: DevOps Handbook the principle of small batch sizes also applies to code reviews * book published October 2016
57. make regular, small, incremental changes 2016: AWS re:Invent * Las Vegas - December 1, 2016 Tim DiLauro AWS Solutions Architect https://www.youtube.com/watch?v=8e2zCHx-K3M which allows you to ultimately move faster
58. 2017: Uber * SRECon, March 2017 Sebastian Yates https://www.youtube.com/watch?v=4A4SLk1PRvg
59. 2017: Uber * SRECon, March 2017 Sebastian Yates https://www.youtube.com/watch?v=4A4SLk1PRvg [service owners are] able to deploy at any schedule they like they can do 40 deploys a day
60. the heart of doing evolutionary architecture is to make small changes * book published September 2017 2017: Building Evolutionary Architectures
61. an architecture that allows small, incremental changes is easier to evolve because developers have a smaller scope of change 2017: Building Evolutionary Architectures * book published September 2017
62. with improved engineering practices comes decreased risk 2017: Building Evolutionary Architectures * book published September 2017
63. Melanie Cebula 2017: Airbnb * FutureStack New York, September 2017 https://www.youtube.com/watch?v=N1BWMW9NEQc
64. * second edition, published January 2018 designing for deployment gives you the ability to make large changes in small steps 2018: Release It!
65. * second edition, published January 2018 to be successful, your software will be deployed early and often smaller, easier deployments mean you can make big changes over a series of small steps 2018: Release It!
66. 2018: Accelerate * book published March 2018
67. * tweet, April 2018 https://twitter.com/bridgetkromhout/status/983739341074845697 2018: Bridget Kromhout
68. 2018: Michele Titolo Microservice deployments should be small. If you have small services, you are probably making small changes. * QCon New York, June 2018
69. 2018: Michele Titolo One of the benefits of small changes is that you can release frequently * QCon New York, June 2018
70. 2018: Flow Commerce * QCon New York, June 2018 Mike Bryzek
71. 2018: Flow Commerce * QCon New York, June 2018 Mike Bryzek
72. 2018: Charity Majors https://charity.wtf/2018/08/19/shipping-software-should-not-be-scary/ * published August 2018 suffice it to say that we now know that smaller and more frequent changes are much safer than larger and less frequent changes ship early, ship often, ship smaller change sets
73. 2019: Financial Times * QCon London, March 2019 https://www.infoq.com/presentations/microservices-financial-times Sarah Wells
74. 2019: Jez Humble * tweet, March 2019 https://twitter.com/jezhumble/status/1106297185119797248
75. 2019: Starbucks * tweet, April 10 2019 https://twitter.com/davidbrunelle/status/1116169369879363584
76. 2019: Starbucks * tweet, April 10 2019 https://twitter.com/davidbrunelle/status/1116169370911141888
77. 2019: Charity Majors * tweet, April 15 2019 https://twitter.com/mipsytipsy/status/1117858831294320640
78. 2019: Charity Majors * published May 1, 2019 https://charity.wtf/2019/05/01/friday-deploy-freezes-are-exactly-like-murdering-puppies/ Fear of deploys is the ultimate technical debt Do not batch up multiple people’s commits into a deploy Good teams ship often
79. 2019: Adrian Cockcroft * CraftConf, May 9 2019 https://www.youtube.com/watch?v=rnygCAvVBj8
80. 2019: Adrian Cockcroft * CraftConf, May 9 2019 https://www.youtube.com/watch?v=rnygCAvVBj8
81. 2019: Adrian Cockcroft * CraftConf, May 9 2019 https://www.youtube.com/watch?v=rnygCAvVBj8
82. 2019: Adrian Cockcroft * CraftConf, May 9 2019 https://www.youtube.com/watch?v=rnygCAvVBj8
83. 2019: Adrian Cockcroft * CraftConf, May 9 2019 https://www.youtube.com/watch?v=rnygCAvVBj8 Number of meetings per deploy: drive to zero You don’t have time to have 10 meetings a day for the 10 times you deploy today.
84. 2019: Adrian Cockcroft * CraftConf, May 9 2019 https://www.youtube.com/watch?v=rnygCAvVBj8 If you have a competitor that is updating 10 times a day and you are updating every 2 weeks, then you will fall behind.
85. 2019: Twilio * CraftConf, May 10 2019 https://www.youtube.com/watch?v=BIkDeTEdbSo Justin Kitagawa
86. Make It So small batches of work frequent deployments