Continuous Integration, Deployment, Delivery

Transcript

1. .co.il www. Continuous Integration, Continuous Deployment & Delivery
2. .co.il www. backlog of finished features not in production daily build ⇒ monthly release
3. .co.il www. Deploy code vs. Release feature dev deadline == deploy date == release date it doesn’t have to be this way …
4. Continuous Delivery A software development discipline where you build software in such a way that the software can be released to production at any time. -- Martin Fowler Code Build Test Deploy Release Monitor Idea Customer
5. Continuous Delivery Reduce the cost, time and risk of delivering incremental changes to users. -- Jez Humble
6. Continuous Integration Tools Jenkins CI / Bamboo / TeamCity CI-aaS - Travis CI / Circle CI / … magnum-ci.com / semaphoreapp.com / codeship.io / drone.io / solanolabs.com / shiningpanda-ci.com hosted-ci.com / fazend.com / appharbor.com / cloudbees.com /clinkerhq.com
7. Continuous Integration / CD ◦ 1997 Extreme Programming extremeprogramming.org/rules/integrateoften.html ◦ 2000 Martin Fowler martinfowler.com/articles/continuousIntegration.html ◦ 2007 Paul Duvall (CI book) ◦ 2010 Jez Humble (CD book)
8. .co.il www. “Between 12:00 AM and 11:59 PM on April 25, 2013, Quora released new versions of the site 46 times. This was a normal day for us.” - Quora / April 2013 “Deployment every 11.6 seconds, 1,079 max in one hour. 10,000 mean number of hosts per deployment, with 30,000 maximum” - Amazon.com / November 2012 “On the Google Consumer Surveys team, 8 minutes after you commit code it's live in production.” - Google / June 2014 “10+ deploys per day.” - John Allspaw / June 2009
9. .co.il www. How long would it take your organization to deploy a change that involved just one single line of code? Do you do this on a repeatable, reliable basis? Continuous Delivery slideshare.net/jezhumble/adopting-continuous-delivery continuousdelivery.com
10. .co.il www. Esty.com is an online marketplace for handmade and vintage items. Etsy.com employs 500+ people and has grown to a community of more than 25 million members and 1,000,000+ online shops who generated 1.35$ billion of sales in 2013. extfiles.etsy.com/progress-report/2013-Etsy-Progress-Report.pdf Deploying etsy.com 40+ times every day by a team of 175+ committers supporting 60+ million unique monthly visitors. youtube.com/watch?v=4pVOXLvoUyE infoq.com/news/2014/03/etsy-deploy-50-times-a-day slideshare.net/beamrider9/continuous-deployment-at-etsy-a-tale-of-two-approaches etsy.com
11. .co.il www. Business 250+ million active customer accounts. 1+ million active retail partners worldwide. Between 100-150 services are accessed to build a page. fool.com/investing/general/2014/05/24/how-many-customers-does-amazon-have. aspx Deployment 1,079 max deploys per hour affecting 30,000 hosts simultaneous receiving a deployment. youtube.com/watch?v=dxk8b9rSKOo amazon.com
12. .co.il www. Netflix.com 36 milliion members, more than one billion hours of video per month. http://www.infoq.com/presentations/netflix-architecture-resilience netflix.com
13. .co.il www. SAP.com FlowCon flowcon.org youtube.com/channel/UCMk1sRo1hnTLMA3kpn6BVKg SAP - youtube.com/watch?v=NJJdPlcCYK4 Before: monthly releases, 2 week qa cycle, late night incidents, 12 month lead time → After: bi-weekly release, <1 day qa, single-button rollback, 1 week lead time
14. .co.il www. Business HP software sells multiple categories of software, serving 94 of Fortune 100 companies. Third largest company in software revenue in the world. en.wikipedia.org/wiki/HP_Software_Division Deployment 1000+ builds per day, 825,000+ change requests in a year, 100,000+ tests run daily. h30614.www3.hp.com/collateral/Barcelona2014/presentations/B5756.pdf hp.com
15. .co.il www. Devops Enterprise Summit devopsenterprise.io Huawei CD - youtube.com/watch?v=G_coTcbj1FU Before: 30 days per feature delivery, 300min prod build, 1440min full test → After: 7 days per feature delivery, 10min prod build, 360min full test Ticketmaster - youtube.com/watch?v=01gfNOwhnro Before: Bi-monthyl/Quarterly releases → After: 109 deployments in September Target.com - youtube.com/watch?v=exrjV9V9vhY 30+ APIs, 80 deployments per week, <10 incidents per month
16. .co.il www. lead time for changes release frequency time to restore service change fail rate How long is the delay between a request for a change, and a production system operating with that change implemented? How long does it take for an abnormal behavior in the system to be restored to the normal standard agreed way of operation? How many changes and features are being released to production in a fixed period of time? How often the system fails or service is disrupted?
17. .co.il www. continuous integration Continuous integration usually consists of • compiling • running unit tests • deployable asset • pushing asset to repository repeated in a loop trigged by each commit
18. .co.il www. Visualise atlassian.com/wallboards/information-radiators.jsp
19. Stoplight
22. .co.il www.
23. .co.il www.
24. .co.il www.
25. .co.il www. Change Management
26. .co.il www.
27. .co.il www. 1950 1998
28. .co.il www. 1. identify 2. store 3. change 4. report
29. .co.il www. servers (vms) / storage operating system application servers application code + dependencies application configuration network monitoring
30. .co.il www.
31. .co.il www.
32. .co.il www. belongs to app code or to cmdb? ◦ packages & versions ◦ mail servers ◦ data centers ◦ database users & passwords ◦ connection strings ◦ integration urls ◦ deployments ◦ batch jobs
33. .co.il www. avoid regressions
34. .co.il www. avoid duplicate effort
35. .co.il www. avoid disaster, have recovery
36. .co.il www. Configuration Management
37. .co.il www.
38. .co.il www. Automation infoq.com/presentations/Infrastructure-as-Code #!/bin/sh NODES="webserver database" for n in $NODES do ssh $n complex-script.sh done
40. zeroturnaround.com/labs/rebel-labs-release-it-ops-devops-productivity-report-2013
41. ◦ open source ◦ manages server config. ◦ cookbooks & recipes ◦ ruby dsl OpsCode Chef
42. redis cookbook cookbooks/redis/recipes/default.rb package "redis-server" do action :upgrade end service "redis-server" do action :nothing supports status: true, restart: true end template "/etc/redis/redis.conf" do source "redis.conf.erb" owner "root" mode "0644" variables({ bind: node[:redis][:bind] }) notifies :restart, "service[redis-server]" end 1. cookbook/redis/attributes/default.rb 2. solo.json ... chef
43. cookbooks/redis/attributes/default.rb default[:redis][:bind] = "127.0.0.1" cookbooks/redis/templates/default/redis.conf.erb # If you want you can bind a single interface, if the bind option is not # specified all the interfaces will listen for incoming connections. bind <%= @bind %> ... template "/etc/redis/redis.conf" do ... variables({ bind: node[:redis][:bind] }) end redis cookbook ... chef
44. running chef-solo $ chef-solo -c solo.rb -j solo.json ... solo.rb cookbook_path "cookbooks" solo.json { "run_list": "recipe[redis]" } ... chef
45. $ sudo chef-solo -c solo.rb -j solo.json Starting Chef Client, version 11.4.0 Compiling Cookbooks... Converging 3 resources Recipe: redis::default * package[redis-server] action upgrade - upgrade package redis-server from uninstalled to 2:2.2.12-1build1 * service[redis-server] action nothing (up to date) * template[/etc/redis/redis.conf] action create (up to date) Chef Client finished, 1 resources updated First run installs redis, creates configuration file. ... chef
46. Second time, everything is already up to date. Idempotency! ƒ[ƒ(x)] ≡ ƒ(x) $ sudo chef-solo -c solo.rb -j solo.json Starting Chef Client, version 11.4.0 Compiling Cookbooks... Converging 3 resources Recipe: redis::default * package[redis-server] action upgrade (up to date) * service[redis-server] action nothing (up to date) * template[/etc/redis/redis.conf] action create (up to date) Chef Client finished, 0 resources updated ... chef
47. making a small change to JSON parameters passed to Chef run. running chef-solo + modification solo.json { "redis": { "bind": "0.0.0.0" }, "run_list": "recipe[redis]" } ... chef
48. $ sudo chef-solo -c solo.rb -j solo.json Starting Chef Client, version 11.4.0 Compiling Cookbooks... Converging 3 resources Recipe: redis::default * package[redis-server] action upgrade (up to date) * service[redis-server] action nothing (up to date) * template[/etc/redis/redis.conf] action create - update template[/etc/redis/redis.conf] from 81b4f1 to 8a6cec --- /etc/redis/redis.conf 2011-07-27 17:26:50.000000000 +0000 +++ /tmp/chef-rendered-template20130406-2537-f8vlv6 2013-04-06 ... @@ -27,7 +27,7 @@ # If you want you can bind a single interface, if the bind option is not # specified all the interfaces will listen for incoming connections. # -bind 127.0.0.1 +bind 0.0.0.0 # Specify the path for the unix socket that will be used to listen for # incoming connections. There is no default, so Redis will not listen * service[redis-server] action restart - restart service service[redis-server] Chef Client finished, 2 resources updated install package run service configuration + restart ... chef
49. .co.il www.
50. .co.il www. We invite you to join Operations Israel Facebook group on on.fb.me/Ops-IL we are hiring at jobs@devops.co.il Thank you! www.devops.co.il
51. .co.il www. Reference: - http://sysadvent.blogspot.co.il/2011/12/day-19-why-use-configuration-management.html - https://www.youtube.com/watch?v=2a2nWELIk-Y Yegor Bugayenko | Kyiv DevOps, September, 2015 - https://twitter.com/thoughtworks/status/658636712709480448 - 20th Century BIG IDEAS in Operations Management 1920s: Ford & Taylor - Moving production line and standardized work 1930s: Shewhart (& Deming) - Statistical Control of Quality 1960s: Ohno - Lean Production System 1980s: Goldratt & Kaplan - Measurement & Theory of Constraints 1990s: Hammer & Dell - Reengineering / Process Focus & “Direct Model”