@jezhumble
continuous delivery: why, what, principles
building quality in
principles and patterns for low-risk releases
overview
Slide 3
Slide 3 text
@jezhumble
what is continuous delivery?
The ability to get changes—features, configuration changes, bug
fixes, experiments—into production or into the hands of users
safely and quickly in a sustainable way.
Slide 4
Slide 4 text
@jezhumble
increase software quality and stability
make releases painless, low risk events
reduce time to market
increase customer and employee satisfaction
reduce cost of ongoing software development
why continuous delivery?
Slide 5
Slide 5 text
scrum-
fall
water-
Slide 6
Slide 6 text
@jezhumble
Jon Jenkins, “Velocity Culture, The Unmet Challenge in Ops” 2011 | http://bit.ly/1vJo1Ya
Slide 7
Slide 7 text
“Evaluating well-designed and executed experiments that
were designed to improve a key metric, only about 1/3 were
successful at improving the key metric!”
do less
“Online Experimentation at Microsoft”, Kohavi et al | http://stanford.io/130uW6X
Slide 8
Slide 8 text
@jezhumble
computers do repetitive tasks, people solve problems
build quality in
work in small batches
everyone is responsible
relentlessly pursue continuous improvement
key principles
Slide 9
Slide 9 text
@jezhumble
releasing frequently
John Allspaw: “Ops Metametrics” http://slidesha.re/dsSZIr
Slide 10
Slide 10 text
@jezhumble
optimize for mtrs
Slide 11
Slide 11 text
@jezhumble
agile manifesto
Our highest priority is to satisfy the customer through early
and continuous delivery of valuable software
Slide 12
Slide 12 text
@jezhumble
production-ready software
Fast, automated feedback on the production readiness of your
applications every time there is a change — to code,
infrastructure, or configuration.
Slide 13
Slide 13 text
@jezhumble
build quality in
“Cease dependence on
mass inspection to
achieve quality. Improve
the process and build
quality into the product in
the first place”
W. Edwards Deming
Slide 14
Slide 14 text
@jezhumble
architecture
patterns and practices
collaboration
continuous delivery
Mainline Server
Develop
Build
Build
pull
Local
Workstation
Build
push
✔
Done!
Slide 17
Slide 17 text
Mainline Server
Develop
Build
Build
pull
Local
Workstation
Build
push
✔
Done!
Everyone Commits
To the Mainline
Every Day
Slide 18
Slide 18 text
@jezhumble
continuous integration at scale
Jason Leyba | http://queue.acm.org/blogposting.cfm?id=78323
Slide 19
Slide 19 text
John Penix | “Large Scale Continuous Testing in the Cloud” | http://bit.ly/1BYMf70
Slide 20
Slide 20 text
@jezhumble
Different Kinds of Testing
Diagram invented by Brian Marick
Slide 21
Slide 21 text
@jezhumble
deployment pipeline
Slide 22
Slide 22 text
@jezhumble
Trade Offs
Slide 23
Slide 23 text
low risk releases
are incremental
blue-green deployments
canary releases
dark launching
production immune system
feature toggles
For more on these patterns, read http://bit.ly/low-risk-releases
Slide 24
Slide 24 text
@jezhumble
STATIC CONTENT
/static/1.1
/static/1.0
DEPENDENT SERVICE
1.0 1.1
Abstraction layer Abstraction layer
APPLICATION
Database
Router /
Load balancer
Interwebs
expand / contract
For more on this pattern, read http://bit.ly/low-risk-releases
blue-green
deployments
For more on this pattern, read https://martinfowler.com/bliki/BlueGreenDeployment.html
Slide 27
Slide 27 text
router
web
server
app
server
DB
server
For more on this pattern, read https://martinfowler.com/bliki/BlueGreenDeployment.html
Slide 28
Slide 28 text
router
v1.1 v1.1 v1.1
web
server
app
server
DB
server
For more on this pattern, read https://martinfowler.com/bliki/BlueGreenDeployment.html
Slide 29
Slide 29 text
router
v1.1 v1.1 v1.1
web
server
app
server
DB
server
v1.2 v1.2 v1.2
For more on this pattern, read https://martinfowler.com/bliki/BlueGreenDeployment.html
Slide 30
Slide 30 text
router
v1.1 v1.1 v1.1
web
server
app
server
DB
server
v1.2 v1.2 v1.2
For more on this pattern, read https://martinfowler.com/bliki/BlueGreenDeployment.html
Slide 31
Slide 31 text
@jezhumble
canary releasing
For more on this pattern, read http://bit.ly/low-risk-releases
Slide 32
Slide 32 text
release !=
deployment
Slide 33
Slide 33 text
[featureToggles]
wobblyFoobars: true
flightyForkHandles: false
Config File
... various UI elements
some.jsp
forkHandle = (featureConfig.isOn(‘flightlyForkHandles)) ?
new FlightyForkHander(aCandle) :
new ForkHandler(aCandle)
other.java
Feature Toggles
For more on this pattern, read https://martinfowler.com/articles/feature-toggles.html
Slide 34
Slide 34 text
dark launching
For more on this pattern, read http://bit.ly/low-risk-releases
Slide 35
Slide 35 text
@jezhumble
For more on this pattern, read http://bit.ly/low-risk-releases
Slide 36
Slide 36 text
@jezhumble
For more on this pattern, read http://bit.ly/low-risk-releases
Slide 37
Slide 37 text
Branch by Abstraction
For more on this pattern, read https://martinfowler.com/bliki/BranchByAbstraction.html
Slide 38
Slide 38 text
Application
New
Library
Library
For more on this pattern, read https://martinfowler.com/bliki/BranchByAbstraction.html
Slide 39
Slide 39 text
Application
New
Library
Library
Interface implements
For more on this pattern, read https://martinfowler.com/bliki/BranchByAbstraction.html
Slide 40
Slide 40 text
Application
New
Library
Library
Interface implements
For more on this pattern, read https://martinfowler.com/bliki/BranchByAbstraction.html
Slide 41
Slide 41 text
Application
New
Library
Library
Interface implements
For more on this pattern, read https://martinfowler.com/bliki/BranchByAbstraction.html
Slide 42
Slide 42 text
Application
New
Library
Interface implements
For more on this pattern, read https://martinfowler.com/bliki/BranchByAbstraction.html
Slide 43
Slide 43 text
For more on this pattern, read https://martinfowler.com/bliki/BranchByAbstraction.html
Slide 44
Slide 44 text
Jesse Robbins, “Master of Disaster” @ Amazon| @jesserobbins | http://oreil.ly/1HRKUVE
”don’t fight stupid
make more awesome!”