Facts and Fallacies of
Continuous Delivery
Agile Tour Brussels 2015
Thierry de Pauw, @tdpauw
Slide 2
Slide 2 text
What is Continuous Delivery
Slide 3
Slide 3 text
"How long would it take your organisation to deploy a change that involves just one single line of code ? Do
you do this on a repeatable, reliable basis ?"
Mary and Tom Poppendieck,
Implementing Lean Software Development
Slide 4
Slide 4 text
Satisfy your customer through early and
continuous delivery of valuable software
Slide 5
Slide 5 text
A Deployment Pipeline is the automated
implementation of your process of getting
software from version control into the hands
of your users.
Slide 6
Slide 6 text
It is a logical extension of Continuous
Integration
Every commit creates a potential release
candidate
Slide 7
Slide 7 text
One of its key objectives is to optimise for
human resource usage
Slide 8
Slide 8 text
The deployment pipeline has 3 purposes:
●every part of the process is visible
●it improves feedback
●it empowers teams
Slide 9
Slide 9 text
The deployment pipeline is part of your value
stream, the process of getting a feature from
the mind of your customers into their hands
Slide 10
Slide 10 text
The entire process, from concept to cash, can
be modelled as a value stream map
Slide 11
Slide 11 text
Fact 1
More than just Tooling
Slide 12
Slide 12 text
Continuous Delivery is a mindset
Slide 13
Slide 13 text
It is about the DevOps Culture and Lean
Software Development
Slide 14
Slide 14 text
The DevOps Culture : The Three Ways
Slide 15
Slide 15 text
The First Way: from Left to Right
Maximise the flow from Development to
Operations to Customers
Slide 16
Slide 16 text
Definition of Done is not when Development
is done coding
Slide 17
Slide 17 text
The aim is single piece flow
Slide 18
Slide 18 text
The Second Way: from Right to Left
Constant flow of fast feedback at all stages
of the value stream
Slide 19
Slide 19 text
To get early feedback, you can't have 1 year
releases
Slide 20
Slide 20 text
The Third Way: a Culture of Improvement
A culture that fosters continual
experimentation, learning from failures and
understanding that repetition and practice
leads to mastery.
Slide 21
Slide 21 text
"If a system is not improving, the result is not steady state. Instead, because of entropy,
organisational performance declines"
Mike Rother,
Toyota Kata: Managing People for Improvement, Adaptiveness and Superior Results
Slide 22
Slide 22 text
The Three Ways are essential to successfully
apply Continuous Delivery
Slide 23
Slide 23 text
Fact 2
Build your binaries once
and only once
Slide 24
Slide 24 text
1. Keep the deployment pipeline efficient
Slide 25
Slide 25 text
2. Always build upon foundations known to
be right
Slide 26
Slide 26 text
It must be possible to deploy the binaries to
every environment
Slide 27
Slide 27 text
Fact 3
Keep everything under Version Control
Slide 28
Slide 28 text
A new team member must be able to
checkout the project from version control, and
run a single command to build and deploy the
application on the local workstation
Slide 29
Slide 29 text
Fact 4
When continuously integrating,
integrate also your internal
dependencies
Slide 30
Slide 30 text
Always use the latest stable version of your
internal dependencies.
Slide 31
Slide 31 text
Fact 5
Feature branches are evil
Slide 32
Slide 32 text
They introduce risk
Slide 33
Slide 33 text
They are time consuming
Slide 34
Slide 34 text
Refactoring becomes difficult
Slide 35
Slide 35 text
They increase the complexity of the internal
dependency management
Slide 36
Slide 36 text
Break down your stories into smaller chunks
of work
Slide 37
Slide 37 text
Implement the user-facing bits last
Slide 38
Slide 38 text
Use Branch by Abstraction
Slide 39
Slide 39 text
Use Feature Toggles
Slide 40
Slide 40 text
Fact 6
Automate the hell out of it
Slide 41
Slide 41 text
"If the build, deploy, test, and release process is not automated,
it is not repeatable"
Jez Humble and David Farley,
Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment
Automation
Slide 42
Slide 42 text
Some things are impossible to automate
Slide 43
Slide 43 text
But the list of things that cannot be
automated is much smaller than many people
think
Slide 44
Slide 44 text
Test Automation
Slide 45
Slide 45 text
Human beings are simply bad at repetitive,
complex tasks that manual regression testing
requires
To identify your bottleneck
You need to model your value stream using
a value stream map
Slide 50
Slide 50 text
Any improvements made anywhere besides
the bottleneck are an illusion
Slide 51
Slide 51 text
Fact 8
Build Quality In !
Slide 52
Slide 52 text
"Your goal is to build quality into the code from the start, not test it in later"
Mary and Tom Poppendieck,
Implementing Lean Software Development
Slide 53
Slide 53 text
When a defect is found, stop the line
Slide 54
Slide 54 text
"Defect tracking systems are queues of partially done work"
Mary and Tom Poppendieck,
Implementing Lean Software Development
Slide 55
Slide 55 text
"(Technical Debt) comes from taking shortcuts, which may make sense in the short-term. But like
financial debt, the compounding interest costs grow over time. If an organisation doesn't pay down its
technical debt, every calorie in the organisation can be spent just paying interest, in the form of
unplanned work."
The Phoenix Project
Slide 56
Slide 56 text
"Delaying testing until after the development process is a sure-fire way to decrease the quality of your
application."
Jez Humble and David Farley,
Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment
Automation
Slide 57
Slide 57 text
Test automation makes the quality of your
application significantly higher, so manual
testing is then only an affirmation of the
functional completeness
Slide 58
Slide 58 text
Fact 9
Deployments become Boring
Slide 59
Slide 59 text
Anti-pattern: Big feature releases as answer
for failing deployments
Slide 60
Slide 60 text
"If it hurts,
do it more often !"
Slide 61
Slide 61 text
Deploy the same way to each environment
Slide 62
Slide 62 text
You have to have a Copy of Production
Slide 63
Slide 63 text
First Fallacy
Deployments cost too much
Slide 64
Slide 64 text
The indirect cost of the downtime during
releases
Slide 65
Slide 65 text
Systems that only operate during "business
hours" are rare
Slide 66
Slide 66 text
SLA 99,5% availability
means
less than 216 minutes of unplanned
downtime per month
Slide 67
Slide 67 text
216 min/month vs 5 hours of planned
downtime
Slide 68
Slide 68 text
Users do not care if the downtime was
planned or unplanned
Slide 69
Slide 69 text
Blue Green Deployments
Slide 70
Slide 70 text
Zero-Down Time Deployments
●Expansion (days before) : add stuff
●Roll out
●Clean up (days after) : remove stuff
Slide 71
Slide 71 text
Fact 10
Rollbacks become obsolete
Slide 72
Slide 72 text
Fix Forward
Slide 73
Slide 73 text
Fact 11
You will not risk customers for an
arbitrary release date
Slide 74
Slide 74 text
For product vendors release dates matter
For the other 99,9%: nobody cares
Slide 75
Slide 75 text
Customers care that the system is available
and works without too many bugs
Slide 76
Slide 76 text
Do not try to rush releases into production
before being ready
Slide 77
Slide 77 text
Fact 12
You build it you run it
Slide 78
Slide 78 text
Everybody in your team (by extension your
organisation) is responsible for the release
process
Slide 79
Slide 79 text
"An organisation that respects software developers as professionals will expect them to design their own jobs [...]. It
will expect them to improve continually the way they do their work as part of a learning process. Finally, it will give
them the time and equipment necessary to do their jobs well. [...] Frontline workers have process design authority and
decision-making responsibility."
Mary and Tom Poppendieck,
Lean Software Development
Slide 80
Slide 80 text
Second Fallacy
That does not work here syndrome
Slide 81
Slide 81 text
Continuous Delivery is all about managing
risk
Slide 82
Slide 82 text
The system prevents you to deploy a build
into production that is not thoroughly tested
and found to be fit
Slide 83
Slide 83 text
Heavy change management processes bring
nothing to the stability of systems
Slide 84
Slide 84 text
Everything that improves throughput
improves stability
Everything that improves innovation also
improves stability
Slide 85
Slide 85 text
Third Fallacy
But we are not Unicorns !
Slide 86
Slide 86 text
"Of the Fortune 500 companies in 1955, 87% are gone. In 1958, the Fortune 500 tenure was 61 years;
now it's only 18 years"
Richard Foster,
Creative Destruction: Why Companies that are built to Last underperform the Market -
and how to Successfully transform Them, 2001
Slide 87
Slide 87 text
One of the primary objections from horses is
that all unicorns were born that way
Slide 88
Slide 88 text
All unicorns were once horses
Slide 89
Slide 89 text
LinkedIn
2011: 2 months feature freeze
Slide 90
Slide 90 text
Etsy
2009: "had to come to grips that they were
living in a sea of their own engineering filth"
Slide 91
Slide 91 text
Facebook
2009: barely able to keep up with user
growth, code deployments were becoming
increasingly dangerous
Slide 92
Slide 92 text
●BNY Mellon, Bank of America, ING
Netherlands
●Target
●UK.gov
Slide 93
Slide 93 text
Resources
Slide 94
Slide 94 text
Hello, I am Thierry de Pauw
Software Engineer, Jack of All Trades
Agile Technical Coach
http://thinkinglabs.io
@tdpauw