Mixing oil and water: devops in an ITSM world

Slide 1

Slide 1 text

Mixing oil + water devops in an ITSM world Lindsay Holmwood @auxesis

Slide 2

Slide 2 text

High-performing IT organisations report experiencing: 46x more frequent deployments

Slide 3

Slide 3 text

High-performing IT organisations report experiencing: 96x faster recovery from failures

Slide 4

Slide 4 text

High-performing IT organisations report experiencing: 5x lower change failure rate

Slide 5

Slide 5 text

High-performing IT organisations report experiencing: 440x shorter lead times

Slide 6

Slide 6 text

High-performing IT organisations report experiencing: 21% less time on unplanned work and rework

Slide 7

Slide 7 text

High-performing IT organisations report experiencing: 44% more time on new work

Slide 8

Slide 8 text

Slide 9

Slide 9 text

High-performing organisations decisively outperform their lower- performing peers in terms of throughput.

Slide 10

Slide 10 text

Let’s demystify

Slide 11

Slide 11 text

What is not devops

Slide 12

Slide 12 text

It’s not cowboying changes into production

Slide 13

Slide 13 text

It’s not tools or Technology

Slide 14

Slide 14 text

It’s not something you buy s/devops/agile/i

Slide 15

Slide 15 text

It’s not a team* *cross-functional, multidisciplinary teams are a good blueprint

Slide 16

Slide 16 text

dev + ops == devops team We’re doing devops!

Slide 17

Slide 17 text

dev & ops & design & UX & product & …

Slide 18

Slide 18 text

Diversity & inclusion

Slide 19

Slide 19 text

It’s not a silver bullet

Slide 20

Slide 20 text

How does DevOps map to ITSM?

Slide 21

Slide 21 text

No content

Slide 22

Slide 22 text

No content

Slide 23

Slide 23 text

Service design Service transition Service operation Service catalogue management Change management Service desk Service level management Release management Application management Availability management Deployment management Operations management Capacity management Service testing & validation Technical management Continuity management Conﬁguration management Event management Security management Incident management Supplier management Request fulﬁllment Problem management Identity management

Slide 24

Slide 24 text

Slide 25

Slide 25 text

What are the benefits?

Slide 26

Slide 26 text

Faster time to market

Slide 27

Slide 27 text

Improved resiliency

Slide 28

Slide 28 text

MTTD + MTTR trending down

Slide 29

Slide 29 text

Improved customer satisfaction

Slide 30

Slide 30 text

Less friction

Slide 31

Slide 31 text

What are the risks?

Slide 32

Slide 32 text

Risk: No executive sponsorship? Don’t bother.

Slide 33

Slide 33 text

There’s a champion somewhere in your organisation.

Slide 34

Slide 34 text

People in the executive   ❤ it because: • it’s new and exciting • it delivers value faster • it brings them closer to the customer

Slide 35

Slide 35 text

People at the coalface   ❤ it because: • it’s new and exciting • it keeps their skills relevant • it brings them closer to the customer

Slide 36

Slide 36 text

Once you’ve got commitment

Slide 37

Slide 37 text

Be the umbrella you want to see in the world.

Slide 38

Slide 38 text

Risk: You have to change culture

Slide 39

Slide 39 text

Culture change is hard

Slide 40

Slide 40 text

What is culture?

Slide 41

Slide 41 text

Schein’s three levels of culture

Slide 42

Slide 42 text

Artifacts Values Assumptions

Slide 43

Slide 43 text

National ↩︎ Organisational ↩︎ Team ↩︎ Occupational

Slide 44

Slide 44 text

Artifacts

Slide 45

Slide 45 text

physical manifestations of culture

Slide 46

Slide 46 text

ceremonies

Slide 47

Slide 47 text

ways of working

Slide 48

Slide 48 text

org charts

Slide 49

Slide 49 text

desk layout

Slide 50

Slide 50 text

processes

Slide 51

Slide 51 text

software

Slide 52

Slide 52 text

most visible parts of an org’s culture

Slide 53

Slide 53 text

easiest part of a culture to adopt

Slide 54

Slide 54 text

Values

Slide 55

Slide 55 text

conscious goals, strategies, and philosophies

Slide 56

Slide 56 text

rules that guide how we interact with people

Slide 57

Slide 57 text

rules that guide how we do our work

Slide 58

Slide 58 text

“we manage risk proactively”

Slide 59

Slide 59 text

“management is available, and listen to our concerns”

Slide 60

Slide 60 text

“we value quality over delivery speed”

Slide 61

Slide 61 text

“nobody will be fired for making an honest mistake”

Slide 62

Slide 62 text

values: lived vs aspirational

Slide 63

Slide 63 text

Communication We have an obligation to communicate.

Slide 64

Slide 64 text

Respect We treat others as we would like to be treated.

Slide 65

Slide 65 text

Integrity We work with customers and prospects openly, honestly, and sincerely.

Slide 66

Slide 66 text

Excellence We are satisfied with nothing less than the very best in everything we do.

Slide 67

Slide 67 text

Ethical We conduct business affairs in accordance with all applicable laws and in a moral and honest manner.

Slide 68

Slide 68 text

No content

Slide 69

Slide 69 text

Work as imagined vs Work as done

Slide 70

Slide 70 text

Be clear about what values are what

Slide 71

Slide 71 text

Systems + processes Values Assumptions

Slide 72

Slide 72 text

Artifacts Values Assumptions

Slide 73

Slide 73 text

They’re a snapshot of our org’s culture

Slide 74

Slide 74 text

They’re a snapshot of our org’s values and assumptions

Slide 75

Slide 75 text

Artifacts influence behaviour

Slide 76

Slide 76 text

Change your org’s values by changing your artifacts

Slide 77

Slide 77 text

Artifact: Make deployment fast and safe

Slide 78

Slide 78 text

Value: “We improve quality by going fast”

Slide 79

Slide 79 text

Artifact: Work happens in cross functional teams

Slide 80

Slide 80 text

Value: “We work together to achieve a shared goal”

Slide 81

Slide 81 text

Artifact: Give autonomy. Eliminate signoff. Grant access. Log everything.

Slide 82

Slide 82 text

Value: “Ask forgiveness, not permission”

Slide 83

Slide 83 text

Artifact: Run post incident reviews Run pre accident investigations

Slide 84

Slide 84 text

Value: “Failure is an opportunity for learning”

Slide 85

Slide 85 text

Risk: You do it and you fail

Slide 86

Slide 86 text

Reputational risk

Slide 87

Slide 87 text

Burn political capital

Slide 88

Slide 88 text

Risk: You don’t do it and you become obsolete

Slide 89

Slide 89 text

“It is not necessary to change. Survival is not mandatory.” – Deming

Slide 90

Slide 90 text

Where can we experiment with devops?

Slide 91

Slide 91 text

Application space is the easiest

Slide 92

Slide 92 text

infrastructure (network, compute, storage) application

Slide 93

Slide 93 text

Take what you learn apply it to infrastructure

Slide 94

Slide 94 text

Variety of options

Slide 95

Slide 95 text

Greenfields Brownfields Blended

Slide 96

Slide 96 text

Greenfields New initiatives

Slide 97

Slide 97 text

Greenfields Digital transformation

Slide 98

Slide 98 text

Greenfield benefits Less baggage

Slide 99

Slide 99 text

Greenfield benefits Buy in from your people

Slide 100

Slide 100 text

Greenfield risks Blow your innovation budget

Slide 101

Slide 101 text

Don’t change what technology you use

Slide 102

Slide 102 text

Change how you use that technology

Slide 103

Slide 103 text

Change how you consume that technology

Slide 104

Slide 104 text

Greenfield risks Create a division between new and old

Slide 105

Slide 105 text

Greenfield risks How does new get absorbed into the old?

Slide 106

Slide 106 text

Disband the team?

Slide 107

Slide 107 text

Throw it over the wall?

Slide 108

Slide 108 text

Build systems to mirror the org structure?

Slide 109

Slide 109 text

Brownfields Existing processes

Slide 110

Slide 110 text

Well defined outputs

Slide 111

Slide 111 text

Something you can refactor piecemeal

Slide 112

Slide 112 text

Brownfields Reboot an existing system or process

Slide 113

Slide 113 text

Brownfield benefits Less experimentation required

Slide 114

Slide 114 text

Brownfield risks Thinking constrained by existing systems

Slide 115

Slide 115 text

Brownfield risks Changing a thing to work in a way it wasn’t designed

Slide 116

Slide 116 text

Brownfield risks Incremental improvements at best

Slide 117

Slide 117 text

Blended Start with

Slide 118

Slide 118 text

Get confidence with new technology approach

Slide 119

Slide 119 text

Get confidence with new delivery approach

Slide 120

Slide 120 text

Blended Then apply to

Slide 121

Slide 121 text

Have exemplars you can point to

Slide 122

Slide 122 text

Take what you learn apply it to infrastructure

Slide 123

Slide 123 text

1. Build the smallest, simplest thing that meets a user need

Slide 124

Slide 124 text

Look at failure demand in your organisation

Slide 125

Slide 125 text

“Shadow IT” is failure demand being met

Slide 126

Slide 126 text

Set delivery constraints

Slide 127

Slide 127 text

⏰ Time Budget Scope (pick 2)

Slide 128

Slide 128 text

2. Start with Continuous Delivery

Slide 129

Slide 129 text

Technical manifestation of devops

Slide 130

Slide 130 text

Align release processes around an outcome

Slide 131

Slide 131 text

NOT

Slide 132

Slide 132 text

Align outcome around a release process

Slide 133

Slide 133 text

Spend tech innovation budget on delivery engineering

Slide 134

Slide 134 text

All changes go through a Continuous Delivery pipeline

Slide 135

Slide 135 text

deploy to production acceptance tests integrate unit tests code done Continuous Delivery Manual Auto Auto Auto

Slide 136

Slide 136 text

shipping multiple times a day

Slide 137

Slide 137 text

Decrease batch size

Slide 138

Slide 138 text

Increase frequency

Slide 139

Slide 139 text

“You build it, you run it” – Vogels, AWS

Slide 140

Slide 140 text

High-performing IT organisations report experiencing: 46x more frequent deployments

Slide 141

Slide 141 text

Code doesn’t create value until it is running in front of a user

Slide 142

Slide 142 text

Nail your automated delivery pipeline from the start

Slide 143

Slide 143 text

Spend tech innovation budget on delivery engineering

Slide 144

Slide 144 text

Write high level acceptance tests (like Cucumber)

Slide 145

Slide 145 text

An executable specification

Slide 146

Slide 146 text

Feature: Refund item Scenario: Jeff returns a faulty microwave Given Jeff has bought a microwave for $100 And he has a receipt When he returns the microwave Then Jeff should be refunded $100

Slide 147

Slide 147 text

3. Use commodity

Slide 148

Slide 148 text

Don’t change what technology you use

Slide 149

Slide 149 text

Change how you use that technology

Slide 150

Slide 150 text

Change how you consume that technology

Slide 151

Slide 151 text

Move innovation up the stack

Slide 152

Slide 152 text

Don’t experiment with IaaS

Slide 153

Slide 153 text

Don’t experiment with Private clouds

Slide 154

Slide 154 text

Don’t experiment with Anything on-prem

Slide 155

Slide 155 text

Experiment with PaaS

Slide 156

Slide 156 text

Experiment with SaaS

Slide 157

Slide 157 text

Experiment with Open Source

Slide 158

Slide 158 text

4. Build in measurement

Slide 159

Slide 159 text

# deploys a day

Slide 160

Slide 160 text

Time to failure detection

Slide 161

Slide 161 text

Deployment lead time

Slide 162

Slide 162 text

Error rates

Slide 163

Slide 163 text

Throughput per class of work

Slide 164

Slide 164 text

Really easy when you buy into PaaS + SaaS

Slide 165

Slide 165 text

Slide 166

Slide 166 text

“People with targets […] will probably meet the targets - even if they have to destroy the enterprise to do it.” – Deming

Slide 167

Slide 167 text

5. Build and listen to feedback loops

Slide 168

Slide 168 text

If you’re experimenting you need to learn from it

Slide 169

Slide 169 text

Construct feedback loops on what works

Slide 170

Slide 170 text

Construct feedback loops on what doesn’t work

Slide 171

Slide 171 text

Construct feedback loops on what is puzzling

Slide 172

Slide 172 text

Activities: Fortnightly retros

Slide 173

Slide 173 text

Activities: PIRs

Slide 174

Slide 174 text

Activities: Failure Fridays

Slide 175

Slide 175 text

“Chaos engineering means experiencing failure on your terms” – Bruce Wong, Twilio

Slide 176

Slide 176 text

What about highly regulated environments?

Slide 177

Slide 177 text

“You can’t do this because of the rules”

Slide 178

Slide 178 text

Policy as a weapon vs Policy as a tool

Slide 179

Slide 179 text

Read the policy yourself

Slide 180

Slide 180 text

Find subject matter experts

Slide 181

Slide 181 text

Document what works and what doesn’t

Slide 182

Slide 182 text

Security

Slide 183

Slide 183 text

“Everyone has a plan until they get punched in the mouth” – Mike Tyson

Slide 184

Slide 184 text

Detection, not prevention

Slide 185

Slide 185 text

Credentials in source control

Slide 186

Slide 186 text

Automated vulnerability detection

Slide 187

Slide 187 text

Automated config validation

Slide 188

Slide 188 text

Hooked into monitoring

Slide 189

Slide 189 text

* CAB as a blocker

Slide 190

Slide 190 text

CABs and CD are incongruous

Slide 191

Slide 191 text

Or are they?

Slide 192

Slide 192 text

Why do we have CABs?

Slide 193

Slide 193 text

Understand what is happening

Slide 194

Slide 194 text

Understand why it’s happening

Slide 195

Slide 195 text

Understand who is affected

Slide 196

Slide 196 text

Ensure due diligence on changes

Slide 197

Slide 197 text

Co-ordinate complex multi-step changes across multiple teams

Slide 198

Slide 198 text

They are a control that mitigates risk

Slide 199

Slide 199 text

ITSM as a model for managing change

Slide 200

Slide 200 text

ITSM as a model for managing complexity

Slide 201

Slide 201 text

Prove to your CAB: your automation gives them the same assurances they’re looking for

Slide 202

Slide 202 text

Submit to your CAB: use a CD pipeline to deploy all future changes

Slide 203

Slide 203 text

Make doing the right thing easy

Slide 204

Slide 204 text

CABs are preventative controls

Slide 205

Slide 205 text

We put all our energy into prevention

Slide 206

Slide 206 text

We have no energy left for detection or response

Slide 207

Slide 207 text

High-performing IT organisations report experiencing: 46x more frequent deployments

Slide 208

Slide 208 text

High-performing IT organisations report experiencing: 96x faster recovery from failures

Slide 209

Slide 209 text

High-performing IT organisations report experiencing: 5x lower change failure rate