Slide 1

Slide 1 text

(without introducing more risk) In Praise of Slow Continuous Delivery Puppet Gareth Rushgrove On-premise software and continuous delivery

Slide 2

Slide 2 text

(without introducing more risk) @garethr

Slide 3

Slide 3 text

(without introducing more risk) Gareth Rushgrove

Slide 4

Slide 4 text

(without introducing more risk) What we’ll cover This talk

Slide 5

Slide 5 text

- What do we mean when we say continuous - The world of Enterprise Software vendors - Why is this different? - Examples of evolving stable software Gareth Rushgrove

Slide 6

Slide 6 text

Note Most of the examples are infrastructure tools, because I’m an infrastructure tools geek. The patterns and practices are more general Gareth Rushgrove

Slide 7

Slide 7 text

(without introducing more risk) What people think of when you say continuous delivery What does good look like?

Slide 8

Slide 8 text

Gareth Rushgrove

Slide 9

Slide 9 text

Gareth Rushgrove “Continuous Delivery is the ability to get changes of all types—including new features, configuration changes, bug fixes and experiments—into production, or into the hands of users, safely and quickly in a sustainable way. Jez Humble, Continuous Delivery

Slide 10

Slide 10 text

Gareth Rushgrove

Slide 11

Slide 11 text

Gareth Rushgrove “10 deploys a day at Flickr John Allspaw and Paul Hammond, VelocityConf, 2009

Slide 12

Slide 12 text

Gareth Rushgrove Jon Jenkins, VelocityConf, 2011 “Amazon deploys every 11.6 seconds

Slide 13

Slide 13 text

How fast are you deploying your software today? Gareth Rushgrove

Slide 14

Slide 14 text

(without introducing more risk) The land the internet forgot Enterprise Software

Slide 15

Slide 15 text

Gareth Rushgrove

Slide 16

Slide 16 text

But everyone wants to be an enterprise software company Gareth Rushgrove

Slide 17

Slide 17 text

Gareth Rushgrove

Slide 18

Slide 18 text

No really. Everyone wants to be an enterprise software company Gareth Rushgrove

Slide 19

Slide 19 text

Gareth Rushgrove 0 50 100 150 200 ORCL IBM SAP VMware Market cap in billions

Slide 20

Slide 20 text

- Puppet Enterprise ships every 3 months - Docker EE ships every 3 months - vSphere ships about every 6 months - Oracle ship a new major every 3 years Gareth Rushgrove

Slide 21

Slide 21 text

Can this be continuous delivery? Gareth Rushgrove

Slide 22

Slide 22 text

Gareth Rushgrove

Slide 23

Slide 23 text

Gareth Rushgrove HP LaserJet Firmware Builds Commits per day Regression tests 1 per week 1 6 weeks 10 per day 100 24 hours 2008 2011

Slide 24

Slide 24 text

Can this be continuous delivery? Gareth Rushgrove

Slide 25

Slide 25 text

(without introducing more risk) Non-SaaS delivery models Why is this different?

Slide 26

Slide 26 text

1. Consuming software via pull vs push Gareth Rushgrove

Slide 27

Slide 27 text

Gareth Rushgrove A typical web app is pushed Deployment Running application Users

Slide 28

Slide 28 text

Gareth Rushgrove New versions replace old Deployment New version Users

Slide 29

Slide 29 text

But deployment of on-premise software is disconnected from it’s consumption by users Gareth Rushgrove

Slide 30

Slide 30 text

Gareth Rushgrove Deployment is disconnected Deployment Package available to download

Slide 31

Slide 31 text

Gareth Rushgrove Deployment is disconnected Deployment New package available Old packages still available

Slide 32

Slide 32 text

Gareth Rushgrove Gareth Rushgrove Consumers choose when to pull Download package Users Local copy per user

Slide 33

Slide 33 text

Problem Users can choose not to update Gareth Rushgrove

Slide 34

Slide 34 text

Problem Other people may choose to package your software if it’s open source Gareth Rushgrove

Slide 35

Slide 35 text

2. The environment permutation explosion Gareth Rushgrove

Slide 36

Slide 36 text

How many platforms does your web applications need to run on? Gareth Rushgrove

Slide 37

Slide 37 text

Gareth Rushgrove 15 supported platforms

Slide 38

Slide 38 text

i386 and x86_64. IBM z Systems and Power. Solaris. RHEL 4 (released 2005). 3 versions of macOS, 4 versions of AIX, 11 versions of Windows across server and desktop. Gareth Rushgrove 80 more platforms!

Slide 39

Slide 39 text

We run about 500 pipelines a day. That’s about 35,000 individual jobs a week. For a development team of less than 100 people Gareth Rushgrove

Slide 40

Slide 40 text

Problem Your users control the environment not you. Limiting the environments you support costs you users Gareth Rushgrove

Slide 41

Slide 41 text

3. Say semantic versioning one more time Gareth Rushgrove

Slide 42

Slide 42 text

What version of GitHub did I just use? Gareth Rushgrove

Slide 43

Slide 43 text

Gareth Rushgrove What version of GitHub Enterprise did I just use?

Slide 44

Slide 44 text

Gareth Rushgrove So Windows 8 is really Windows 6.2?

Slide 45

Slide 45 text

Versions numbers are marketing too Gareth Rushgrove

Slide 46

Slide 46 text

Docker 17.03, Ubuntu 16.04, PE 2017.1, Office 2016, Office 365, OpenStack Mitaka, Newton, Ocata, etc. Gareth Rushgrove

Slide 47

Slide 47 text

(without introducing more risk) CalVer

Slide 48

Slide 48 text

Gareth Rushgrove Constantly incrementing values

Slide 49

Slide 49 text

(without introducing more risk) Recommended reading

Slide 50

Slide 50 text

Problem Versioning packaged software is a wicked problem Gareth Rushgrove A wicked problem is a problem that is difficult or impossible to solve because of incomplete, contradictory, and changing requirements that are often difficult to recognise.

Slide 51

Slide 51 text

4. How many versions of your software can you support at once? Gareth Rushgrove

Slide 52

Slide 52 text

The ideal of continuous delivery is one Gareth Rushgrove

Slide 53

Slide 53 text

How long do you support that one version? Until you ship the next one (maybe later the same day?) Gareth Rushgrove

Slide 54

Slide 54 text

(without introducing more risk) RHEL support life-cycle Gareth Rushgrove

Slide 55

Slide 55 text

(without introducing more risk) RHEL Support dates Gareth Rushgrove

Slide 56

Slide 56 text

RHEL 4 end of life is this month. It’s been supported for 12 years Gareth Rushgrove

Slide 57

Slide 57 text

Gareth Rushgrove 12 years

Slide 58

Slide 58 text

(without introducing more risk) Docker support scheme

Slide 59

Slide 59 text

Release quarterly, each release supported for a year means 4 supported major versions Gareth Rushgrove

Slide 60

Slide 60 text

Gareth Rushgrove “ Docker not supporting a version for longer than a year is a no-go. This is not enterprise ready. Hacker News comment

Slide 61

Slide 61 text

Gareth Rushgrove “In large enterprise it will take 6 months to get a version certified to work with tooling so need 3 year at the minimum to put any effort on it. Hacker News comment

Slide 62

Slide 62 text

Gareth Rushgrove “Docker initial release was just 4 years ago, your asking for a support timeframe that's around the same length as the age of the project itself. Hacker News comment

Slide 63

Slide 63 text

Problem User expectations vary wildly across organisations, sectors and types of software Gareth Rushgrove

Slide 64

Slide 64 text

5. Licenses matter Gareth Rushgrove

Slide 65

Slide 65 text

Who has every had to do a license audit? Gareth Rushgrove

Slide 66

Slide 66 text

(without introducing more risk) Puppet Enterprise licenses

Slide 67

Slide 67 text

Puppet Server (just one component of Puppet Enterprise) has 170 different open source components, each with individual license terms Gareth Rushgrove

Slide 68

Slide 68 text

In total we ship third-party software with 19 different licenses, from MIT, Apache2 and BSD to EDL 1.0, EPL 1.0, Artistic 1.0 and 2.0, Ruby, Boost Software License and more Gareth Rushgrove

Slide 69

Slide 69 text

Problem Does this commit break the law? Is this release legal? Who has legal checks in their continuous delivery pipeline? Gareth Rushgrove

Slide 70

Slide 70 text

(without introducing more risk) Adopting continuous delivery patterns Evolving stable software

Slide 71

Slide 71 text

1. Feature flags and conducting experiments Gareth Rushgrove

Slide 72

Slide 72 text

Allow users to opt-in to some functionality Gareth Rushgrove

Slide 73

Slide 73 text

(without introducing more risk) Docker experimental flag

Slide 74

Slide 74 text

(without introducing more risk) Puppet future parser

Slide 75

Slide 75 text

Dark launch features and test in the background Gareth Rushgrove

Slide 76

Slide 76 text

(without introducing more risk) Terraform experiments Gareth Rushgrove

Slide 77

Slide 77 text

2. Releasing every commit Gareth Rushgrove

Slide 78

Slide 78 text

(without introducing more risk) Puppet nightly builds

Slide 79

Slide 79 text

Gareth Rushgrove Puppet docs “Our automated systems create new “nightly” repositories for builds that pass our acceptance testing on the most popular platforms.

Slide 80

Slide 80 text

(without introducing more risk) gsutil cat gs://kubernetes-release-dev/ci/latest-green.txt

Slide 81

Slide 81 text

The difference between being able to ship every day vs actually shipping every day Gareth Rushgrove

Slide 82

Slide 82 text

3. The importance of product analytics Gareth Rushgrove

Slide 83

Slide 83 text

Part of Continuous Delivery is Continuous Improvement. And without data about usage how do you know what to improve? Gareth Rushgrove

Slide 84

Slide 84 text

Do you expect a web application adopting continuous delivery to use things like: - Google Analytics - Kissmetrics - New Relic - AppDynamics - Dynatrace Gareth Rushgrove

Slide 85

Slide 85 text

Do you expect an open source CLI based tool to do the same? Gareth Rushgrove

Slide 86

Slide 86 text

(without introducing more risk) HashiCorp Checkpoint

Slide 87

Slide 87 text

Gareth Rushgrove “Consul makes use of a HashiCorp service called Checkpoint which is used to check for updates and critical security bulletins. Only anonymous information, which cannot be used to identify the user or host, is sent to Checkpoint Consul FAQ

Slide 88

Slide 88 text

Gareth Rushgrove Docker for Mac statistics

Slide 89

Slide 89 text

(without introducing more risk) Puppet Enterprise data

Slide 90

Slide 90 text

Gareth Rushgrove “Data like this helps us understand how you use the product, which in turn helps us improve the product to meet your needs How does sharing this data benefit PE users?

Slide 91

Slide 91 text

Problem Not every one is connected to the internet Gareth Rushgrove

Slide 92

Slide 92 text

(without introducing more risk) Offline data gathering

Slide 93

Slide 93 text

Questions around packaged software, product analytics, network usage, firewalls and user consent can be tricky Gareth Rushgrove

Slide 94

Slide 94 text

(without introducing more risk) If all you remember is Conclusions

Slide 95

Slide 95 text

Enterprise user expectations clash with some of the expected implementations of the practices of continuous delivery Gareth Rushgrove

Slide 96

Slide 96 text

But the practices of continuous delivery are still relevant to enterprise software Gareth Rushgrove

Slide 97

Slide 97 text

Continuous Delivery is about speed Gareth Rushgrove

Slide 98

Slide 98 text

But speed is a relative measure that depends on a frame of reference Gareth Rushgrove My Physics degree coming in handy

Slide 99

Slide 99 text

Always remember to put your users and your context ahead of any absolute measures Gareth Rushgrove Especially those you read on the internet

Slide 100

Slide 100 text

(without introducing more risk) Questions? And thanks for listening