Continuous Delivery at scale - social and technical practices for fast flow - Conflux - Agile Manchester 2023

Slide 1

Slide 1 text

Continuous Delivery at scale Social and technical practices for fast ﬂow Matthew Skelton, Founder at Conﬂux K03 Keynote at Agile Manchester | 10 May 2023

Slide 2

Slide 2 text

Photo goes here Matthew Skelton Founder at Conﬂux Co-author of Team Topologies 2

Slide 3

Slide 3 text

Team Topologies Organizing business and technology teams for fast ﬂow Matthew Skelton & Manuel Pais IT Revolution Press, September 2019 Order via stores worldwide: teamtopologies.com/book 3

Slide 4

Slide 4 text

Slide 5

Slide 5 text

5 Fast flow: multiple ongoing streams of changes to software-enriched services done safely and sustainably https://confluxhq.com/fast-flow

Slide 6

Slide 6 text

Fast ﬂow delivers vital business outcomes 6

Slide 7

Slide 7 text

7 Improved eﬃciency due to removed dependencies and reduced rework

Slide 8

Slide 8 text

8 Shorter time from hypothesis to user value

Slide 9

Slide 9 text

9 Greater transparency and traceability in service development and operation

Slide 10

Slide 10 text

10 Quicker problem diagnosis and service restoration

Slide 11

Slide 11 text

11 Happier teams

Slide 12

Slide 12 text

12 Improved eﬃciency due to removed dependencies and reduced rework Shorter time from hypothesis to user value Greater transparency and traceability in service development and operation Quicker problem diagnosis and service restoration Happier teams

Slide 13

Slide 13 text

What social and technical practices are needed for fast ﬂow at scale? 13

Slide 14

Slide 14 text

14 Make space for learning/sharing Deﬁne the Platform Address operability early on Good logging is foundational Empower teams to improve

Slide 15

Slide 15 text

Case study: context & challenges Guiding principles for fast ﬂow What we did to help fast ﬂow Results and key takeaways 15

Slide 16

Slide 16 text

Case study: context & challenges 16

Slide 17

Slide 17 text

Case Study UK Home Oﬃce (GOV.UK) 17

Slide 18

Slide 18 text

Case Study 18

Slide 19

Slide 19 text

Case Study 19 700+ people in total 70+ engineering teams, 7 locations Time-critical delivery (#Brexit) Most of 2018

Slide 20

Slide 20 text

Case Study 20 Increase speed & safety of delivery Multi-year programme New Brexit-related work Track & control infra costs

Slide 21

Slide 21 text

Case Study 21 (Traditional DC + VMs + JVM) AWS + Kubernetes + JVM ELK for logs and transaction traces [ ]

Slide 22

Slide 22 text

Case Study Scale existing approach to 7x size 22

Slide 23

Slide 23 text

Case Study Spread awareness of eﬀective practices 23

Slide 24

Slide 24 text

Case Study Cross internal and external boundaries 24

Slide 25

Slide 25 text

Case Study “Continuous Delivery” 25

Slide 26

Slide 26 text

Case Study Avoid “one-size-ﬁts-all” 26

Slide 27

Slide 27 text

Case Study Deﬁne the Platform 27

Slide 28

Slide 28 text

Case Study Improve logging quality 28

Slide 29

Slide 29 text

Case Study Multi-supplier delivery model 29

Slide 30

Slide 30 text

What do we aim for in the context of fast ﬂow? 30

Slide 31

Slide 31 text

31 Continuously ‘untangle’ business concepts

Slide 32

Slide 32 text

32 Find and adjust team & system boundaries for ﬂow

Slide 33

Slide 33 text

33 Minimize hand-oﬀs

Slide 34

Slide 34 text

34 Avoid blocking dependencies

Slide 35

Slide 35 text

35 Move some decision-making to teams

Slide 36

Slide 36 text

36 ➔ Untangle business concepts ➔ Adjust boundaries for ﬂow ➔ Minimize hand-oﬀs ➔ Remove blocking dependencies ➔ Move decision-making to teams

Slide 37

Slide 37 text

37 Example: deployment pipeline architecture

Slide 38

Slide 38 text

Slide 39

Slide 39 text

Slide 40

Slide 40 text

Slide 41

Slide 41 text

Slide 42

Slide 42 text

42 The architecture of deployment pipelines is a key consideration for fast ﬂow

Slide 43

Slide 43 text

43 “Show me your deployment pipelines and I will predict your team boundaries” 💥

Slide 44

Slide 44 text

Guiding principles for fast ﬂow 45

Slide 45

Slide 45 text

46 Reliable Software Releases Through Build, Test, and Deployment Automation Jez Humble & Dave Farley, 2010 Continuous Delivery

Slide 46

Slide 46 text

Continuous Delivery with Windows and .NET 47 Matthew Skelton & Chris O’Dell O’Reilly, 2016 cdwithwindows.net ** Now a key text for MSc in Software Engineering at UCL **

Slide 47

Slide 47 text

Team Topologies Organizing business and technology teams for fast ﬂow Matthew Skelton & Manuel Pais IT Revolution Press, September 2019 Order via stores worldwide: teamtopologies.com/book 48

Slide 48

Slide 48 text

49 Re-aligned architecture

Slide 49

Slide 49 text

50 Fast feedback via deployment pipelines

Slide 50

Slide 50 text

51 Good technical practices (TDD, …)

Slide 51

Slide 51 text

52 Team ownership of software & services

Slide 52

Slide 52 text

53 Conﬁguration in version control (Git)

Slide 53

Slide 53 text

54 Cloud-native: transparent in operation

Slide 54

Slide 54 text

55 Cloud-native: designed for automation

Slide 55

Slide 55 text

56 Continuous testing performance scanning deployment monitoring right-sizing integration

Slide 56

Slide 56 text

Team Topologies Organizing business and technology teams for fast ﬂow Matthew Skelton & Manuel Pais IT Revolution Press, September 2019 Order via stores worldwide: teamtopologies.com/book 57

Slide 57

Slide 57 text

What we did to help fast ﬂow 58

Slide 58

Slide 58 text

Case Study New ﬂow-based operating model 59

Slide 59

Slide 59 text

60 Flow of change

Slide 60

Slide 60 text

Case Study Focus on the "operator experience" 61

Slide 61

Slide 61 text

Case Study Deﬁne the platform 62

Slide 62

Slide 62 text

Case Study Engineering Guild + Engineering Working Group 64

Slide 63

Slide 63 text

Case Study Weekly lunchtime tech talks 65

Slide 64

Slide 64 text

Case Study ‘Continuous Delivery for people who don’t write code’ Slides: https://www.slideshare.net/matthewskelton/continuous-delivery-for-p eople-who-do-not-write-code-matthew-skelton-conﬂux 66

Slide 65

Slide 65 text

Case Study Team-ﬁrst techniques Example: Run Book Dialogue Sheet 67

Slide 66

Slide 66 text

68 runbooktemplate.info Run Book dialogue sheets

Slide 67

Slide 67 text

Case Study Engineering Standards 69

Slide 68

Slide 68 text

Slide 69

Slide 69 text

Engineering Standards 71 Minimal but important constraints Dynamic for reviewing and updating Emphasise team engagement

Slide 70

Slide 70 text

Case Study Engineering Assessments 72

Slide 71

Slide 71 text

Engineering Assessments 73 6 dimensions [in 2018]: Team Health, Deployment, Continuous Delivery, Flow, Operability, Testing SoftwareDeliveryAssessment.com

Slide 72

Slide 72 text

Engineering Assessments 74 Criteria taken from existing sources: Spotify Team Health Check, DevOps for the Modern Enterprise, Continuous Delivery, Accelerate, Team Guide to Software Operability, Team Guide to Software Testability

Slide 73

Slide 73 text

Slide 74

Slide 74 text

Slide 75

Slide 75 text

Slide 76

Slide 76 text

Engineering Assessments 78 Team self-assessment Facilitated to guide discussions “Viral” facilitation model Results collated across teams

Slide 77

Slide 77 text

Slide 78

Slide 78 text

80 SoftwareDeliveryAssessment.com

Slide 79

Slide 79 text

Many of these principles and practices appear in the Team Topologies book 💡 82

Slide 80

Slide 80 text

Team Topologies Organizing business and technology teams for fast ﬂow Matthew Skelton & Manuel Pais IT Revolution Press, September 2019 Order via stores worldwide: teamtopologies.com/book 83

Slide 81

Slide 81 text

Results and key takeaways 84

Slide 82

Slide 82 text

Case Study 85 Software releases From 2 successful release candidate builds per week to 8 builds per day Multiple, independent routes to live

Slide 83

Slide 83 text

Case Study 86 Operability Ops teams love the new "operator experience" with logging & dashboards No major operational problems with new releases

Slide 84

Slide 84 text

Case Study 87 Collaboration Logging beginning to be seen as a key observability tool Better cross-team awareness of how systems work through telemetry

Slide 85

Slide 85 text

Case Study Moving towards Continuous Delivery at DDaT See public blog post at https://hodigital.blog.gov.uk/2018/08/10/moving-tow ards-continuous-delivery-in-ddat/ 88

Slide 86

Slide 86 text

Case Study 89 (Many other people were also involved)

Slide 87

Slide 87 text

Case Study 90 Ideas and patterns from Team Topologies now being used in this department

Slide 88

Slide 88 text

What social and technical practices are needed for fast ﬂow at scale? 91

Slide 89

Slide 89 text

92 Make space for learning/sharing Promote good work Help to develop skills in speaking Lunchtime talks (food) Interplay of working group + guild

Slide 90

Slide 90 text

93 Deﬁne the Platform Service Level Expectations Who owns what? What's missing? Use DevEx to drive improvements Thinnest Viable Platform

Slide 91

Slide 91 text

94 Address operability early on Operational aspects are also features Add operational checks into the deployment pipeline Team-ﬁrst: RunBookTemplate.info

Slide 92

Slide 92 text

95 Good logging is foundational Use a well-deﬁned event space Searchable EventIDs increase context Enables collaboration between teams Invest in logging infra (Saas/people/&c.)

Slide 93

Slide 93 text

96 Empower teams to improve Co-create the engineering standards Look for organisational blockers “No-blame” is paramount SoftwareDeliveryAssessment.com

Slide 94

Slide 94 text

98 The architecture of deployment pipelines is a key consideration for fast ﬂow

Slide 95

Slide 95 text

Team Topologies Organizing business and technology teams for fast ﬂow Matthew Skelton & Manuel Pais IT Revolution Press, September 2019 Order via stores worldwide: teamtopologies.com/book 100

Slide 96

Slide 96 text

expert-led group learning ❏ P602 - Software Delivery Assessment ❏ P616 - software reliability for product managers ❏ P630 - Find possible streams with Independent Service Heuristics ❏ P634 - Peer Learning - Writing Tech Talks ❏ P641f - Preparing to adopt Team Topologies ❏ P641i - Skills paths for fast ﬂow - leaders and HR 101 online workshops for up to 20 people

Slide 97

Slide 97 text

102 expert-led group learning for adopting fast ﬂow and Team Topologies conﬂuxhq.com