DevOps is just a beginning

This is the updated version of a prior talk called "DevOps is not enough". To reflect the update, I slightly changed the title - basically without changing the meaning. The slide deck starts with a little definition of the term "DevOps" based on the three ways as they were described in the book "The Phoenix Project". After this short introduction the claim of the title is picked up: "Why is DevOps just a beginning?".

In order to explain this claim, the history of economic markets and of IT are briefly explained. The bottomline is that almost all markets supported by IT have drastically changed in the years since IT became relevant for companies. Additionally, IT itself has changed dramatically in this period of time. Therefore, most of the past common knowledge and best practices became counter-productive meanwhile because they solve a completely different problem, i.e., the problems of the times when the markets and IT itself were totally different.

The conclusion from the short examination of history is that we basically have to re-think IT as a whole, which is discussed briefly in the next section of the talk. This section first takes a look at the new drivers that inflict change on IT. Then it derives the new goals of IT and shows some (typical) building blocks.

Having this new idea of IT at hand, the role of DevOps in it is finally considered. Starting with DevOps and its continuous pursuit of shortening cycle times in order to optimize outcome, DevOps can be used to drive the change of IT. This is exemplarily shown by starting with DevOps and then see, which question arise from that and what solutions it leads to. In the end of the example, many of the building blocks of the new IT are in place - just by starting with DevOps and continuously improving.

This is a very dense talk covering a lot of ground in order to lead to the final observation that DevOps can (and should) be used to drive the required change of IT and many details have been left out to fit the talk in less than an hour. Also the voice track is missing of course, but I still hope that it provides you with some useful information.

Transcript

1. DevOps is just a beginning Rethinking IT from scratch (updated 2020 edition) Uwe Friedrichsen – codecentric AG – 2012-2019
2. Uwe Friedrichsen CTO @ codecentric https://twitter.com/ufried https://www.speakerdeck.com/ufried https://medium.com/@ufried
3. What is DevOps?
4. DevOps is about automating build and deployment Isn’t that just a new term for “Agile”? DevOps is about more tooling! You build it, you run it! You need Kubernetes for DevOps It’s about teams that mediate between Dev and Ops It’s a mindset thing … DevOps
5. DevOps is about automating build and deployment Isn’t that just a new term for “Agile”? DevOps is about more tooling! You build it, you run it! You need Kubernetes for DevOps It’s about teams that mediate between Dev and Ops It’s a mindset thing … DevOps Yikes! No! No, you don’t. You confuse cause and (potential) effect No, not at all. Nope. Different driver, different goals. No. That’s CI/CD. Well, isn’t that always true?
6. Let’s check the “DevOps bible” https://itrevolution.com/book/the-phoenix-project/ https://itrevolution.com/the-three-ways-principles-underpinning-devops/
7. The 3 ways of DevOps
8. Systems thinking • Minimize cycle times – maximize flow • Holistic view – optimize for global goals • Never pass defects downstream • No local optimizations • Seek to increase flow Ops Dev Business IT value chain Customer Holistic optimization
9. Amplify feedback loops • Create right to left feedback loops • Create quality at source – avoid downstream defects • Respond to all customers – internal and external • Shorten and amplify all feedback loops • Embed knowledge where needed Ops Dev Business IT value chain Customer
10. Culture of continual learning • Foster continual experimentation • Experiment, take risks and learn from success and failure • Understand that repetition and practice is needed for mastery • Allocate time for improvement of daily work • Create rituals that reward teams for taking risks • Introduce faults into the system to increase resilience Ops Dev Business IT value chain Customer
11. But why is DevOps just a beginning?
12. First, a bit of background …
13. Evolution of markets
14. Formal part of value creation Solution: machine Dynamic part of value creation Solution: man sluggishness/low dynamic high dynamic high dynamic The historical course of market dynamics and the recent rise of highly dynamic and complex markets The dominance of high dynamics and complexity is neither good nor bad. It‘s a historical fact. t 1970/80 today Age of crafts manu- facturing Age of tayloristic industry Age of global markets 1850/1900 Spacious markets, little competition Local markets, high customi- zation Outperformers exercise market pressure over conventional companies We call the graph shown here the “Taylor Bathtub”. The “bathtub” curve Source: BetaCodex Network Associates, “Organize for complexity”, BetaCodex Network White Paper 12 & 13
15. Formal part of value creation Solution: machine Dynamic part of value creation Solution: man sluggishness/low dynamic high dynamic high dynamic The historical course of market dynamics and the recent rise of highly dynamic and complex markets The dominance of high dynamics and complexity is neither good nor bad. It‘s a historical fact. t 1970/80 today Age of crafts manu- facturing Age of tayloristic industry Age of global markets 1850/1900 Spacious markets, little competition Local markets, high customi- zation Outperformers exercise market pressure over conventional companies We call the graph shown here the “Taylor Bathtub”. Pre-industrial era Source: BetaCodex Network Associates, “Organize for complexity”, BetaCodex Network White Paper 12 & 13 Tailor-made solutions Mastery is key to success
16. Formal part of value creation Solution: machine Dynamic part of value creation Solution: man sluggishness/low dynamic high dynamic high dynamic The historical course of market dynamics and the recent rise of highly dynamic and complex markets The dominance of high dynamics and complexity is neither good nor bad. It‘s a historical fact. t 1970/80 today Age of crafts manu- facturing Age of tayloristic industry Age of global markets 1850/1900 Spacious markets, little competition Local markets, high customi- zation Outperformers exercise market pressure over conventional companies We call the graph shown here the “Taylor Bathtub”. Industrial era Source: BetaCodex Network Associates, “Organize for complexity”, BetaCodex Network White Paper 12 & 13 Cost-efficiently scale production Getting more done with less people is key to success
17. Formal part of value creation Solution: machine Dynamic part of value creation Solution: man sluggishness/low dynamic high dynamic high dynamic The historical course of market dynamics and the recent rise of highly dynamic and complex markets The dominance of high dynamics and complexity is neither good nor bad. It‘s a historical fact. t 1970/80 today Age of crafts manu- facturing Age of tayloristic industry Age of global markets 1850/1900 Spacious markets, little competition Local markets, high customi- zation Outperformers exercise market pressure over conventional companies We call the graph shown here the “Taylor Bathtub”. Post-industrial era Source: BetaCodex Network Associates, “Organize for complexity”, BetaCodex Network White Paper 12 & 13 Continuously respond to changing demands Continuous market adaption is key to success
18. Key drivers Pre-industrial era • No clear driver • “Whatever pays the bill” Industrial era • Cost-efficiency • Scalability • Repeatability • Stability • Efficiency & scale Post-industrial era • Cycle times • Adaptability • Flexibility • Resilience • Effectiveness & speed
19. Evolution of IT
20. 1960 1970 1980 1990 2000 2010 2020 Complicated (Business functions) Complex (Business processes) Highly complex (Business nervous system) Software crisis Software engineering PC LAN Internet Business Support of IT Selective Holistic Complicated Complex “Moore’s law” Mobile IoT
21. IT has changed a lot over the decades ...
22. 1960 1970 1980 1990 2000 2010 2020 Complicated (Business functions) Complex (Business processes) Highly complex (Business nervous system) Software crisis PC LAN Internet Business Support of IT Selective Holistic Complicated Complex “Moore’s law” Mobile IoT Software engineering ... but still we strive to control our IT of today ... ... based on the concepts we developed for an IT almost 50 years ago
23. Formal part of value creation Solution: machine Dynamic part of value creation Solution: man sluggishness/low dynamic high dynamic high dynamic The historical course of market dynamics and the recent rise of highly dynamic and complex markets The dominance of high dynamics and complexity is neither good nor bad. It‘s a historical fact. t 1970/80 today Age of crafts manu- facturing Age of tayloristic industry Age of global markets 1850/1900 Spacious markets, little competition Local markets, high customi- zation Outperformers exercise market pressure over conventional companies We call the graph shown here the “Taylor Bathtub”. Remember the “bathtub” curve? Source: BetaCodex Network Associates, “Organize for complexity”, BetaCodex Network White Paper 12 & 13
24. 1960 1970 1980 1990 2000 2010 2020 Complicated (Business functions) Complex (Business processes) Highly complex (Business nervous system) Software crisis Software engineering PC LAN Internet Business Support of IT Selective Holistic Complicated Complex “Moore’s law” Mobile IoT Also the business we support with IT today ... ... is very different from the business we supported back then
25. What is the role of IT today?
26. IT today is ... • ... the nervous system of the business • ... an enabler of (disruptive) new business models • ... an integral part of the business model (“digitization”) • ... the medium for the continuous customer communication
27. And now?
28. 1 We live in an age of uncertainty
29. 2 We need to move fast and learn all the time to succeed in the face of uncertainty
30. 3 IT today is a key success factor to survive in a post-industrial market
31. 4 The traditional IT “best practices” are counterproductive because they solve a completely different problem
32. We need to rethink IT!
33. Rethinking IT
34. What are the change drivers?
35. Change drivers • Post-industrial markets • Highly dynamic, consumer-driven markets • Economic Darwinism • Digital transformation • IT as (integral part of) a product, API-driven business models • Contextual computing, ubiquitous computing • Disruptive technologies (Innovation through disruption) • Mobile, cloud, smart data, IoT, serverless, AI, …
36. What are the new goals?
37. New IT goals • Speed – Minimize cycle times through IT value chain • Flexibility – Respond flexibly and nimbly to changing needs • Effectiveness – Focus on outcome, not output • Efficiency – Ensure required throughput • Robustness – Build highly available, yet adaptable solutions • Improve continuously – Entropy never sleeps!
38. What are the new building blocks?
39. New IT Process DevOps Continuous design Flow (batch size 1) Agile Lean … Governance Beyond Budgeting Outcome-driven Decentralized control Lean EAM … People Empathy T-shaped Responsibility Curiosity Mastery … Organization DevOps Market capability teams Autonomy & purpose Decentralized responsibility Automation … Technology Cloud native Serverless Automation Resilience Smart data … Improvement DevOps Continuous improvement Systemic optimization Inspect and adapt …
40. The role of DevOps in a new IT
41. Repetition: The 3 ways of DevOps • Systems thinking • Amplify feedback loops • Culture of continual experimentation & learning
42. DevOps acts as a change catalyst
43. Eh, WTF?
44. Okay, let me explain …
45. “Let us start with DevOps …”
46. DevOps
47. “But our IT organization is too slow. We need to organize differently …”
48. DevOps Market capability teams (plus optional platform teams)
49. “How can we support the teams to become faster?”
50. Market capability teams (plus optional platform teams) DevOps Autonomy (decentralized responsibility)
51. “How can we manage those teams?”
52. Market capability teams (plus optional platform teams) Autonomy (decentralized responsibility) DevOps Control via purpose (vision, goals, constraints)
53. “How can we measure (control) If the teams are on the right track?”
54. Market capability teams (plus optional platform teams) Autonomy (decentralized responsibility) Control via purpose (vision, goals, constraints) DevOps Measure outcome (not output)
55. “Hmm, we need to rethink our overall governance model …”
56. Market capability teams (plus optional platform teams) Autonomy (decentralized responsibility) Control via purpose (vision, goals, constraints) Measure outcome (not output) DevOps Beyond budgeting (and BetaCodex)
57. “How can we support autonomy on the architectural level?”
58. Market capability teams (plus optional platform teams) Autonomy (decentralized responsibility) Control via purpose (vision, goals, constraints) Measure outcome (not output) Beyond budgeting (and BetaCodex) DevOps Cloud native (Microservices)
59. “How can we support teams getting faster on the technological level?”
60. Market capability teams (plus optional platform teams) Autonomy (decentralized responsibility) Control via purpose (vision, goals, constraints) Cloud native (Microservices) Measure outcome (not output) Beyond budgeting (and BetaCodex) DevOps Heterogeneity
61. “How can we ensure the required speed and flexibility on the platform level?”
62. Market capability teams (plus optional platform teams) Autonomy (decentralized responsibility) Control via purpose (vision, goals, constraints) Cloud native (Microservices) Heterogeneity Measure outcome (not output) Beyond budgeting (and BetaCodex) DevOps Cloud & Serverless
63. “How can we guarantee high speed and high-quality delivery?”
64. Market capability teams (plus optional platform teams) Autonomy (decentralized responsibility) Control via purpose (vision, goals, constraints) Cloud native (Microservices) Heterogeneity Cloud & Serverless Measure outcome (not output) Beyond budgeting (and BetaCodex) DevOps Continuous Delivery
65. “How can we guarantee high availability in production?”
66. Market capability teams (plus optional platform teams) Autonomy (decentralized responsibility) Control via purpose (vision, goals, constraints) Cloud native (Microservices) Continuous Delivery Heterogeneity Cloud & Serverless Measure outcome (not output) Beyond budgeting (and BetaCodex) DevOps Resilience (incl. chaos engineering)
67. “How can we reliably manage all those moving parts in production?”
68. Market capability teams (plus optional platform teams) Autonomy (decentralized responsibility) Control via purpose (vision, goals, constraints) Cloud native (Microservices) Continuous Delivery Heterogeneity Cloud & Serverless Resilience (incl. chaos engineering) Measure outcome (not output) Beyond budgeting (and BetaCodex) DevOps Automation & Observability
69. “How can we make sure the services created by different teams work smoothly together?”
70. Market capability teams (plus optional platform teams) Autonomy (decentralized responsibility) Control via purpose (vision, goals, constraints) Cloud native (Microservices) Continuous Delivery Heterogeneity Cloud & Serverless Resilience (incl. chaos engineering) Automation & Observability Measure outcome (not output) Beyond budgeting (and BetaCodex) DevOps Lean EAM
71. “How can we make our efforts sustainable?”
72. Market capability teams (plus optional platform teams) Autonomy (decentralized responsibility) Control via purpose (vision, goals, constraints) Cloud native (Microservices) Continuous Delivery Heterogeneity Cloud & Serverless Resilience (incl. chaos engineering) Automation & Observability Measure outcome (not output) Beyond budgeting (and BetaCodex) Lean EAM DevOps Continuous improvement
73. “How do we get enough input for continuous improvement?”
74. Market capability teams (plus optional platform teams) Autonomy (decentralized responsibility) Control via purpose (vision, goals, constraints) Cloud native (Microservices) Continuous Delivery Heterogeneity Cloud & Serverless Resilience (incl. chaos engineering) Automation & Observability Measure outcome (not output) Beyond budgeting (and BetaCodex) Lean EAM Continuous improvement DevOps Quick feedback loops (OODA loop)
75. “How can we establish those quick feedback loops?”
76. Market capability teams (plus optional platform teams) Autonomy (decentralized responsibility) Control via purpose (vision, goals, constraints) Cloud native (Microservices) Continuous Delivery Heterogeneity Cloud & Serverless Resilience (incl. chaos engineering) Automation & Observability Measure outcome (not output) Beyond budgeting (and BetaCodex) Lean EAM Continuous improvement DevOps Quick feedback loops (OODA loop) Flow (batch size 1)
77. “How what does this all mean for the people involved?”
78. Market capability teams (plus optional platform teams) Autonomy (decentralized responsibility) Control via purpose (vision, goals, constraints) Cloud native (Microservices) Continuous Delivery Heterogeneity Cloud & Serverless Resilience (incl. chaos engineering) Automation & Observability Measure outcome (not output) Beyond budgeting (and BetaCodex) Flow (batch size 1) Lean EAM Continuous improvement DevOps Quick feedback loops (OODA loop) Mastery
79. Market capability teams (plus optional platform teams) Autonomy (decentralized responsibility) Control via purpose (vision, goals, constraints) Cloud native (Microservices) Continuous Delivery Heterogeneity Cloud & Serverless Resilience (incl. chaos engineering) Automation & Observability Mastery Measure outcome (not output) Beyond budgeting (and BetaCodex) Flow (batch size 1) Lean EAM Continuous improvement DevOps Quick feedback loops (OODA loop) T-Shaped people (being empathetic)
80. Market capability teams (plus optional platform teams) Autonomy (decentralized responsibility) Control via purpose (vision, goals, constraints) Cloud native (Microservices) Continuous Delivery Heterogeneity Cloud & Serverless Resilience (incl. chaos engineering) Automation & Observability Mastery Measure outcome (not output) Beyond budgeting (and BetaCodex) Flow (batch size 1) Lean EAM Continuous improvement T-Shaped people (being empathetic) DevOps Quick feedback loops (OODA loop) Curiosity
81. Market capability teams (plus optional platform teams) Autonomy (decentralized responsibility) Control via purpose (vision, goals, constraints) Cloud native (Microservices) Continuous Delivery Heterogeneity Cloud & Serverless Resilience (incl. chaos engineering) Automation & Observability Mastery Measure outcome (not output) Beyond budgeting (and BetaCodex) Flow (batch size 1) Lean EAM Continuous improvement T-Shaped people (being empathetic) DevOps Quick feedback loops (OODA loop) Curiosity
82. What can learn from it?
83. 1 All the topics are interrelated
84. 2 A single topic on its own has a limited effect
85. 3 The effects of the topics multiply up
86. 4 DevOps drives the implementation of the new IT
87. A little warning at the end
88. This is a model for reasoning, not a dogma!
89. Reality will be more complex
90. DevOps is a revolution in your head, but an evolution in implementation
91. Wrap-up
92. Wrap-up • Markets have changed • IT has changed • The role of IT has changed • New drivers • New goals • New building blocks • DevOps acts as a change catalyst
93. Some recommended reading
94. Uwe Friedrichsen CTO @ codecentric https://twitter.com/ufried https://www.speakerdeck.com/ufried https://medium.com/@ufried