The IT multiverse - A tourist guide with insider tips

This talk was created as a keynote for a conference. Thus, it is meant rather as food for thought and offering some different point of view than concrete advice how to do things.

It starts with a journey through the "IT multiverse", visiting several types of IT organizations - first, along the evolution path of enterprise IT, then visiting startups and those who became successful and finally several bordering universes.

While the ideas found on that round flight can be used for several (often strategic) purposes, I use it here for a more limited use case, which is trying to understand where an individual is, where (s)he wants to go and how to get there.

In order to reason about those questions, I create a little model that tries to extract the general type of IT organization model (including the forces that act on it), its viability and typical evolution paths. This model can then be used to ponder the questions mentioned before (without overdoing it).

Finally, I offer a few more personal tips that might be useful on your way through the crazy, but fascinating multiverse called IT.

As always, unfortunately the voice track is missing (and there is no recording of the talk yet), but hopefully the slides in themselves also offer you a bit food for thought ... ;)

Transcript

1. The IT multiverse A tourist guide with insider tips Uwe Friedrichsen – codecentric AG – 1993-2019
2. Uwe Friedrichsen CTO @ codecentric https://twitter.com/ufried https://www.speakerdeck.com/ufried https://medium.com/@ufried
3. The dawn of IT …
5. Source: https://commons.wikimedia.org/wiki/File:ZuseZ22BerlinTechnikMuseum.jpg No machine-readable author provided. JuergenG assumed (based on copyright claims). [CC BY-SA 3.0]
7. Software as add-on
8. Software development as … … an ad hoc activity
9. Ad hoc • Development on demand • Peer to peer communication • No clear processes • No clear practices (“Whatever goes”) • Lots of manual work • Lots of DIY development (Do it yourself)
10. Ad hoc
11. Software as core capability
12. Software crisis
13. Software engineering
14. Software development process
15. Software development as … … an activity driven by a comprehensive software development process
16. Process driven • Development as a division-of-labor activity • Highly regulated communication • Detailed processes • Hierarchical organization • Some automation, manual work defined and controlled • Standard Software and Enterprise Frameworks
17. Ad hoc Process driven
18. Source: The CHAOS Report (1994) – The Standish Group
19. IT is just a cost center
20. Software development as … … an outsourced activity
21. Outsourcing • IT not considered differentiating (“cost center”) • Expensive and error-prone (see chaos report) • Outsourcing to specialized IT suppliers • Expectations often not met
22. Ad hoc Outsourcing Process driven
23. Source: https://agilemanifesto.org/
24. Source: Heart of Agile, https://heartofagile.com/
25. Software development as … … a collaborative activity
26. Actual Agile • Collaborative software development • Inspect and adapt • Continuous delivery • Continuous feedback • Continuous learning • Continuous improvement • Usually working for selected small teams only
27. Ad hoc Outsourcing Process driven Actual Agile
28. Scrum took over …
29. … and got blended with enterprise habits
30. Software development as … … cargo cult “Agile”
31. Cargo cult “Agile” • Gaps in Scrum filled up with existing process and roles • Usually ignored outside the development team • Inspect and adapt cycle not implemented • No continuous delivery • No or little feedback • No or little learning • No or little improvement
32. Ad hoc Outsourcing Process driven Actual Agile Cargo cult “Agile”
33. Source: Scaled Agile, Inc., https://www.scaledagileframework.com/
34. Software development as … … scaled “Agile”
35. Scaled “Agile” • SAFe won the race • Addresses enterprise’s yearning for perceived certainty • Turned cargo-cult Agile into non-Agile • Process-driven development in a new disguise • Other, more agile approaches available (e.g., SWARM *) • Will remain niches at best * https://dannorth.net/2018/01/26/in-praise-of-swarming/
36. Ad hoc Outsourcing Process driven Scaled “Agile” Actual Agile Cargo cult “Agile”
37. A bit earlier in another universe …
38. Dot-com
39. Software development as … … traditional startup activity
40. Startup 1.0 • IT-intensive business offering • Needed to go fast • Used a minimal enterprise IT stack • Focus on fast feedback
41. Ad hoc Outsourcing Startup 1.0 Process driven Scaled “Agile” Actual Agile Cargo cult “Agile”
42. “Whoa, we cannot scale and stay fast!”
43. DevOps & Microservices
44. Software development as … … collaborative activity of multiple loosely coupled capability teams
45. DevOps 1.0 • Needed to keep going fast while (hyper-)scaling business • DevOps on the organizational and governance level • Loosely coupled cross-functional capability teams • Microservices as supporting architecture style
46. Ad hoc Outsourcing Startup 1.0 Process driven Scaled “Agile” Actual Agile DevOps 1.0 Cargo cult “Agile”
47. “We want that, too!”
48. Software development as … … cargo cult “DevOps”, a.k.a. CrapOps / NoOps
49. CrapOps / NoOps • Result of multiple misunderstandings • Mimicked DevOps instructions without understanding ideas • Copied technology stack without understanding drivers • Filled gaps with old corporate habits • Filled gaps with old corporate habits • “We need to control everything centrally!” • “How can we use it to save money/lay off people?” • Made companies neither faster nor better nor more scalable • IT additionally drowns in non-understood tech stack
50. Ad hoc Outsourcing Startup 1.0 Process driven Scaled “Agile” Actual Agile DevOps 1.0 CrapOps/ NoOps Cargo cult “Agile”
51. Meanwhile in the DevOps universe …
52. “Our teams do lots of redundant work”
53. Software development as … … collaborative activity of multiple loosely coupled capability teams supported by platform teams
54. DevOps 2.0 (w/ platform services) • Redundant tasks picked up by platform teams • Platform teams act as internal service providers • Capability teams still have decision authority • Emerging platform self-service marketplace
55. Ad hoc Outsourcing Startup 1.0 Process driven DevOps 2.0 Scaled “Agile” Actual Agile DevOps 1.0 CrapOps/ NoOps Cargo cult “Agile”
56. “Existing solutions do not scale enough”
57. “Let’s create our own scalable solutions”
58. “We want that, too!”
59. Software development as … … startup activity with strong NIH syndrome
60. Startup 2.0 (NIH infested) • Wannabe Netflix syndrome • Infatuated with technology • Strong NIH syndrome (Not invented here) • Strong DIY syndrome (Do it yourself) • Bloated IT departments
61. Ad hoc Outsourcing Startup 1.0 Startup 2.0 Process driven DevOps 2.0 Scaled “Agile” Actual Agile DevOps 1.0 CrapOps/ NoOps Cargo cult “Agile”
62. Meanwhile in the DevOps universe …
63. “Hey, cloud helps us to go even faster”
64. “Let’s go for managed services!”
65. Software development as … … collaborative activity of multiple loosely coupled capability teams, going cloud native all-in
66. DevOps 3.0 (cloud native all-in) • Leveraging managed services on all levels • Integrating using FaaS technology • Only building differentiating logic on their own • Actively reducing vertical integration depth • Deliberately moving right on a Wardley map * • Going faster without compromising quality * https://medium.com/wardleymaps/on-being-lost-2ef5f05eb1ec
67. Ad hoc Outsourcing Startup 1.0 Startup 2.0 Process driven DevOps 2.0 Scaled “Agile” Actual Agile DevOps 1.0 CrapOps/ NoOps DevOps 3.0 Cargo cult “Agile”
68. “Hmm, that’s good advice”
69. Software development as … … startup activity with minimal IT integration depth
70. Startup 3.0 (Use over make) • Adopt cloud native all-in (managed services) • Go for low code/no code environments where possible • Maximize reduction of vertical IT integration depth • Focus on customer experience, not IT impressiveness
71. Ad hoc Outsourcing Startup 1.0 Startup 3.0 Startup 2.0 Process driven DevOps 2.0 Scaled “Agile” Actual Agile DevOps 1.0 CrapOps/ NoOps DevOps 3.0 Cargo cult “Agile”
72. Exploring the borders a bit …
73. The IoT universe
74. IoT (Internet of Things) • A.k.a. the wild, wild west of IT • Widely heterogeneous, not yet settled landscape • Disruptive change in IT • Business models still emerging • Fascinating, but very volatile domain
75. Ad hoc Outsourcing Startup 1.0 Startup 3.0 Startup 2.0 Process driven DevOps 2.0 Scaled “Agile” Actual Agile IoT DevOps 1.0 CrapOps/ NoOps DevOps 3.0 Cargo cult “Agile”
76. The data science universe
77. Data Science (incl. AI/ML/DL) • Important complementing aspect of modern IT • Often a bit decoupled from normal software development • Different mindset (exploration first) • Math often frightening for average developer • Needs to be integrated better in normal IT activities
78. Ad hoc Outsourcing Startup 1.0 Startup 3.0 Startup 2.0 Process driven DevOps 2.0 Scaled “Agile” Actual Agile IoT DevOps 1.0 CrapOps/ NoOps Data Science DevOps 3.0 Cargo cult “Agile”
79. The scientific computing universe
80. Scientific computing • We do not really understand what they are doing • Looks interesting, but not actually applicable for us • Quite loosely connected to commercial computing • We decide to ignore them – and hope it’s safe … ;)
81. Ad hoc Outsourcing Startup 1.0 Startup 3.0 Startup 2.0 Process driven Scientific computing DevOps 2.0 Scaled “Agile” Actual Agile IoT DevOps 1.0 CrapOps/ NoOps Data Science DevOps 3.0 Cargo cult “Agile”
82. The industrial computing universe
83. Industrial computing • Focus on embedded computing for many years • IIoT acts as a game changer • Gets closer to traditional IT at high speed • Multi-billion dollar/euro market emerges • Still mostly unnoticed/ignored by traditional IT
84. Ad hoc Industrial computing Outsourcing Startup 1.0 Startup 3.0 Startup 2.0 Process driven Scientific computing DevOps 2.0 Scaled “Agile” Actual Agile IoT DevOps 1.0 CrapOps/ NoOps Data Science DevOps 3.0 Cargo cult “Agile”
85. Ad hoc Industrial computing Outsourcing Startup 1.0 Startup 3.0 Startup 2.0 Process driven Scientific computing DevOps 2.0 Scaled “Agile” Actual Agile IoT DevOps 1.0 CrapOps/ NoOps Data Science DevOps 3.0 Cargo cult “Agile”
86. There are a lot more universes …
87. … but let us stop here and try to use the knowledge we have discovered so far
88. Ad hoc Industrial computing Outsourcing Startup 1.0 Startup 3.0 Startup 2.0 Process driven Scientific computing DevOps 2.0 Scaled “Agile” Actual Agile IoT DevOps 1.0 CrapOps/ NoOps Data Science DevOps 3.0 Cargo cult “Agile”
89. Where are you right now? Where would you like to be? How do you get there?
90. Therefore let us look at the universes from a bit different angle drivers – forces – viability - paths
91. But first a (tiny) bit of theory …
92. Formal part of value creation Solution: machine Dynamic part of value creation Solution: man sluggishness/low dynamic high dynamic high dynamic The historical course of market dynamics and the recent rise of highly dynamic and complex markets The dominance of high dynamics and complexity is neither good nor bad. It‘s a historical fact. t 1970/80 today Age of crafts manu- facturing Age of tayloristic industry Age of global markets 1850/1900 Spacious markets, little competition Local markets, high customi- zation Outperformers exercise market pressure over conventional companies We call the graph shown here the “Taylor Bathtub”. The “bathtub” curve Source: BetaCodex Network Associates, “Organize for complexity”, BetaCodex Network White Paper 12 & 13
93. Key drivers Pre-industrial era • No clear driver • “Whatever pays the bill” Industrial era • Cost-efficiency • Scalability • Repeatability • Stability • Efficiency & scale Post-industrial era • Cycle times • Adaptability • Flexibility • Resilience • Effectiveness & speed
94. This model can also be applied to IT
95. Properties Pre-industrial Industrial Post-industrial Market Emerging market Wide & slow market Narrow & fast market Strategic goal None (“Get the bill paid!”) Cost-efficiency Short cycle times System theory Obvious Complicated Complex Work model Ad hoc Processes Collaboration Organization model Peer to peer Hierarchies Autonomous teams Practices “Whatever goes” ITIL, V-Modell, SAFe DevOps, Kanban, XP, Agile Individuals Experts (Cube-shaped) Specialists (I-shaped) Generalists (T-Shaped) Automation Low Medium-high Very high Manual work Ad hoc Defined and controlled Avoided Programming model Do it yourself Standard Software, Enterprise Frameworks Cloud-native, Serverless, Managed Services
96. Another, related point of view
97. Taking this “magic triangle” ...
98. Good Fast Cheap Optimizing for quality and cycle times will result in higher costs Optimizing for quality and costs will result in long cycle times Optimizing for cycle times and costs will result in reduced quality
99. ... will lead you to this model
100. Pre-industrial No clear driver Good Fast Cheap Industrial Deliver large batches at minimized costs towards slow contexts Post-industrial Quickly adapt to ever-changing needs of dynamic, fast-moving contexts Startup Test hypotheses and pivot as fast as possible to discover a product-context fit
101. Now we are ready to look at the universes from a bit different angle drivers – forces – viability - paths
102. Ad hoc • Pre-Industrial • Suitable for companies with little IT demand • Truck factor typically < 3 • Does not scale • Usually develops into process-driven IT
103. Process driven • Industrial • Predominant in larger companies • Self-reinforcing system • Strong attractor – requires energy not to drift into • Requires huge amount of energy to escape from • Usually terminal state • Sometimes disguised as “Agile” (à scaled “Agile”) or alike
104. Outsourcing • Industrial • Very fashionable IT strategy 10 - 20 years ago • Valid option for non-differentiating parts of IT • Okay, if lead time for changes are irrelevant • Yet, often a result of a general misperception of IT by decision makers who do not really understand IT • As a general IT strategy, usually not sustainable
105. Actual Agile • Post-industrial • Addresses needs of highly dynamic environments • Rather a developer ideal than IT reality • Needs protected environment to survive • E.g., strong leader shielding the team from outer forces • Usually assimilated by process-driven IT (cargo cult “Agile”)
106. Cargo cult “Agile” • Industrial • Result of corporate IT assimilating actual Agile • Implementing rituals without understanding ideas • Often just "group therapy for frustrated developers" • Overall usually no real improvements realized • Will eventually drift into process-driven IT or scaled "Agile"
107. Scaled “Agile” • Industrial • Result of scaling cargo cult “Agile” to corporate level • Can help larger corporates to add a tiny dose of flexibility into their overly rigid processes (there is value in doing this) • Not agile, but process-driven IT in a disguise • Usually terminal state • Sometimes drifts back to pure process-driven IT
108. Startup 1.0 • Startup • Vertical IT integration depth needs to be considered • Will be superseded by startup 3.0 as future default • Still, will remain as a niche • With growth, usually develops • either into process-driven IT • or into DevOps 1.0
109. Startup 2.0 (NIH infested) • Startup • Very deep vertical IT integration depth • Okay for unicorns with unique problems • Not sustainable for everyone else • Either evolution to Startup 3.0 or extinction over time • With growth, usually develops • either into process-driven IT • or into DevOps 1.0/2.0
110. Startup 3.0 (Use over make) • Startup • Most up-to-date implementation of Startup • Enables extremely short cycle times at very low cost • With growth, usually develops • either into process-driven IT • or into DevOps 3.0
111. DevOps 1.0 • Post-industrial • Effective, but not very efficient • Lots of duplicate work in different teams • Usually evolves into DevOps 2.0/3.0 over time
112. DevOps 2.0 (w/ platform services) • Post-industrial • Brings a healthy dose of efficiency back into DevOps • Must not be confused with Ops 0.5 • Platform services as marketplace, not as prescriptions • Sustainable model • Still, often energy required to fend off industrial forces • Drift towards DevOps 3.0 should be considered
113. DevOps 3.0 (cloud native all-in) • Post-industrial • Most up-to-date implementation of DevOps • Enables very short cycle times without compromising quality • Sustainable model • Still, often energy required to fend off industrial forces
114. CrapOps / NoOps • Industrial • Cargo cult – implementing rituals only • Often augmented with lots of new technology • Overstrained teams as typical effect • Less speed, more errors • Not sustainable • Usually drifts back to process-driven IT or scaled "Agile"
115. IoT (Internet of Things) • Often pre-industrial or startup • Complementing universe • Currently brings more volatility to the connected universe • Will become more stable with settling of IoT landscape • Will cease to exist as a distinct universe over time • Instead will become integral part of other universes
116. Data Science (incl. AI/ML/DL) • Pre-industrial most of the time • Complementing universe • Brings more trial and error attitude to connected universes • Thus, often experiences repulsive force from industrial IT • Will cease to exist as a distinct universe over time • Instead will become integral part of other universes
117. Scientific computing • Not really connected to the commercial IT universes • Only accidental connections from and to this universe
118. Industrial computing • Industrial • Target IT market with growing relevance • Widespread conservative technology adoption patterns • Very affine to standard software solutions • Especially post-industrial IT will need to learn to respond to it • Sustainable model • Will at least partially merge with other universes over time
119. Summing the different perspective up …
120. Key Model Extra details Sustainability Model Extra details Sustainability Pre-industrial Industrial Post-industrial Startup Sustainable Partially sustainable Sustainable only in niches Not sustainable Usually drifts into different universe over time Terminal state Not viable in the long run Will merge with other universe Unstable (will drift forth and back) Requires protection Mostly unknown
121. Ad hoc Industrial computing Outsourcing Startup 1.0 Startup 3.0 Startup 2.0 Process driven Scientific computing DevOps 2.0 Scaled “Agile” Actual Agile IoT DevOps 1.0 CrapOps/ NoOps Data Science DevOps 3.0 Cargo cult “Agile”
122. Any insider tips for moving between universes?
123. Moving away from process driven • Extremely hard to overcome the inertia • It’s not the individuals, it’s systemic reinforcement • Easier to change the job than to change the company • Sometimes you can establish a protected niche • Still requires continuous efforts to protect your niche from the company’s immune system that will fight it all the time • Usually only works with upper level management support • Focus on culture, not processes or tools
124. Evolving from startup • Organizational growth leads to new challenges • Typical response: Process, hierarchy, bureaucracy • Inevitably leads to process-driven IT • Alternative response: Beta organization • Leads to DevOps • The second path is a lot harder (but worth the effort)
125. How do I become actually Agile? • Aim for DevOps (in its original meaning!) • DevOps provides better framing for IT than, e.g., Scrum • Creates relentless focus on value creation, delivery, feedback (from where value is realized) and continuous improvement • Working Agile environment emerges as a “side effect”
126. We could go on for a long time …
127. … yet, after all, always remember …
128. Ad hoc Industrial computing Outsourcing Startup 1.0 Startup 3.0 Startup 2.0 Process driven Scientific computing DevOps 2.0 Scaled “Agile” Actual Agile IoT DevOps 1.0 CrapOps/ NoOps Data Science DevOps 3.0 Cargo cult “Agile” This is only a model
129. Reality is more complex, has more shades of grey …
130. Use the model to support your reasoning, but do not carry it too far … ;)
131. Any other useful tips for my journey?
132. Technology
133. Technology • Go cloud native – use managed services • DIY is fun, but takes too long and does not create value • Focus on the frontend • That is where the battle for the customer is won today • Mind the IoT train • Accept data as the currency of the 21st century • Do not forget security
134. Simplify! From requirements over architecture over code to infrastructure
135. Take it easy Go your own way and do not take things too seriously
136. One step at a time
137. Love it, change it or leave it
138. A better tomorrow How can we reduce our ecological footprint with IT?
139. Safe travels and have a great journey!
140. Uwe Friedrichsen CTO @ codecentric https://twitter.com/ufried https://www.speakerdeck.com/ufried https://medium.com/@ufried