The IT multiverse - A tourist guide with insider tips

Slide 1

Slide 1 text

The IT multiverse A tourist guide with insider tips Uwe Friedrichsen – codecentric AG – 1993-2019

Slide 2

Slide 2 text

Uwe Friedrichsen CTO @ codecentric https://twitter.com/ufried https://www.speakerdeck.com/ufried https://medium.com/@ufried

Slide 3

Slide 3 text

The dawn of IT …

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

Source: https://commons.wikimedia.org/wiki/File:ZuseZ22BerlinTechnikMuseum.jpg No machine-readable author provided. JuergenG assumed (based on copyright claims). [CC BY-SA 3.0]

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

Software as add-on

Slide 8

Slide 8 text

Software development as … … an ad hoc activity

Slide 9

Slide 9 text

Ad hoc • Development on demand • Peer to peer communication • No clear processes • No clear practices (“Whatever goes”) • Lots of manual work • Lots of DIY development (Do it yourself)

Slide 10

Slide 10 text

Ad hoc

Slide 11

Slide 11 text

Software as core capability

Slide 12

Slide 12 text

Software crisis

Slide 13

Slide 13 text

Software engineering

Slide 14

Slide 14 text

Software development process

Slide 15

Slide 15 text

Software development as … … an activity driven by a comprehensive software development process

Slide 16

Slide 16 text

Process driven • Development as a division-of-labor activity • Highly regulated communication • Detailed processes • Hierarchical organization • Some automation, manual work defined and controlled • Standard Software and Enterprise Frameworks

Slide 17

Slide 17 text

Ad hoc Process driven

Slide 18

Slide 18 text

Source: The CHAOS Report (1994) – The Standish Group

Slide 19

Slide 19 text

IT is just a cost center

Slide 20

Slide 20 text

Software development as … … an outsourced activity

Slide 21

Slide 21 text

Outsourcing • IT not considered differentiating (“cost center”) • Expensive and error-prone (see chaos report) • Outsourcing to specialized IT suppliers • Expectations often not met

Slide 22

Slide 22 text

Ad hoc Outsourcing Process driven

Slide 23

Slide 23 text

Source: https://agilemanifesto.org/

Slide 24

Slide 24 text

Source: Heart of Agile, https://heartofagile.com/

Slide 25

Slide 25 text

Software development as … … a collaborative activity

Slide 26

Slide 26 text

Actual Agile • Collaborative software development • Inspect and adapt • Continuous delivery • Continuous feedback • Continuous learning • Continuous improvement • Usually working for selected small teams only

Slide 27

Slide 27 text

Ad hoc Outsourcing Process driven Actual Agile

Slide 28

Slide 28 text

Scrum took over …

Slide 29

Slide 29 text

… and got blended with enterprise habits

Slide 30

Slide 30 text

Software development as … … cargo cult “Agile”

Slide 31

Slide 31 text

Cargo cult “Agile” • Gaps in Scrum filled up with existing process and roles • Usually ignored outside the development team • Inspect and adapt cycle not implemented • No continuous delivery • No or little feedback • No or little learning • No or little improvement

Slide 32

Slide 32 text

Ad hoc Outsourcing Process driven Actual Agile Cargo cult “Agile”

Slide 33

Slide 33 text

Source: Scaled Agile, Inc., https://www.scaledagileframework.com/

Slide 34

Slide 34 text

Software development as … … scaled “Agile”

Slide 35

Slide 35 text

Scaled “Agile” • SAFe won the race • Addresses enterprise’s yearning for perceived certainty • Turned cargo-cult Agile into non-Agile • Process-driven development in a new disguise • Other, more agile approaches available (e.g., SWARM *) • Will remain niches at best * https://dannorth.net/2018/01/26/in-praise-of-swarming/

Slide 36

Slide 36 text

Ad hoc Outsourcing Process driven Scaled “Agile” Actual Agile Cargo cult “Agile”

Slide 37

Slide 37 text

A bit earlier in another universe …

Slide 38

Slide 38 text

Dot-com

Slide 39

Slide 39 text

Software development as … … traditional startup activity

Slide 40

Slide 40 text

Startup 1.0 • IT-intensive business offering • Needed to go fast • Used a minimal enterprise IT stack • Focus on fast feedback

Slide 41

Slide 41 text

Ad hoc Outsourcing Startup 1.0 Process driven Scaled “Agile” Actual Agile Cargo cult “Agile”

Slide 42

Slide 42 text

“Whoa, we cannot scale and stay fast!”

Slide 43

Slide 43 text

DevOps & Microservices

Slide 44

Slide 44 text

Software development as … … collaborative activity of multiple loosely coupled capability teams

Slide 45

Slide 45 text

DevOps 1.0 • Needed to keep going fast while (hyper-)scaling business • DevOps on the organizational and governance level • Loosely coupled cross-functional capability teams • Microservices as supporting architecture style

Slide 46

Slide 46 text

Ad hoc Outsourcing Startup 1.0 Process driven Scaled “Agile” Actual Agile DevOps 1.0 Cargo cult “Agile”

Slide 47

Slide 47 text

“We want that, too!”

Slide 48

Slide 48 text

Software development as … … cargo cult “DevOps”, a.k.a. CrapOps / NoOps

Slide 49

Slide 49 text

CrapOps / NoOps • Result of multiple misunderstandings • Mimicked DevOps instructions without understanding ideas • Copied technology stack without understanding drivers • Filled gaps with old corporate habits • Filled gaps with old corporate habits • “We need to control everything centrally!” • “How can we use it to save money/lay off people?” • Made companies neither faster nor better nor more scalable • IT additionally drowns in non-understood tech stack

Slide 50

Slide 50 text

Ad hoc Outsourcing Startup 1.0 Process driven Scaled “Agile” Actual Agile DevOps 1.0 CrapOps/ NoOps Cargo cult “Agile”

Slide 51

Slide 51 text

Meanwhile in the DevOps universe …

Slide 52

Slide 52 text

“Our teams do lots of redundant work”

Slide 53

Slide 53 text

Software development as … … collaborative activity of multiple loosely coupled capability teams supported by platform teams

Slide 54

Slide 54 text

DevOps 2.0 (w/ platform services) • Redundant tasks picked up by platform teams • Platform teams act as internal service providers • Capability teams still have decision authority • Emerging platform self-service marketplace

Slide 55

Slide 55 text

Ad hoc Outsourcing Startup 1.0 Process driven DevOps 2.0 Scaled “Agile” Actual Agile DevOps 1.0 CrapOps/ NoOps Cargo cult “Agile”

Slide 56

Slide 56 text

“Existing solutions do not scale enough”

Slide 57

Slide 57 text

“Let’s create our own scalable solutions”

Slide 58

Slide 58 text

“We want that, too!”

Slide 59

Slide 59 text

Software development as … … startup activity with strong NIH syndrome

Slide 60

Slide 60 text

Startup 2.0 (NIH infested) • Wannabe Netflix syndrome • Infatuated with technology • Strong NIH syndrome (Not invented here) • Strong DIY syndrome (Do it yourself) • Bloated IT departments

Slide 61

Slide 61 text

Ad hoc Outsourcing Startup 1.0 Startup 2.0 Process driven DevOps 2.0 Scaled “Agile” Actual Agile DevOps 1.0 CrapOps/ NoOps Cargo cult “Agile”

Slide 62

Slide 62 text

Meanwhile in the DevOps universe …

Slide 63

Slide 63 text

“Hey, cloud helps us to go even faster”

Slide 64

Slide 64 text

“Let’s go for managed services!”

Slide 65

Slide 65 text

Software development as … … collaborative activity of multiple loosely coupled capability teams, going cloud native all-in

Slide 66

Slide 66 text

DevOps 3.0 (cloud native all-in) • Leveraging managed services on all levels • Integrating using FaaS technology • Only building differentiating logic on their own • Actively reducing vertical integration depth • Deliberately moving right on a Wardley map * • Going faster without compromising quality * https://medium.com/wardleymaps/on-being-lost-2ef5f05eb1ec

Slide 67

Slide 67 text

Ad hoc Outsourcing Startup 1.0 Startup 2.0 Process driven DevOps 2.0 Scaled “Agile” Actual Agile DevOps 1.0 CrapOps/ NoOps DevOps 3.0 Cargo cult “Agile”

Slide 68

Slide 68 text

“Hmm, that’s good advice”

Slide 69

Slide 69 text

Software development as … … startup activity with minimal IT integration depth

Slide 70

Slide 70 text

Startup 3.0 (Use over make) • Adopt cloud native all-in (managed services) • Go for low code/no code environments where possible • Maximize reduction of vertical IT integration depth • Focus on customer experience, not IT impressiveness

Slide 71

Slide 71 text

Ad hoc Outsourcing Startup 1.0 Startup 3.0 Startup 2.0 Process driven DevOps 2.0 Scaled “Agile” Actual Agile DevOps 1.0 CrapOps/ NoOps DevOps 3.0 Cargo cult “Agile”

Slide 72

Slide 72 text

Exploring the borders a bit …

Slide 73

Slide 73 text

The IoT universe

Slide 74

Slide 74 text

IoT (Internet of Things) • A.k.a. the wild, wild west of IT • Widely heterogeneous, not yet settled landscape • Disruptive change in IT • Business models still emerging • Fascinating, but very volatile domain

Slide 75

Slide 75 text

Ad hoc Outsourcing Startup 1.0 Startup 3.0 Startup 2.0 Process driven DevOps 2.0 Scaled “Agile” Actual Agile IoT DevOps 1.0 CrapOps/ NoOps DevOps 3.0 Cargo cult “Agile”

Slide 76

Slide 76 text

The data science universe

Slide 77

Slide 77 text

Data Science (incl. AI/ML/DL) • Important complementing aspect of modern IT • Often a bit decoupled from normal software development • Different mindset (exploration first) • Math often frightening for average developer • Needs to be integrated better in normal IT activities

Slide 78

Slide 78 text

Ad hoc Outsourcing Startup 1.0 Startup 3.0 Startup 2.0 Process driven DevOps 2.0 Scaled “Agile” Actual Agile IoT DevOps 1.0 CrapOps/ NoOps Data Science DevOps 3.0 Cargo cult “Agile”

Slide 79

Slide 79 text

The scientific computing universe

Slide 80

Slide 80 text

Scientific computing • We do not really understand what they are doing • Looks interesting, but not actually applicable for us • Quite loosely connected to commercial computing • We decide to ignore them – and hope it’s safe … ;)

Slide 81

Slide 81 text

Ad hoc Outsourcing Startup 1.0 Startup 3.0 Startup 2.0 Process driven Scientific computing DevOps 2.0 Scaled “Agile” Actual Agile IoT DevOps 1.0 CrapOps/ NoOps Data Science DevOps 3.0 Cargo cult “Agile”

Slide 82

Slide 82 text

The industrial computing universe

Slide 83

Slide 83 text

Industrial computing • Focus on embedded computing for many years • IIoT acts as a game changer • Gets closer to traditional IT at high speed • Multi-billion dollar/euro market emerges • Still mostly unnoticed/ignored by traditional IT

Slide 84

Slide 84 text

Ad hoc Industrial computing Outsourcing Startup 1.0 Startup 3.0 Startup 2.0 Process driven Scientific computing DevOps 2.0 Scaled “Agile” Actual Agile IoT DevOps 1.0 CrapOps/ NoOps Data Science DevOps 3.0 Cargo cult “Agile”

Slide 85

Slide 85 text

Slide 86

Slide 86 text

There are a lot more universes …

Slide 87

Slide 87 text

… but let us stop here and try to use the knowledge we have discovered so far

Slide 88

Slide 88 text

Slide 89

Slide 89 text

Where are you right now? Where would you like to be? How do you get there?

Slide 90

Slide 90 text

Therefore let us look at the universes from a bit different angle drivers – forces – viability - paths

Slide 91

Slide 91 text

But first a (tiny) bit of theory …

Slide 92

Slide 92 text

Formal part of value creation Solution: machine Dynamic part of value creation Solution: man sluggishness/low dynamic high dynamic high dynamic The historical course of market dynamics and the recent rise of highly dynamic and complex markets The dominance of high dynamics and complexity is neither good nor bad. It‘s a historical fact. t 1970/80 today Age of crafts manu- facturing Age of tayloristic industry Age of global markets 1850/1900 Spacious markets, little competition Local markets, high customi- zation Outperformers exercise market pressure over conventional companies We call the graph shown here the “Taylor Bathtub”. The “bathtub” curve Source: BetaCodex Network Associates, “Organize for complexity”, BetaCodex Network White Paper 12 & 13

Slide 93

Slide 93 text

Key drivers Pre-industrial era • No clear driver • “Whatever pays the bill” Industrial era • Cost-efficiency • Scalability • Repeatability • Stability • Efficiency & scale Post-industrial era • Cycle times • Adaptability • Flexibility • Resilience • Effectiveness & speed

Slide 94

Slide 94 text

This model can also be applied to IT

Slide 95

Slide 95 text

Properties Pre-industrial Industrial Post-industrial Market Emerging market Wide & slow market Narrow & fast market Strategic goal None (“Get the bill paid!”) Cost-efficiency Short cycle times System theory Obvious Complicated Complex Work model Ad hoc Processes Collaboration Organization model Peer to peer Hierarchies Autonomous teams Practices “Whatever goes” ITIL, V-Modell, SAFe DevOps, Kanban, XP, Agile Individuals Experts (Cube-shaped) Specialists (I-shaped) Generalists (T-Shaped) Automation Low Medium-high Very high Manual work Ad hoc Defined and controlled Avoided Programming model Do it yourself Standard Software, Enterprise Frameworks Cloud-native, Serverless, Managed Services

Slide 96

Slide 96 text

Another, related point of view

Slide 97

Slide 97 text

Taking this “magic triangle” ...

Slide 98

Slide 98 text

Good Fast Cheap Optimizing for quality and cycle times will result in higher costs Optimizing for quality and costs will result in long cycle times Optimizing for cycle times and costs will result in reduced quality

Slide 99

Slide 99 text

... will lead you to this model

Slide 100

Slide 100 text

Pre-industrial No clear driver Good Fast Cheap Industrial Deliver large batches at minimized costs towards slow contexts Post-industrial Quickly adapt to ever-changing needs of dynamic, fast-moving contexts Startup Test hypotheses and pivot as fast as possible to discover a product-context fit

Slide 101

Slide 101 text

Now we are ready to look at the universes from a bit different angle drivers – forces – viability - paths

Slide 102

Slide 102 text

Ad hoc • Pre-Industrial • Suitable for companies with little IT demand • Truck factor typically < 3 • Does not scale • Usually develops into process-driven IT

Slide 103

Slide 103 text

Process driven • Industrial • Predominant in larger companies • Self-reinforcing system • Strong attractor – requires energy not to drift into • Requires huge amount of energy to escape from • Usually terminal state • Sometimes disguised as “Agile” (à scaled “Agile”) or alike

Slide 104

Slide 104 text

Outsourcing • Industrial • Very fashionable IT strategy 10 - 20 years ago • Valid option for non-differentiating parts of IT • Okay, if lead time for changes are irrelevant • Yet, often a result of a general misperception of IT by decision makers who do not really understand IT • As a general IT strategy, usually not sustainable

Slide 105

Slide 105 text

Actual Agile • Post-industrial • Addresses needs of highly dynamic environments • Rather a developer ideal than IT reality • Needs protected environment to survive • E.g., strong leader shielding the team from outer forces • Usually assimilated by process-driven IT (cargo cult “Agile”)

Slide 106

Slide 106 text

Cargo cult “Agile” • Industrial • Result of corporate IT assimilating actual Agile • Implementing rituals without understanding ideas • Often just "group therapy for frustrated developers" • Overall usually no real improvements realized • Will eventually drift into process-driven IT or scaled "Agile"

Slide 107

Slide 107 text

Scaled “Agile” • Industrial • Result of scaling cargo cult “Agile” to corporate level • Can help larger corporates to add a tiny dose of flexibility into their overly rigid processes (there is value in doing this) • Not agile, but process-driven IT in a disguise • Usually terminal state • Sometimes drifts back to pure process-driven IT

Slide 108

Slide 108 text

Startup 1.0 • Startup • Vertical IT integration depth needs to be considered • Will be superseded by startup 3.0 as future default • Still, will remain as a niche • With growth, usually develops • either into process-driven IT • or into DevOps 1.0

Slide 109

Slide 109 text

Startup 2.0 (NIH infested) • Startup • Very deep vertical IT integration depth • Okay for unicorns with unique problems • Not sustainable for everyone else • Either evolution to Startup 3.0 or extinction over time • With growth, usually develops • either into process-driven IT • or into DevOps 1.0/2.0

Slide 110

Slide 110 text

Startup 3.0 (Use over make) • Startup • Most up-to-date implementation of Startup • Enables extremely short cycle times at very low cost • With growth, usually develops • either into process-driven IT • or into DevOps 3.0

Slide 111

Slide 111 text

DevOps 1.0 • Post-industrial • Effective, but not very efficient • Lots of duplicate work in different teams • Usually evolves into DevOps 2.0/3.0 over time

Slide 112

Slide 112 text

DevOps 2.0 (w/ platform services) • Post-industrial • Brings a healthy dose of efficiency back into DevOps • Must not be confused with Ops 0.5 • Platform services as marketplace, not as prescriptions • Sustainable model • Still, often energy required to fend off industrial forces • Drift towards DevOps 3.0 should be considered

Slide 113

Slide 113 text

DevOps 3.0 (cloud native all-in) • Post-industrial • Most up-to-date implementation of DevOps • Enables very short cycle times without compromising quality • Sustainable model • Still, often energy required to fend off industrial forces

Slide 114

Slide 114 text

CrapOps / NoOps • Industrial • Cargo cult – implementing rituals only • Often augmented with lots of new technology • Overstrained teams as typical effect • Less speed, more errors • Not sustainable • Usually drifts back to process-driven IT or scaled "Agile"

Slide 115

Slide 115 text

IoT (Internet of Things) • Often pre-industrial or startup • Complementing universe • Currently brings more volatility to the connected universe • Will become more stable with settling of IoT landscape • Will cease to exist as a distinct universe over time • Instead will become integral part of other universes

Slide 116

Slide 116 text

Data Science (incl. AI/ML/DL) • Pre-industrial most of the time • Complementing universe • Brings more trial and error attitude to connected universes • Thus, often experiences repulsive force from industrial IT • Will cease to exist as a distinct universe over time • Instead will become integral part of other universes

Slide 117

Slide 117 text

Scientific computing • Not really connected to the commercial IT universes • Only accidental connections from and to this universe

Slide 118

Slide 118 text

Industrial computing • Industrial • Target IT market with growing relevance • Widespread conservative technology adoption patterns • Very affine to standard software solutions • Especially post-industrial IT will need to learn to respond to it • Sustainable model • Will at least partially merge with other universes over time

Slide 119

Slide 119 text

Summing the different perspective up …

Slide 120

Slide 120 text

Key Model Extra details Sustainability Model Extra details Sustainability Pre-industrial Industrial Post-industrial Startup Sustainable Partially sustainable Sustainable only in niches Not sustainable Usually drifts into different universe over time Terminal state Not viable in the long run Will merge with other universe Unstable (will drift forth and back) Requires protection Mostly unknown

Slide 121

Slide 121 text

Slide 122

Slide 122 text

Any insider tips for moving between universes?

Slide 123

Slide 123 text

Moving away from process driven • Extremely hard to overcome the inertia • It’s not the individuals, it’s systemic reinforcement • Easier to change the job than to change the company • Sometimes you can establish a protected niche • Still requires continuous efforts to protect your niche from the company’s immune system that will fight it all the time • Usually only works with upper level management support • Focus on culture, not processes or tools

Slide 124

Slide 124 text

Evolving from startup • Organizational growth leads to new challenges • Typical response: Process, hierarchy, bureaucracy • Inevitably leads to process-driven IT • Alternative response: Beta organization • Leads to DevOps • The second path is a lot harder (but worth the effort)

Slide 125

Slide 125 text

How do I become actually Agile? • Aim for DevOps (in its original meaning!) • DevOps provides better framing for IT than, e.g., Scrum • Creates relentless focus on value creation, delivery, feedback (from where value is realized) and continuous improvement • Working Agile environment emerges as a “side effect”

Slide 126

Slide 126 text

We could go on for a long time …

Slide 127

Slide 127 text

… yet, after all, always remember …

Slide 128

Slide 128 text

Slide 129

Slide 129 text

Reality is more complex, has more shades of grey …

Slide 130

Slide 130 text

Use the model to support your reasoning, but do not carry it too far … ;)

Slide 131

Slide 131 text

Any other useful tips for my journey?

Slide 132

Slide 132 text

Technology

Slide 133

Slide 133 text

Technology • Go cloud native – use managed services • DIY is fun, but takes too long and does not create value • Focus on the frontend • That is where the battle for the customer is won today • Mind the IoT train • Accept data as the currency of the 21st century • Do not forget security

Slide 134

Slide 134 text

Simplify! From requirements over architecture over code to infrastructure

Slide 135

Slide 135 text

Take it easy Go your own way and do not take things too seriously

Slide 136

Slide 136 text

One step at a time

Slide 137

Slide 137 text

Love it, change it or leave it

Slide 138

Slide 138 text

A better tomorrow How can we reduce our ecological footprint with IT?

Slide 139

Slide 139 text

Safe travels and have a great journey!

Slide 140

Slide 140 text

Uwe Friedrichsen CTO @ codecentric https://twitter.com/ufried https://www.speakerdeck.com/ufried https://medium.com/@ufried