Slides from my talk at Agile Cambridge 2016
I never set out to be a sysadmin but after 15 years in technology management it’s the hat I put on most often. And one I encourage people to wear on a daily basis.
Without a shared understanding of what it takes to run a software system in production, delivery falters and failure demand rises.
The way I’ve gone about fostering this shared understanding is to start by establishing a culture of trust. One manifestation of this is a permissive approach to root and admin permissions with constraints only added for what they enable.
I’ve also done away with staging. For me Staging had become a ‘Pot Roast’ - something you have just because you’ve always had one in the past. As an experiment at Honeycomb we did without and haven’t looked back. It makes our deployment pipeline simpler and quicker, discourages batching of features and saves us money.
I encourage a ‘Do It Yourself’ attitude amongst the people I work with. If the bottleneck or impediment is somewhere in the stack that you don’t feel at home in but is where the need presents itself, a willingness to learn skills outside of your domain and generalise can get things moving far quicker than handing off to a specialist.
There are limits and risks to this. A shallow knowledge of something can lead to mistakes. A while back I naively created an AWS VPC with a public CIDR block. This mistake was not discovered for quite a while and then took quite a bit of effort to put right.
There is a tension between the speed and agility of generalisation and the robustness and focus of specialisation. When I do introduce specialisation I look for the people involved, those providing the specialist capability and those with the need it addresses to collaborate to establish the interface between them.
To ensure interfaces such as this work, that resources are balanced against demand, you need instrumentation. I urge developers to make Instrumentation and Monitoring first class citizens and to get them into the code from the get go.
All of these practices are there to encourage an engagement with the system as a whole wether you work as a generalist across a number of disciplines or as a specialist focused on specific concerns.
worldofchris
77web
maruto
ivargrimstad
euxn23
tnagao7
yujisoftware
qnighy
maaretp
mayuki
har1101
ysk8hori
rmw
holman
tanoku
aarron
bcantrill
wjessup
mongodb
lauravandoore
marktimemedia
denniskardys
morganepeng
productmarketing
![Thank You! [email protected] @worldofchris @honeycomb_tv](https://files.speakerdeck.com/presentations/43a81598c02643e9a619d60300029392/slide_38.jpg){kind=link}