Halt and Don't Catch Fire - SREcon18 Europe

Every young systems team (regardless if it’s in a startup or not) is burdened with someone else’s decisions, good and bad.

Startups have the uniqueness where most of the people that contributed to their technical debt are still there. They are flexible, and, in many cases, still can create a solid foundation (both cultural and technological) for the future. On the other hand, they are fragile, change fast, and frequently adopt new technologies prematurely. New systems teams in medium-sized companies face even harder challenges. They must keep everything running, take care of an unknown complex infrastructure and development team while dealing with the lack of original information.

We will discuss how we can slow down the generation of technical debt in new teams, put it in perspective using practical examples and real-life stories, and how small iterations, and changes in culture can lead to more sustainable systems and teams.

Transcript

1. Halt and Don’t Catch Fire effie mouzeli SREcon18 Düsseldorf
2. Startups Small organisations Newly formed “sysadms”* teams Solo systems engineers Who Cares? * ops/systems/admin/cluster/SRE/<insert more> 1
3. Technical Debt • Lack processes and experience • Early adoption of new tech • Re-inventing the wheel, and cargo cult • Frequent change of requirements • Overestimation and Time 2
4. How we got here? • Short term planning • Duct tape and bubblegum • Early adopters :) • No documentation • No processes • No boundaries • No backups Startups 3
5. How we got here? • Duct tape and bubblegum :) • Lack of original information • *Some* documentation • *Some* backups • Waiting for a(nother) hero • Communication issues • “Don’t touch this” systems !Startups 4
6. Challenges of new teams • Who are you? • Who are they? • Unknown infrastructure • Finding information • Unaware of what’s coming 5
7. The 5 Stages of Technical Debt • Denial • Anger • Bargaining • Depression • Acceptance 6
8. Take a breath • Ask the right questions • Go through the backlog (if any) • Write down everything • Create a complete dashboard • Prioritise • DON’T SKIP weekly/biweekly meetings • Build relationships • Rinse and repeat 7
9. Slowing it down • Take care of repetitive tasks • Choose software wisely • Know your current limits • Keep track of what’s going on around you, and talk • Break bad habits • Be consistent and apply rules 8
10. How debt can kill a team • Burnout • Blame game • Tension and toxicity • Martyrs and messiahs 9
11. Creating a culture • Being a guard • Junior engineers • Adopt good practices • Make mistakes, it’s human • There’s no shame in asking • Following the Golden Rule* can go a long way * https://en.wikipedia.org/wiki/Golden_Rule 10
12. Takeaways • Technical debt in small environments sucks • Stop, ask the right questions, and breathe • It will never be eliminated • Identify the debt that kills you and the one that burdens you • Our systems will never be perfect, just manageable 11
13. TL;DR How we got here? Go through the stages Ask the right questions Document the problems Try not to kill your team Create a sustainable culture 12
14. Thank you! emouzeli@runbox.no jijiki @freenode
15. images • https://www.deviantart.com/sudeepdash/art/IT-Crowd-200295794 • http://fuckyeah-pixels.tumblr.com/post/31749217421 • https://mymodernmet.com/pop-culture-pixel-people/ • http://fuckyeah-pixels.tumblr.com/post/52333978134 • http://fuckyeah-pixels.tumblr.com/post/47682887619 • http://pixelartmaker.com/art/adb1880541bb6ea 10