Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The Great PagerDuty DevOps Heist of 2016

The Great PagerDuty DevOps Heist of 2016

This is a talk I gave at DevOpsDays Stockholm 2017
https://www.devopsdays.org/events/2017-stockholm/program/arup-chakrabarti/

Arup Chakrabarti

May 09, 2017
Tweet

More Decks by Arup Chakrabarti

Other Decks in Technology

Transcript

  1. @arupchak The Great PagerDuty DevOps Heist
 of 2016 Arup Chakrabarti

    Director of Engineering DEVOPSDAYS STOCKHOLM
  2. @arupchak Org Chart VP of Engineering Operations Servers Databases LB’s

    Reliability Backend services System reliability Product Customer facing Frontend services
  3. @arupchak “The Heist” • What we were selling to teams

    • Business Success depends on innovation • Innovation depends on rate of change • We will increase the rate of change by having engineers own more
  4. @arupchak Org Chart VP of Engineering Infrastructure Products SRE Platform

    Security Product Team Alpha Servers Databases Operations Product Team Beta Servers Databases Operations
  5. @arupchak More Problems • People working with new people •

    People working on things they were not hired for • Changed responsibilities for everyone
  6. @arupchak The Job Changed • For previously Dev focused people

    • They now owned the full vertical stack • Code It. Ship It. Own It. • The previously Ops focused people • They now had to empower others to do their job • Make the right thing the easy thing
  7. @arupchak What we built • Infrastructure Tools • Self Service

    Server Provisioning • Self Service Metrics and Telemetry • Self Service Deployments • Self Service Common Infrastructure Services • Documentation • Where we could not automate easily
  8. @arupchak The Hard Part of Leadership • Some changes are

    not for everyone • Some people who thrived in the old ways, will fail in the new ways • They are not trying to be jerks • Expect 10% attrition or managed exits
  9. @arupchak Anecdotes • Intern replaced our Kafka cluster • Engineers

    practicing MySQL recovery • Engineers building out test coverage metrics
  10. @arupchak “The Heist” • What we were selling to teams

    • Business Success depends on innovation • Innovation depends on rate of change • We will increase the rate of change by having engineers own more
  11. @arupchak “The Heist” • What did it cost the teams?

    • All Engineers go on-call • Code it. Ship it. Own it • People leaving