Upgrade to Pro — share decks privately, control downloads, hide ads and more …

untitled config game

Ryn Daniels
February 03, 2020

untitled config game

Keynote presented at Config Management Camp 2020

Ryn Daniels

February 03, 2020
Tweet

More Decks by Ryn Daniels

Other Decks in Technology

Transcript

  1. It's a lovely morning in the data center, and you

    are a horrible goose devop. @rynchantress ConfigMgmtCamp
  2. • 200 beauBful and unique snowflake servers • 0 automaBon

    • 1 very sad new sysadmin Back in the day... @rynchantress ConfigMgmtCamp
  3. • What is the system doing? • What is the

    system supposed to be doing? • What versions of things exist in the system? • What is and is not installed? • What things changed and when and by whom? System State @rynchantress ConfigMgmtCamp
  4. Me watching all the other sysadmins with their cool configura<on

    management while I sit in the data center with a bash script @rynchantress ConfigMgmtCamp
  5. known system state + known config change = new known

    system state @rynchantress ConfigMgmtCamp
  6. • "Known" state wasn't actually known • Configuring system state

    didn't account for unknown applicaBon state • Undo in configuraBon state couldn't undo changes to applicaBon state Wat. @rynchantress ConfigMgmtCamp
  7. Application state @rynchantress ConfigMgmtCamp • What is the applicaBon doing?

    • What does the applicaBon care about? • What things changed and when and by whom? • It's always a caching problem.
  8. @rynchantress ConfigMgmtCamp Me with my several years of experience and

    smart colleagues and tested tooling, ge>ng ready to do a no-op
  9. • System state interacted with configuraBon change to cause new

    system state • Unknown and inconsistent configuraBon state across infrastructure • Uncontrolled source state changes combined with inconsistent configuraBon state Wat. @rynchantress ConfigMgmtCamp
  10. • ApplicaBon state • System state • ConfiguraBon state •

    Source state Let's talk about state @rynchantress ConfigMgmtCamp
  11. Configuration State @rynchantress ConfigMgmtCamp • What did humans tell the

    config management to do? • What does the config management system know about the world? • How is the config management tool running?
  12. Source State @rynchantress ConfigMgmtCamp • Sources of packages • Sources

    of applicaBon code • Sources of config management code • Whatever else you can put in source control, have fun!
  13. 1.Don't forget about applicaBon state 2.Knowns aren't always known 3.Monitor

    your configuraBon state 4.Don't leave your states unsupervised horrible lessons learned @rynchantress ConfigMgmtCamp
  14. @rynchantress ConfigMgmtCamp Me with my several more years of experience

    and other smart colleagues and tested tooling, ge>ng ready to do a devops
  15. known system state + known config change = new known

    system state @rynchantress ConfigMgmtCamp
  16. • Wildly and unknowably different configuraBon states • Current configuraBon

    state not fully captured in source state • Wat even was in the terraform plan output again?! Wat. @rynchantress ConfigMgmtCamp
  17. • ApplicaBon state • System state • ConfiguraBon state •

    Source state • Terraform state Let's talk about state @rynchantress ConfigMgmtCamp
  18. • ApplicaBon state • System state • ConfiguraBon state •

    Source state • Terraform state Let's talk about state @rynchantress ConfigMgmtCamp
  19. 1.Don't forget about applicaBon state 2.Knowns aren't always known 3.Monitor

    your configuraBon state 4.Don't leave your states unsupervised 5.Enforce source state completeness and config state consistency horrible lessons learned @rynchantress ConfigMgmtCamp
  20. • ApplicaBon state • System state • ConfiguraBon state •

    Source state • Terraform state • Human state Let's talk about state @rynchantress ConfigMgmtCamp
  21. @rynchantress ConfigMgmtCamp Me with so many shiny new monitoring and

    observability tools, ssh-ing into a server like a professional
  22. 1.Don't forget about applicaBon state 2.Knowns aren't always known 3.Monitor

    your configuraBon state 4.Don't leave your states unsupervised 5.Enforce source state completeness and config state consistency 6.Get rid of unreliable tools 7.Make sure your tooling is usable by humans horrible lessons learned @rynchantress ConfigMgmtCamp
  23. @rynchantress ConfigMgmtCamp THANK YOU! Special thanks to House House for

    creating Untitled Goose Game and Samuel Fine for creating the Untitled Goose Game Generator