untitled config game

2396958133b7324fc7afe505dfa02572?s=47 Ryn Daniels
February 03, 2020

untitled config game

Keynote presented at Config Management Camp 2020

2396958133b7324fc7afe505dfa02572?s=128

Ryn Daniels

February 03, 2020
Tweet

Transcript

  1. ConfigMgmtCamp 2020 Ryn Daniels they/them @rynchantress untitled config game

  2. It's a lovely morning in the data center, and you

    are a horrible goose devop. @rynchantress ConfigMgmtCamp
  3. •Professional computerer •Knows several things about Terraform •Very good at

    puns Me now... @rynchantress ConfigMgmtCamp
  4. • 200 beauBful and unique snowflake servers • 0 automaBon

    • 1 very sad new sysadmin Back in the day... @rynchantress ConfigMgmtCamp
  5. Computers were a mistake Computers are bad, actually @rynchantress ConfigMgmtCamp

  6. • ApplicaBon state • System state Let's talk about state

    @rynchantress ConfigMgmtCamp
  7. • ApplicaBon state • System state Let's talk about state

    @rynchantress ConfigMgmtCamp
  8. • What is the system doing? • What is the

    system supposed to be doing? • What versions of things exist in the system? • What is and is not installed? • What things changed and when and by whom? System State @rynchantress ConfigMgmtCamp
  9. Me watching all the other sysadmins with their cool configura<on

    management while I sit in the data center with a bash script @rynchantress ConfigMgmtCamp
  10. Bash scripts don't manage system state well. @rynchantress ConfigMgmtCamp

  11. @rynchantress ConfigMgmtCamp Me ge>ng ready to do a configura<on management

  12. @rynchantress ConfigMgmtCamp

  13. @rynchantress ConfigMgmtCamp

  14. known system state + known config change = new known

    system state @rynchantress ConfigMgmtCamp
  15. @rynchantress ConfigMgmtCamp

  16. • "Known" state wasn't actually known • Configuring system state

    didn't account for unknown applicaBon state • Undo in configuraBon state couldn't undo changes to applicaBon state Wat. @rynchantress ConfigMgmtCamp
  17. • ApplicaBon state • System state Let's talk about state

    @rynchantress ConfigMgmtCamp
  18. Application state @rynchantress ConfigMgmtCamp • What is the applicaBon doing?

    • What does the applicaBon care about? • What things changed and when and by whom? • It's always a caching problem.
  19. @rynchantress ConfigMgmtCamp

  20. 1.Don't forget about applicaBon state 2.Knowns aren't always known horrible

    lessons learned @rynchantress ConfigMgmtCamp
  21. @rynchantress ConfigMgmtCamp

  22. @rynchantress ConfigMgmtCamp Me with my several years of experience and

    smart colleagues and tested tooling, ge>ng ready to do a no-op
  23. @rynchantress ConfigMgmtCamp

  24. known config state + no-op = same known config state

    @rynchantress ConfigMgmtCamp
  25. @rynchantress ConfigMgmtCamp

  26. • System state interacted with configuraBon change to cause new

    system state • Unknown and inconsistent configuraBon state across infrastructure • Uncontrolled source state changes combined with inconsistent configuraBon state Wat. @rynchantress ConfigMgmtCamp
  27. • ApplicaBon state • System state • ConfiguraBon state •

    Source state Let's talk about state @rynchantress ConfigMgmtCamp
  28. Configuration State @rynchantress ConfigMgmtCamp • What did humans tell the

    config management to do? • What does the config management system know about the world? • How is the config management tool running?
  29. Source State @rynchantress ConfigMgmtCamp • Sources of packages • Sources

    of applicaBon code • Sources of config management code • Whatever else you can put in source control, have fun!
  30. @rynchantress ConfigMgmtCamp

  31. 1.Don't forget about applicaBon state 2.Knowns aren't always known 3.Monitor

    your configuraBon state 4.Don't leave your states unsupervised horrible lessons learned @rynchantress ConfigMgmtCamp
  32. @rynchantress ConfigMgmtCamp

  33. @rynchantress ConfigMgmtCamp Me with my several more years of experience

    and other smart colleagues and tested tooling, ge>ng ready to do a devops
  34. @rynchantress ConfigMgmtCamp

  35. known system state + known config change = new known

    system state @rynchantress ConfigMgmtCamp
  36. @rynchantress ConfigMgmtCamp

  37. • Wildly and unknowably different configuraBon states • Current configuraBon

    state not fully captured in source state • Wat even was in the terraform plan output again?! Wat. @rynchantress ConfigMgmtCamp
  38. • ApplicaBon state • System state • ConfiguraBon state •

    Source state • Terraform state Let's talk about state @rynchantress ConfigMgmtCamp
  39. • ApplicaBon state • System state • ConfiguraBon state •

    Source state • Terraform state Let's talk about state @rynchantress ConfigMgmtCamp
  40. @rynchantress ConfigMgmtCamp

  41. 1.Don't forget about applicaBon state 2.Knowns aren't always known 3.Monitor

    your configuraBon state 4.Don't leave your states unsupervised 5.Enforce source state completeness and config state consistency horrible lessons learned @rynchantress ConfigMgmtCamp
  42. @rynchantress ConfigMgmtCamp

  43. @rynchantress ConfigMgmtCamp

  44. • ApplicaBon state • System state • ConfiguraBon state •

    Source state • Terraform state • Human state Let's talk about state @rynchantress ConfigMgmtCamp
  45. Let's talk about human factors @rynchantress ConfigMgmtCamp

  46. @rynchantress ConfigMgmtCamp

  47. Computers were a mistake @rynchantress ConfigMgmtCamp fewer sources of state

  48. @rynchantress ConfigMgmtCamp Me with so many shiny new monitoring and

    observability tools, ssh-ing into a server like a professional
  49. Computers were a mistake @rynchantress ConfigMgmtCamp surface incorrect state

  50. Computers were a mistake @rynchantress ConfigMgmtCamp make things human-readable

  51. @rynchantress ConfigMgmtCamp

  52. @rynchantress ConfigMgmtCamp

  53. 1.Don't forget about applicaBon state 2.Knowns aren't always known 3.Monitor

    your configuraBon state 4.Don't leave your states unsupervised 5.Enforce source state completeness and config state consistency 6.Get rid of unreliable tools 7.Make sure your tooling is usable by humans horrible lessons learned @rynchantress ConfigMgmtCamp
  54. @rynchantress ConfigMgmtCamp THANK YOU! Special thanks to House House for

    creating Untitled Goose Game and Samuel Fine for creating the Untitled Goose Game Generator