Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Continuously testing infrastructure

Continuously testing infrastructure

Talk from #puppetconf 2014 all about moving towards infrastructure as code and why policy driven development is a cool idea.

Gareth Rushgrove

September 24, 2014
Tweet

More Decks by Gareth Rushgrove

Other Decks in Programming

Transcript

  1. Continuously Testing
    Infrastructure
    Puppet Conf, San Francisco, 2014
    Gareth Rushgrove
    Beyond Module Testing

    View Slide

  2. @garethr

    View Slide

  3. Gareth Rushgrove

    View Slide

  4. Gareth Rushgrove

    View Slide

  5. Gareth Rushgrove

    View Slide

  6. Not talking about

    View Slide

  7. Finished software
    Gareth Rushgrove

    View Slide

  8. Testing individual modules
    Gareth Rushgrove

    View Slide

  9. puppet-lint, puppet-syntax,
    rspec-puppet, beaker
    Gareth Rushgrove

    View Slide

  10. Gareth Rushgrove

    View Slide

  11. Am talking about

    View Slide

  12. Experiments
    Gareth Rushgrove

    View Slide

  13. Testing images and
    containers
    Gareth Rushgrove

    View Slide

  14. Test driving infrastructure
    as a service
    Gareth Rushgrove

    View Slide

  15. Testing with PuppetDB
    Gareth Rushgrove

    View Slide

  16. Testing
    images and
    containers
    1

    View Slide

  17. Gareth Rushgrove

    View Slide

  18. Packer builds images
    based on a JSON
    template
    Gareth Rushgrove

    View Slide

  19. Gareth Rushgrove

    View Slide

  20. It has some Puppet
    integration too
    Gareth Rushgrove

    View Slide

  21. Gareth Rushgrove

    View Slide

  22. But how do we know the
    image works?
    Gareth Rushgrove

    View Slide

  23. Lets add some tests!
    Gareth Rushgrove

    View Slide

  24. Gareth Rushgrove

    View Slide

  25. shaunduncan/packer-provisioner-host-command
    Gareth Rushgrove

    View Slide

  26. serverspec.org
    Gareth Rushgrove

    View Slide

  27. Gareth Rushgrove

    View Slide

  28. Gareth Rushgrove

    View Slide

  29. Gareth Rushgrove

    View Slide

  30. Serverspec also supports
    port, file, ppa, selinux,
    user, group, lxc, iptables,
    cron and more
    Gareth Rushgrove

    View Slide

  31. Only publish the image if
    the tests pass
    Gareth Rushgrove

    View Slide

  32. Run tests automatically
    with a continuous
    integration system
    Gareth Rushgrove

    View Slide

  33. Gareth Rushgrove

    View Slide

  34. Gareth Rushgrove

    View Slide

  35. garethr/packer-serverspec-example
    Gareth Rushgrove

    View Slide

  36. Gareth Rushgrove

    View Slide

  37. Same approach works
    with containers too
    Gareth Rushgrove

    View Slide

  38. Gareth Rushgrove

    View Slide

  39. garethr/docker-spec-example
    Gareth Rushgrove

    View Slide

  40. Test drive
    your IaaS
    2

    View Slide

  41. Test driven development
    Gareth Rushgrove

    View Slide

  42. First the developer writes
    an automated test case
    that defines a desired
    improvement or new
    function
    Gareth Rushgrove

    View Slide

  43. Then produces the
    minimum amount of code
    to pass that test
    Gareth Rushgrove

    View Slide

  44. And finally refactors the
    new code
    Gareth Rushgrove

    View Slide

  45. Gareth Rushgrove
    First the developer writes
    an automated test case
    that defines a desired
    improvement or new
    function

    View Slide

  46. Your infrastructure should!
    have an API
    Gareth Rushgrove

    View Slide

  47. What if we write
    assertions against!
    that API?
    Gareth Rushgrove

    View Slide

  48. Aside: Clojure
    2.1

    View Slide

  49. Gareth Rushgrove

    View Slide

  50. Great for building DSLs
    Gareth Rushgrove

    View Slide

  51. Don’t worry, you could
    write the examples in any
    language
    Gareth Rushgrove

    View Slide

  52. Policy driven development
    Gareth Rushgrove

    View Slide

  53. I don’t want to launch too
    many nodes, they’re
    expensive
    Gareth Rushgrove
    Policy

    View Slide

  54. Gareth Rushgrove

    View Slide

  55. I don’t want any stopped
    nodes, they are costing
    me money
    Gareth Rushgrove
    Policy

    View Slide

  56. Gareth Rushgrove

    View Slide

  57. Large nodes are really
    expensive, so limit their
    usage
    Gareth Rushgrove
    Policy

    View Slide

  58. Gareth Rushgrove

    View Slide

  59. We should be backing up
    every node
    Gareth Rushgrove
    Policy

    View Slide

  60. Gareth Rushgrove

    View Slide

  61. I only want nodes in
    London and !
    San Francisco
    Gareth Rushgrove
    Policy

    View Slide

  62. Gareth Rushgrove

    View Slide

  63. All our nodes should be
    named environment-name
    Gareth Rushgrove
    Policy

    View Slide

  64. Gareth Rushgrove

    View Slide

  65. garethr/digitalocean-expect
    Gareth Rushgrove

    View Slide

  66. Gareth Rushgrove

    View Slide

  67. Now we have the tests,
    we can provision some
    infrastructure
    Gareth Rushgrove

    View Slide

  68. Aside: Provisioning
    with Puppet
    2.2

    View Slide

  69. Gareth Rushgrove

    View Slide

  70. Gareth Rushgrove

    View Slide

  71. puppetlabs/gce_compute
    Gareth Rushgrove

    View Slide

  72. Gareth Rushgrove

    View Slide

  73. Gareth Rushgrove

    View Slide

  74. garethr/digitalocean
    Gareth Rushgrove

    View Slide

  75. Gareth Rushgrove

    View Slide

  76. bobtfish/aws_api
    Gareth Rushgrove

    View Slide

  77. Testing with
    PuppetDB
    3

    View Slide

  78. Aside: PuppetDB
    3.1

    View Slide

  79. puppetlabs/puppetdb
    Gareth Rushgrove

    View Slide

  80. PuppetDB can store a lot
    of data about your
    infrastructure
    Gareth Rushgrove

    View Slide

  81. The most recent facts
    from every node
    Gareth Rushgrove

    View Slide

  82. The most recent catalog
    for every node
    Gareth Rushgrove

    View Slide

  83. A wide range of metrics
    Gareth Rushgrove

    View Slide

  84. Gareth Rushgrove

    View Slide

  85. I want to run the same
    operating system on all
    hosts
    Gareth Rushgrove
    Policy

    View Slide

  86. Gareth Rushgrove

    View Slide

  87. Security enforcing
    packages should be
    installed everywhere
    Gareth Rushgrove
    Policy

    View Slide

  88. Gareth Rushgrove

    View Slide

  89. I want to limit how many
    puppet resources I’m
    using
    Gareth Rushgrove
    Policy

    View Slide

  90. Gareth Rushgrove

    View Slide

  91. We should avoid heavy I/
    O load on the database by
    maintaining a high catalog
    duplication rate
    Gareth Rushgrove
    Policy

    View Slide

  92. Gareth Rushgrove

    View Slide

  93. garethr/puppetdb-expect
    Gareth Rushgrove

    View Slide

  94. Testing based on
    PuppetDB
    3.2

    View Slide

  95. PuppetDB is a great
    source of context for tests
    Gareth Rushgrove

    View Slide

  96. Generate serverspec tests
    from PuppetDB data
    Gareth Rushgrove

    View Slide

  97. Automatically detect
    hosts, and generate
    commands
    Gareth Rushgrove

    View Slide

  98. Gareth Rushgrove

    View Slide

  99. Match puppet resources to
    serverspec resources
    Gareth Rushgrove

    View Slide

  100. Gareth Rushgrove

    View Slide

  101. For instance on a Puppet
    Enterprise master
    Gareth Rushgrove

    View Slide

  102. Gareth Rushgrove

    View Slide

  103. Run serverspec tests on
    all puppet managed hosts
    Gareth Rushgrove

    View Slide

  104. Gareth Rushgrove

    View Slide

  105. garethr/serverspec-puppetdb
    Gareth Rushgrove

    View Slide

  106. Conclusions

    View Slide

  107. Is this monitoring?
    Gareth Rushgrove

    View Slide

  108. We’re still moving towards
    infrastructure as code
    Gareth Rushgrove

    View Slide

  109. Infrastructure as code
    rather than infrastructure
    from code
    Gareth Rushgrove

    View Slide

  110. Taking about policy as
    code might help
    communicate intent
    Gareth Rushgrove

    View Slide

  111. Questions?
    And thanks for listening

    View Slide