Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Continuously testing infrastructure

Continuously testing infrastructure

Talk from #puppetconf 2014 all about moving towards infrastructure as code and why policy driven development is a cool idea.

Gareth Rushgrove

September 24, 2014
Tweet

More Decks by Gareth Rushgrove

Other Decks in Programming

Transcript

  1. Continuously Testing
    Infrastructure
    Puppet Conf, San Francisco, 2014
    Gareth Rushgrove
    Beyond Module Testing

    View full-size slide

  2. Gareth Rushgrove

    View full-size slide

  3. Gareth Rushgrove

    View full-size slide

  4. Gareth Rushgrove

    View full-size slide

  5. Not talking about

    View full-size slide

  6. Finished software
    Gareth Rushgrove

    View full-size slide

  7. Testing individual modules
    Gareth Rushgrove

    View full-size slide

  8. puppet-lint, puppet-syntax,
    rspec-puppet, beaker
    Gareth Rushgrove

    View full-size slide

  9. Gareth Rushgrove

    View full-size slide

  10. Am talking about

    View full-size slide

  11. Experiments
    Gareth Rushgrove

    View full-size slide

  12. Testing images and
    containers
    Gareth Rushgrove

    View full-size slide

  13. Test driving infrastructure
    as a service
    Gareth Rushgrove

    View full-size slide

  14. Testing with PuppetDB
    Gareth Rushgrove

    View full-size slide

  15. Testing
    images and
    containers
    1

    View full-size slide

  16. Gareth Rushgrove

    View full-size slide

  17. Packer builds images
    based on a JSON
    template
    Gareth Rushgrove

    View full-size slide

  18. Gareth Rushgrove

    View full-size slide

  19. It has some Puppet
    integration too
    Gareth Rushgrove

    View full-size slide

  20. Gareth Rushgrove

    View full-size slide

  21. But how do we know the
    image works?
    Gareth Rushgrove

    View full-size slide

  22. Lets add some tests!
    Gareth Rushgrove

    View full-size slide

  23. Gareth Rushgrove

    View full-size slide

  24. shaunduncan/packer-provisioner-host-command
    Gareth Rushgrove

    View full-size slide

  25. serverspec.org
    Gareth Rushgrove

    View full-size slide

  26. Gareth Rushgrove

    View full-size slide

  27. Gareth Rushgrove

    View full-size slide

  28. Gareth Rushgrove

    View full-size slide

  29. Serverspec also supports
    port, file, ppa, selinux,
    user, group, lxc, iptables,
    cron and more
    Gareth Rushgrove

    View full-size slide

  30. Only publish the image if
    the tests pass
    Gareth Rushgrove

    View full-size slide

  31. Run tests automatically
    with a continuous
    integration system
    Gareth Rushgrove

    View full-size slide

  32. Gareth Rushgrove

    View full-size slide

  33. Gareth Rushgrove

    View full-size slide

  34. garethr/packer-serverspec-example
    Gareth Rushgrove

    View full-size slide

  35. Gareth Rushgrove

    View full-size slide

  36. Same approach works
    with containers too
    Gareth Rushgrove

    View full-size slide

  37. Gareth Rushgrove

    View full-size slide

  38. garethr/docker-spec-example
    Gareth Rushgrove

    View full-size slide

  39. Test drive
    your IaaS
    2

    View full-size slide

  40. Test driven development
    Gareth Rushgrove

    View full-size slide

  41. First the developer writes
    an automated test case
    that defines a desired
    improvement or new
    function
    Gareth Rushgrove

    View full-size slide

  42. Then produces the
    minimum amount of code
    to pass that test
    Gareth Rushgrove

    View full-size slide

  43. And finally refactors the
    new code
    Gareth Rushgrove

    View full-size slide

  44. Gareth Rushgrove
    First the developer writes
    an automated test case
    that defines a desired
    improvement or new
    function

    View full-size slide

  45. Your infrastructure should!
    have an API
    Gareth Rushgrove

    View full-size slide

  46. What if we write
    assertions against!
    that API?
    Gareth Rushgrove

    View full-size slide

  47. Aside: Clojure
    2.1

    View full-size slide

  48. Gareth Rushgrove

    View full-size slide

  49. Great for building DSLs
    Gareth Rushgrove

    View full-size slide

  50. Don’t worry, you could
    write the examples in any
    language
    Gareth Rushgrove

    View full-size slide

  51. Policy driven development
    Gareth Rushgrove

    View full-size slide

  52. I don’t want to launch too
    many nodes, they’re
    expensive
    Gareth Rushgrove
    Policy

    View full-size slide

  53. Gareth Rushgrove

    View full-size slide

  54. I don’t want any stopped
    nodes, they are costing
    me money
    Gareth Rushgrove
    Policy

    View full-size slide

  55. Gareth Rushgrove

    View full-size slide

  56. Large nodes are really
    expensive, so limit their
    usage
    Gareth Rushgrove
    Policy

    View full-size slide

  57. Gareth Rushgrove

    View full-size slide

  58. We should be backing up
    every node
    Gareth Rushgrove
    Policy

    View full-size slide

  59. Gareth Rushgrove

    View full-size slide

  60. I only want nodes in
    London and !
    San Francisco
    Gareth Rushgrove
    Policy

    View full-size slide

  61. Gareth Rushgrove

    View full-size slide

  62. All our nodes should be
    named environment-name
    Gareth Rushgrove
    Policy

    View full-size slide

  63. Gareth Rushgrove

    View full-size slide

  64. garethr/digitalocean-expect
    Gareth Rushgrove

    View full-size slide

  65. Gareth Rushgrove

    View full-size slide

  66. Now we have the tests,
    we can provision some
    infrastructure
    Gareth Rushgrove

    View full-size slide

  67. Aside: Provisioning
    with Puppet
    2.2

    View full-size slide

  68. Gareth Rushgrove

    View full-size slide

  69. Gareth Rushgrove

    View full-size slide

  70. puppetlabs/gce_compute
    Gareth Rushgrove

    View full-size slide

  71. Gareth Rushgrove

    View full-size slide

  72. Gareth Rushgrove

    View full-size slide

  73. garethr/digitalocean
    Gareth Rushgrove

    View full-size slide

  74. Gareth Rushgrove

    View full-size slide

  75. bobtfish/aws_api
    Gareth Rushgrove

    View full-size slide

  76. Testing with
    PuppetDB
    3

    View full-size slide

  77. Aside: PuppetDB
    3.1

    View full-size slide

  78. puppetlabs/puppetdb
    Gareth Rushgrove

    View full-size slide

  79. PuppetDB can store a lot
    of data about your
    infrastructure
    Gareth Rushgrove

    View full-size slide

  80. The most recent facts
    from every node
    Gareth Rushgrove

    View full-size slide

  81. The most recent catalog
    for every node
    Gareth Rushgrove

    View full-size slide

  82. A wide range of metrics
    Gareth Rushgrove

    View full-size slide

  83. Gareth Rushgrove

    View full-size slide

  84. I want to run the same
    operating system on all
    hosts
    Gareth Rushgrove
    Policy

    View full-size slide

  85. Gareth Rushgrove

    View full-size slide

  86. Security enforcing
    packages should be
    installed everywhere
    Gareth Rushgrove
    Policy

    View full-size slide

  87. Gareth Rushgrove

    View full-size slide

  88. I want to limit how many
    puppet resources I’m
    using
    Gareth Rushgrove
    Policy

    View full-size slide

  89. Gareth Rushgrove

    View full-size slide

  90. We should avoid heavy I/
    O load on the database by
    maintaining a high catalog
    duplication rate
    Gareth Rushgrove
    Policy

    View full-size slide

  91. Gareth Rushgrove

    View full-size slide

  92. garethr/puppetdb-expect
    Gareth Rushgrove

    View full-size slide

  93. Testing based on
    PuppetDB
    3.2

    View full-size slide

  94. PuppetDB is a great
    source of context for tests
    Gareth Rushgrove

    View full-size slide

  95. Generate serverspec tests
    from PuppetDB data
    Gareth Rushgrove

    View full-size slide

  96. Automatically detect
    hosts, and generate
    commands
    Gareth Rushgrove

    View full-size slide

  97. Gareth Rushgrove

    View full-size slide

  98. Match puppet resources to
    serverspec resources
    Gareth Rushgrove

    View full-size slide

  99. Gareth Rushgrove

    View full-size slide

  100. For instance on a Puppet
    Enterprise master
    Gareth Rushgrove

    View full-size slide

  101. Gareth Rushgrove

    View full-size slide

  102. Run serverspec tests on
    all puppet managed hosts
    Gareth Rushgrove

    View full-size slide

  103. Gareth Rushgrove

    View full-size slide

  104. garethr/serverspec-puppetdb
    Gareth Rushgrove

    View full-size slide

  105. Is this monitoring?
    Gareth Rushgrove

    View full-size slide

  106. We’re still moving towards
    infrastructure as code
    Gareth Rushgrove

    View full-size slide

  107. Infrastructure as code
    rather than infrastructure
    from code
    Gareth Rushgrove

    View full-size slide

  108. Taking about policy as
    code might help
    communicate intent
    Gareth Rushgrove

    View full-size slide

  109. Questions?
    And thanks for listening

    View full-size slide