Continuously testing infrastructure

Talk from #puppetconf 2014 all about moving towards infrastructure as code and why policy driven development is a cool idea.

Transcript

1. Continuously Testing
   Infrastructure
   Puppet Conf, San Francisco, 2014
   Gareth Rushgrove
   Beyond Module Testing
   

   View Slide

2. @garethr
   

   View Slide

3. Gareth Rushgrove
   

   View Slide

4. Gareth Rushgrove
   

   View Slide

5. Gareth Rushgrove
   

   View Slide

6. Not talking about
   

   View Slide

7. Finished software
   Gareth Rushgrove
   

   View Slide

8. Testing individual modules
   Gareth Rushgrove
   

   View Slide

9. puppet-lint, puppet-syntax,
   rspec-puppet, beaker
   Gareth Rushgrove
   

   View Slide

10. Gareth Rushgrove
    

    View Slide

11. Am talking about
    

    View Slide

12. Experiments
    Gareth Rushgrove
    

    View Slide

13. Testing images and
    containers
    Gareth Rushgrove
    

    View Slide

14. Test driving infrastructure
    as a service
    Gareth Rushgrove
    

    View Slide

15. Testing with PuppetDB
    Gareth Rushgrove
    

    View Slide

16. Testing
    images and
    containers
    1
    

    View Slide

17. Gareth Rushgrove
    

    View Slide

18. Packer builds images
    based on a JSON
    template
    Gareth Rushgrove
    

    View Slide

19. Gareth Rushgrove
    

    View Slide

20. It has some Puppet
    integration too
    Gareth Rushgrove
    

    View Slide

21. Gareth Rushgrove
    

    View Slide

22. But how do we know the
    image works?
    Gareth Rushgrove
    

    View Slide

23. Lets add some tests!
    Gareth Rushgrove
    

    View Slide

24. Gareth Rushgrove
    

    View Slide

25. shaunduncan/packer-provisioner-host-command
    Gareth Rushgrove
    

    View Slide

26. serverspec.org
    Gareth Rushgrove
    

    View Slide

27. Gareth Rushgrove
    

    View Slide

28. Gareth Rushgrove
    

    View Slide

29. Gareth Rushgrove
    

    View Slide

30. Serverspec also supports
    port, file, ppa, selinux,
    user, group, lxc, iptables,
    cron and more
    Gareth Rushgrove
    

    View Slide

31. Only publish the image if
    the tests pass
    Gareth Rushgrove
    

    View Slide

32. Run tests automatically
    with a continuous
    integration system
    Gareth Rushgrove
    

    View Slide

33. Gareth Rushgrove
    

    View Slide

34. Gareth Rushgrove
    

    View Slide

35. garethr/packer-serverspec-example
    Gareth Rushgrove
    

    View Slide

36. Gareth Rushgrove
    

    View Slide

37. Same approach works
    with containers too
    Gareth Rushgrove
    

    View Slide

38. Gareth Rushgrove
    

    View Slide

39. garethr/docker-spec-example
    Gareth Rushgrove
    

    View Slide

40. Test drive
    your IaaS
    2
    

    View Slide

41. Test driven development
    Gareth Rushgrove
    

    View Slide

42. First the developer writes
    an automated test case
    that defines a desired
    improvement or new
    function
    Gareth Rushgrove
    

    View Slide

43. Then produces the
    minimum amount of code
    to pass that test
    Gareth Rushgrove
    

    View Slide

44. And finally refactors the
    new code
    Gareth Rushgrove
    

    View Slide

45. Gareth Rushgrove
    First the developer writes
    an automated test case
    that defines a desired
    improvement or new
    function
    

    View Slide

46. Your infrastructure should!
    have an API
    Gareth Rushgrove
    

    View Slide

47. What if we write
    assertions against!
    that API?
    Gareth Rushgrove
    

    View Slide

48. Aside: Clojure
    2.1
    

    View Slide

49. Gareth Rushgrove
    

    View Slide

50. Great for building DSLs
    Gareth Rushgrove
    

    View Slide

51. Don’t worry, you could
    write the examples in any
    language
    Gareth Rushgrove
    

    View Slide

52. Policy driven development
    Gareth Rushgrove
    

    View Slide

53. I don’t want to launch too
    many nodes, they’re
    expensive
    Gareth Rushgrove
    Policy
    

    View Slide

54. Gareth Rushgrove
    

    View Slide

55. I don’t want any stopped
    nodes, they are costing
    me money
    Gareth Rushgrove
    Policy
    

    View Slide

56. Gareth Rushgrove
    

    View Slide

57. Large nodes are really
    expensive, so limit their
    usage
    Gareth Rushgrove
    Policy
    

    View Slide

58. Gareth Rushgrove
    

    View Slide

59. We should be backing up
    every node
    Gareth Rushgrove
    Policy
    

    View Slide

60. Gareth Rushgrove
    

    View Slide

61. I only want nodes in
    London and !
    San Francisco
    Gareth Rushgrove
    Policy
    

    View Slide

62. Gareth Rushgrove
    

    View Slide

63. All our nodes should be
    named environment-name
    Gareth Rushgrove
    Policy
    

    View Slide

64. Gareth Rushgrove
    

    View Slide

65. garethr/digitalocean-expect
    Gareth Rushgrove
    

    View Slide

66. Gareth Rushgrove
    

    View Slide

67. Now we have the tests,
    we can provision some
    infrastructure
    Gareth Rushgrove
    

    View Slide

68. Aside: Provisioning
    with Puppet
    2.2
    

    View Slide

69. Gareth Rushgrove
    

    View Slide

70. Gareth Rushgrove
    

    View Slide

71. puppetlabs/gce_compute
    Gareth Rushgrove
    

    View Slide

72. Gareth Rushgrove
    

    View Slide

73. Gareth Rushgrove
    

    View Slide

74. garethr/digitalocean
    Gareth Rushgrove
    

    View Slide

75. Gareth Rushgrove
    

    View Slide

76. bobtfish/aws_api
    Gareth Rushgrove
    

    View Slide

77. Testing with
    PuppetDB
    3
    

    View Slide

78. Aside: PuppetDB
    3.1
    

    View Slide

79. puppetlabs/puppetdb
    Gareth Rushgrove
    

    View Slide

80. PuppetDB can store a lot
    of data about your
    infrastructure
    Gareth Rushgrove
    

    View Slide

81. The most recent facts
    from every node
    Gareth Rushgrove
    

    View Slide

82. The most recent catalog
    for every node
    Gareth Rushgrove
    

    View Slide

83. A wide range of metrics
    Gareth Rushgrove
    

    View Slide

84. Gareth Rushgrove
    

    View Slide

85. I want to run the same
    operating system on all
    hosts
    Gareth Rushgrove
    Policy
    

    View Slide

86. Gareth Rushgrove
    

    View Slide

87. Security enforcing
    packages should be
    installed everywhere
    Gareth Rushgrove
    Policy
    

    View Slide

88. Gareth Rushgrove
    

    View Slide

89. I want to limit how many
    puppet resources I’m
    using
    Gareth Rushgrove
    Policy
    

    View Slide

90. Gareth Rushgrove
    

    View Slide

91. We should avoid heavy I/
    O load on the database by
    maintaining a high catalog
    duplication rate
    Gareth Rushgrove
    Policy
    

    View Slide

92. Gareth Rushgrove
    

    View Slide

93. garethr/puppetdb-expect
    Gareth Rushgrove
    

    View Slide

94. Testing based on
    PuppetDB
    3.2
    

    View Slide

95. PuppetDB is a great
    source of context for tests
    Gareth Rushgrove
    

    View Slide

96. Generate serverspec tests
    from PuppetDB data
    Gareth Rushgrove
    

    View Slide

97. Automatically detect
    hosts, and generate
    commands
    Gareth Rushgrove
    

    View Slide

98. Gareth Rushgrove
    

    View Slide

99. Match puppet resources to
    serverspec resources
    Gareth Rushgrove
    

    View Slide

100. Gareth Rushgrove
     

     View Slide

101. For instance on a Puppet
     Enterprise master
     Gareth Rushgrove
     

     View Slide

102. Gareth Rushgrove
     

     View Slide

103. Run serverspec tests on
     all puppet managed hosts
     Gareth Rushgrove
     

     View Slide

104. Gareth Rushgrove
     

     View Slide

105. garethr/serverspec-puppetdb
     Gareth Rushgrove
     

     View Slide

106. Conclusions
     

     View Slide

107. Is this monitoring?
     Gareth Rushgrove
     

     View Slide

108. We’re still moving towards
     infrastructure as code
     Gareth Rushgrove
     

     View Slide

109. Infrastructure as code
     rather than infrastructure
     from code
     Gareth Rushgrove
     

     View Slide

110. Taking about policy as
     code might help
     communicate intent
     Gareth Rushgrove
     

     View Slide

111. Questions?
     And thanks for listening
     

     View Slide