TDD your Puppet Modules

This short talk will briefly show you how you can test your puppet modules. The talk was given @kaufda office on the 22th of January, 2014. More infos under http://www.meetup.com/Puppet-User-Group-Berlin/events/145643562/

Transcript

1. TDD your
   Puppet
   Modules
   Testing is necessary
   Matthias Günther
   (@wikimatze)
   

   View Slide

2. Testing like a dummy
   

   View Slide

3. Tools
   puppet cucumber
   puppet-lint
   rspec-puppet
   

   View Slide

4. Cucumber
   Feature: Catalog policy
   In order to ensure basic correctness
   I want all catalogs to obey my policy
   Scenario Outline: Generic policy for all server roles
   Given a node with role ""
   When I compile its catalog
   Then compilation should succeed
   And puppet should ensure all packages are up-to-date
   Examples:
   | server_role |
   | dhcp_server |
   

   View Slide

5. puppet-lint
   # manifests/init.pp
   class git::init {
   include git:: package
   }
   

   View Slide

6. Running puppet-lint
   $ puppet-lint manifests/init.pp
   ERROR: two-space soft tabs not used on line 10
   WARNING: unquoted resource title on line 10
   

   View Slide

7. Modules
   

   View Slide

8. Puppet module
   structure
   |-- manifests
   | |-- init.pp
   | |-- package.pp
   | `-- params.pp
   |-- Rakefile
   `-- spec
   |-- classes
   |-- defines
   |-- fixtures
   | |-- manifests
   | | `-- site.pp
   | `-- modules
   | `-- git
   | `-- manifests -> ../../../../manifests
   |-- functions
   |-- hosts
   `-- spec_helper.rb
   

   View Slide

9. RSpec Test
   # spec/classes/init_spec.pp
   require 'spec_helper'
   describe "git::init" do
   it { should create_class('git::package') }
   end
   

   View Slide

10. Running the test
    $ rake spec
    git::init
    example at ./spec/classes/init_spec.rb:4 (FAILED - 1)
    Failures:
    1) git::init
    Failure/Error: it { should create_class('git::install')}
    Puppet::Error:
    Could not find class git::init
    # ./spec/classes/init_spec.rb:4:in `block
    Finished in 0.20564 seconds
    1 examples, 1 failure
    

    View Slide

11. Code to pass the test
    class git::init {
    include git::install
    }
    

    View Slide

12. Cool
    

    View Slide

13. Jenkins module:
    installation
    # === Authors
    #
    # * Matthias Günther
    #
    class jenkins::installation {
    exec{ 'update system before jenkins installation':
    command => 'apt-get update',
    require => Class[Jenkins::Setup],
    }
    exec{ 'install jenkins':
    command => 'sudo apt-get install jenkins -y',
    require => Exec['update system before jenkins installation'],
    }
    }
    

    View Slide

14. Jenkins module:
    installation spec
    require 'spec_helper'
    describe "jenkins::installation" do
    it { should contain_exec('update system before jenkins installation').with(
    'command' => 'apt-get update'
    ) }
    it do should contain_exec("install jenkins").with(
    'command' => 'sudo apt-get install jenkins -y',
    'require' => "Exec[update system before jenkins installation]"
    ) end
    end
    

    View Slide

15. Enable lint task
    require 'rspec/core/rake_task'
    require 'puppet-lint/tasks/puppet-lint'
    RSpec::Core::RakeTask.new(:spec) do |t|
    t.pattern = 'spec/*/*_spec.rb'
    end
    task :default => %w(spec lint)
    

    View Slide

16. Running lint
    $ rake
    apache2::init
    ...
    Finished in 1.45 seconds
    7 examples, 0 failures
    manifests/package.pp - WARNING: class not documented on line 1
    

    View Slide

17. Get rid of certain line
    warnings
    PuppetLint.configuration.send("disable_80chars")
    PuppetLint.configuration.send("disable_documentation")
    

    View Slide

18. Overview of all our
    modules
    ~/git/development(master ✔) ls
    ant composer dev_user hosts lessc myhammer_directories npm_packages php rabbitmq_myhammer_config shellscripts sysctl vcs_user_key
    apache credentials faxchecker htop libcurl3 mysql_client ntpdate php5_geoip remove_existing_repositories.sh ssh_keys timezone vim
    api-apache2 cronolog freetds_bin imagemagick mailsystem ngrep openjdk_jdk php_configuration repositories.txt stdlib tinyproxy
    apt crons_qa git jstestdriver mc nodejs openjdk_jre puppetfix rsync subversion update_repositories_cron_script.sh
    build.xml developmentresolve git_repositories_import.sh lcov memcached npm pear rabbitmq ruby symlinks varnish
    => manageable <=
    

    View Slide

19. Continous Integration
    

    View Slide

20. We use Jenkins …
    

    View Slide

21. … with script trigger
    

    View Slide

22. The content of the
    script
    #!/bin/bash -x
    REPOS=`ls -d /mnt/revo/jenkins/jobs/Puppet/workspace/*/`;
    for repo in $REPOS
    do
    cd $repo;
    echo $repo;
    remote=$(git ls-remote -h origin master | awk '{print $1}')
    local=$(git rev-parse HEAD)
    printf "Local : %s\nRemote: %s\n" $local $remote
    if [ $local == $remote ]; then
    echo "Commits match."
    else
    echo "Changes available => run test."
    exit 1;
    fi
    cd ..
    done
    

    View Slide

23. Summary
    your arms => rspec-
    puppet
    your eyes => lint
    your ears => jenkins
    

    View Slide

24. Refactor your specs
    require 'spec_helper'
    describe "jenkins::plugins" do
    let (:base_url) { 'http://updates.jenkins-ci.org/download/plugins/' }
    let (:destination) { '/home/vagrant' }
    it do should contain_exec('get active-directory').with(
    'command' => "sudo wget #{base_url}active-directory/1.33/active-directory.hpi -P #{destination}"
    ) end
    it do should contain_exec('get ansicolor').with(
    'command' => "sudo wget #{base_url}ansicolor/0.3.1/ansicolor.hpi -P #{destination}"
    ) end
    it do should contain_exec('get ant').with(
    'command' => "sudo wget #{base_url}ant/1.2/ant.hpi -P #{destination}"
    ) end
    it do should contain_exec('get claim').with(
    'command' => "sudo wget #{base_url}claim/2.2/claim.hpi -P #{destination}"
    ) end
    it do should contain_exec('get ci-game').with(
    'command' => "sudo wget #{base_url}ci-game/1.19/ci-game.hpi -P #{destination}"
    ) end
    it do should contain_exec('get cvs').with(
    'command' => "sudo wget #{base_url}cvs/2.9/cvs.hpi -P #{destination}"
    ) end
    it do should contain_exec('get analysis-core').with(
    'command' => "sudo wget #{base_url}analysis-core/1.49/analysis-core.hpi -P #{destination}"
    ) end
    it do should contain_exec('get beer').with(
    'command' => "sudo wget #{base_url}beer/1.2/beer.hpi -P #{destination}"
    ) end
    it do should contain_exec('get buildtriggerbadge').with(
    'command' => "sudo wget #{base_url}buildtriggerbadge/1.0/buildtriggerbadge.hpi -P #{destination}"
    ) end
    it do should contain_exec('get chucknorris').with(
    'command' => "sudo wget #{base_url}chucknorris/0.5/chucknorris.hpi -P #{destination}"
    ) end
    it do should contain_exec('get credentials').with(
    'command' => "sudo wget #{base_url}credentials/1.4/credential.hpi -P #{destination}"
    ) end
    it do should contain_exec('get dashboard view').with(
    'command' => "sudo wget #{base_url}dashboard-view/2.7/dashboard-view.hpi -P #{destination}"
    ) end
    it do should contain_exec('get dry').with(
    'command' => "sudo wget #{base_url}dry/2.34/dry.hpi -P #{destination}"
    ) end
    it do should contain_exec('get checkstyle').with(
    'command' => "sudo wget #{base_url}checkstyle/3.35/checkstyle.hpi -P #{destination}"
    ) end
    it do should contain_exec('get cloverphp').with(
    'command' => "sudo wget #{base_url}cloverphp/0.3.3/cloverphp.hpi -P #{destination}"
    ) end
    it do should contain_exec('get javadoc').with(
    'command' => "sudo wget #{base_url}javadoc/1.1/javadoc.hpi -P #{destination}"
    ) end
    it do should contain_exec('get plot').with(
    'command' => "sudo wget #{base_url}plot/1.5/plot.hpi -P #{destination}"
    ) end
    it do should contain_exec('get groovy-postbuild').with(
    'command' => "sudo wget #{base_url}groovy-postbuild/1.8/groovy-postbuild.hpi -P #{destination}"
    ) end
    it do should contain_exec('get htmlpublisher').with(
    'command' => "sudo wget #{base_url}htmlpublisher/1.2/htmlpublisher.hpi -P #{destination}"
    ) end
    it do should contain_exec('get gcm-notification').with(
    'command' => "sudo wget #{base_url}gcm-notification/1.0/gcm-notification.hpi -P #{destination}"
    ) end
    it do should contain_exec('get thinBackup').with(
    'command' => "sudo wget #{base_url}thinBackup/1.7.2/thinBackup.hpi -P #{destination}"
    ) end
    it do should contain_exec('get greenballs').with(
    'command' => "sudo wget #{base_url}greenballs/1.12/greenballs.hpi -P #{destination}"
    ) end
    it do should contain_exec('get trac').with(
    'command' => "sudo wget #{base_url}trac/1.13/trac.hpi -P #{destination}"
    ) end
    it do should contain_exec('get external-monitor-job').with(
    'command' => "sudo wget #{base_url}external-monitor-job/1.1/external-monitor-job.hpi -P #{destination}"
    ) end
    it do should contain_exec('get git').with(
    'command' => "sudo wget #{base_url}git/1.4.0/git.hpi -P #{destination}"
    ) end
    it do should contain_exec('get xfpanel').with(
    'command' => "sudo wget #{base_url}xfpanel/1.2.2/xfpanel.hpi -P #{destination}"
    ) end
    it do should contain_exec('get git-parameter').with(
    'command' => "sudo wget #{base_url}git-parameter/0.2/git-parameter.hpi -P #{destination}"
    ) end
    it do should contain_exec('get git-client').with(
    'command' => "sudo wget #{base_url}git-client/1.0.7/git-client.hpi -P #{destination}"
    ) end
    it do should contain_exec('get dry').with(
    'command' => "sudo wget #{base_url}dry/2.34/dry.hpi -P #{destination}"
    ) end
    it do should contain_exec('get jdepend').with(
    'command' => "sudo wget #{base_url}jdepend/1.2.3/jdepend.hpi -P #{destination}"
    ) end
    it do should contain_exec('get pmd').with(
    'command' => "sudo wget #{base_url}pmd/3.34/pmd.hpi -P #{destination}"
    ) end
    it do should contain_exec('get global-build-stats').with(
    'command' => "sudo wget #{base_url}global-build-stats/1.3/global-build-stats.hpi -P #{destination}"
    ) end
    it do should contain_exec('get instant-messaging').with(
    'command' => "sudo wget #{base_url}instant-messaging/1.25/instant-messaging.hpi -P #{destination}"
    ) end
    it do should contain_exec('get dry').with(
    'command' => "sudo wget #{base_url}dry/2.34/dry.hpi -P #{destination}"
    ) end
    it do should contain_exec('get jquery').with(
    'command' => "sudo wget #{base_url}jquery/1.7.2-1/jquery.hpi -P #{destination}"
    ) end
    it do should contain_exec('get mailer').with(
    'command' => "sudo wget #{base_url}mailer/1.5/mailer.hpi -P #{destination}"
    ) end
    it do should contain_exec('get sonar').with(
    'command' => "sudo wget #{base_url}sonar/2.1/sonar.hpi -P #{destination}"
    ) end
    it do should contain_exec('get sonar').with(
    'command' => "sudo wget #{base_url}sonar/2.1/sonar.hpi -P #{destination}"
    ) end
    it do should contain_exec('get subversion').with(
    'command' => "sudo wget #{base_url}subversion/1.50/subversion.hpi -P #{destination}"
    ) end
    it do should contain_exec('get translation').with(
    'command' => "sudo wget #{base_url}translation/1.10/translation.hpi -P #{destination}"
    ) end
    it do should contain_exec('get jenkinswalldisplay').with(
    'command' => "sudo wget #{base_url}jenkinswalldisplay/0.6.17/jenkinswalldisplay.hpi -P #{destination}"
    ) end
    it do should contain_exec('get jira').with(
    'command' => "sudo wget #{base_url}jira/1.36/jira.hpi -P #{destination}"
    ) end
    it do should contain_exec('get ldap').with(
    'command' => "sudo wget #{base_url}ldap/1.5/ldap.hpi -P #{destination}"
    ) end
    it do should contain_exec('get maven-plugin').with(
    'command' => "sudo wget #{base_url}maven-plugin/1.519/maven-plugin.hpi -P #{destination}"
    ) end
    it do should contain_exec('get pam-auth').with(
    'command' => "sudo wget #{base_url}pam-auth/1.1/pam-auth.hpi -P #{destination}"
    ) end
    it do should contain_exec('get plot').with(
    'command' => "sudo wget #{base_url}plot/1.5/plot.hpi -P #{destination}"
    ) end
    it do should contain_exec('get ssh-credentials').with(
    'command' => "sudo wget #{base_url}ssh-credentials/0.4/ssh-credentials.hpi -P #{destination}"
    ) end
    it do should contain_exec('get ssh-slaves').with(
    'command' => "sudo wget #{base_url}ssh-slaves/0.27/ssh-slaves.hpi -P #{destination}"
    ) end
    it do should contain_exec('get tasks').with(
    'command' => "sudo wget #{base_url}tasks/4.36/tasks.hpi -P #{destination}"
    ) end
    it do should contain_exec('get test-stability').with(
    'command' => "sudo wget #{base_url}test-stability/1.0/test-stability.hpi -P #{destination}"
    ) end
    it do should contain_exec('get violations').with(
    'command' => "sudo wget #{base_url}violations/0.7.11/violations.hpi -P #{destination}"
    ) end
    it do should contain_exec('get xunit').with(
    'command' => "sudo wget #{base_url}xunit/1.60/xunit.hpi -P #{destination}"
    ) end
    it do should contain_exec('move plugins into /var/lib/jenkins/plugins').with(
    'command' => 'sudo mv /home/vagrant/*.hpi /var/lib/jenkins/plugins/'
    ) end
    it do should contain_exec('restart jenkins').with(
    'command' => 'service jenkins restart',
    'require' => "Exec[move plugins into /var/lib/jenkins/plugins]"
    ) end
    end
    

    View Slide

25. Pic Credits I
    Testing like a dummy by
    lincolnblues: http://
    www.flickr.com/photos/
    lincolnblues/1413187481
    Modules by Mike Mozart:
    http://www.flickr.com/photos/
    jeepersmedia/11884693514
    

    View Slide

26. Pic Credits II
    Cool by Wally Gobetz:
    http://www.flickr.com/photos/
    wallyg/8779127308
    Continous Integration by
    susan: http://www.flickr.com/
    photos/
    certified_su/229016531/
    

    View Slide

27. Pic Credits III
    We use Jenkins by
    Matthias Günther: http://
    www.flickr.com/photos/
    wikimatze/12072648986
    Script Trigger by
    Matthias Günther: http://
    www.flickr.com/photos/
    wikimatze/12072648136
    

    View Slide

28. About
    def talks
    "wikimatze.de/talks"
    end
    def twitter
    @wikimatze
    end
    def usergroup
    "vimberlin.de"
    end
    def book
    "padrinobook.com"
    end
    def github
    "matthias-guenther"
    end
    

    View Slide