How do you Puppet ?

Infrastructure as Code @8thcolor

Transcript

1. How do you Puppet ? Don't knit your strings
2. Disclaim me I'm Stephan Mestach "previously" a Java guy 6 months ago, I knew nearly nothing about Puppet, Linux and Ruby. I work at 8th color, a Ruby shop, Puppet is written and extended with Ruby
3. At the beginning There was a shell script with some manual steps in between, manual fixes at the end. Reproducibility : 0 Maintainability : 0 Testability : 0 Reusability : 0
4. At the end cap go HOST='vps31124.ovh.net' USER='root' Reproducibility : 98% Maintainability : 98% Testability : 98% Reusability : 98% we will talk about the 2% later ;)
5. DevOps@8thcolor • better communication between dev and ops in our case dev = ops ;) • use humans for tasks that matters when you are 3, you don't want to provision or deploy manually, you want to code new features, do code review, prepare launch,... • automation, automation, automation provisioning, adding new components, adding new servers were a pain => infrastructure as code
6. Puppet DSL to manage server resources package { 'apache2': ensure => present, } service { 'apache2': ensure => true, enable => true, subscribe => File["/etc/apache2/apache2.conf"] } other resources like directory/file/erb template/cron/user/group/... Puppet apply - Idempotent execution - Execution order based on dependency graph deduced by the subscribe, require,... apt-get install apache2 or yum install apache2 or nothing systemV , upstart commands apply facts : osfamilty,... resource : presence, state, properties
7. Infrastructure as code Deserves your development best practices : - Version control, IDE, reuse, ci-build - Pull request, code review - Issue tracking - Test, refactoring - Patterns and antipatterns - Profiling - Style guide and best practices
8. Development environment veewee librarian-puppet vagrant up vagrant provision rspec-puppet Command line and scripting environment for VirtualBox unit-testing for puppet : no cost of provisioning, only building catalog, mock facts Iso of distribution, few settings and 30 minutes later a running vagrant box bundler, reuse existing modules
9. Reuse existing modules http://librarian-puppet.com/ = bundler for puppet modules You can all stop using git submodules now == Puppetfile == mod 'apt', git: "git://github.com/puppetlabs/puppetlabs-apt.git" mod 'stdlib', git: 'https://github.com/puppetlabs/puppetlabs-stdlib.git' mod 'logrotate', git: 'https://github.com/rodjek/puppet-logrotate.git' mod 'ssh', git: 'https://github.com/blt/puppet-ssh.git' mod 'postgresql', git: 'https://github.com/puppetlabs/puppet-postgresql.git' mod 'rvm', git: 'https://github.com/angusscown/puppet-rvm.git' mod 'locales', git: 'https://github.com/attachmentgenie/puppet-module-locales.git' mod 'jenkins', git: 'https://github.com/mestachs/puppet-jenkins.git' mod 'firewall', git: 'https://github.com/puppetlabs/puppetlabs-firewall.git' mod 'redis', git: 'https://github.com/HIGHLANDSTECHNOLOGIESSAS/puppet-module-redis.git' mod 'mongodb', git: 'https://github.com/puppetlabs/puppetlabs-mongodb.git' mod 'vcsrepo', git: 'https://github.com/puppetlabs/puppetlabs-vcsrepo.git' mod 'apache', git: 'https://github.com/puppetlabs/puppetlabs-apache.git'
10. Puppet - Rspec ease the refactoring, we moved from a home grown httpd module to the puppet-labs one. first validation of puppet catalogue
11. Continuous build with Jenkins - puppet-rspec - puppet-lint http://docs.puppetlabs.com/guides/style_guide.html Continuous apply... agent less, https://wiki.jenkins-ci.org/display/JENKINS/Build+Pipeline+Plugin Continuous...
12. Patterns Package -> Config ~> Service - deal with usual ordering issue - make the module reusable http://www.slideshare.net/PuppetLabs/modern-module-development-ken-barber-2012-edinburgh-puppet-camp Prefer composition over inheritance - node extends vs include class - class vs define Layout : params, firewall,... - http://www.example42.com/?q=understandExample42PuppetModules Anchor - http://projects.puppetlabs.com/projects/puppet/wiki/Anchor_Pattern
13. Profiling –evaltrace Info: /Stage[main]/Ruby/Rvm_system_ruby[ruby-1.9.3-p362]: Evaluated in 261.84 seconds Info: /Stage[main]/Ruby/Rvm_gem[ruby-1.9.3-p362@rails3.2.11/passenger]: Evaluated in 18.14 seconds Info: /Stage[main]/Rvm::Passenger::Apache::Ubuntu::Post/Exec[passenger-install-apache2-module]: ...43.45 Info: /Stage[main]/Ruby/Rvm_gem[ruby-1.9.3-p362@rails3.2.11/rails]: Evaluated in 89.32 seconds https://github.com/rodjek/puppet-profiler vagrant apt-cache https://gist.github.com/3798773 https://github.com/fgrehm/vagrant-cachier rvm pre-built binairies Info: /Stage[main]/Ruby/Exec[rvm_system_ruby_mount-1.9.3-p374]: Evaluated in 69.66 seconds
14. Style guide / best practices http://docs.puppetlabs.com/guides/style_guide.html https://github.com/alphagov/styleguides/blob/master/puppet.md http://sysphere.org/~anrxc/j/archives/2013/06/24/hosting_with_puppet_- _design/index.html
15. Capistrano apply cap go HOST='vps31124.ovh.net' USER='root' Small infrastructure, agent less https://github.com/pitluga/supply_drop https://github.com/aussielunix/jenkins-appliance
16. What about the Reproducibility 98% Time and network dependencies - accidentally being "beta" tester, - new rvm dependencies,... - early adopter of new Puppet release Vagrant vs OVH - some packages pre-installed by Veewee e.g. RubyGems - network interface eth0, eth1 OVH vs OVH (openvz vs vmware) - overcommit, network interface naming
17. What about the Reproducibility 98% 2 puppet-apply syndrome - missing Puppet dependencies --graph and --graphdir '/tmp/vagrant-puppet/modules-0/graphs' and gephi Too Weak requirement require => User['...'] but more than this require a user with home directory/ssh config/... http://projects.puppetlabs.com/projects/puppet/wiki/Anchor_Pattern
18. What about the Testability 98% puppet-rspec is good but not enough eg: init.d script with debian cartridge ideally provision a vagrant box in a continuous build and check that the services are running, the application responding,... http://serverspec.org/ https://github.com/bryanwb/tk-demo
19. What about the Reusability 98% We provision - our ci build - our staging / production infrastructure - our batsd / batsd-dash instance - our errbit instance - partially my laptop Need to investigate - hiera to avoid "copy/paste/modify" between staging and production - a better way to manage the firewall
20. What about the Maintainability 98% - librarian-puppet : cross fingers just like for bundle update - Puppet : started with 3.0, upgrade 3.1, now at 3.2.1 - Vagrant no more as a gem, not upgraded yet - secret still handled by hand SSH keys SSL certificates (default to snake_oil)
21. Credits and links Puppet labs - puppet - https://puppetlabs.com/ , http://docs.puppetlabs.com/learning/ rodjek (Tim Sharpe) - puppet-profiler, rspec-puppet, puppet-lint, librarian-puppet - https://github.com/rodjek Patrick Debois - Devops, Veewee,... - https://github.com/jedi4ever/veewee Jon Peltier - clock image (whatever) - http://peltiertech.com/WordPress/time-is-on-my-side/ Hashicorp, Mitchell Hashimoto - Vagrant - http://www.vagrantup.com/ Koshuke & co - jenkins - http://jenkins-ci.org/ Capistrano - https://github.com/capistrano/capistrano aussielunix - jenkins appliance - https://github.com/aussielunix/jenkins-appliance Example42 - puppet modules - http://www.example42.com/ Etst - statsd, ... - http://techblog.netflix.com/ Noah Lorang / 37signal - batsd - https://github.com/noahhl/batsd Boxen - http://boxen.github.com/, http://garylarizza.com/blog/2013/02/15/puppet-plus-github-equals-laptop-love/ Michael Coyne - batsd-dash - https://github.com/mikeycgto/batsd-dash Errbit team - errbit error catcher - https://github.com/errbit/errbit Gareth Rushgrove - devops weekly, a lot of modules - http://devopsweekly.com/ https://github.com/garethr wayneeseguin - Ruby Version Manager - http://rvm.io/ Sam Stephenson - Ruby environment - https://github.com/sstephenson/rbenv Deploying Rails - http://pragprog.com/book/cbdepra/deploying-rails alternatives to puppet : ansible - http://www.ansibleworks.com/ chef - http://www.opscode.com/chef/ cfengine - http: //cfengine.com/ commodit - http://www.comodit.com/ saltstack - http://saltstack.com/community.html sprinkle - https: //github.com/crafterm/sprinkle ...