Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Workshop Puppet IPC12

Joshua Thijssen
October 15, 2012
370

Workshop Puppet IPC12

Joshua Thijssen

October 15, 2012
Tweet

Transcript

  1. Joshua Thijssen / Netherlands Freelance consultant and trainer @ NoxLogic

    & TechAdemy Development in PHP, Python, C, Java Lead developer of Saffire Blog: http://adayinthelifeof.nl Email: [email protected] Twitter: @jaytaph whoami 2
  2. .plan ➡ A bit of history / intro about why

    use Puppet and vagrant. ➡ Install / using vagrant ➡ Intro on writing puppet manifests ➡ Actually writing puppet manifests ➡ Misc (monitoring, mcollective, enc etc) 3
  3. Puppet for sysadmins ➡ Control your infrastructure from a single

    point (of failure). ➡ Documented upgrades through version control. ➡ Easy upgrades. ➡ Acceptance infrastructure environments. 15
  4. LAMPGMVNMCSTRAH Linux Apache MySQL PHP Gearman MongoDB CouchDB Solr Tika

    Redis ActiveMQ Hadoop Varnish Nginx Memcache 20 Puppet for developers
  5. ➡ How do you make sure all developers are using

    the same versions of your components? ➡ The same configuration? ➡ Even the same components! ➡ New developers? New development install ➡ Keep development, acceptance, production in sync? 21 Puppet for developers
  6. My first vagrant 26 [default] Importing base box 'lucid32'... [default]

    The guest additions on this VM do not match the install version of VirtualBox! This may cause things such as forwarded ports, shared folders, and more to not work properly. If any of those things fail on this machine, please update the guest additions and repackage the box. Guest Additions Version: 4.1.14 VirtualBox Version: 4.2.0 [default] Matching MAC address for NAT networking... [default] Clearing any previously set forwarded ports... [default] Forwarding ports... [default] -- 22 => 2222 (adapter 1) [default] Creating shared folders metadata... [default] Clearing any previously set network interfaces... [default] Booting VM... [default] Waiting for VM to boot. This can take a few minutes. [default] VM booted and ready for use! [default] Mounting shared folders... [default] -- v-root: /vagrant
  7. ➡ Downloads (optionally) the requested base box ➡ Deploys and

    boots up a new VM. ➡ Runs optional provisioner (puppet / chef / shell) 31
  8. Multi VM’s Vagrant::Config.run do |config| config.vm.box = 'centos-62-64-puppet' config.vm.box_url =

    'http://../centos-6.2-64bit-puppet-vbox.4.1.12.box' config.vm.define :web do |web_config| web_config.vm.host_name = 'web.example.org' web_config.vm.forward_port 80 8080 ... end config.vm.define :database do |db_config| db_config.vm.host_name = 'db.example.org' db_config.vm.forward_port 3306 3306 ... end end Vagrantfile 32
  9. Joind.in example Vagrant::Config.run do |config| # We define one box

    (joindin), but config.vm.define :joindin do |ji_config| ji_config.vm.box = 'centos-62-64-puppet' ji_config.vm.box_url = 'http://.../centos-6.2-64bit-puppet-vbox.4.1.12.box' ji_config.vm.host_name = "joind.in" ji_config.vm.forward_port 80, 8080 # config.vm.share_folder "v-data", "/vagrant_data", "../data" ji_config.vm.boot_mode = :gui ji_config.vm.provision :puppet do |puppet| puppet.manifests_path = "puppet/manifests" puppet.module_path = "puppet/modules" puppet.manifest_file = "joindin.pp" puppet.options = [ '--verbose', ] end end end 33 https://github.com/joindin/joind.in
  10. Quick tips ➡ Use 32bit boxes. Only 64bit when you

    need to, or when you are sure all developers can run them. ➡ Use NFS mounts on linux / osx. (Can’t on windows) config.vm.share_folder(“v-root”, “/vagrant”, “.”, :nfs => (RUBY_PLATFORM =~ /linux/ or RUBY_PLATFORM =~ /darwin/)) 34
  11. Base boxes ➡ Package from current images ➡ Download them

    (http://vagrantbox.es) ➡ Minimal install (netinstall) ➡ vagrant user + “public” private key ➡ ssh server 35
  12. Base boxes 36 $ vagrant box list lucid32 centos-63-32bit-puppet $

    vagrant box add lucid32 lucid32.box $ vagrant box add centos-63-32bit-puppet centos63.box $ vagrant package $ vagrant package --vagrantfile Vagrantfile.pkg --include README.txt
  13. Shared directories ➡ Work from local directory (IDE) ➡ Run

    remote (33.33.33.10) ➡ /vagrant default shared ➡ NFS, vboxfs ➡ watch out with file permissions! 37
  14. ➡ Open source configuration management tool. ➡ Puppet Labs (Reductive

    Labs) ➡ Written in Ruby ➡ Open source: https://github.com/puppetlabs ➡ Commercial version available (puppet enterprise) 39
  15. ➡ Don’t tell HOW to do stuff. ➡ Tell WHAT

    to do. ¹ “yum install httpd” “apt-get install apache2” “install and run the apache webserver” 40
  16. Puppet CA ➡ Certificate signing ➡ Creates, signs, checks x509

    certificates ➡ So you don’t have to worry about them 46
  17. List all nodes 47 root@puppetmaster:~# puppet cert --list --all +

    puppetmaster.noxlogic.local (74:A7:C8:27:72:0D:C1:DD:B8:71:0D:4F:37:69:3D:0C) puppetnode1.noxlogic.local (09:9D:1E:01:D0:A7:BA:FB:8C:F4:2D:96:78:34:54:44)
  18. Sign a node 48 root@puppetmaster:~# puppet cert --sign puppetnode1.noxlogic.local ....

    root@puppetmaster:~# puppet cert --list --all + puppetmaster.noxlogic.local (74:A7:C8:27:72:0D:C1:DD:B8:71:0D:4F:37:69:3D:0C) + puppetnode1.noxlogic.local (CC:50:49:98:1D:F9:06:36:0E:6E:31:F5:27:D8:50:D8)
  19. Puppet agent ➡ Runs on every node that will be

    managed by puppet as a daemon (or crontab, or mcollective). ➡ Calls the puppet master every 30 minutes for updates. ➡ Receives and executes a “catalog”. 49
  20. Facter ➡ Runs on nodes to gather system information. ➡

    Returns $variables to be used on the puppet master in the manifest files. 50
  21. Facter 51 [root@puppetnode1 ~]# facter --puppet architecture => x86_64 fqdn

    => puppetnode1.noxlogic.local interfaces => eth1,eth2,lo ipaddress_eth1 => 192.168.1.114 ipaddress_eth2 => 192.168.56.200 kernel => Linux kernelmajversion => 2.6 operatingsystem => CentOS operatingsystemrelease => 6.0 processor0 => Intel(R) Core(TM)2 Duo CPU T7500 @ 2.20GHz puppetversion => 2.6.9
  22. Facts ➡ You can create your own facts: ➡ project

    names ➡ master / slave databases ➡ zend server ➡ directadmin / plesk 52
  23. 53 zendserver.rb: Facter.add(“Zendserver”) do confine :kernel => :linux setcode do

    if FileTest.exists?(“/usr/local/zend/bin”) “true” else “false” end end end Facts
  24. Title Text 56 package { “strace” : ensure => present,

    } file { “/home/jaytaph/secret-ingredient.txt” : ensure => present, mode => 0600, user => ‘jaytaph’, group => ‘noxlogic’, content => “beer”, }
  25. package { “httpd” : ensure => present, } service {

    “httpd”: running => true, enable => true, } require => Package[“httpd”], 57
  26. Centos / Redhat service: httpd package: httpd config: /etc/httpd/conf/httpd.conf vhosts:

    /etc/httpd/conf.d/*.conf Debian / Ubuntu service: apache2 package: apache2 config: /etc/apache2/httpd.conf vhosts: /etc/apache2/sites-available 58
  27. class webserver { package { “apache”: case $operatingsystem { centos,

    redhat { $packagename = “httpd” } debian, ubuntu { $packagename = “apache2” } default : { fail(‘I don’t know this OS/distro’) } } name => $packagename, ensure => installed, } service { “apache” : running => true, enable => true, require => Package[“apache”], } } 59
  28. [root@puppetnode1 ~]# facter --puppet architecture => x86_64 fqdn => puppetnode1.noxlogic.local

    interfaces => eth1,eth2,lo ipaddress_eth1 => 192.168.1.114 ipaddress_eth2 => 192.168.56.200 kernel => Linux kernelmajversion => 2.6 operatingsystem => CentOS operatingsystemrelease => 6.0 processor0 => Intel(R) Core(TM)2 Duo CPU T7500 @ 2.20GHz puppetversion => 2.6.9 60
  29. Puppet resources • augeas • computer • cron • exec

    • file • filebucket • group • host • interface • k5login • macauthorization • mailalias • maillist • mcx • mount • nagios_command • nagios_contact • nagios_contactgroup • nagios_host • nagios_hostdependency • nagios_hostescalation • nagios_hostextinfo • nagios_hostgroup • nagios_service • nagios_servicedependency • nagios_serviceescalation • nagios_serviceextinfo • nagios_servicegroup • nagios_timeperiod • notify • package • resources • router • schedule • scheduled_task • selboolean • selmodule • service • ssh_authorized_key • sshkey • stage • tidy • user • vlan • yumrepo • zfs • zone • zpool 61
  30. node “web01.example.org” { include webserver } node /^db\d+\.example\.org$/ { package

    { “mysql-server” : ensure => installed, } } 62 /etc/puppet/manifests/site.pp:
  31. node “web01.example.local” { $webserver_name = “web01.example.local” $webserver_alias = “www.example.local” $webserver_docroot

    = “/var/www/web01” include webserver } node “web02.example.local” { $webserver_name = “web02.example.local” $webserver_alias = “crm.example.local” $webserver_docroot = “/var/www/web02” include webserver } 63
  32. Multi VM vagrant with puppet installed 64 $ git clone

    git://gist.github.com/3887842.git vagrant003 $ cd vagrant003 $ vagrant up
  33. 66

  34. Distributed monitoring ➡ Export resources ➡ @@nagios_host ➡ @@nagios_service ➡

    Collect on (monitoring) server: ➡ Nagios_host <<| |>> ➡ Nagios_service <<| |>> 68
  35. 71

  36. 72

  37. 73

  38. 74

  39. --- classes: common: puppet: ntp: ntpserver: 0.pool.ntp.org aptsetup: additional_apt_repos: -

    deb localrepo.example.com/ubuntu lucid production - deb localrepo.example.com/ubuntu lucid vendor parameters: ntp_servers: - 0.pool.ntp.org - ntp.example.com mail_server: mail.example.com iburst: true environment: production output YAML:
  40. ➡ Input is nodename. ➡ Output is YAML structure. ➡

    *CAN* mix site.pp and ENC, but wouldn’t recommend it. (http://docs.puppetlabs.com/ guides/external_nodes.html#how-merging- works) ➡ Possible to store nodes inside databases, LDAP etc..
  41. Title Text 81 ACTIVEMQ Client MCollective Server Node Middleware Client

    MCollective Server MCollective Server Collective
  42. 82 $ mco ping node1.phpconference.org time=51.48 ms node2.phpconference.org time=91.23 ms

    puppetmaster.phpconference.org time=91.60 ms ---- ping statistics ---- 3 replies max: 91.60 min: 51.48 avg: 78.10
  43. Title Text 83 $ mco facts kernel Report for fact:

    kernel Linux found 3 times Finished processing 3 / 3 hosts in 47.99 ms $ mco facts hostname Report for fact: hostname node1 found 1 times node2 found 1 times puppetmaster found 1 times Finished processing 3 / 3 hosts in 50.65 ms
  44. ➡ find all (zombie) processes in your collective. ➡ find

    servers with 80% of utilized memory and running MySQL. ➡ restart all apache webservers in the UK, with less than 4GB of memory, except the ones running on debian 6.0 85
  45. ➡ Run or deploy software ➡ Restart services ➡ Start

    puppet agent ➡ Upgrade your system ➡ Write your own agents! 86