Share
Embed
Workshop Puppet IPC12
Joshua Thijssen
October 15, 2012
Transcript
86 slides
-
Puppet for Dummies IPC - Mainz - Germany 14-17 october
2012 workshop -
Joshua Thijssen / Netherlands Freelance consultant and trainer @ NoxLogic
& TechAdemy Development in PHP, Python, C, Java Lead developer of Saffire Blog: http://adayinthelifeof.nl Email: [email protected] Twitter: @jaytaph whoami 2 -
.plan ➡ A bit of history / intro about why
use Puppet and vagrant. ➡ Install / using vagrant ➡ Intro on writing puppet manifests ➡ Actually writing puppet manifests ➡ Misc (monitoring, mcollective, enc etc) 3 -
11 How are we going to manage our projects, servers,
clouds, customers properly? -
Puppet for sysadmins ➡ Control your infrastructure from a single
point (of failure). ➡ Documented upgrades through version control. ➡ Easy upgrades. ➡ Acceptance infrastructure environments. 15 -
LAMPGMVNMCSTRAH Linux Apache MySQL PHP Gearman MongoDB CouchDB Solr Tika
Redis ActiveMQ Hadoop Varnish Nginx Memcache 20 Puppet for developers -
➡ How do you make sure all developers are using
the same versions of your components? ➡ The same configuration? ➡ Even the same components! ➡ New developers? New development install ➡ Keep development, acceptance, production in sync? 21 Puppet for developers -
My first vagrant 25 $ git clone git://gist.github.com/3863869.git vagrant001 $
cd vagrant001 $ vagrant up -
My first vagrant 26 [default] Importing base box 'lucid32'... [default]
The guest additions on this VM do not match the install version of VirtualBox! This may cause things such as forwarded ports, shared folders, and more to not work properly. If any of those things fail on this machine, please update the guest additions and repackage the box. Guest Additions Version: 4.1.14 VirtualBox Version: 4.2.0 [default] Matching MAC address for NAT networking... [default] Clearing any previously set forwarded ports... [default] Forwarding ports... [default] -- 22 => 2222 (adapter 1) [default] Creating shared folders metadata... [default] Clearing any previously set network interfaces... [default] Booting VM... [default] Waiting for VM to boot. This can take a few minutes. [default] VM booted and ready for use! [default] Mounting shared folders... [default] -- v-root: /vagrant -
Actually, it’s a simple tool that speaks to the virtualbox
api (but it’s still awesome) 30 -
➡ Downloads (optionally) the requested base box ➡ Deploys and
boots up a new VM. ➡ Runs optional provisioner (puppet / chef / shell) 31 -
Multi VM’s Vagrant::Config.run do |config| config.vm.box = 'centos-62-64-puppet' config.vm.box_url =
'http://../centos-6.2-64bit-puppet-vbox.4.1.12.box' config.vm.define :web do |web_config| web_config.vm.host_name = 'web.example.org' web_config.vm.forward_port 80 8080 ... end config.vm.define :database do |db_config| db_config.vm.host_name = 'db.example.org' db_config.vm.forward_port 3306 3306 ... end end Vagrantfile 32 -
Joind.in example Vagrant::Config.run do |config| # We define one box
(joindin), but config.vm.define :joindin do |ji_config| ji_config.vm.box = 'centos-62-64-puppet' ji_config.vm.box_url = 'http://.../centos-6.2-64bit-puppet-vbox.4.1.12.box' ji_config.vm.host_name = "joind.in" ji_config.vm.forward_port 80, 8080 # config.vm.share_folder "v-data", "/vagrant_data", "../data" ji_config.vm.boot_mode = :gui ji_config.vm.provision :puppet do |puppet| puppet.manifests_path = "puppet/manifests" puppet.module_path = "puppet/modules" puppet.manifest_file = "joindin.pp" puppet.options = [ '--verbose', ] end end end 33 https://github.com/joindin/joind.in -
Quick tips ➡ Use 32bit boxes. Only 64bit when you
need to, or when you are sure all developers can run them. ➡ Use NFS mounts on linux / osx. (Can’t on windows) config.vm.share_folder(“v-root”, “/vagrant”, “.”, :nfs => (RUBY_PLATFORM =~ /linux/ or RUBY_PLATFORM =~ /darwin/)) 34 -
Base boxes ➡ Package from current images ➡ Download them
(http://vagrantbox.es) ➡ Minimal install (netinstall) ➡ vagrant user + “public” private key ➡ ssh server 35 -
Base boxes 36 $ vagrant box list lucid32 centos-63-32bit-puppet $
vagrant box add lucid32 lucid32.box $ vagrant box add centos-63-32bit-puppet centos63.box $ vagrant package $ vagrant package --vagrantfile Vagrantfile.pkg --include README.txt -
Shared directories ➡ Work from local directory (IDE) ➡ Run
remote (33.33.33.10) ➡ /vagrant default shared ➡ NFS, vboxfs ➡ watch out with file permissions! 37 -
➡ Open source configuration management tool. ➡ Puppet Labs (Reductive
Labs) ➡ Written in Ruby ➡ Open source: https://github.com/puppetlabs ➡ Commercial version available (puppet enterprise) 39 -
➡ Don’t tell HOW to do stuff. ➡ Tell WHAT
to do. ¹ “yum install httpd” “apt-get install apache2” “install and run the apache webserver” 40 -
➡ puppet master (puppetmasterd) ➡ puppet cert (puppetca) ➡ puppet
agent (puppetd) ➡ facter 44 -
Puppet master ➡ Central server ➡ File & Configuration server
➡ REST(ish) over HTTPS interface 45 -
Puppet CA ➡ Certificate signing ➡ Creates, signs, checks x509
certificates ➡ So you don’t have to worry about them 46 -
List all nodes 47 root@puppetmaster:~# puppet cert --list --all +
puppetmaster.noxlogic.local (74:A7:C8:27:72:0D:C1:DD:B8:71:0D:4F:37:69:3D:0C) puppetnode1.noxlogic.local (09:9D:1E:01:D0:A7:BA:FB:8C:F4:2D:96:78:34:54:44) -
Sign a node 48 root@puppetmaster:~# puppet cert --sign puppetnode1.noxlogic.local ....
root@puppetmaster:~# puppet cert --list --all + puppetmaster.noxlogic.local (74:A7:C8:27:72:0D:C1:DD:B8:71:0D:4F:37:69:3D:0C) + puppetnode1.noxlogic.local (CC:50:49:98:1D:F9:06:36:0E:6E:31:F5:27:D8:50:D8) -
Puppet agent ➡ Runs on every node that will be
managed by puppet as a daemon (or crontab, or mcollective). ➡ Calls the puppet master every 30 minutes for updates. ➡ Receives and executes a “catalog”. 49 -
Facter ➡ Runs on nodes to gather system information. ➡
Returns $variables to be used on the puppet master in the manifest files. 50 -
Facter 51 [root@puppetnode1 ~]# facter --puppet architecture => x86_64 fqdn
=> puppetnode1.noxlogic.local interfaces => eth1,eth2,lo ipaddress_eth1 => 192.168.1.114 ipaddress_eth2 => 192.168.56.200 kernel => Linux kernelmajversion => 2.6 operatingsystem => CentOS operatingsystemrelease => 6.0 processor0 => Intel(R) Core(TM)2 Duo CPU T7500 @ 2.20GHz puppetversion => 2.6.9 -
Facts ➡ You can create your own facts: ➡ project
names ➡ master / slave databases ➡ zend server ➡ directadmin / plesk 52 -
53 zendserver.rb: Facter.add(“Zendserver”) do confine :kernel => :linux setcode do
if FileTest.exists?(“/usr/local/zend/bin”) “true” else “false” end end end Facts -
Multi VM vagrant 54 $ git clone git://gist.github.com/3887712.git vagrant002 $
cd vagrant002 $ vagrant up -
Title Text 56 package { “strace” : ensure => present,
} file { “/home/jaytaph/secret-ingredient.txt” : ensure => present, mode => 0600, user => ‘jaytaph’, group => ‘noxlogic’, content => “beer”, } -
package { “httpd” : ensure => present, } service {
“httpd”: running => true, enable => true, } require => Package[“httpd”], 57 -
Centos / Redhat service: httpd package: httpd config: /etc/httpd/conf/httpd.conf vhosts:
/etc/httpd/conf.d/*.conf Debian / Ubuntu service: apache2 package: apache2 config: /etc/apache2/httpd.conf vhosts: /etc/apache2/sites-available 58 -
class webserver { package { “apache”: case $operatingsystem { centos,
redhat { $packagename = “httpd” } debian, ubuntu { $packagename = “apache2” } default : { fail(‘I don’t know this OS/distro’) } } name => $packagename, ensure => installed, } service { “apache” : running => true, enable => true, require => Package[“apache”], } } 59 -
[root@puppetnode1 ~]# facter --puppet architecture => x86_64 fqdn => puppetnode1.noxlogic.local
interfaces => eth1,eth2,lo ipaddress_eth1 => 192.168.1.114 ipaddress_eth2 => 192.168.56.200 kernel => Linux kernelmajversion => 2.6 operatingsystem => CentOS operatingsystemrelease => 6.0 processor0 => Intel(R) Core(TM)2 Duo CPU T7500 @ 2.20GHz puppetversion => 2.6.9 60 -
Puppet resources • augeas • computer • cron • exec
• file • filebucket • group • host • interface • k5login • macauthorization • mailalias • maillist • mcx • mount • nagios_command • nagios_contact • nagios_contactgroup • nagios_host • nagios_hostdependency • nagios_hostescalation • nagios_hostextinfo • nagios_hostgroup • nagios_service • nagios_servicedependency • nagios_serviceescalation • nagios_serviceextinfo • nagios_servicegroup • nagios_timeperiod • notify • package • resources • router • schedule • scheduled_task • selboolean • selmodule • service • ssh_authorized_key • sshkey • stage • tidy • user • vlan • yumrepo • zfs • zone • zpool 61 -
node “web01.example.org” { include webserver } node /^db\d+\.example\.org$/ { package
{ “mysql-server” : ensure => installed, } } 62 /etc/puppet/manifests/site.pp: -
node “web01.example.local” { $webserver_name = “web01.example.local” $webserver_alias = “www.example.local” $webserver_docroot
= “/var/www/web01” include webserver } node “web02.example.local” { $webserver_name = “web02.example.local” $webserver_alias = “crm.example.local” $webserver_docroot = “/var/www/web02” include webserver } 63 -
Multi VM vagrant with puppet installed 64 $ git clone
git://gist.github.com/3887842.git vagrant003 $ cd vagrant003 $ vagrant up -
Distributed monitoring ➡ Export resources ➡ @@nagios_host ➡ @@nagios_service ➡
Collect on (monitoring) server: ➡ Nagios_host <<| |>> ➡ Nagios_service <<| |>> 68 -
--- classes: common: puppet: ntp: ntpserver: 0.pool.ntp.org aptsetup: additional_apt_repos: -
deb localrepo.example.com/ubuntu lucid production - deb localrepo.example.com/ubuntu lucid vendor parameters: ntp_servers: - 0.pool.ntp.org - ntp.example.com mail_server: mail.example.com iburst: true environment: production output YAML: -
➡ Input is nodename. ➡ Output is YAML structure. ➡
*CAN* mix site.pp and ENC, but wouldn’t recommend it. (http://docs.puppetlabs.com/ guides/external_nodes.html#how-merging- works) ➡ Possible to store nodes inside databases, LDAP etc.. -
Title Text ➡ Marionette Collective ➡ server orchestration or parallel
job execution system 80 -
Title Text 81 ACTIVEMQ Client MCollective Server Node Middleware Client
MCollective Server MCollective Server Collective -
82 $ mco ping node1.phpconference.org time=51.48 ms node2.phpconference.org time=91.23 ms
puppetmaster.phpconference.org time=91.60 ms ---- ping statistics ---- 3 replies max: 91.60 min: 51.48 avg: 78.10 -
Title Text 83 $ mco facts kernel Report for fact:
kernel Linux found 3 times Finished processing 3 / 3 hosts in 47.99 ms $ mco facts hostname Report for fact: hostname node1 found 1 times node2 found 1 times puppetmaster found 1 times Finished processing 3 / 3 hosts in 50.65 ms -
➡ find all (zombie) processes in your collective. ➡ find
servers with 80% of utilized memory and running MySQL. ➡ restart all apache webservers in the UK, with less than 4GB of memory, except the ones running on debian 6.0 85 -
➡ Run or deploy software ➡ Restart services ➡ Start
puppet agent ➡ Upgrade your system ➡ Write your own agents! 86