Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Puppet for dummies - PHP|Tek 2012

Puppet for dummies - PHP|Tek 2012

1761ecd7fe763583553dde43e62c47bd?s=128

Joshua Thijssen

May 23, 2012
Tweet

More Decks by Joshua Thijssen

Other Decks in Technology

Transcript

  1. Puppet for Dummies PHP|Tek - Chicago - USA May 23,

    2012 woensdag 23 mei 12
  2. Joshua Thijssen Freelance consultant, developer and trainer @ NoxLogic /

    Techademy Development in PHP, Python, Perl, C, Java and some sysadmin Blog: http://adayinthelifeof.nl Email: jthijssen@noxlogic.nl Twitter: @jaytaph oh hai! 2 woensdag 23 mei 12
  3. What is puppet and why should I care? 3 woensdag

    23 mei 12
  4. “People are finally figuring out puppet and how it gets

    you to the pub by 4pm. Note that I’ve been at this pub since 2pm.” - Jorge Castro 4 woensdag 23 mei 12
  5. 5 woensdag 23 mei 12

  6. Puppet is a (not necessarily the) solution for the following

    problem: How do we setup, manage, synchronize, and upgrade our internal and external infrastructure? 6 woensdag 23 mei 12
  7. Sysadmin! Y U no fix problem! 7 woensdag 23 mei

    12
  8. Sysadmin! Y U no fix problem! NO 7 woensdag 23

    mei 12
  9. LAMP-stack 8 woensdag 23 mei 12

  10. LAMP-stack Linux Apache MySQL PHP 8 woensdag 23 mei 12

  11. LAMPGMVNMCSTRAH-stack 9 woensdag 23 mei 12

  12. LAMPGMVNMCSTRAH-stack Linux Apache MySQL PHP Gearman MongoDB CouchDB Solr Tika

    Redis ActiveMQ Hadoop Varnish Ngnix Memcache 9 woensdag 23 mei 12
  13. 10 woensdag 23 mei 12

  14. 10 How do we control our infrastructure? woensdag 23 mei

    12
  15. ➡ Solution 1: We don’t, 10 How do we control

    our infrastructure? woensdag 23 mei 12
  16. ➡ Solution 1: We don’t, ➡ Solution 2: We outsource,

    10 How do we control our infrastructure? woensdag 23 mei 12
  17. ➡ Solution 1: We don’t, ➡ Solution 2: We outsource,

    ➡ Solution 3: We automate the process. 10 How do we control our infrastructure? woensdag 23 mei 12
  18. ‣ Solution 1: we don’t 11 woensdag 23 mei 12

  19. ➡ It’s not funny: you find it more often than

    not. Especially inside small development companies. ‣ Solution 1: we don’t 11 woensdag 23 mei 12
  20. ➡ It’s not funny: you find it more often than

    not. Especially inside small development companies. ➡ Internal sysadmin, but he’s too busy with development to do sysadmin. ‣ Solution 1: we don’t 11 woensdag 23 mei 12
  21. ➡ It’s not funny: you find it more often than

    not. Especially inside small development companies. ➡ Internal sysadmin, but he’s too busy with development to do sysadmin. ➡ We only act on escalation ‣ Solution 1: we don’t 11 woensdag 23 mei 12
  22. ➡ It’s not funny: you find it more often than

    not. Especially inside small development companies. ➡ Internal sysadmin, but he’s too busy with development to do sysadmin. ➡ We only act on escalation ➡ reactive, not proactive ‣ Solution 1: we don’t 11 woensdag 23 mei 12
  23. ‣ Solution 2: we outsource 12 woensdag 23 mei 12

  24. ➡ Expensive $LA’s. ‣ Solution 2: we outsource 12 woensdag

    23 mei 12
  25. ➡ Expensive $LA’s. ➡ What about INTERNAL servers like your

    development systems and infrastructure? ‣ Solution 2: we outsource 12 woensdag 23 mei 12
  26. ➡ Expensive $LA’s. ➡ What about INTERNAL servers like your

    development systems and infrastructure? ➡ Fight between stability and agility. ‣ Solution 2: we outsource 12 woensdag 23 mei 12
  27. ➡ Expensive $LA’s. ➡ What about INTERNAL servers like your

    development systems and infrastructure? ➡ Fight between stability and agility. ➡ Does your hosting company decide on whether you can use PHP5.3??? ‣ Solution 2: we outsource 12 woensdag 23 mei 12
  28. ‣ Solution 3: we do it ourselves and automate 13

    woensdag 23 mei 12
  29. ➡ We are in charge. ‣ Solution 3: we do

    it ourselves and automate 13 woensdag 23 mei 12
  30. ➡ We are in charge. ➡ You can do what

    you like ‣ Solution 3: we do it ourselves and automate 13 woensdag 23 mei 12
  31. ➡ We are in charge. ➡ You can do what

    you like ➡ Use: cfEngine, chef, puppet. ‣ Solution 3: we do it ourselves and automate 13 woensdag 23 mei 12
  32. ➡ We are in charge. ➡ You can do what

    you like ➡ Use: cfEngine, chef, puppet. ➡ When done right, maintenance should not be difficult. ‣ Solution 3: we do it ourselves and automate 13 woensdag 23 mei 12
  33. PUPPET 14 woensdag 23 mei 12

  34. ➡ Open source configuration management tool. ➡ Written in Ruby

    ➡ Open source: https://github.com/puppetlabs ➡ Commercial version available (puppet enterprise) 15 woensdag 23 mei 12
  35. ➡ Don’t tell HOW to do stuff. ➡ Tell WHAT

    to do. ¹ ¹ It’s not actually true, but good enough for now... 16 woensdag 23 mei 12
  36. ➡ Don’t tell HOW to do stuff. ➡ Tell WHAT

    to do. ¹ ¹ It’s not actually true, but good enough for now... “yum install httpd” “apt-get install apache2” 16 woensdag 23 mei 12
  37. ➡ Don’t tell HOW to do stuff. ➡ Tell WHAT

    to do. ¹ ¹ It’s not actually true, but good enough for now... “yum install httpd” “apt-get install apache2” “install and run the apache webserver” 16 woensdag 23 mei 12
  38. 17 Schematic representation of a puppet infrastructure woensdag 23 mei

    12
  39. Puppet 18 woensdag 23 mei 12

  40. Puppet CA Puppet Master Puppet Agent https 19 woensdag 23

    mei 12
  41. Puppet CA Puppet Master Puppet Agent Puppet Agent Puppet Agent

    https 19 woensdag 23 mei 12
  42. ➡ Agent “calls” the puppet master. 20 woensdag 23 mei

    12
  43. ➡ Agent “calls” the puppet master. ➡ Agent sends “facts”

    to the master. 20 woensdag 23 mei 12
  44. ➡ Agent “calls” the puppet master. ➡ Agent sends “facts”

    to the master. ➡ Master creates “catalog” from the manifests and facts, sends to agent. 20 woensdag 23 mei 12
  45. ➡ Agent “calls” the puppet master. ➡ Agent sends “facts”

    to the master. ➡ Master creates “catalog” from the manifests and facts, sends to agent. ➡ Agent sets up system according to the catalog. 20 woensdag 23 mei 12
  46. ➡ Agent “calls” the puppet master. ➡ Agent sends “facts”

    to the master. ➡ Master creates “catalog” from the manifests and facts, sends to agent. ➡ Agent sets up system according to the catalog. ➡ Agent reports status to master. 20 woensdag 23 mei 12
  47. ➡ Catalogs are “compiled” manifests 21 woensdag 23 mei 12

  48. ➡ Catalogs are “compiled” manifests ➡ Manifests are puppet definitions

    21 woensdag 23 mei 12
  49. ➡ Catalogs are “compiled” manifests ➡ Manifests are puppet definitions

    ➡ <filename>.pp 21 woensdag 23 mei 12
  50. ➡ Catalogs are “compiled” manifests ➡ Manifests are puppet definitions

    ➡ <filename>.pp ➡ Puppet DSL 21 woensdag 23 mei 12
  51. ➡ Catalogs are “compiled” manifests ➡ Manifests are puppet definitions

    ➡ <filename>.pp ➡ Puppet DSL ➡ De-cla-ra-tive language 21 woensdag 23 mei 12
  52. ➡ Catalogs are “compiled” manifests ➡ Manifests are puppet definitions

    ➡ <filename>.pp ➡ Puppet DSL ➡ De-cla-ra-tive language ➡ Version your manifests! (git/svn) 21 woensdag 23 mei 12
  53. package { “strace” : ensure => present, } file {

    “/home/jaytaph/secret-ingredient.txt” : ensure => present, mode => 0600, user => ‘jaytaph’, group => ‘noxlogic’, content => “beer”, } 22 woensdag 23 mei 12
  54. package { “httpd” : ensure => present, } service {

    “httpd”: running => true, enable => true, } 23 woensdag 23 mei 12
  55. package { “httpd” : ensure => present, } service {

    “httpd”: running => true, enable => true, } require => Package[“httpd”], 23 woensdag 23 mei 12
  56. ‣ Different distributions, different names Centos / Redhat service: httpd

    package: httpd config: /etc/httpd/conf/httpd.conf vhosts: /etc/httpd/conf.d/*.conf Debian / Ubuntu service: apache2 package: apache2 config: /etc/apache2/httpd.conf vhosts: /etc/apache2/sites-available 24 woensdag 23 mei 12
  57. class apache { package { “apache”: case $operatingsystem { centos,

    redhat { $packagename = “httpd” } debian, ubuntu { $packagename = “apache2” } default : { fail(‘I don’t know this OS/distro’) } } name => $packagename, ensure => installed, } service { “apache” : running => true, enable => true, require => Package[“apache”], } } 25 woensdag 23 mei 12
  58. [root@puppetnode1 ~]# facter --puppet architecture => x86_64 fqdn => puppetnode1.noxlogic.local

    interfaces => eth1,eth2,lo ipaddress_eth1 => 192.168.1.114 ipaddress_eth2 => 192.168.56.200 kernel => Linux kernelmajversion => 2.6 operatingsystem => CentOS operatingsystemrelease => 6.0 processor0 => Intel(R) Core(TM)2 Duo CPU T7500 @ 2.20GHz puppetversion => 2.6.9 ‣ A simple list with info (also useable in your own tools) 26 woensdag 23 mei 12
  59. node “web01.example.org” { include apache } node /^db\d+\.example\.org$/ { package

    { “mysql-server” : ensure => installed, } } 27 /etc/puppet/manifests/site.pp: woensdag 23 mei 12
  60. node “web01.example.local” { $webserver_name = “web01.example.local” $webserver_alias = “www.example.local” $webserver_docroot

    = “/var/www/web01” include apache } node “web02.example.local” { $webserver_name = “web02.example.local” $webserver_alias = “crm.example.local” $webserver_docroot = “/var/www/web02” include apache } 28 woensdag 23 mei 12
  61. http://docs.puppetlabs.com/references/stable/type.html 29 What can Puppet configure / control? woensdag 23

    mei 12
  62. http://docs.puppetlabs.com/references/stable/type.html ➡ Almost everything. ➡ Standard 48 different resource types

    ➡ Ranging from “file” to “cron” to “ssh_key” to “user” to “selinux”. ➡ Can control your Cisco routers and windows machines too (sortakinda) 30 woensdag 23 mei 12
  63. 31 class joindin::web { include apache # include phpmyadmin if

    needed if $params::phpmyadmin == true { include joindin::web::phpmyadmin } # Configure apache virtual host apache::vhost { $params::host : docroot => '/vagrant/src', template => 'joindin/vhost.conf.erb', port => $params::port, require => Package["apache"], } https://github.com/jaytaph/joind.in/tree/puppet woensdag 23 mei 12
  64. 32 ... # Install PHP modules php::module { 'mysql': }

    php::module { "pecl-xdebug" : require => File["EpelRepo"], # xdebug is in the epel repo } # Set development values to our php.ini augeas { 'set-php-ini-values': context => '/files/etc/php.ini', changes => [ 'set PHP/error_reporting "E_ALL | E_STRICT"', 'set PHP/display_errors On', 'set PHP/display_startup_errors On', 'set PHP/html_errors On', 'set Date/date.timezone Europe/London', ], require => Package['php'], notify => Service['apache'], } } # End class woensdag 23 mei 12
  65. 33 ➡ Puppet went from v0.25 to v2.6. ➡ REST

    interface since 2.6. XMLRPC before that. ➡ One binary to rule them all (puppet). ➡ Puppet v2.7 switched from GPLv2 to apache2.0 license. woensdag 23 mei 12
  66. 34 So how does Puppet benefit me as a DEVELOPER?

    woensdag 23 mei 12
  67. 35 ➡ Keep all developers in sync ➡ Keep your

    DTAP in sync ➡ Lets infrastructure be a part of your project woensdag 23 mei 12
  68. Vagrant http://vagrantup.com/ http://vagrantup.com/images/vagrant_chilling.png 36 woensdag 23 mei 12

  69. Vagrant is a tool for building and distributing virtualized development

    environments. 37 woensdag 23 mei 12
  70. Vagrant::Config.run do |config| config.vm.box = 'centos-62-64-puppet' config.vm.box_url = 'http://../centos-6.2-64bit-puppet-vbox.4.1.12.box' #

    Forward a port from the guest to the host, which allows for outside # computers to access the VM, whereas host only networking does not. config.vm.forward_port 80, 8080 config.vm.provision :puppet do |puppet| puppet.manifests_path = "puppet/manifests" puppet.module_path = "puppet/modules" puppet.manifest_file = "main.pp" puppet.options = [ '--verbose', ] end end Vagrantfile 38 woensdag 23 mei 12
  71. # git clone git@github.com:jaytaph/myproject.git # vagrant up 39 woensdag 23

    mei 12
  72. ➡ Downloads (optionally) the base box ➡ Deploys and boots

    up a new VM ➡ Runs the provisioner (puppet) ➡ Profit! 40 woensdag 23 mei 12
  73. Multi VM’s Vagrant::Config.run do |config| config.vm.box = 'centos-62-64-puppet' config.vm.box_url =

    'http://../centos-6.2-64bit-puppet-vbox.4.1.12.box' config.vm.define :web do |web_config| web_config.vm.host_name = 'web.example.org' web_config.vm.forward_port 80 8080 ... end config.vm.define :database do |db_config| db_config.vm.host_name = 'db.example.org' db_config.vm.forward_port 3306 3306 ... end end Vagrantfile 41 woensdag 23 mei 12
  74. 42 woensdag 23 mei 12

  75. ➡ Puppet agent “calls” the master every 30 minutes. ➡

    But what about realtime command & control? ➡ “Puppet kick”... (meh) ➡ MCollective (Marionette Collective) 43 woensdag 23 mei 12
  76. ➡ Which systems running a database and have 16GB or

    less? ➡ Which systems are using <50% of available memory? ➡ Restart all apache services in timezone GMT+5. 44 woensdag 23 mei 12
  77. ACTIVEMQ Client MCollective Server Node Middleware Client MCollective Server MCollective

    Server ‣ Middleware takes care of distribution, ‣ queued, broadcast etc.. Collective 45 woensdag 23 mei 12
  78. http://docs.puppetlabs.com/mcollective/reference/basic/subcollectives.html 46 woensdag 23 mei 12

  79. Filter out nodes based on facts $ mc-facts operatingsystem Report

    for fact: operatingsystem CentOS found 3 times Debian found 14 times Solaris found 4 times $ mc-facts -W operatingsystem=Centos operatingsystemrelease Report for fact: operatingsystemrelease 6.0 found 1 times 5.6 found 2 times 47 woensdag 23 mei 12
  80. ➡ Display all running processes ➡ Run or deploy software

    ➡ Restart services ➡ Start puppet agent ➡ Upgrade your systems ➡ Write your own agents! 48 woensdag 23 mei 12
  81. -ETOOMUCHINFO Let’s recap 49 woensdag 23 mei 12

  82. ➡ Configuration management tool. ➡ Focusses on “what” instead of

    “how”. ➡ Scales from 1 to 100K+ systems. ➡ Uses descriptive manifests. ➡ Vagrant for setting up your development environments. 50 woensdag 23 mei 12
  83. ➡ Useful for sysadmins and developers. ➡ Keeps your infrastructure

    in sync. ➡ Keeps your infrastructure versioned. ➡ Infrastructure as part of your projects. ➡ MCollective controls your hosts based on facts, not names. 51 woensdag 23 mei 12
  84. There is no reason NOT to manage your infrastructure. Having

    only 3 servers is NOT a reason. 52 You will be able to join the rest of us in the pub early. Don’t “install” development environments, build them! woensdag 23 mei 12
  85. http://farm1.static.flickr.com/73/163450213_18478d3aa6_d.jpg 53 woensdag 23 mei 12

  86. Please rate my talk on joind.in: http://joind.in/6515 Thank you 54

    Find me on twitter: @jaytaph Find me for development and training: www.noxlogic.nl Find me on email: jthijssen@noxlogic.nl Find me for blogs: www.adayinthelifeof.nl woensdag 23 mei 12