Tweet
Tweet

Slide 1

Slide 1 text

Puppet for Dummies PHP|Tek - Chicago - USA May 23, 2012 woensdag 23 mei 12

Slide 2

Slide 2 text

Joshua Thijssen Freelance consultant, developer and trainer @ NoxLogic / Techademy Development in PHP, Python, Perl, C, Java and some sysadmin Blog: http://adayinthelifeof.nl Email: [email protected] Twitter: @jaytaph oh hai! 2 woensdag 23 mei 12

Slide 3

Slide 3 text

What is puppet and why should I care? 3 woensdag 23 mei 12

Slide 4

Slide 4 text

“People are finally figuring out puppet and how it gets you to the pub by 4pm. Note that I’ve been at this pub since 2pm.” - Jorge Castro 4 woensdag 23 mei 12

Slide 5

Slide 5 text

5 woensdag 23 mei 12

Slide 6

Slide 6 text

Puppet is a (not necessarily the) solution for the following problem: How do we setup, manage, synchronize, and upgrade our internal and external infrastructure? 6 woensdag 23 mei 12

Slide 7

Slide 7 text

Sysadmin! Y U no fix problem! 7 woensdag 23 mei 12

Slide 8

Slide 8 text

Sysadmin! Y U no fix problem! NO 7 woensdag 23 mei 12

Slide 9

Slide 9 text

LAMP-stack 8 woensdag 23 mei 12

Slide 10

Slide 10 text

LAMP-stack Linux Apache MySQL PHP 8 woensdag 23 mei 12

Slide 11

Slide 11 text

LAMPGMVNMCSTRAH-stack 9 woensdag 23 mei 12

Slide 12

Slide 12 text

LAMPGMVNMCSTRAH-stack Linux Apache MySQL PHP Gearman MongoDB CouchDB Solr Tika Redis ActiveMQ Hadoop Varnish Ngnix Memcache 9 woensdag 23 mei 12

Slide 13

Slide 13 text

10 woensdag 23 mei 12

Slide 14

Slide 14 text

10 How do we control our infrastructure? woensdag 23 mei 12

Slide 15

Slide 15 text

➡ Solution 1: We don’t, 10 How do we control our infrastructure? woensdag 23 mei 12

Slide 16

Slide 16 text

➡ Solution 1: We don’t, ➡ Solution 2: We outsource, 10 How do we control our infrastructure? woensdag 23 mei 12

Slide 17

Slide 17 text

➡ Solution 1: We don’t, ➡ Solution 2: We outsource, ➡ Solution 3: We automate the process. 10 How do we control our infrastructure? woensdag 23 mei 12

Slide 18

Slide 18 text

‣ Solution 1: we don’t 11 woensdag 23 mei 12

Slide 19

Slide 19 text

➡ It’s not funny: you find it more often than not. Especially inside small development companies. ‣ Solution 1: we don’t 11 woensdag 23 mei 12

Slide 20

Slide 20 text

➡ It’s not funny: you find it more often than not. Especially inside small development companies. ➡ Internal sysadmin, but he’s too busy with development to do sysadmin. ‣ Solution 1: we don’t 11 woensdag 23 mei 12

Slide 21

Slide 21 text

➡ It’s not funny: you find it more often than not. Especially inside small development companies. ➡ Internal sysadmin, but he’s too busy with development to do sysadmin. ➡ We only act on escalation ‣ Solution 1: we don’t 11 woensdag 23 mei 12

Slide 22

Slide 22 text

➡ It’s not funny: you find it more often than not. Especially inside small development companies. ➡ Internal sysadmin, but he’s too busy with development to do sysadmin. ➡ We only act on escalation ➡ reactive, not proactive ‣ Solution 1: we don’t 11 woensdag 23 mei 12

Slide 23

Slide 23 text

‣ Solution 2: we outsource 12 woensdag 23 mei 12

Slide 24

Slide 24 text

➡ Expensive $LA’s. ‣ Solution 2: we outsource 12 woensdag 23 mei 12

Slide 25

Slide 25 text

➡ Expensive $LA’s. ➡ What about INTERNAL servers like your development systems and infrastructure? ‣ Solution 2: we outsource 12 woensdag 23 mei 12

Slide 26

Slide 26 text

➡ Expensive $LA’s. ➡ What about INTERNAL servers like your development systems and infrastructure? ➡ Fight between stability and agility. ‣ Solution 2: we outsource 12 woensdag 23 mei 12

Slide 27

Slide 27 text

➡ Expensive $LA’s. ➡ What about INTERNAL servers like your development systems and infrastructure? ➡ Fight between stability and agility. ➡ Does your hosting company decide on whether you can use PHP5.3??? ‣ Solution 2: we outsource 12 woensdag 23 mei 12

Slide 28

Slide 28 text

‣ Solution 3: we do it ourselves and automate 13 woensdag 23 mei 12

Slide 29

Slide 29 text

➡ We are in charge. ‣ Solution 3: we do it ourselves and automate 13 woensdag 23 mei 12

Slide 30

Slide 30 text

➡ We are in charge. ➡ You can do what you like ‣ Solution 3: we do it ourselves and automate 13 woensdag 23 mei 12

Slide 31

Slide 31 text

➡ We are in charge. ➡ You can do what you like ➡ Use: cfEngine, chef, puppet. ‣ Solution 3: we do it ourselves and automate 13 woensdag 23 mei 12

Slide 32

Slide 32 text

➡ We are in charge. ➡ You can do what you like ➡ Use: cfEngine, chef, puppet. ➡ When done right, maintenance should not be difficult. ‣ Solution 3: we do it ourselves and automate 13 woensdag 23 mei 12

Slide 33

Slide 33 text

PUPPET 14 woensdag 23 mei 12

Slide 34

Slide 34 text

➡ Open source configuration management tool. ➡ Written in Ruby ➡ Open source: https://github.com/puppetlabs ➡ Commercial version available (puppet enterprise) 15 woensdag 23 mei 12

Slide 35

Slide 35 text

➡ Don’t tell HOW to do stuff. ➡ Tell WHAT to do. ¹ ¹ It’s not actually true, but good enough for now... 16 woensdag 23 mei 12

Slide 36

Slide 36 text

➡ Don’t tell HOW to do stuff. ➡ Tell WHAT to do. ¹ ¹ It’s not actually true, but good enough for now... “yum install httpd” “apt-get install apache2” 16 woensdag 23 mei 12

Slide 37

Slide 37 text

➡ Don’t tell HOW to do stuff. ➡ Tell WHAT to do. ¹ ¹ It’s not actually true, but good enough for now... “yum install httpd” “apt-get install apache2” “install and run the apache webserver” 16 woensdag 23 mei 12

Slide 38

Slide 38 text

17 Schematic representation of a puppet infrastructure woensdag 23 mei 12

Slide 39

Slide 39 text

Puppet 18 woensdag 23 mei 12

Slide 40

Slide 40 text

Puppet CA Puppet Master Puppet Agent https 19 woensdag 23 mei 12

Slide 41

Slide 41 text

Puppet CA Puppet Master Puppet Agent Puppet Agent Puppet Agent https 19 woensdag 23 mei 12

Slide 42

Slide 42 text

➡ Agent “calls” the puppet master. 20 woensdag 23 mei 12

Slide 43

Slide 43 text

➡ Agent “calls” the puppet master. ➡ Agent sends “facts” to the master. 20 woensdag 23 mei 12

Slide 44

Slide 44 text

➡ Agent “calls” the puppet master. ➡ Agent sends “facts” to the master. ➡ Master creates “catalog” from the manifests and facts, sends to agent. 20 woensdag 23 mei 12

Slide 45

Slide 45 text

➡ Agent “calls” the puppet master. ➡ Agent sends “facts” to the master. ➡ Master creates “catalog” from the manifests and facts, sends to agent. ➡ Agent sets up system according to the catalog. 20 woensdag 23 mei 12

Slide 46

Slide 46 text

➡ Agent “calls” the puppet master. ➡ Agent sends “facts” to the master. ➡ Master creates “catalog” from the manifests and facts, sends to agent. ➡ Agent sets up system according to the catalog. ➡ Agent reports status to master. 20 woensdag 23 mei 12

Slide 47

Slide 47 text

➡ Catalogs are “compiled” manifests 21 woensdag 23 mei 12

Slide 48

Slide 48 text

➡ Catalogs are “compiled” manifests ➡ Manifests are puppet definitions 21 woensdag 23 mei 12

Slide 49

Slide 49 text

➡ Catalogs are “compiled” manifests ➡ Manifests are puppet definitions ➡ <filename>.pp 21 woensdag 23 mei 12

Slide 50

Slide 50 text

➡ Catalogs are “compiled” manifests ➡ Manifests are puppet definitions ➡ <filename>.pp ➡ Puppet DSL 21 woensdag 23 mei 12

Slide 51

Slide 51 text

➡ Catalogs are “compiled” manifests ➡ Manifests are puppet definitions ➡ <filename>.pp ➡ Puppet DSL ➡ De-cla-ra-tive language 21 woensdag 23 mei 12

Slide 52

Slide 52 text

➡ Catalogs are “compiled” manifests ➡ Manifests are puppet definitions ➡ <filename>.pp ➡ Puppet DSL ➡ De-cla-ra-tive language ➡ Version your manifests! (git/svn) 21 woensdag 23 mei 12

Slide 53

Slide 53 text

package { “strace” : ensure => present, } file { “/home/jaytaph/secret-ingredient.txt” : ensure => present, mode => 0600, user => ‘jaytaph’, group => ‘noxlogic’, content => “beer”, } 22 woensdag 23 mei 12

Slide 54

Slide 54 text

package { “httpd” : ensure => present, } service { “httpd”: running => true, enable => true, } 23 woensdag 23 mei 12

Slide 55

Slide 55 text

package { “httpd” : ensure => present, } service { “httpd”: running => true, enable => true, } require => Package[“httpd”], 23 woensdag 23 mei 12

Slide 56

Slide 56 text

‣ Different distributions, different names Centos / Redhat service: httpd package: httpd config: /etc/httpd/conf/httpd.conf vhosts: /etc/httpd/conf.d/*.conf Debian / Ubuntu service: apache2 package: apache2 config: /etc/apache2/httpd.conf vhosts: /etc/apache2/sites-available 24 woensdag 23 mei 12

Slide 57

Slide 57 text

class apache { package { “apache”: case $operatingsystem { centos, redhat { $packagename = “httpd” } debian, ubuntu { $packagename = “apache2” } default : { fail(‘I don’t know this OS/distro’) } } name => $packagename, ensure => installed, } service { “apache” : running => true, enable => true, require => Package[“apache”], } } 25 woensdag 23 mei 12

Slide 58

Slide 58 text

[root@puppetnode1 ~]# facter --puppet architecture => x86_64 fqdn => puppetnode1.noxlogic.local interfaces => eth1,eth2,lo ipaddress_eth1 => 192.168.1.114 ipaddress_eth2 => 192.168.56.200 kernel => Linux kernelmajversion => 2.6 operatingsystem => CentOS operatingsystemrelease => 6.0 processor0 => Intel(R) Core(TM)2 Duo CPU T7500 @ 2.20GHz puppetversion => 2.6.9 ‣ A simple list with info (also useable in your own tools) 26 woensdag 23 mei 12

Slide 59

Slide 59 text

node “web01.example.org” { include apache } node /^db\d+\.example\.org$/ { package { “mysql-server” : ensure => installed, } } 27 /etc/puppet/manifests/site.pp: woensdag 23 mei 12

Slide 60

Slide 60 text

node “web01.example.local” { $webserver_name = “web01.example.local” $webserver_alias = “www.example.local” $webserver_docroot = “/var/www/web01” include apache } node “web02.example.local” { $webserver_name = “web02.example.local” $webserver_alias = “crm.example.local” $webserver_docroot = “/var/www/web02” include apache } 28 woensdag 23 mei 12

Slide 61

Slide 61 text

http://docs.puppetlabs.com/references/stable/type.html 29 What can Puppet configure / control? woensdag 23 mei 12

Slide 62

Slide 62 text

http://docs.puppetlabs.com/references/stable/type.html ➡ Almost everything. ➡ Standard 48 different resource types ➡ Ranging from “file” to “cron” to “ssh_key” to “user” to “selinux”. ➡ Can control your Cisco routers and windows machines too (sortakinda) 30 woensdag 23 mei 12

Slide 63

Slide 63 text

31 class joindin::web { include apache # include phpmyadmin if needed if $params::phpmyadmin == true { include joindin::web::phpmyadmin } # Configure apache virtual host apache::vhost { $params::host : docroot => '/vagrant/src', template => 'joindin/vhost.conf.erb', port => $params::port, require => Package["apache"], } https://github.com/jaytaph/joind.in/tree/puppet woensdag 23 mei 12

Slide 64

Slide 64 text

32 ... # Install PHP modules php::module { 'mysql': } php::module { "pecl-xdebug" : require => File["EpelRepo"], # xdebug is in the epel repo } # Set development values to our php.ini augeas { 'set-php-ini-values': context => '/files/etc/php.ini', changes => [ 'set PHP/error_reporting "E_ALL | E_STRICT"', 'set PHP/display_errors On', 'set PHP/display_startup_errors On', 'set PHP/html_errors On', 'set Date/date.timezone Europe/London', ], require => Package['php'], notify => Service['apache'], } } # End class woensdag 23 mei 12

Slide 65

Slide 65 text

33 ➡ Puppet went from v0.25 to v2.6. ➡ REST interface since 2.6. XMLRPC before that. ➡ One binary to rule them all (puppet). ➡ Puppet v2.7 switched from GPLv2 to apache2.0 license. woensdag 23 mei 12

Slide 66

Slide 66 text

34 So how does Puppet benefit me as a DEVELOPER? woensdag 23 mei 12

Slide 67

Slide 67 text

35 ➡ Keep all developers in sync ➡ Keep your DTAP in sync ➡ Lets infrastructure be a part of your project woensdag 23 mei 12

Slide 68

Slide 68 text

Vagrant http://vagrantup.com/ http://vagrantup.com/images/vagrant_chilling.png 36 woensdag 23 mei 12

Slide 69

Slide 69 text

Vagrant is a tool for building and distributing virtualized development environments. 37 woensdag 23 mei 12

Slide 70

Slide 70 text

Vagrant::Config.run do |config| config.vm.box = 'centos-62-64-puppet' config.vm.box_url = 'http://../centos-6.2-64bit-puppet-vbox.4.1.12.box' # Forward a port from the guest to the host, which allows for outside # computers to access the VM, whereas host only networking does not. config.vm.forward_port 80, 8080 config.vm.provision :puppet do |puppet| puppet.manifests_path = "puppet/manifests" puppet.module_path = "puppet/modules" puppet.manifest_file = "main.pp" puppet.options = [ '--verbose', ] end end Vagrantfile 38 woensdag 23 mei 12

Slide 71

Slide 71 text

# git clone [email protected]:jaytaph/myproject.git # vagrant up 39 woensdag 23 mei 12

Slide 72

Slide 72 text

➡ Downloads (optionally) the base box ➡ Deploys and boots up a new VM ➡ Runs the provisioner (puppet) ➡ Profit! 40 woensdag 23 mei 12

Slide 73

Slide 73 text

Multi VM’s Vagrant::Config.run do |config| config.vm.box = 'centos-62-64-puppet' config.vm.box_url = 'http://../centos-6.2-64bit-puppet-vbox.4.1.12.box' config.vm.define :web do |web_config| web_config.vm.host_name = 'web.example.org' web_config.vm.forward_port 80 8080 ... end config.vm.define :database do |db_config| db_config.vm.host_name = 'db.example.org' db_config.vm.forward_port 3306 3306 ... end end Vagrantfile 41 woensdag 23 mei 12

Slide 74

Slide 74 text

42 woensdag 23 mei 12

Slide 75

Slide 75 text

➡ Puppet agent “calls” the master every 30 minutes. ➡ But what about realtime command & control? ➡ “Puppet kick”... (meh) ➡ MCollective (Marionette Collective) 43 woensdag 23 mei 12

Slide 76

Slide 76 text

➡ Which systems running a database and have 16GB or less? ➡ Which systems are using <50% of available memory? ➡ Restart all apache services in timezone GMT+5. 44 woensdag 23 mei 12

Slide 77

Slide 77 text

ACTIVEMQ Client MCollective Server Node Middleware Client MCollective Server MCollective Server ‣ Middleware takes care of distribution, ‣ queued, broadcast etc.. Collective 45 woensdag 23 mei 12

Slide 78

Slide 78 text

http://docs.puppetlabs.com/mcollective/reference/basic/subcollectives.html 46 woensdag 23 mei 12

Slide 79

Slide 79 text

Filter out nodes based on facts $ mc-facts operatingsystem Report for fact: operatingsystem CentOS found 3 times Debian found 14 times Solaris found 4 times $ mc-facts -W operatingsystem=Centos operatingsystemrelease Report for fact: operatingsystemrelease 6.0 found 1 times 5.6 found 2 times 47 woensdag 23 mei 12

Slide 80

Slide 80 text

➡ Display all running processes ➡ Run or deploy software ➡ Restart services ➡ Start puppet agent ➡ Upgrade your systems ➡ Write your own agents! 48 woensdag 23 mei 12

Slide 81

Slide 81 text

-ETOOMUCHINFO Let’s recap 49 woensdag 23 mei 12

Slide 82

Slide 82 text

➡ Configuration management tool. ➡ Focusses on “what” instead of “how”. ➡ Scales from 1 to 100K+ systems. ➡ Uses descriptive manifests. ➡ Vagrant for setting up your development environments. 50 woensdag 23 mei 12

Slide 83

Slide 83 text

➡ Useful for sysadmins and developers. ➡ Keeps your infrastructure in sync. ➡ Keeps your infrastructure versioned. ➡ Infrastructure as part of your projects. ➡ MCollective controls your hosts based on facts, not names. 51 woensdag 23 mei 12

Slide 84

Slide 84 text

There is no reason NOT to manage your infrastructure. Having only 3 servers is NOT a reason. 52 You will be able to join the rest of us in the pub early. Don’t “install” development environments, build them! woensdag 23 mei 12

Slide 85

Slide 85 text

http://farm1.static.flickr.com/73/163450213_18478d3aa6_d.jpg 53 woensdag 23 mei 12

Slide 86

Slide 86 text

Please rate my talk on joind.in: http://joind.in/6515 Thank you 54 Find me on twitter: @jaytaph Find me for development and training: www.noxlogic.nl Find me on email: [email protected] Find me for blogs: www.adayinthelifeof.nl woensdag 23 mei 12