Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Puppet for dummies - PHP|Tek 2012

Puppet for dummies - PHP|Tek 2012

Joshua Thijssen

May 23, 2012
Tweet

More Decks by Joshua Thijssen

Other Decks in Technology

Transcript

  1. Puppet for
    Dummies
    PHP|Tek - Chicago - USA
    May 23, 2012
    woensdag 23 mei 12

    View Slide

  2. Joshua Thijssen
    Freelance consultant, developer and
    trainer @ NoxLogic / Techademy
    Development in PHP, Python, Perl,
    C, Java and some sysadmin
    Blog: http://adayinthelifeof.nl
    Email: [email protected]
    Twitter: @jaytaph
    oh hai!
    2
    woensdag 23 mei 12

    View Slide

  3. What is puppet and why should I care?
    3
    woensdag 23 mei 12

    View Slide

  4. “People are finally figuring out puppet and
    how it gets you to the pub by 4pm.
    Note that I’ve been at this pub since 2pm.”
    - Jorge Castro
    4
    woensdag 23 mei 12

    View Slide

  5. 5
    woensdag 23 mei 12

    View Slide

  6. Puppet is a (not necessarily the) solution for
    the following problem:
    How do we setup, manage, synchronize,
    and upgrade our internal and external
    infrastructure?
    6
    woensdag 23 mei 12

    View Slide

  7. Sysadmin!
    Y U no fix problem!
    7
    woensdag 23 mei 12

    View Slide

  8. Sysadmin!
    Y U no fix problem!
    NO
    7
    woensdag 23 mei 12

    View Slide

  9. LAMP-stack
    8
    woensdag 23 mei 12

    View Slide

  10. LAMP-stack
    Linux
    Apache
    MySQL
    PHP
    8
    woensdag 23 mei 12

    View Slide

  11. LAMPGMVNMCSTRAH-stack
    9
    woensdag 23 mei 12

    View Slide

  12. LAMPGMVNMCSTRAH-stack
    Linux
    Apache
    MySQL
    PHP
    Gearman
    MongoDB
    CouchDB
    Solr
    Tika
    Redis
    ActiveMQ
    Hadoop
    Varnish
    Ngnix
    Memcache
    9
    woensdag 23 mei 12

    View Slide

  13. 10
    woensdag 23 mei 12

    View Slide

  14. 10
    How do we control our infrastructure?
    woensdag 23 mei 12

    View Slide

  15. ➡ Solution 1: We don’t,
    10
    How do we control our infrastructure?
    woensdag 23 mei 12

    View Slide

  16. ➡ Solution 1: We don’t,
    ➡ Solution 2: We outsource,
    10
    How do we control our infrastructure?
    woensdag 23 mei 12

    View Slide

  17. ➡ Solution 1: We don’t,
    ➡ Solution 2: We outsource,
    ➡ Solution 3: We automate the process.
    10
    How do we control our infrastructure?
    woensdag 23 mei 12

    View Slide

  18. ‣ Solution 1: we don’t
    11
    woensdag 23 mei 12

    View Slide

  19. ➡ It’s not funny: you find it more often
    than not. Especially inside small
    development companies.
    ‣ Solution 1: we don’t
    11
    woensdag 23 mei 12

    View Slide

  20. ➡ It’s not funny: you find it more often
    than not. Especially inside small
    development companies.
    ➡ Internal sysadmin, but he’s too busy
    with development to do sysadmin.
    ‣ Solution 1: we don’t
    11
    woensdag 23 mei 12

    View Slide

  21. ➡ It’s not funny: you find it more often
    than not. Especially inside small
    development companies.
    ➡ Internal sysadmin, but he’s too busy
    with development to do sysadmin.
    ➡ We only act on escalation
    ‣ Solution 1: we don’t
    11
    woensdag 23 mei 12

    View Slide

  22. ➡ It’s not funny: you find it more often
    than not. Especially inside small
    development companies.
    ➡ Internal sysadmin, but he’s too busy
    with development to do sysadmin.
    ➡ We only act on escalation
    ➡ reactive, not proactive
    ‣ Solution 1: we don’t
    11
    woensdag 23 mei 12

    View Slide

  23. ‣ Solution 2: we outsource
    12
    woensdag 23 mei 12

    View Slide

  24. ➡ Expensive $LA’s.
    ‣ Solution 2: we outsource
    12
    woensdag 23 mei 12

    View Slide

  25. ➡ Expensive $LA’s.
    ➡ What about INTERNAL servers like your
    development systems and
    infrastructure?
    ‣ Solution 2: we outsource
    12
    woensdag 23 mei 12

    View Slide

  26. ➡ Expensive $LA’s.
    ➡ What about INTERNAL servers like your
    development systems and
    infrastructure?
    ➡ Fight between stability and agility.
    ‣ Solution 2: we outsource
    12
    woensdag 23 mei 12

    View Slide

  27. ➡ Expensive $LA’s.
    ➡ What about INTERNAL servers like your
    development systems and
    infrastructure?
    ➡ Fight between stability and agility.
    ➡ Does your hosting company decide on
    whether you can use PHP5.3???
    ‣ Solution 2: we outsource
    12
    woensdag 23 mei 12

    View Slide

  28. ‣ Solution 3: we do it ourselves and automate
    13
    woensdag 23 mei 12

    View Slide

  29. ➡ We are in charge.
    ‣ Solution 3: we do it ourselves and automate
    13
    woensdag 23 mei 12

    View Slide

  30. ➡ We are in charge.
    ➡ You can do what you like
    ‣ Solution 3: we do it ourselves and automate
    13
    woensdag 23 mei 12

    View Slide

  31. ➡ We are in charge.
    ➡ You can do what you like
    ➡ Use: cfEngine, chef, puppet.
    ‣ Solution 3: we do it ourselves and automate
    13
    woensdag 23 mei 12

    View Slide

  32. ➡ We are in charge.
    ➡ You can do what you like
    ➡ Use: cfEngine, chef, puppet.
    ➡ When done right, maintenance should
    not be difficult.
    ‣ Solution 3: we do it ourselves and automate
    13
    woensdag 23 mei 12

    View Slide

  33. PUPPET
    14
    woensdag 23 mei 12

    View Slide

  34. ➡ Open source configuration management tool.
    ➡ Written in Ruby
    ➡ Open source: https://github.com/puppetlabs
    ➡ Commercial version available (puppet enterprise)
    15
    woensdag 23 mei 12

    View Slide

  35. ➡ Don’t tell HOW to do stuff.
    ➡ Tell WHAT to do.
    ¹
    ¹ It’s not actually true, but good enough for now...
    16
    woensdag 23 mei 12

    View Slide

  36. ➡ Don’t tell HOW to do stuff.
    ➡ Tell WHAT to do.
    ¹
    ¹ It’s not actually true, but good enough for now...
    “yum install httpd”
    “apt-get install apache2”
    16
    woensdag 23 mei 12

    View Slide

  37. ➡ Don’t tell HOW to do stuff.
    ➡ Tell WHAT to do.
    ¹
    ¹ It’s not actually true, but good enough for now...
    “yum install httpd”
    “apt-get install apache2”
    “install and run the apache webserver”
    16
    woensdag 23 mei 12

    View Slide

  38. 17
    Schematic representation of a puppet infrastructure
    woensdag 23 mei 12

    View Slide

  39. Puppet
    18
    woensdag 23 mei 12

    View Slide

  40. Puppet CA
    Puppet
    Master
    Puppet
    Agent
    https
    19
    woensdag 23 mei 12

    View Slide

  41. Puppet CA
    Puppet
    Master
    Puppet
    Agent
    Puppet
    Agent
    Puppet
    Agent
    https
    19
    woensdag 23 mei 12

    View Slide

  42. ➡ Agent “calls” the puppet master.
    20
    woensdag 23 mei 12

    View Slide

  43. ➡ Agent “calls” the puppet master.
    ➡ Agent sends “facts” to the master.
    20
    woensdag 23 mei 12

    View Slide

  44. ➡ Agent “calls” the puppet master.
    ➡ Agent sends “facts” to the master.
    ➡ Master creates “catalog” from the manifests
    and facts, sends to agent.
    20
    woensdag 23 mei 12

    View Slide

  45. ➡ Agent “calls” the puppet master.
    ➡ Agent sends “facts” to the master.
    ➡ Master creates “catalog” from the manifests
    and facts, sends to agent.
    ➡ Agent sets up system according to the
    catalog.
    20
    woensdag 23 mei 12

    View Slide

  46. ➡ Agent “calls” the puppet master.
    ➡ Agent sends “facts” to the master.
    ➡ Master creates “catalog” from the manifests
    and facts, sends to agent.
    ➡ Agent sets up system according to the
    catalog.
    ➡ Agent reports status to master.
    20
    woensdag 23 mei 12

    View Slide

  47. ➡ Catalogs are “compiled” manifests
    21
    woensdag 23 mei 12

    View Slide

  48. ➡ Catalogs are “compiled” manifests
    ➡ Manifests are puppet definitions
    21
    woensdag 23 mei 12

    View Slide

  49. ➡ Catalogs are “compiled” manifests
    ➡ Manifests are puppet definitions
    ➡ <filename>.pp
    21
    woensdag 23 mei 12

    View Slide

  50. ➡ Catalogs are “compiled” manifests
    ➡ Manifests are puppet definitions
    ➡ <filename>.pp
    ➡ Puppet DSL
    21
    woensdag 23 mei 12

    View Slide

  51. ➡ Catalogs are “compiled” manifests
    ➡ Manifests are puppet definitions
    ➡ <filename>.pp
    ➡ Puppet DSL
    ➡ De-cla-ra-tive language
    21
    woensdag 23 mei 12

    View Slide

  52. ➡ Catalogs are “compiled” manifests
    ➡ Manifests are puppet definitions
    ➡ <filename>.pp
    ➡ Puppet DSL
    ➡ De-cla-ra-tive language
    ➡ Version your manifests! (git/svn)
    21
    woensdag 23 mei 12

    View Slide

  53. package { “strace” :
    ensure => present,
    }
    file { “/home/jaytaph/secret-ingredient.txt” :
    ensure => present,
    mode => 0600,
    user => ‘jaytaph’,
    group => ‘noxlogic’,
    content => “beer”,
    }
    22
    woensdag 23 mei 12

    View Slide

  54. package { “httpd” :
    ensure => present,
    }
    service { “httpd”:
    running => true,
    enable => true,
    }
    23
    woensdag 23 mei 12

    View Slide

  55. package { “httpd” :
    ensure => present,
    }
    service { “httpd”:
    running => true,
    enable => true,
    }
    require => Package[“httpd”],
    23
    woensdag 23 mei 12

    View Slide

  56. ‣ Different distributions, different names
    Centos / Redhat
    service: httpd
    package: httpd
    config: /etc/httpd/conf/httpd.conf
    vhosts: /etc/httpd/conf.d/*.conf
    Debian / Ubuntu
    service: apache2
    package: apache2
    config: /etc/apache2/httpd.conf
    vhosts: /etc/apache2/sites-available
    24
    woensdag 23 mei 12

    View Slide

  57. class apache {
    package { “apache”:
    case $operatingsystem {
    centos, redhat { $packagename = “httpd” }
    debian, ubuntu { $packagename = “apache2” }
    default : { fail(‘I don’t know this OS/distro’) }
    }
    name => $packagename,
    ensure => installed,
    }
    service { “apache” :
    running => true,
    enable => true,
    require => Package[“apache”],
    }
    }
    25
    woensdag 23 mei 12

    View Slide

  58. [[email protected] ~]# facter --puppet
    architecture => x86_64
    fqdn => puppetnode1.noxlogic.local
    interfaces => eth1,eth2,lo
    ipaddress_eth1 => 192.168.1.114
    ipaddress_eth2 => 192.168.56.200
    kernel => Linux
    kernelmajversion => 2.6
    operatingsystem => CentOS
    operatingsystemrelease => 6.0
    processor0 => Intel(R) Core(TM)2 Duo CPU T7500 @ 2.20GHz
    puppetversion => 2.6.9
    ‣ A simple list with info (also useable in your own tools)
    26
    woensdag 23 mei 12

    View Slide

  59. node “web01.example.org” {
    include apache
    }
    node /^db\d+\.example\.org$/ {
    package { “mysql-server” :
    ensure => installed,
    }
    }
    27
    /etc/puppet/manifests/site.pp:
    woensdag 23 mei 12

    View Slide

  60. node “web01.example.local” {
    $webserver_name = “web01.example.local”
    $webserver_alias = “www.example.local”
    $webserver_docroot = “/var/www/web01”
    include apache
    }
    node “web02.example.local” {
    $webserver_name = “web02.example.local”
    $webserver_alias = “crm.example.local”
    $webserver_docroot = “/var/www/web02”
    include apache
    }
    28
    woensdag 23 mei 12

    View Slide

  61. http://docs.puppetlabs.com/references/stable/type.html
    29
    What can Puppet configure / control?
    woensdag 23 mei 12

    View Slide

  62. http://docs.puppetlabs.com/references/stable/type.html
    ➡ Almost everything.
    ➡ Standard 48 different resource types
    ➡ Ranging from “file” to “cron” to “ssh_key”
    to “user” to “selinux”.
    ➡ Can control your Cisco routers and
    windows machines too (sortakinda)
    30
    woensdag 23 mei 12

    View Slide

  63. 31
    class joindin::web {
    include apache
    # include phpmyadmin if needed
    if $params::phpmyadmin == true {
    include joindin::web::phpmyadmin
    }
    # Configure apache virtual host
    apache::vhost { $params::host :
    docroot => '/vagrant/src',
    template => 'joindin/vhost.conf.erb',
    port => $params::port,
    require => Package["apache"],
    }
    https://github.com/jaytaph/joind.in/tree/puppet
    woensdag 23 mei 12

    View Slide

  64. 32
    ...
    # Install PHP modules
    php::module { 'mysql': }
    php::module { "pecl-xdebug" :
    require => File["EpelRepo"], # xdebug is in the epel repo
    }
    # Set development values to our php.ini
    augeas { 'set-php-ini-values':
    context => '/files/etc/php.ini',
    changes => [
    'set PHP/error_reporting "E_ALL | E_STRICT"',
    'set PHP/display_errors On',
    'set PHP/display_startup_errors On',
    'set PHP/html_errors On',
    'set Date/date.timezone Europe/London',
    ],
    require => Package['php'],
    notify => Service['apache'],
    }
    } # End class
    woensdag 23 mei 12

    View Slide

  65. 33
    ➡ Puppet went from v0.25 to v2.6.
    ➡ REST interface since 2.6. XMLRPC before
    that.
    ➡ One binary to rule them all (puppet).
    ➡ Puppet v2.7 switched from GPLv2 to
    apache2.0 license.
    woensdag 23 mei 12

    View Slide

  66. 34
    So how does Puppet benefit me as a
    DEVELOPER?
    woensdag 23 mei 12

    View Slide

  67. 35
    ➡ Keep all developers in sync
    ➡ Keep your DTAP in sync
    ➡ Lets infrastructure be a part of your
    project
    woensdag 23 mei 12

    View Slide

  68. Vagrant
    http://vagrantup.com/
    http://vagrantup.com/images/vagrant_chilling.png 36
    woensdag 23 mei 12

    View Slide

  69. Vagrant is a tool for building and
    distributing virtualized development
    environments.
    37
    woensdag 23 mei 12

    View Slide

  70. Vagrant::Config.run do |config|
    config.vm.box = 'centos-62-64-puppet'
    config.vm.box_url = 'http://../centos-6.2-64bit-puppet-vbox.4.1.12.box'
    # Forward a port from the guest to the host, which allows for outside
    # computers to access the VM, whereas host only networking does not.
    config.vm.forward_port 80, 8080
    config.vm.provision :puppet do |puppet|
    puppet.manifests_path = "puppet/manifests"
    puppet.module_path = "puppet/modules"
    puppet.manifest_file = "main.pp"
    puppet.options = [
    '--verbose',
    ]
    end
    end
    Vagrantfile
    38
    woensdag 23 mei 12

    View Slide

  71. # git clone [email protected]:jaytaph/myproject.git
    # vagrant up
    39
    woensdag 23 mei 12

    View Slide

  72. ➡ Downloads (optionally) the base box
    ➡ Deploys and boots up a new VM
    ➡ Runs the provisioner (puppet)
    ➡ Profit!
    40
    woensdag 23 mei 12

    View Slide

  73. Multi VM’s
    Vagrant::Config.run do |config|
    config.vm.box = 'centos-62-64-puppet'
    config.vm.box_url = 'http://../centos-6.2-64bit-puppet-vbox.4.1.12.box'
    config.vm.define :web do |web_config|
    web_config.vm.host_name = 'web.example.org'
    web_config.vm.forward_port 80 8080
    ...
    end
    config.vm.define :database do |db_config|
    db_config.vm.host_name = 'db.example.org'
    db_config.vm.forward_port 3306 3306
    ...
    end
    end
    Vagrantfile
    41
    woensdag 23 mei 12

    View Slide

  74. 42
    woensdag 23 mei 12

    View Slide

  75. ➡ Puppet agent “calls” the master every 30 minutes.
    ➡ But what about realtime command & control?
    ➡ “Puppet kick”... (meh)
    ➡ MCollective (Marionette Collective)
    43
    woensdag 23 mei 12

    View Slide

  76. ➡ Which systems running a database and have
    16GB or less?
    ➡ Which systems are using <50% of available
    memory?
    ➡ Restart all apache services in timezone
    GMT+5.
    44
    woensdag 23 mei 12

    View Slide

  77. ACTIVEMQ
    Client
    MCollective
    Server
    Node
    Middleware
    Client
    MCollective
    Server
    MCollective
    Server
    ‣ Middleware takes care of distribution,
    ‣ queued, broadcast etc..
    Collective
    45
    woensdag 23 mei 12

    View Slide

  78. http://docs.puppetlabs.com/mcollective/reference/basic/subcollectives.html
    46
    woensdag 23 mei 12

    View Slide

  79. Filter out nodes based on facts
    $ mc-facts operatingsystem
    Report for fact: operatingsystem
    CentOS found 3 times
    Debian found 14 times
    Solaris found 4 times
    $ mc-facts -W operatingsystem=Centos operatingsystemrelease
    Report for fact: operatingsystemrelease
    6.0 found 1 times
    5.6 found 2 times
    47
    woensdag 23 mei 12

    View Slide

  80. ➡ Display all running processes
    ➡ Run or deploy software
    ➡ Restart services
    ➡ Start puppet agent
    ➡ Upgrade your systems
    ➡ Write your own agents!
    48
    woensdag 23 mei 12

    View Slide

  81. -ETOOMUCHINFO
    Let’s recap
    49
    woensdag 23 mei 12

    View Slide

  82. ➡ Configuration management tool.
    ➡ Focusses on “what” instead of “how”.
    ➡ Scales from 1 to 100K+ systems.
    ➡ Uses descriptive manifests.
    ➡ Vagrant for setting up your development
    environments.
    50
    woensdag 23 mei 12

    View Slide

  83. ➡ Useful for sysadmins and developers.
    ➡ Keeps your infrastructure in sync.
    ➡ Keeps your infrastructure versioned.
    ➡ Infrastructure as part of your projects.
    ➡ MCollective controls your hosts based
    on facts, not names.
    51
    woensdag 23 mei 12

    View Slide

  84. There is no reason NOT to manage your infrastructure.
    Having only 3 servers is NOT a reason.
    52
    You will be able to join the rest of us in the pub early.
    Don’t “install” development environments, build them!
    woensdag 23 mei 12

    View Slide

  85. http://farm1.static.flickr.com/73/163450213_18478d3aa6_d.jpg 53
    woensdag 23 mei 12

    View Slide

  86. Please rate my talk on joind.in:
    http://joind.in/6515
    Thank you
    54
    Find me on twitter: @jaytaph
    Find me for development and training: www.noxlogic.nl
    Find me on email: [email protected]
    Find me for blogs: www.adayinthelifeof.nl
    woensdag 23 mei 12

    View Slide