Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Puppet for dummies - DPC 12

Puppet for dummies - DPC 12

Joshua Thijssen

May 09, 2012
Tweet

More Decks by Joshua Thijssen

Other Decks in Technology

Transcript

  1. Puppet for
    Dummies
    DPC - Amsterdam - Netherlands
    9 may 2012

    View Slide

  2. Joshua Thijssen
    Freelance consultant, developer and
    trainer @ NoxLogic / Techademy
    Development in PHP, Python, Perl,
    C, Java and some sysadmin
    Blog: http://adayinthelifeof.nl
    Email: [email protected]
    Twitter: @jaytaph
    oh hai!
    2

    View Slide

  3. What is puppet and why should I care?
    3

    View Slide

  4. “People are finally figuring out puppet and
    how it gets you to the pub by 4pm.
    Note that I’ve been at this pub since 2pm.”
    - Jorge Castro
    4

    View Slide

  5. 5

    View Slide

  6. Puppet is a (not necessarily the) solution for
    the following problem:
    How do we setup, manage, synchronize,
    and upgrade our internal and external
    infrastructure?
    6

    View Slide

  7. Sysadmin!
    Y U no fix problem!
    NO
    7

    View Slide

  8. LAMP-stack
    Linux
    Apache
    MySQL
    PHP
    8

    View Slide

  9. LAMPGMVNMCSTRAH-stack
    Linux
    Apache
    MySQL
    PHP
    Gearman
    MongoDB
    CouchDB
    Solr
    Tika
    Redis
    ActiveMQ
    Hadoop
    Varnish
    Ngnix
    Memcache
    9

    View Slide

  10. Your application
    is not special
    anymore.
    You are just
    another cog in
    the machine
    10
    http://www.sxc.hu/photo/400612

    View Slide

  11. ➡ Solution 1: We don’t,
    ➡ Solution 2: We outsource,
    ➡ Solution 3: We automate the process.
    11
    How do we control our infrastructure?

    View Slide

  12. ➡ It’s not funny: you find it more often
    than not. Especially inside small
    development companies.
    ➡ Internal sysadmin, but he’s too busy
    with development to do sysadmin.
    ➡ We only act on escalation
    ➡ reactive, not proactive
    ‣ Solution 1: we don’t
    12

    View Slide

  13. ➡ Expensive $LA’s.
    ➡ What about INTERNAL servers like your
    development systems and
    infrastructure?
    ➡ Fight between stability and agility.
    ➡ Does your hosting company decide on
    whether you can use PHP5.3???
    ‣ Solution 2: we outsource
    13

    View Slide

  14. ➡ We are in charge.
    ➡ You can do what you like
    ➡ Use: cfEngine, chef, puppet.
    ➡ When done right, maintenance should
    not be difficult.
    ‣ Solution 3: we do it ourselves and automate
    14

    View Slide

  15. PUPPET
    15

    View Slide

  16. ➡ Open source configuration management tool.
    ➡ Puppet Labs (Reductive Labs)
    ➡ Written in Ruby
    ➡ Open source: https://github.com/puppetlabs
    ➡ Commercial version available (puppet enterprise)
    16

    View Slide

  17. ➡ Don’t tell HOW to do stuff.
    ➡ Tell WHAT to do.
    ¹
    ¹ It’s not actually true, but good enough for now...
    “yum install httpd”
    “apt-get install apache2”
    “install and run the apache webserver”
    17

    View Slide

  18. 18
    Schematic representation of a puppet infrastructure

    View Slide

  19. Puppet
    19

    View Slide

  20. Puppet CA
    Puppet
    Master
    Puppet
    Agent
    Puppet
    Agent
    Puppet
    Agent
    https
    20

    View Slide

  21. ➡ Agent “calls” the puppet master.
    ➡ Agent sends “facts” to the master.
    ➡ Master creates “catalog” from the manifests
    and facts, sends to agent.
    ➡ Agent sets up system according to the
    catalog.
    ➡ Agent reports status to master.
    21

    View Slide

  22. ➡ Catalogs are “compiled” manifests
    ➡ Manifests are puppet definitions
    ➡ <filename>.pp
    ➡ Puppet DSL
    ➡ De-cla-ra-tive language
    ➡ Version your manifests! (git/svn)
    22

    View Slide

  23. package { “strace” :
    ensure => present,
    }
    file { “/home/jaytaph/secret-ingredient.txt” :
    ensure => present,
    mode => 0600,
    user => ‘jaytaph’,
    group => ‘noxlogic’,
    content => “beer”,
    }
    23

    View Slide

  24. package { “httpd” :
    ensure => present,
    }
    service { “httpd”:
    running => true,
    enable => true,
    }
    require => Package[“httpd”],
    24

    View Slide

  25. Centos / Redhat
    service: httpd
    package: httpd
    config: /etc/httpd/conf/httpd.conf
    vhosts: /etc/httpd/conf.d/*.conf
    Debian / Ubuntu
    service: apache2
    package: apache2
    config: /etc/apache2/httpd.conf
    vhosts: /etc/apache2/sites-available
    25

    View Slide

  26. class webserver {
    package { “apache”:
    case $operatingsystem {
    centos, redhat { $packagename = “httpd” }
    debian, ubuntu { $packagename = “apache2” }
    default : { fail(‘I don’t know this OS/distro’) }
    }
    name => $packagename,
    ensure => installed,
    }
    service { “apache” :
    running => true,
    enable => true,
    require => Package[“apache”],
    }
    }
    26

    View Slide

  27. [[email protected] ~]# facter --puppet
    architecture => x86_64
    fqdn => puppetnode1.noxlogic.local
    interfaces => eth1,eth2,lo
    ipaddress_eth1 => 192.168.1.114
    ipaddress_eth2 => 192.168.56.200
    kernel => Linux
    kernelmajversion => 2.6
    operatingsystem => CentOS
    operatingsystemrelease => 6.0
    processor0 => Intel(R) Core(TM)2 Duo CPU T7500 @ 2.20GHz
    puppetversion => 2.6.9
    27

    View Slide

  28. node “web01.example.org” {
    include webserver
    }
    node /^db\d+\.example\.org$/ {
    package { “mysql-server” :
    ensure => installed,
    }
    }
    28
    /etc/puppet/manifests/site.pp:

    View Slide

  29. node “web01.example.local” {
    $webserver_name = “web01.example.local”
    $webserver_alias = “www.example.local”
    $webserver_docroot = “/var/www/web01”
    include webserver
    }
    node “web02.example.local” {
    $webserver_name = “web02.example.local”
    $webserver_alias = “crm.example.local”
    $webserver_docroot = “/var/www/web02”
    include webserver
    }
    29

    View Slide

  30. 30
    What can Puppet configure / control?

    View Slide

  31. http://docs.puppetlabs.com/references/stable/type.html
    ➡ (Almost) everything.
    ➡ Standard 48 different resource types
    ➡ Ranging from “file” to “cron” to “ssh_key”
    to “user” to “selinux”.
    ➡ Can control your Cisco routers and
    windows machines too (sortakinda)
    31

    View Slide

  32. 32
    https://github.com/joindin/joind.in

    View Slide

  33. 33
    class params {
    $host = 'dev.joind.in'
    $port = '80'
    # Database credentials.
    #Test database will be setup with _test prefixed to these variables.
    $dbname = 'joindin'
    $dbuser = 'joindin'
    $dbpass = 'password'
    # True if phpmyadmin needs to be installed, false if not.
    $phpmyadmin = true
    # Uncomment to turn on debugging
    #$debug = 'on'
    }
    manifests/params.pp

    View Slide

  34. 34
    node default {
    include params
    include joindin
    }
    manifests/joindin.pp

    View Slide

  35. 35

    View Slide

  36. 36
    modules/joindin/manifests/init.pp
    class joindin {
    include joindin::setup
    include joindin::sql
    include joindin::web
    include joindin::app
    }

    View Slide

  37. 37
    class joindin::web {
    include apache
    # include phpmyadmin if needed
    if $params::phpmyadmin == true {
    include joindin::web::phpmyadmin
    }
    # Configure apache virtual host
    apache::vhost { $params::host :
    docroot => '/vagrant/src',
    template => 'joindin/vhost.conf.erb',
    port => $params::port,
    require => Package["apache"],
    }
    modules/joindin/manifests/web.pp

    View Slide

  38. 38
    ...
    # Install PHP modules
    php::module { 'mysql': }
    php::module { "pecl-xdebug" :
    require => File["EpelRepo"], # xdebug is in the epel repo
    }
    # Set development values to our php.ini
    augeas { 'set-php-ini-values':
    context => '/files/etc/php.ini',
    changes => [
    'set PHP/error_reporting "E_ALL | E_STRICT"',
    'set PHP/display_errors On',
    'set PHP/display_startup_errors On',
    'set PHP/html_errors On',
    'set Date/date.timezone Europe/London',
    ],
    require => Package['php'],
    notify => Service['apache'],
    }
    } # End class

    View Slide

  39. 39
    ➡ Puppet went from v0.25 to v2.6.
    ➡ REST interface since 2.6. XMLRPC before
    that.
    ➡ One binary to rule them all (puppet).
    ➡ Puppet v2.7 switched from GPLv2 to
    apache2.0 license.

    View Slide

  40. 40
    So how does Puppet benefit me as a
    DEVELOPER?

    View Slide

  41. 41
    ➡ Keep all developers in sync
    ➡ Keep your DTAP in sync
    ➡ Lets infrastructure be a part of your
    project

    View Slide

  42. Vagrant
    http://vagrantup.com/
    http://vagrantup.com/images/vagrant_chilling.png 42

    View Slide

  43. Vagrant is a tool for building and
    distributing virtualized development
    environments.
    43

    View Slide

  44. Vagrant::Config.run do |config|
    config.vm.box = 'centos-62-64-puppet'
    config.vm.box_url = 'http://../centos-6.2-64bit-puppet-vbox.4.1.12.box'
    # Forward a port from the guest to the host, which allows for outside
    # computers to access the VM, whereas host only networking does not.
    config.vm.forward_port 80, 8080
    config.vm.provision :puppet do |puppet|
    puppet.manifests_path = "puppet/manifests"
    puppet.module_path = "puppet/modules"
    puppet.manifest_file = "main.pp"
    puppet.options = [
    '--verbose',
    ]
    end
    end
    Vagrantfile
    44

    View Slide

  45. # git clone [email protected]:jaytaph/myproject.git
    # vagrant up
    45

    View Slide

  46. ➡ Downloads (optionally) the base box
    ➡ Deploys and boots up a new VM
    ➡ Runs the provisioner (puppet)
    ➡ Profit!
    46

    View Slide

  47. Multi VM’s
    Vagrant::Config.run do |config|
    config.vm.box = 'centos-62-64-puppet'
    config.vm.box_url = 'http://../centos-6.2-64bit-puppet-vbox.4.1.12.box'
    config.vm.define :web do |web_config|
    web_config.vm.host_name = 'web.example.org'
    web_config.vm.forward_port 80 8080
    ...
    end
    config.vm.define :database do |db_config|
    db_config.vm.host_name = 'db.example.org'
    db_config.vm.forward_port 3306 3306
    ...
    end
    end
    Vagrantfile
    47

    View Slide

  48. 48

    View Slide

  49. ➡ Puppet agent “calls” the master every 30 minutes.
    ➡ But what about realtime command & control?
    ➡ “Puppet kick”... (meh)
    ➡ MCollective (Marionette Collective)
    49

    View Slide

  50. ➡ Which systems running a database and have
    16GB or less?
    ➡ Which systems are using <50% of available
    memory?
    ➡ Restart all apache services in timezone
    GMT+5.
    50

    View Slide

  51. ACTIVEMQ
    Client
    MCollective
    Server
    Node
    Middleware
    Client
    MCollective
    Server
    MCollective
    Server
    Collective
    51

    View Slide

  52. http://docs.puppetlabs.com/mcollective/reference/basic/subcollectives.html 52

    View Slide

  53. Filter out nodes based on facts
    $ mc-facts operatingsystem
    Report for fact: operatingsystem
    CentOS found 3 times
    Debian found 14 times
    Solaris found 4 times
    $ mc-facts -W operatingsystem=Centos operatingsystemrelease
    Report for fact: operatingsystemrelease
    6.0 found 1 times
    5.6 found 2 times
    53

    View Slide

  54. ➡ Display all running processes
    ➡ Run or deploy software
    ➡ Restart services
    ➡ Start puppet agent
    ➡ Upgrade your systems
    ➡ Write your own agents!
    54

    View Slide

  55. -ETOOMUCHINFO
    Let’s recap
    55

    View Slide

  56. ➡ Configuration management tool.
    ➡ Focusses on “what” instead of “how”.
    ➡ Scales from 1 to 100K+ systems.
    ➡ Uses descriptive manifests.
    ➡ Vagrant for setting up your development
    environments.
    56

    View Slide

  57. ➡ Useful for sysadmins and developers.
    ➡ Keeps your infrastructure in sync.
    ➡ Keeps your infrastructure versioned.
    ➡ Infrastructure as part of your projects.
    ➡ MCollective controls your hosts based
    on facts, not names.
    57

    View Slide

  58. There is no reason NOT to manage your infrastructure.
    Having only 3 servers is NOT a reason.
    58
    You will be able to join the rest of us in the pub early.
    Don’t “install” development environments, build them!

    View Slide

  59. http://farm1.static.flickr.com/73/163450213_18478d3aa6_d.jpg 59

    View Slide

  60. Please rate my talk on joind.in:
    http://joind.in/6254
    Thank you
    60
    Find me on twitter: @jaytaph
    Find me for development and training: www.noxlogic.nl
    Find me on email: [email protected]
    Find me for blogs: www.adayinthelifeof.nl

    View Slide