Puppet for dummies - PHP|Tek 2012

Transcript

1. Puppet for
   Dummies
   PHP|Tek - Chicago - USA
   May 23, 2012
   woensdag 23 mei 12
   

   View Slide

2. Joshua Thijssen
   Freelance consultant, developer and
   trainer @ NoxLogic / Techademy
   Development in PHP, Python, Perl,
   C, Java and some sysadmin
   Blog: http://adayinthelifeof.nl
   Email: jthijssen@noxlogic.nl
   Twitter: @jaytaph
   oh hai!
   2
   woensdag 23 mei 12
   

   View Slide

3. What is puppet and why should I care?
   3
   woensdag 23 mei 12
   

   View Slide

4. “People are finally figuring out puppet and
   how it gets you to the pub by 4pm.
   Note that I’ve been at this pub since 2pm.”
   - Jorge Castro
   4
   woensdag 23 mei 12
   

   View Slide

5. 5
   woensdag 23 mei 12
   

   View Slide

6. Puppet is a (not necessarily the) solution for
   the following problem:
   How do we setup, manage, synchronize,
   and upgrade our internal and external
   infrastructure?
   6
   woensdag 23 mei 12
   

   View Slide

7. Sysadmin!
   Y U no fix problem!
   7
   woensdag 23 mei 12
   

   View Slide

8. Sysadmin!
   Y U no fix problem!
   NO
   7
   woensdag 23 mei 12
   

   View Slide

9. LAMP-stack
   8
   woensdag 23 mei 12
   

   View Slide

10. LAMP-stack
    Linux
    Apache
    MySQL
    PHP
    8
    woensdag 23 mei 12
    

    View Slide

11. LAMPGMVNMCSTRAH-stack
    9
    woensdag 23 mei 12
    

    View Slide

12. LAMPGMVNMCSTRAH-stack
    Linux
    Apache
    MySQL
    PHP
    Gearman
    MongoDB
    CouchDB
    Solr
    Tika
    Redis
    ActiveMQ
    Hadoop
    Varnish
    Ngnix
    Memcache
    9
    woensdag 23 mei 12
    

    View Slide

13. 10
    woensdag 23 mei 12
    

    View Slide

14. 10
    How do we control our infrastructure?
    woensdag 23 mei 12
    

    View Slide

15. ➡ Solution 1: We don’t,
    10
    How do we control our infrastructure?
    woensdag 23 mei 12
    

    View Slide

16. ➡ Solution 1: We don’t,
    ➡ Solution 2: We outsource,
    10
    How do we control our infrastructure?
    woensdag 23 mei 12
    

    View Slide

17. ➡ Solution 1: We don’t,
    ➡ Solution 2: We outsource,
    ➡ Solution 3: We automate the process.
    10
    How do we control our infrastructure?
    woensdag 23 mei 12
    

    View Slide

18. ‣ Solution 1: we don’t
    11
    woensdag 23 mei 12
    

    View Slide

19. ➡ It’s not funny: you find it more often
    than not. Especially inside small
    development companies.
    ‣ Solution 1: we don’t
    11
    woensdag 23 mei 12
    

    View Slide

20. ➡ It’s not funny: you find it more often
    than not. Especially inside small
    development companies.
    ➡ Internal sysadmin, but he’s too busy
    with development to do sysadmin.
    ‣ Solution 1: we don’t
    11
    woensdag 23 mei 12
    

    View Slide

21. ➡ It’s not funny: you find it more often
    than not. Especially inside small
    development companies.
    ➡ Internal sysadmin, but he’s too busy
    with development to do sysadmin.
    ➡ We only act on escalation
    ‣ Solution 1: we don’t
    11
    woensdag 23 mei 12
    

    View Slide

22. ➡ It’s not funny: you find it more often
    than not. Especially inside small
    development companies.
    ➡ Internal sysadmin, but he’s too busy
    with development to do sysadmin.
    ➡ We only act on escalation
    ➡ reactive, not proactive
    ‣ Solution 1: we don’t
    11
    woensdag 23 mei 12
    

    View Slide

23. ‣ Solution 2: we outsource
    12
    woensdag 23 mei 12
    

    View Slide

24. ➡ Expensive $LA’s.
    ‣ Solution 2: we outsource
    12
    woensdag 23 mei 12
    

    View Slide

25. ➡ Expensive $LA’s.
    ➡ What about INTERNAL servers like your
    development systems and
    infrastructure?
    ‣ Solution 2: we outsource
    12
    woensdag 23 mei 12
    

    View Slide

26. ➡ Expensive $LA’s.
    ➡ What about INTERNAL servers like your
    development systems and
    infrastructure?
    ➡ Fight between stability and agility.
    ‣ Solution 2: we outsource
    12
    woensdag 23 mei 12
    

    View Slide

27. ➡ Expensive $LA’s.
    ➡ What about INTERNAL servers like your
    development systems and
    infrastructure?
    ➡ Fight between stability and agility.
    ➡ Does your hosting company decide on
    whether you can use PHP5.3???
    ‣ Solution 2: we outsource
    12
    woensdag 23 mei 12
    

    View Slide

28. ‣ Solution 3: we do it ourselves and automate
    13
    woensdag 23 mei 12
    

    View Slide

29. ➡ We are in charge.
    ‣ Solution 3: we do it ourselves and automate
    13
    woensdag 23 mei 12
    

    View Slide

30. ➡ We are in charge.
    ➡ You can do what you like
    ‣ Solution 3: we do it ourselves and automate
    13
    woensdag 23 mei 12
    

    View Slide

31. ➡ We are in charge.
    ➡ You can do what you like
    ➡ Use: cfEngine, chef, puppet.
    ‣ Solution 3: we do it ourselves and automate
    13
    woensdag 23 mei 12
    

    View Slide

32. ➡ We are in charge.
    ➡ You can do what you like
    ➡ Use: cfEngine, chef, puppet.
    ➡ When done right, maintenance should
    not be difficult.
    ‣ Solution 3: we do it ourselves and automate
    13
    woensdag 23 mei 12
    

    View Slide

33. PUPPET
    14
    woensdag 23 mei 12
    

    View Slide

34. ➡ Open source configuration management tool.
    ➡ Written in Ruby
    ➡ Open source: https://github.com/puppetlabs
    ➡ Commercial version available (puppet enterprise)
    15
    woensdag 23 mei 12
    

    View Slide

35. ➡ Don’t tell HOW to do stuff.
    ➡ Tell WHAT to do.
    ¹
    ¹ It’s not actually true, but good enough for now...
    16
    woensdag 23 mei 12
    

    View Slide

36. ➡ Don’t tell HOW to do stuff.
    ➡ Tell WHAT to do.
    ¹
    ¹ It’s not actually true, but good enough for now...
    “yum install httpd”
    “apt-get install apache2”
    16
    woensdag 23 mei 12
    

    View Slide

37. ➡ Don’t tell HOW to do stuff.
    ➡ Tell WHAT to do.
    ¹
    ¹ It’s not actually true, but good enough for now...
    “yum install httpd”
    “apt-get install apache2”
    “install and run the apache webserver”
    16
    woensdag 23 mei 12
    

    View Slide

38. 17
    Schematic representation of a puppet infrastructure
    woensdag 23 mei 12
    

    View Slide

39. Puppet
    18
    woensdag 23 mei 12
    

    View Slide

40. Puppet CA
    Puppet
    Master
    Puppet
    Agent
    https
    19
    woensdag 23 mei 12
    

    View Slide

41. Puppet CA
    Puppet
    Master
    Puppet
    Agent
    Puppet
    Agent
    Puppet
    Agent
    https
    19
    woensdag 23 mei 12
    

    View Slide

42. ➡ Agent “calls” the puppet master.
    20
    woensdag 23 mei 12
    

    View Slide

43. ➡ Agent “calls” the puppet master.
    ➡ Agent sends “facts” to the master.
    20
    woensdag 23 mei 12
    

    View Slide

44. ➡ Agent “calls” the puppet master.
    ➡ Agent sends “facts” to the master.
    ➡ Master creates “catalog” from the manifests
    and facts, sends to agent.
    20
    woensdag 23 mei 12
    

    View Slide

45. ➡ Agent “calls” the puppet master.
    ➡ Agent sends “facts” to the master.
    ➡ Master creates “catalog” from the manifests
    and facts, sends to agent.
    ➡ Agent sets up system according to the
    catalog.
    20
    woensdag 23 mei 12
    

    View Slide

46. ➡ Agent “calls” the puppet master.
    ➡ Agent sends “facts” to the master.
    ➡ Master creates “catalog” from the manifests
    and facts, sends to agent.
    ➡ Agent sets up system according to the
    catalog.
    ➡ Agent reports status to master.
    20
    woensdag 23 mei 12
    

    View Slide

47. ➡ Catalogs are “compiled” manifests
    21
    woensdag 23 mei 12
    

    View Slide

48. ➡ Catalogs are “compiled” manifests
    ➡ Manifests are puppet definitions
    21
    woensdag 23 mei 12
    

    View Slide

49. ➡ Catalogs are “compiled” manifests
    ➡ Manifests are puppet definitions
    ➡ <filename>.pp
    21
    woensdag 23 mei 12
    

    View Slide

50. ➡ Catalogs are “compiled” manifests
    ➡ Manifests are puppet definitions
    ➡ <filename>.pp
    ➡ Puppet DSL
    21
    woensdag 23 mei 12
    

    View Slide

51. ➡ Catalogs are “compiled” manifests
    ➡ Manifests are puppet definitions
    ➡ <filename>.pp
    ➡ Puppet DSL
    ➡ De-cla-ra-tive language
    21
    woensdag 23 mei 12
    

    View Slide

52. ➡ Catalogs are “compiled” manifests
    ➡ Manifests are puppet definitions
    ➡ <filename>.pp
    ➡ Puppet DSL
    ➡ De-cla-ra-tive language
    ➡ Version your manifests! (git/svn)
    21
    woensdag 23 mei 12
    

    View Slide

53. package { “strace” :
    ensure => present,
    }
    file { “/home/jaytaph/secret-ingredient.txt” :
    ensure => present,
    mode => 0600,
    user => ‘jaytaph’,
    group => ‘noxlogic’,
    content => “beer”,
    }
    22
    woensdag 23 mei 12
    

    View Slide

54. package { “httpd” :
    ensure => present,
    }
    service { “httpd”:
    running => true,
    enable => true,
    }
    23
    woensdag 23 mei 12
    

    View Slide

55. package { “httpd” :
    ensure => present,
    }
    service { “httpd”:
    running => true,
    enable => true,
    }
    require => Package[“httpd”],
    23
    woensdag 23 mei 12
    

    View Slide

56. ‣ Different distributions, different names
    Centos / Redhat
    service: httpd
    package: httpd
    config: /etc/httpd/conf/httpd.conf
    vhosts: /etc/httpd/conf.d/*.conf
    Debian / Ubuntu
    service: apache2
    package: apache2
    config: /etc/apache2/httpd.conf
    vhosts: /etc/apache2/sites-available
    24
    woensdag 23 mei 12
    

    View Slide

57. class apache {
    package { “apache”:
    case $operatingsystem {
    centos, redhat { $packagename = “httpd” }
    debian, ubuntu { $packagename = “apache2” }
    default : { fail(‘I don’t know this OS/distro’) }
    }
    name => $packagename,
    ensure => installed,
    }
    service { “apache” :
    running => true,
    enable => true,
    require => Package[“apache”],
    }
    }
    25
    woensdag 23 mei 12
    

    View Slide

58. [root@puppetnode1 ~]# facter --puppet
    architecture => x86_64
    fqdn => puppetnode1.noxlogic.local
    interfaces => eth1,eth2,lo
    ipaddress_eth1 => 192.168.1.114
    ipaddress_eth2 => 192.168.56.200
    kernel => Linux
    kernelmajversion => 2.6
    operatingsystem => CentOS
    operatingsystemrelease => 6.0
    processor0 => Intel(R) Core(TM)2 Duo CPU T7500 @ 2.20GHz
    puppetversion => 2.6.9
    ‣ A simple list with info (also useable in your own tools)
    26
    woensdag 23 mei 12
    

    View Slide

59. node “web01.example.org” {
    include apache
    }
    node /^db\d+\.example\.org$/ {
    package { “mysql-server” :
    ensure => installed,
    }
    }
    27
    /etc/puppet/manifests/site.pp:
    woensdag 23 mei 12
    

    View Slide

60. node “web01.example.local” {
    $webserver_name = “web01.example.local”
    $webserver_alias = “www.example.local”
    $webserver_docroot = “/var/www/web01”
    include apache
    }
    node “web02.example.local” {
    $webserver_name = “web02.example.local”
    $webserver_alias = “crm.example.local”
    $webserver_docroot = “/var/www/web02”
    include apache
    }
    28
    woensdag 23 mei 12
    

    View Slide

61. http://docs.puppetlabs.com/references/stable/type.html
    29
    What can Puppet configure / control?
    woensdag 23 mei 12
    

    View Slide

62. http://docs.puppetlabs.com/references/stable/type.html
    ➡ Almost everything.
    ➡ Standard 48 different resource types
    ➡ Ranging from “file” to “cron” to “ssh_key”
    to “user” to “selinux”.
    ➡ Can control your Cisco routers and
    windows machines too (sortakinda)
    30
    woensdag 23 mei 12
    

    View Slide

63. 31
    class joindin::web {
    include apache
    # include phpmyadmin if needed
    if $params::phpmyadmin == true {
    include joindin::web::phpmyadmin
    }
    # Configure apache virtual host
    apache::vhost { $params::host :
    docroot => '/vagrant/src',
    template => 'joindin/vhost.conf.erb',
    port => $params::port,
    require => Package["apache"],
    }
    https://github.com/jaytaph/joind.in/tree/puppet
    woensdag 23 mei 12
    

    View Slide

64. 32
    ...
    # Install PHP modules
    php::module { 'mysql': }
    php::module { "pecl-xdebug" :
    require => File["EpelRepo"], # xdebug is in the epel repo
    }
    # Set development values to our php.ini
    augeas { 'set-php-ini-values':
    context => '/files/etc/php.ini',
    changes => [
    'set PHP/error_reporting "E_ALL | E_STRICT"',
    'set PHP/display_errors On',
    'set PHP/display_startup_errors On',
    'set PHP/html_errors On',
    'set Date/date.timezone Europe/London',
    ],
    require => Package['php'],
    notify => Service['apache'],
    }
    } # End class
    woensdag 23 mei 12
    

    View Slide

65. 33
    ➡ Puppet went from v0.25 to v2.6.
    ➡ REST interface since 2.6. XMLRPC before
    that.
    ➡ One binary to rule them all (puppet).
    ➡ Puppet v2.7 switched from GPLv2 to
    apache2.0 license.
    woensdag 23 mei 12
    

    View Slide

66. 34
    So how does Puppet benefit me as a
    DEVELOPER?
    woensdag 23 mei 12
    

    View Slide

67. 35
    ➡ Keep all developers in sync
    ➡ Keep your DTAP in sync
    ➡ Lets infrastructure be a part of your
    project
    woensdag 23 mei 12
    

    View Slide

68. Vagrant
    http://vagrantup.com/
    http://vagrantup.com/images/vagrant_chilling.png 36
    woensdag 23 mei 12
    

    View Slide

69. Vagrant is a tool for building and
    distributing virtualized development
    environments.
    37
    woensdag 23 mei 12
    

    View Slide

70. Vagrant::Config.run do |config|
    config.vm.box = 'centos-62-64-puppet'
    config.vm.box_url = 'http://../centos-6.2-64bit-puppet-vbox.4.1.12.box'
    # Forward a port from the guest to the host, which allows for outside
    # computers to access the VM, whereas host only networking does not.
    config.vm.forward_port 80, 8080
    config.vm.provision :puppet do |puppet|
    puppet.manifests_path = "puppet/manifests"
    puppet.module_path = "puppet/modules"
    puppet.manifest_file = "main.pp"
    puppet.options = [
    '--verbose',
    ]
    end
    end
    Vagrantfile
    38
    woensdag 23 mei 12
    

    View Slide

71. # git clone git@github.com:jaytaph/myproject.git
    # vagrant up
    39
    woensdag 23 mei 12
    

    View Slide

72. ➡ Downloads (optionally) the base box
    ➡ Deploys and boots up a new VM
    ➡ Runs the provisioner (puppet)
    ➡ Profit!
    40
    woensdag 23 mei 12
    

    View Slide

73. Multi VM’s
    Vagrant::Config.run do |config|
    config.vm.box = 'centos-62-64-puppet'
    config.vm.box_url = 'http://../centos-6.2-64bit-puppet-vbox.4.1.12.box'
    config.vm.define :web do |web_config|
    web_config.vm.host_name = 'web.example.org'
    web_config.vm.forward_port 80 8080
    ...
    end
    config.vm.define :database do |db_config|
    db_config.vm.host_name = 'db.example.org'
    db_config.vm.forward_port 3306 3306
    ...
    end
    end
    Vagrantfile
    41
    woensdag 23 mei 12
    

    View Slide

74. 42
    woensdag 23 mei 12
    

    View Slide

75. ➡ Puppet agent “calls” the master every 30 minutes.
    ➡ But what about realtime command & control?
    ➡ “Puppet kick”... (meh)
    ➡ MCollective (Marionette Collective)
    43
    woensdag 23 mei 12
    

    View Slide

76. ➡ Which systems running a database and have
    16GB or less?
    ➡ Which systems are using <50% of available
    memory?
    ➡ Restart all apache services in timezone
    GMT+5.
    44
    woensdag 23 mei 12
    

    View Slide

77. ACTIVEMQ
    Client
    MCollective
    Server
    Node
    Middleware
    Client
    MCollective
    Server
    MCollective
    Server
    ‣ Middleware takes care of distribution,
    ‣ queued, broadcast etc..
    Collective
    45
    woensdag 23 mei 12
    

    View Slide

78. http://docs.puppetlabs.com/mcollective/reference/basic/subcollectives.html
    46
    woensdag 23 mei 12
    

    View Slide

79. Filter out nodes based on facts
    $ mc-facts operatingsystem
    Report for fact: operatingsystem
    CentOS found 3 times
    Debian found 14 times
    Solaris found 4 times
    $ mc-facts -W operatingsystem=Centos operatingsystemrelease
    Report for fact: operatingsystemrelease
    6.0 found 1 times
    5.6 found 2 times
    47
    woensdag 23 mei 12
    

    View Slide

80. ➡ Display all running processes
    ➡ Run or deploy software
    ➡ Restart services
    ➡ Start puppet agent
    ➡ Upgrade your systems
    ➡ Write your own agents!
    48
    woensdag 23 mei 12
    

    View Slide

81. -ETOOMUCHINFO
    Let’s recap
    49
    woensdag 23 mei 12
    

    View Slide

82. ➡ Configuration management tool.
    ➡ Focusses on “what” instead of “how”.
    ➡ Scales from 1 to 100K+ systems.
    ➡ Uses descriptive manifests.
    ➡ Vagrant for setting up your development
    environments.
    50
    woensdag 23 mei 12
    

    View Slide

83. ➡ Useful for sysadmins and developers.
    ➡ Keeps your infrastructure in sync.
    ➡ Keeps your infrastructure versioned.
    ➡ Infrastructure as part of your projects.
    ➡ MCollective controls your hosts based
    on facts, not names.
    51
    woensdag 23 mei 12
    

    View Slide

84. There is no reason NOT to manage your infrastructure.
    Having only 3 servers is NOT a reason.
    52
    You will be able to join the rest of us in the pub early.
    Don’t “install” development environments, build them!
    woensdag 23 mei 12
    

    View Slide

85. http://farm1.static.flickr.com/73/163450213_18478d3aa6_d.jpg 53
    woensdag 23 mei 12
    

    View Slide

86. Please rate my talk on joind.in:
    http://joind.in/6515
    Thank you
    54
    Find me on twitter: @jaytaph
    Find me for development and training: www.noxlogic.nl
    Find me on email: jthijssen@noxlogic.nl
    Find me for blogs: www.adayinthelifeof.nl
    woensdag 23 mei 12
    

    View Slide