Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Vagrant, Puppet & Chef #FTW

Thijs Feryn
October 24, 2013

Vagrant, Puppet & Chef #FTW

My Vagrant talk at #FOWA 2013 in London

Thijs Feryn

October 24, 2013
Tweet

More Decks by Thijs Feryn

Other Decks in Technology

Transcript

  1. View Slide

  2. Hi, my name is Thijs

    View Slide

  3. I’m @ThijsFeryn
    on Twitter

    View Slide

  4. I’m an
    at
    Evangelist

    View Slide

  5. I’m a
    at
    board member

    View Slide

  6. View Slide

  7. Create and configure
    lightweight,
    reproducible, and
    portable development
    environments.

    View Slide

  8. Written in Ruby
    By Mitchell Hashimoto
    in 2010

    View Slide

  9. A layer on top of

    View Slide

  10. View Slide

  11. View Slide

  12. WHY?
    WHY?
    WHY?

    View Slide

  13. View Slide

  14. •Environment per project
    •Dev ~= Test ~= Staging ~= Prod
    •Easy to define & transport
    •Easy to tear down
    •Provisionable: infrastructure as code
    •Versionable
    •Shared across the team

    View Slide

  15. $"vagrant
    Usage:"vagrant"[.v]"[.h]"command"[]
    """".v,"..version""Print"the"version"and"exit.
    """".h,"..help"""""Print"this"help.
    Available"subcommands:
    """""box
    """""destroy
    """""halt
    """""init
    """""package
    """""plugin
    """""provision
    """""reload
    """""resume
    """""ssh
    """""ssh.config
    """""status
    """""suspend
    """""up

    View Slide

  16. $"vagrant"init".h
    Usage:"vagrant"init"[box.name]"[box.url]
    Generates a Vagrantfile
    “vagrant
    init”, innit?

    View Slide

  17. •Select base box
    •Choose virtualization provider
    •Configure VM parameters
    •Configure networking
    •Tweak SSH settings
    •Mount local folders
    •Provision machine
    What does a Vagrantfile do?

    View Slide

  18. $"vagrant"init
    A"`Vagrantfile`"has"been"placed"in"this"
    directory."You"are"now
    ready"to"`vagrant"up`"your"first"virtual"
    environment!"Please"read
    the"comments"in"the"Vagrantfile"as"well"
    as"documentation"on
    `vagrantup.com`"for"more"information"on"
    using"Vagrant.
    #".*."mode:"ruby".*.
    #"vi:"set"ft=ruby":
    Vagrant.configure("2")"do"|config|
    ""config.vm.box"=""base"
    end

    View Slide

  19. $"vagrant"init"precise32"http://
    files.vagrantup.com/precise32.box
    A"`Vagrantfile`"has"been"placed"in"this"
    directory."You"are"now
    ready"to"`vagrant"up`"your"first"virtual"
    environment!"Please"read
    the"comments"in"the"Vagrantfile"as"well"as"
    documentation"on
    `vagrantup.com`"for"more"information"on"
    using"Vagrant.
    #".*."mode:"ruby".*.
    #"vi:"set"ft=ruby":
    Vagrant.configure("2")"do"|config|
    ""config.vm.box"=""precise32"
    ""config.vm.box_url"=""http://
    files.vagrantup.com/precise32.box"""
    end

    View Slide

  20. $vagrant"up
    Bringing"machine"'default'"up"with"'virtualbox'"provider...
    [default]"Box"'precise32'"was"not"found."Fetching"box"from"specified"URL"for
    the"provider"'virtualbox'."Note"that"if"the"URL"does"not"have
    a"box"for"this"provider,"you"should"interrupt"Vagrant"now"and"add
    the"box"yourself."Otherwise"Vagrant"will"attempt"to"download"the
    full"box"prior"to"discovering"this"error.
    Downloading"or"copying"the"box...
    Extracting"
    box...################################################################%"
    (Rate:"/s,"Estimated"time"remaining:"))
    Successfully"added"box"'precise32'"with"provider"'virtualbox'!
    [default]"Importing"base"box"'precise32'...
    [default]"Matching"MAC"address"for"NAT"networking...
    [default]"Setting"the"name"of"the"VM...
    [default]"Clearing"any"previously"set"forwarded"ports...
    [default]"Creating"shared"folders"metadata...
    [default]"Clearing"any"previously"set"network"interfaces...
    [default]"Preparing"network"interfaces"based"on"configuration...
    [default]"Forwarding"ports...
    [default]".."22"=>"2222"(adapter"1)
    [default]"Booting"VM...
    [default]"Waiting"for"VM"to"boot."This"can"take"a"few"minutes.
    [default]"VM"booted"and"ready"for"use!
    [default]"Configuring"and"enabling"network"interfaces...
    [default]"Mounting"shared"folders...
    [default]".."/vagrant

    View Slide

  21. Boxes

    View Slide

  22. http://vagrantbox.es

    View Slide

  23. https://github.com/
    jedi4ever/veewee
    Build
    your own
    boxes

    View Slide

  24. $"vagrant"box
    Usage:"vagrant"box""[]
    Available"subcommands:
    """""add
    """""list
    """""remove
    """""repackage

    View Slide

  25. $"vagrant"box"list
    precise32"""""""(virtualbox)
    $"vagrant"box"remove"precise32"virtualbox
    Removing"box"'precise32'"with"provider"'virtualbox'...
    $"vagrant"box"add"precise32"http://files.vagrantup.com/
    precise32.box"..provider"virtualbox
    Downloading"or"copying"the"box...
    Extracting"
    box...#####################################################
    ###########%"(Rate:"/s,"Estimated"time"remaining:"))
    Successfully"added"box"'precise32'"with"provider"
    'virtualbox'!
    $"vagrant"box"repackage"precise32"virtualbox
    $"ls".lh
    .rw.r..r..""1"thijs""staff"""275M"10"okt"16:06"package.box

    View Slide

  26. $"vagrant"box"list
    precise32"""""""(virtualbox)
    $"vagrant"box"remove"precise32"virtualbox
    Removing"box"'precise32'"with"provider"'virtualbox'...
    $"vagrant"box"add"precise32"http://files.vagrantup.com/
    precise32.box"..provider"virtualbox
    Downloading"or"copying"the"box...
    Extracting"
    box...#####################################################
    ###########%"(Rate:"/s,"Estimated"time"remaining:"))
    Successfully"added"box"'precise32'"with"provider"
    'virtualbox'!
    $"vagrant"box"repackage"precise32"virtualbox
    $"ls".lh
    .rw.r..r..""1"thijs""staff"""275M"10"okt"16:06"package.box
    #".*."mode:"ruby".*.
    #"vi:"set"ft=ruby":
    Vagrant.configure("2")"do"|config|
    ""config.vm.box"=""precise32"
    ""config.vm.box_url"=""http://
    files.vagrantup.com/precise32.box"""
    end
    Vagrantfile
    can do this
    too

    View Slide

  27. $"tar"xvzf"package.box
    x"./box.disk1.vmdk
    x"./box.ovf
    x"./metadata.json
    x"./Vagrantfile
    Content
    of a box
    file

    View Slide

  28. Vagrant up & running

    View Slide

  29. $"vagrant"up
    Bringing"machine"'default'"up"with"'virtualbox'"provider...
    [default]"Importing"base"box"'precise32'...
    [default]"Matching"MAC"address"for"NAT"networking...
    [default]"Setting"the"name"of"the"VM...
    [default]"Clearing"any"previously"set"forwarded"ports...
    [default]"Creating"shared"folders"metadata...
    [default]"Clearing"any"previously"set"network"interfaces...
    [default]"Preparing"network"interfaces"based"on"configuration...
    [default]"Forwarding"ports...
    [default]".."22"=>"2222"(adapter"1)
    [default]"Booting"VM...
    [default]"Waiting"for"VM"to"boot."This"can"take"a"few"minutes.
    [default]"VM"booted"and"ready"for"use!
    [default]"Configuring"and"enabling"network"interfaces...
    [default]"Mounting"shared"folders...
    [default]".."/vagrant
    When
    VM doesn’t
    exist

    View Slide

  30. $"vagrant"up
    Bringing"machine"'default'"up"with"'virtualbox'"provider...
    [default]"VirtualBox"VM"is"already"running.
    $"vagrant"status
    Current"machine"states:
    default""""""""""""""""""running"(virtualbox)
    When VM
    exists & is
    running

    View Slide

  31. $"vagrant"suspend
    [default]"Saving"VM"state"and"suspending"execution...
    $"vagrant"status
    Current"machine"states:
    default""""""""""""""""""saved"(virtualbox)
    $"vagrant"up
    Bringing"machine"'default'"up"with"'virtualbox'"provider...
    [default]"Resuming"suspended"VM...
    [default]"Booting"VM...
    [default]"Waiting"for"VM"to"boot."This"can"take"a"few"
    minutes.
    [default]"VM"booted"and"ready"for"use!
    Suspend &
    resume

    View Slide

  32. $"vagrant"halt
    [default]"Attempting"graceful"shutdown"of"VM...
    $"vagrant"status
    Current"machine"states:
    default""""""""""""""""""poweroff"(virtualbox)
    $"vagrant"up
    Bringing"machine"'default'"up"with"'virtualbox'"provider...
    [default]"Setting"the"name"of"the"VM...
    [default]"Clearing"any"previously"set"forwarded"ports...
    [default]"Creating"shared"folders"metadata...
    [default]"Clearing"any"previously"set"network"interfaces...
    [default]"Preparing"network"interfaces"based"on"configuration...
    [default]"Forwarding"ports...
    [default]".."22"=>"2222"(adapter"1)
    [default]"Booting"VM...
    [default]"Waiting"for"VM"to"boot."This"can"take"a"few"minutes.
    [default]"VM"booted"and"ready"for"use!
    [default]"Configuring"and"enabling"network"interfaces...
    [default]"Mounting"shared"folders...
    [default]".."/vagrant
    Stop & start

    View Slide

  33. $"vagrant"destroy
    Are"you"sure"you"want"to"destroy"the"'default'"VM?"[y/N]"y
    [default]"Forcing"shutdown"of"VM...
    [default]"Destroying"VM"and"associated"drives...
    $"vagrant"status
    Current"machine"states:
    default""""""""""""""""""not"created"(virtualbox)
    Destroy

    View Slide

  34. Connect

    View Slide

  35. $"vagrant"ssh
    Linux"debian.7"3.2.0.4.amd64"#1"SMP"Debian"3.2.46.1"x86_64
    The"programs"included"with"the"Debian"GNU/Linux"system"are"
    free"software;
    the"exact"distribution"terms"for"each"program"are"described"
    in"the
    individual"files"in"/usr/share/doc/*/copyright.
    Debian"GNU/Linux"comes"with"ABSOLUTELY"NO"WARRANTY,"to"the"
    extent
    permitted"by"applicable"law.
    Last"login:"Mon"Jul"15"13:13:36"2013"from"10.0.2.2
    [email protected]:~$"sudo"su
    [email protected]:/home/vagrant
    No login or
    password
    required

    View Slide

  36. [email protected]:/home/vagrant#"cat".ssh/authorized_keys
    ssh.rsa"
    AAAAB3NzaC1yc2EAAAABIwAAAQEA6NF8iallvQVp22WDkTkyrtvp9eWW6A8
    YVr+kz4TjGYe7gHzIw
    +niNltGEFHzD8+v1I2YJ6oXevct1YeS0o9HZyN1Q9qgCgzUFtdOKLv6Iedp
    lqoPkcmF0aYet2PkEDo3MlTBckFXPITAMzF8dJSIFo9D8HfdOV0IAdx4O7P
    tixWKn5y2hMNG0zQPyUecp4pzC6kivAIhyfHilFR61RGL
    +GPXQ2MWZWFYbAGjyiYJnAmCP3NOTd0jMZEnDkbUvxhMmBYSdETk1rRgm
    +R4LOzFUGaHqHDLKLX
    +FIPKcF96hrucXzcWyLbIbEgE98OHlnVYCzRdK8jlqm8tehUc9c9WhQ=="
    vagrant"insecure"public"key
    Public key

    View Slide

  37. $"vagrant"ssh.config
    Host"default
    ""HostName"127.0.0.1
    ""User"vagrant
    ""Port"2222
    ""UserKnownHostsFile"/dev/null
    ""StrictHostKeyChecking"no
    ""PasswordAuthentication"no
    ""IdentityFile""/Users/thijs/.vagrant.d/insecure_private_key"
    ""IdentitiesOnly"yes
    ""LogLevel"FATAL
    Username,
    host, port &
    private key
    location

    View Slide

  38. $"cat"~/.vagrant.d/insecure_private_key
    .....BEGIN"RSA"PRIVATE"KEY.....
    MIIEogIBAAKCAQEA6NF8iallvQVp22WDkTkyrtvp9eWW6A8YVr+kz4TjGYe7gHzI
    w+niNltGEFHzD8+v1I2YJ6oXevct1YeS0o9HZyN1Q9qgCgzUFtdOKLv6IedplqoP
    kcmF0aYet2PkEDo3MlTBckFXPITAMzF8dJSIFo9D8HfdOV0IAdx4O7PtixWKn5y2
    hMNG0zQPyUecp4pzC6kivAIhyfHilFR61RGL+GPXQ2MWZWFYbAGjyiYJnAmCP3NO
    Td0jMZEnDkbUvxhMmBYSdETk1rRgm+R4LOzFUGaHqHDLKLX+FIPKcF96hrucXzcW
    yLbIbEgE98OHlnVYCzRdK8jlqm8tehUc9c9WhQIBIwKCAQEA4iqWPJXtzZA68mKd
    ELs4jJsdyky+ewdZeNds5tjcnHU5zUYE25K+ffJED9qUWICcLZDc81TGWjHyAqD1
    Bw7XpgUwFgeUJwUlzQurAv+/ySnxiwuaGJfhFM1CaQHzfXphgVml+fZUvnJUTvzf
    TK2Lg6EdbUE9TarUlBf/xPfuEhMSlIE5keb/Zz3/LUlRg8yDqz5w+QWVJ4utnKnK
    iqwZN0mwpwU7YSyJhlT4YV1F3n4YjLswM5wJs2oqm0jssQu/BT0tyEXNDYBLEF4A
    sClaWuSJ2kjq7KhrrYXzagqhnSei9ODYFShJu8UWVec3Ihb5ZXlzO6vdNQ1J9Xsf
    4m+2ywKBgQD6qFxx/Rv9CNN96l/4rb14HKirC2o/orApiHmHDsURs5rUKDx0f9iP
    cXN7S1uePXuJRK/5hsubaOCx3Owd2u9gD6Oq0CsMkE4CUSiJcYrMANtx54cGH7Rk
    EjFZxK8xAv1ldELEyxrFqkbE4BKd8QOt414qjvTGyAK+OLD3M2QdCQKBgQDtx8pN
    CAxR7yhHbIWT1AH66+XWN8bXq7l3RO/ukeaci98JfkbkxURZhtxV/HHuvUhnPLdX
    3TwygPBYZFNo4pzVEhzWoTtnEtrFueKxyc3+LjZpuo+mBlQ6ORtfgkr9gBVphXZG
    YEzkCD3lVdl8L4cw9BVpKrJCs1c5taGjDgdInQKBgHm/fVvv96bJxc9x1tffXAcj
    3OVdUN0UgXNCSaf/3A/phbeBQe9xS+3mpc4r6qvx+iy69mNBeNZ0xOitIjpjBo2+
    dBEjSBwLk5q5tJqHmy/jKMJL4n9ROlx93XS+njxgibTvU6Fp9w+NOFD/HvxB3Tcz
    6+jJF85D5BNAG3DBMKBjAoGBAOAxZvgsKN+JuENXsST7F89Tck2iTcQIT8g5rwWC
    P9Vt74yboe2kDT531w8+egz7nAmRBKNM751U/95P9t88EDacDI/Z2OwnuFQHCPDF
    llYOUI+SpLJ6/vURRbHSnnn8a/XG+nzedGH5JGqEJNQsz+xT2axM0/W/CRknmGaJ
    kda/AoGANWrLCz708y7VYgAtW2Uf1DPOIYMdvo6fxIB5i9ZfISgcJ/bbCUkFrhoH
    +vq/5CIWxCPp0f85R4qxxQ5ihxJ0YDQT9Jpx4TMss4PSavPaBH3RXow5Ohe+bYoQ
    NE5OgEXk2wVfZczCZpigBKbKZHNYcelXtTt/nP3rsCuGcM4h53s=
    .....END"RSA"PRIVATE"KEY.....
    Private key

    View Slide

  39. The portability
    of a Vagrantfile

    View Slide

  40. Add
    Vagrantfile to
    VCS

    View Slide

  41. Share Vagrantfile
    Share dev env

    View Slide

  42. Infrastructure as code

    View Slide

  43. #".*."mode:"ruby".*.
    #"vi:"set"ft=ruby":
    Vagrant.configure("2")"do"|config|
    ""config.vm.box"=""precise32"
    ""config.vm.box_url"=""http://
    files.vagrantup.com/precise32.box"""
    end
    You probably
    remember this one

    View Slide

  44. Networking

    View Slide

  45. config.vm.network":forwarded_port,"guest:"80,"host:"8080
    config.vm.network":public_network
    config.vm.network":private_network,"ip:""192.168.10.10"
    Networking
    Bridged network
    Port forwarding
    Private IP space

    View Slide

  46. #".*."mode:"ruby".*.
    #"vi:"set"ft=ruby":
    Vagrant.configure("2")"do"|config|
    ""config.vm.box"=""precise32"
    ""config.vm.box_url"=""http://files.vagrantup.com/
    precise32.box"""
    ""config.vm.network":private_network,"ip:""192.168.10.10"
    end
    $"vagrant"up
    ...
    [default]"Preparing"network"interfaces"based"on"
    configuration...
    [default]"Forwarding"ports...
    [default]".."22"=>"2222"(adapter"1)
    Yes, there is
    port forwarding
    involved

    View Slide

  47. #".*."mode:"ruby".*.
    #"vi:"set"ft=ruby":
    Vagrant.configure("2")"do"|config|
    ""config.vm.box"=""precise32"
    ""config.vm.box_url"=""http://files.vagrantup.com/
    precise32.box"""
    ""config.vm.network":forwarded_port,"guest:"80,"host:"8080
    end
    $"vagrant"up
    ...
    [default]"Preparing"network"interfaces"based"on"
    configuration...
    [default]"Forwarding"ports...
    [default]".."22"=>"2222"(adapter"1)
    [default]".."80"=>"8080"(adapter"1) There’s
    port 80

    View Slide

  48. #".*."mode:"ruby".*.
    #"vi:"set"ft=ruby":
    Vagrant.configure("2")"do"|config|
    ""config.vm.box"=""precise32"
    ""config.vm.box_url"=""http://files.vagrantup.com/
    precise32.box"""
    ""config.vm.network":public_network
    end

    View Slide

  49. $"vagrant"up
    ...
    Bringing"machine"'default'"up"with"'virtualbox'"
    provider...
    [default]"Clearing"any"previously"set"forwarded"ports...
    [default]"Creating"shared"folders"metadata...
    [default]"Clearing"any"previously"set"network"
    interfaces...
    [default]"Available"bridged"network"interfaces:
    1)"en1:"Wi.Fi"(AirPort)
    2)"en0:"Ethernet
    3)"p2p0
    What"interface"should"the"network"bridge"to?"1
    [default]"Preparing"network"interfaces"based"on"
    configuration...
    [default]"Forwarding"ports...
    [default]".."22"=>"2222"(adapter"1)
    Choose
    interface

    View Slide

  50. config.vm.network""public_network",":bridge"=>"'en1:"Wi.
    Fi"(AirPort)'
    Assign interface in Vagrantfile

    View Slide

  51. Synced folders

    View Slide

  52. Local filesystem available in VM

    View Slide

  53. Save directly from your editor/IDE

    View Slide

  54. $"vagrant"up
    ...
    [default]"Mounting"shared"folders...
    [default]".."/vagrant
    config.vm.synced_folder""web/",""/var/www"
    $"vagrant"up
    ...
    [default]"Mounting"shared"folders...
    [default]".."/vagrant
    [default]".."/var/www
    Extra mount
    Default mount

    View Slide

  55. VM properties

    View Slide

  56. config.vm.hostname"=""mymachine"
    config.vm.provider":virtualbox"do"|v|
    ""v.customize"["modifyvm",":id,"'..chipset',"'ich9']
    ""v.customize"["modifyvm",":id,""..natdnshostresolver1",""on"]
    ""v.customize"["modifyvm",":id,""..ioapic",""on"]
    ""v.customize"["modifyvm",":id,""..memory","2048]
    ""v.customize"["modifyvm",":id,""..cpus",""4"]
    ""#v.gui"="true
    end
    VM properties

    View Slide

  57. Provisioning

    View Slide

  58. •Add specific software
    •Create configuration files
    •Execute commands
    •Create users
    •Manage services
    •Automatically executed on vagrant up
    Provisioning

    View Slide

  59. Having an
    exact copy of your
    production
    environment

    View Slide

  60. •Shell
    •Ansible
    •Chef Solo
    •Chef Client
    •Puppet Apply
    •Puppet Agent
    Provisioning

    View Slide

  61. Shell

    View Slide

  62. Shell
    config.vm.provision":shell,":inline"=>""mount"
    .t"tmpfs".o"size=50m,mode=0777"tmpfs"/vagrant/
    app/cache"
    Mount RAM disk volume

    View Slide

  63. Shell
    $script"="<echo"I"am"provisioning...
    date">"/etc/vagrant_provisioned_at
    SCRIPT
    Vagrant.configure("2")"do"|config|
    ""config.vm.provision""shell","inline:"$script
    end
    Inject Ruby

    View Slide

  64. Shell
    Vagrant.configure("2")"do"|config|
    ""config.vm.provision""shell","path:""script.sh"
    end
    Point to script in your current folder
    Vagrant.configure("2")"do"|config|
    ""config.vm.provision""shell","path:""https://
    example.com/provisioner.sh"
    end
    Point to external script

    View Slide

  65. View Slide

  66. Written in Ruby
    By Opscode
    in 2009

    View Slide

  67. Written in Ruby
    By Puppet Labs
    in 2005

    View Slide

  68. •Written in Ruby
    •Open source with enterprise revenue model
    •Similar features
    •Both have a standalone and server-side edition
    •Supported by a large community
    •Modularized components
    •Use packages for software installs
    •Use templating for custom files
    •Filesystem methods
    •...
    Chef & Puppet

    View Slide

  69. Chef vs Puppet
    Chef Puppet
    Modules
    Actions
    Language
    Running order
    Approach
    Programming style
    Cookbooks Modules
    Recipes Manifests
    Ruby extended with DSL DSL
    Sequential “Random”
    Define actions Define state
    Procedural “OO-like”

    View Slide

  70. Let’s do some Chef!

    View Slide

  71. •Download cookbooks (https://github.com/opscode-cookbooks)
    •Configure chef.cookbooks_path in Vagrantfile
    •Add recipes using chef.add_recipe in
    Vagrantfile
    •Configure attributes with chef.json
    •Group custom actions in custom cookbook
    How to use Chef Solo

    View Slide

  72. config.vm.provision":chef_solo"do"|chef|
    """chef.cookbooks_path"="./tools/chef/cookbooks"
    """chef.add_recipe""mysql::server"
    """chef.json"="{"
    """""mysql""=>"{
    """""""""server_root_password""=>""foo",
    """""""""server_repl_password""=>""foo",
    """""""""server_debian_password""=>""foo"
    """""}""""""""""""""
    """}
    end
    Vagrantfile

    View Slide

  73. Cookbook layout
    cookbook
    ├──"README.md
    ├──"attributes
    ├──"definitions
    ├──"files
    │"""└──"default
    ├──"libraries
    ├──"metadata.rb
    ├──"providers
    ├──"recipes
    │"""└──"default.rb
    ├──"resources
    └──"templates
    """"└──"default

    View Slide

  74. MySQL attributes
    default['mysql']['port']"""""""""""""""""""""""="3306
    default['mysql']['nice']"""""""""""""""""""""""="0
    case"node['platform_family']
    when"'debian'
    ""default['mysql']['server']['packages']""""="%w[mysql.server]
    ""default['mysql']['service_name']""""""""""="'mysql'
    ""default['mysql']['basedir']"""""""""""""""="'/usr'
    ""default['mysql']['data_dir']""""""""""""""="'/var/lib/mysql'
    ""default['mysql']['root_group']""""""""""""="'root'
    ""default['mysql']['mysqladmin_bin']""""""""="'/usr/bin/mysqladmin'
    ""default['mysql']['mysql_bin']"""""""""""""="'/usr/bin/mysql'
    ""default['mysql']['conf_dir']""""""""""""""="'/etc/mysql'
    ""default['mysql']['confd_dir']"""""""""""""="'/etc/mysql/conf.d'
    ""default['mysql']['socket']""""""""""""""""="'/var/run/mysqld/
    mysqld.sock'
    ""default['mysql']['pid_file']""""""""""""""="'/var/run/mysqld/
    mysqld.pid'
    ""default['mysql']['old_passwords']"""""""""="0
    ""default['mysql']['grants_path']"""""""""""="'/etc/mysql/grants.sql'
    ...

    View Slide

  75. MySQL server recipe
    ...
    group"'mysql'"do
    ""action":create
    end
    user"'mysql'"do
    ""comment"'MySQL"Server'
    ""gid"""""'mysql'
    ""system""true
    ""home""""node['mysql']['data_dir']
    ""shell"""'/sbin/nologin'
    end
    node['mysql']['server']['packages'].each"do"|name|
    ""package"name"do
    """"action""":install
    """"notifies":start,"'service[mysql]',":immediately
    ""end
    end
    ...

    View Slide

  76. MySQL template
    [client]
    host"""""="localhost
    user"""""="debian.sys.maint
    password"="<%="node['mysql']['server_debian_password']"%>
    socket"""="<%="node['mysql']['socket']"%>
    [mysql_upgrade]
    host"""""="localhost
    user"""""="debian.sys.maint
    password"="<%="node['mysql']['server_debian_password']"%>
    socket"""="<%="node['mysql']['socket']"%>
    basedir""="/usr

    View Slide

  77. •Cron
    •Directory
    •Execute
    •File
    •Git
    •Group
    •Link
    •Log
    •Package
    •Service
    •Template
    •User
    •...
    Typical Chef resources

    View Slide

  78. Let’s do it ourselves
    execute"'update"apt'"do
    """"command""apt.get"update"
    """"action":run
    end
    package"'mysql.server'"do
    """"action""":install
    """"notifies":start,"'service[mysql]',":immediately
    end
    package"'apache2'"do
    """"action""":install
    """"notifies":start,"'service[apache2]',":delayed
    end
    package"'php5'"do
    """"action""":install
    """"notifies":reload,"'service[apache2]',":delayed
    end
    ./tools/chef/cookbooks/project/recipes/default.rb

    View Slide

  79. Let’s do it ourselves
    execute"'assign.root.password'"do
    """"command""/usr/bin/mysqladmin".u"root"password"'#{node['project']
    ['server_root_password']}'"
    """"action":run
    """"only_if""/usr/bin/mysql".u"root".e"'show"databases;'"
    end
    service"'mysql'"do
    """"service_name"'mysql'
    """"supports":status"=>"true,":restart"=>"true,":reload"=>"true
    """"action""":enable
    end
    service"'apache2'"do
    """"service_name"'apache2'
    """"supports":status"=>"true,":restart"=>"true,":reload"=>"true
    """"action""":enable
    end
    ./tools/chef/cookbooks/project/recipes/default.rb

    View Slide

  80. Let’s do it ourselves
    #".*."mode:"ruby".*.
    #"vi:"set"ft=ruby":
    Vagrant.configure("2")"do"|config|
    config.vm.box"=""precise32"
    config.vm.box_url"=""http://files.vagrantup.com/
    precise32.box"
    config.vm.synced_folder""web/",""/var/www"
    config.vm.provision":chef_solo"do"|chef
    chef.cookbooks_path"="./tools/chef/cookbooks"
    chef.add_recipe""project"
    chef.json"="{"
    "project""=>"{
    "server_root_password""=>""foo"
    }""""""""""""""
    }
    end"
    end

    View Slide

  81. $"vagrant"up
    ...
    [default]"Mounting"shared"folders...
    [default]".."/vagrant
    [default]".."/tmp/vagrant.chef.1/chef.solo.1/cookbooks
    [default]"Running"provisioner:"chef_solo...
    Generating"chef"JSON"and"uploading...
    Running"chef.solo...
    ...
    Provisioning

    View Slide

  82. [2013.10.21T10:37:22+00:00]"INFO:"***"Chef"11.4.4"***
    [2013.10.21T10:37:22+00:00]"INFO:"Setting"the"run_list"to"
    ["recipe[project]"]"from"JSON
    [2013.10.21T10:37:22+00:00]"INFO:"Run"List"is"[recipe[project]]
    [2013.10.21T10:37:22+00:00]"INFO:"Run"List"expands"to"[project]
    [2013.10.21T10:37:22+00:00]"INFO:"Starting"Chef"Run"for"debian.7.1.0
    [2013.10.21T10:37:22+00:00]"INFO:"Running"start"handlers
    [2013.10.21T10:37:22+00:00]"INFO:"Start"handlers"complete.
    [2013.10.21T10:37:24+00:00]"INFO:"Processing"package[mysql.server]"action"
    install"(project::default"line"1)
    [2013.10.21T10:37:59+00:00]"INFO:"package[mysql.server]"sending"start"
    action"to"service[mysql]"(immediate)
    [2013.10.21T10:37:59+00:00]"INFO:"Processing"service[mysql]"action"start"
    (project::default"line"10)
    [2013.10.21T10:37:59+00:00]"INFO:"Processing"execute[assign.root.password]"
    action"run"(project::default"line"5)
    [2013.10.21T10:37:59+00:00]"INFO:"execute[assign.root.password]"ran"
    successfully
    [2013.10.21T10:37:59+00:00]"INFO:"Processing"service[mysql]"action"enable"
    (project::default"line"10)
    [2013.10.21T10:37:59+00:00]"INFO:"Chef"Run"complete"in"37.208173462"seconds
    [2013.10.21T10:37:59+00:00]"INFO:"Running"report"handlers
    [2013.10.21T10:37:59+00:00]"INFO:"Report"handlers"complete

    View Slide

  83. Let’s do some Puppet!

    View Slide

  84. •Download modules (https://github.com/puppetlabs/)
    •Configure puppet.module_path,
    puppet.manifests_path & puppet.manifest_file in
    Vagrantfile
    •Provisioning flow happens in the main manifest
    •Configure attributes with puppet.facter
    How to use Puppet Apply

    View Slide

  85. config.vm.provision":puppet"do"|puppet|
    """puppet.manifests_path"=""./tools/puppet/manifests"
    """puppet.module_path"=""./tools/puppet/modules"
    """puppet.manifest_file""=""init.pp"
    """puppet.options"="['..verbose']
    end
    Vagrantfile
    include"mysql::server
    init.pp
    class"{"'::mysql::server':
    ""root_password"=>"'foo'
    }
    Override password
    Default

    View Slide

  86. Module layout
    module
    ├──"README.md
    ├──"files
    ├──"lib
    ├──"metadata.json
    ├──"spec
    ├──"manifests
    │"""└──"init.pp
    │"""└──"params.pp
    ├──"resources
    └──"templates
    └──"tests

    View Slide

  87. MySQL params
    class"mysql::params"{
    ""$manage_config_file"""""="true
    ""$old_root_password""""""="''
    ""$purge_conf_dir"""""""""="false
    ""$restart""""""""""""""""="false
    ""$root_password""""""""""="'UNSET'
    ""$server_package_ensure""="'present'
    ""$server_service_manage""="true
    ""$server_service_enabled"="true
    ""#"mysql::bindings
    ""$bindings_enable"""""""""="false
    ""$java_package_ensure"""""="'present'
    ""$java_package_provider"""="undef
    ""$perl_package_ensure"""""="'present'
    ""$perl_package_provider"""="undef
    ""$php_package_ensure""""""="'present'
    ""$php_package_provider""""="undef
    ""$python_package_ensure"""="'present'
    ""$python_package_provider"="undef
    ""$ruby_package_ensure"""""="'present'
    ""$ruby_package_provider"""="undef

    View Slide

  88. MySQL params 2
    ...
    case"$::osfamily"{
    """"'RedHat':"{
    """"""if"$::operatingsystem"=="'Fedora'"and"
    (is_integer($::operatingsystemrelease)"and"$::operatingsystemrelease">="
    19"or"$::operatingsystemrelease"==""Rawhide")"{
    """"""""$client_package_name"="'mariadb'
    """"""""$server_package_name"="'mariadb.server'
    """"""}"else"{
    """"""""$client_package_name"="'mysql'
    """"""""$server_package_name"="'mysql.server'
    """"""}
    """"""$basedir"""""""""""""="'/usr'
    """"""$config_file"""""""""="'/etc/my.cnf'
    """"""$datadir"""""""""""""="'/var/lib/mysql'
    """"""$log_error"""""""""""="'/var/log/mysqld.log'
    """"""$pidfile"""""""""""""="'/var/run/mysqld/mysqld.pid'
    """"""$root_group""""""""""="'root'
    """"""$server_service_name"="'mysqld'
    """"""$socket""""""""""""""="'/var/lib/mysql/mysql.sock'
    """"""$ssl_ca""""""""""""""="'/etc/mysql/cacert.pem'
    """"""$ssl_cert""""""""""""="'/etc/mysql/server.cert.pem'
    """"""$ssl_key"""""""""""""="'/etc/mysql/server.key.pem'

    View Slide

  89. MySQL server manifest
    class"mysql::server"(
    ""$config_file"""""""""""""="$mysql::params::config_file,
    ""$manage_config_file""""""="$mysql::params::manage_config_file,
    ""$old_root_password"""""""="$mysql::params::old_root_password,
    ""$override_options""""""""="{},
    ""$package_ensure""""""""""="$mysql::params::server_package_ensure,
    ""$package_name""""""""""""="$mysql::params::server_package_name,
    ""$purge_conf_dir""""""""""="$mysql::params::purge_conf_dir,
    ""$remove_default_accounts"="false,
    ""$restart"""""""""""""""""="$mysql::params::restart,
    ""$root_group""""""""""""""="$mysql::params::root_group,
    ""$root_password"""""""""""="$mysql::params::root_password,
    ""$service_enabled"""""""""="
    $mysql::params::server_service_enabled,
    ""$service_manage""""""""""="$mysql::params::server_service_manage,
    ""$service_name""""""""""""="$mysql::params::server_service_name,
    ""$service_provider""""""""="
    $mysql::params::server_service_provider,
    ""#"Deprecated"parameters
    ""$enabled"""""""""""""""""="undef,
    ""$manage_service""""""""""="undef
    )"inherits"mysql::params"{
    ...

    View Slide

  90. MySQL template
    [client]
    user=root
    host=localhost
    <%"unless"scope.lookupvar('mysql::server::root_password')"
    =="'UNSET'".%>
    password='<%="
    scope.lookupvar('mysql::server::root_password')"%>'
    <%"end".%>
    socket=<%="@options['client']['socket']".%>
    ERB style too!

    View Slide

  91. •Computer
    •Cron
    •Exec
    •File
    •Filebucket
    •Group
    •Host
    •Interface
    •Package
    •Service
    •Sshkey
    •User
    Typical Puppet resources

    View Slide

  92. Let’s do it ourselves
    exec"{""apt.update":
    """"command"=>""/usr/bin/apt.get"update",
    }
    package"{"'mysql.server':
    """"ensure""=>"present,
    """"require"=>"Exec['apt.update'],
    """"notify"=>"Service['mysql'],
    }
    package"{"'apache2':
    """"ensure""=>"present,
    """"require"=>"Exec['apt.update'],
    """"notify"=>"Package['php5'],
    }
    package"{"'php5':
    """"ensure""=>"present,
    """"require"=>"Exec['apt.update'],
    """"notify"=>"Service['apache2'],
    }
    ./tools/puppet/manifests/init.pp

    View Slide

  93. Let’s do it ourselves
    exec"{"'assign.root.password':
    """"command"=>""/usr/bin/mysqladmin".u"root"password"$root_password",
    """"require"=>"Package["mysql.server"],
    """"onlyif""=>""/usr/bin/mysql".u"root".e"'show"databases;'"
    }
    """"
    service"{""mysql":
    """"name"""""""=>""mysql",
    """"ensure"""""=>"running,
    """"enable"""""=>"true,
    """"hasrestart"=>"true,
    """"require""""=>"Package["mysql.server"],
    }""""
    service"{""apache2":
    """"name"""""""=>""apache2",
    """"ensure"""""=>"running,
    """"enable"""""=>"true,
    """"hasrestart"=>"true,
    """"require""""=>"Package["apache2"],
    }
    ./tools/puppet/manifests/init.pp

    View Slide

  94. Let’s do it ourselves
    #".*."mode:"ruby".*.
    #"vi:"set"ft=ruby":
    Vagrant.configure("2")"do"|config|
    config.vm.box"=""precise32"
    config.vm.box_url"=""http://files.vagrantup.com/
    precise32.box"
    config.vm.synced_folder""web/",""/var/www"
    config.vm.provision":puppet"do"|puppet|
    """puppet.manifests_path"=""puppet/manifests"
    """puppet.module_path"=""puppet/modules"
    """puppet.manifest_file""=""init.pp"
    """puppet.facter"="{
    """"root_password""=>""foo",
    """}
    """puppet.options"="['..verbose']
    end"
    end

    View Slide

  95. $"vagrant"up
    ...
    [default]"Mounting"shared"folders...
    [default]".."/vagrant
    [default]".."/tmp/vagrant.puppet/manifests
    [default]".."/tmp/vagrant.puppet/modules.0
    [default]"Running"provisioner:"puppet...
    Running"Puppet"with"init.pp...
    Notice:"/Stage[main]//Exec[apt.update]/returns:"executed"
    successfully
    Notice:"/Stage[main]//Package[mysql.server]/ensure:"ensure"
    changed"'purged'"to"'present'
    Service[mysql]
    Notice:"/Stage[main]//Service[mysql]:"Triggered"'refresh'"from"
    1"events
    Notice:"/Stage[main]//Exec[assign.root.password]/returns:"
    executed"successfully
    Notice:"/Stage[main]//Package[apache2]/ensure:"ensure"changed"
    'purged'"to"'present'
    Notice:"/Stage[main]//Package[php5]/ensure:"ensure"changed"
    'purged'"to"'present'
    Notice:"/Stage[main]//Service[apache2]:"Triggered"'refresh'"
    from"1"events
    Notice:"Finished"catalog"run"in"222.55"seconds
    Provisioning

    View Slide

  96. Re-provision VM
    $"vagrant"provision
    When
    machine is
    running

    View Slide

  97. Chef or Puppet?

    View Slide

  98. Problems with
    provisioning

    View Slide

  99. •Provisioning is often slow (especially when
    installing lots of packages)
    •Quality of public cookbooks/manifests
    •Support on cookbooks/manifests
    •Writing it yourself can be difficult
    •Some cookbooks/manifests only work on
    certain Linux distros
    Problems with provisioning

    View Slide

  100. Re-distribute current VM
    $"vagrant"package
    •Possible solution for slow provisioning
    •Is not the same as vagrant box repackage
    •Use exported box as new base box
    •No provisioning required on startup
    •Possibility of doing “light” provisioning
    instead

    View Slide

  101. Multi-machine setup

    View Slide

  102. #".*."mode:"ruby".*.
    #"vi:"set"ft=ruby":
    Vagrant.configure("2")"do"|config|
    ""config.vm.provision""shell","inline:""/usr/bin/apt.get"update"""""
    ""config.vm.box"=""debian.7.1.0"
    ""
    ""config.vm.define""web","primary:"true"do"|web|
    """"""web.vm.hostname"=""web"
    """"""web.vm.network":private_network,"ip:""192.168.33.10"
    """"""web.vm.synced_folder""web/",""/var/www"""""""
    """"""web.vm.provision":puppet"do"|puppet|
    """"""""""puppet.manifests_path"=""./tools/puppet/manifests"
    """"""""""puppet.module_path"=""./tools/puppet/modules"
    """"""""""puppet.manifest_file""=""web.pp"
    """"""""""puppet.options"="['..verbose']""""
    """"""end""""""
    ""end
    ""config.vm.define""db""do"|db|"
    """"""db.vm.hostname"=""db"
    """"""db.vm.network":private_network,"ip:""192.168.33.11"
    """"""db.vm.provision":puppet"do"|puppet|
    """"""""""puppet.manifests_path"=""./tools/puppet/manifests"
    """"""""""puppet.module_path"=""./tools/puppet/modules"
    """"""""""puppet.manifest_file""=""db.pp"
    """"""""""puppet.options"="['..verbose']
    """"""""""puppet.facter"="{
    """""""""""""""root_password""=>""foo",
    """"""""""}""""""
    """"""end""""""
    ""end""
    end

    View Slide

  103. $"vagrant"up
    Bringing"machine"'web'"up"with"'virtualbox'"provider...
    Bringing"machine"'db'"up"with"'virtualbox'"provider...
    [web]"Importing"base"box"'debian.7.1.0'...
    [web]"Matching"MAC"address"for"NAT"networking...
    [web]"Setting"the"name"of"the"VM...
    [web]"Clearing"any"previously"set"forwarded"ports...
    [web]"Creating"shared"folders"metadata...
    [web]"Clearing"any"previously"set"network"interfaces...
    [web]"Preparing"network"interfaces"based"on"configuration...
    [web]"Forwarding"ports...
    [web]".."22"=>"2222"(adapter"1)
    [web]"Booting"VM...
    [web]"Waiting"for"VM"to"boot."This"can"take"a"few"minutes.
    [web]"VM"booted"and"ready"for"use!
    [web]"Configuring"and"enabling"network"interfaces...
    [web]"Mounting"shared"folders...
    [web]".."/vagrant
    [web]".."/var/www
    [web]".."/tmp/vagrant.puppet/manifests
    [web]".."/tmp/vagrant.puppet/modules.0
    [web]"Running"provisioner:"shell...
    [web]"Running:"inline"script
    [web]"Running"provisioner:"puppet...
    Running"Puppet"with"web.pp...

    View Slide

  104. [db]"Importing"base"box"'debian.7.1.0'...
    [db]"Matching"MAC"address"for"NAT"networking...
    [db]"Setting"the"name"of"the"VM...
    [db]"Clearing"any"previously"set"forwarded"ports...
    [db]"Fixed"port"collision"for"22"=>"2222."Now"on"port"2200.
    [db]"Creating"shared"folders"metadata...
    [db]"Clearing"any"previously"set"network"interfaces...
    [db]"Preparing"network"interfaces"based"on"configuration...
    [db]"Forwarding"ports...
    [db]".."22"=>"2200"(adapter"1)
    [db]"Booting"VM...
    [db]"Waiting"for"VM"to"boot."This"can"take"a"few"minutes.
    [db]"VM"booted"and"ready"for"use!
    [db]"Configuring"and"enabling"network"interfaces...
    [db]"Mounting"shared"folders...
    [db]".."/vagrant
    [db]".."/tmp/vagrant.puppet/manifests
    [db]".."/tmp/vagrant.puppet/modules.0
    [db]"Running"provisioner:"shell...
    [db]"Running:"inline"script
    [db]"Running"provisioner:"puppet...
    Running"Puppet"with"db.pp...

    View Slide

  105. Control the machines

    View Slide

  106. vagrant"up
    vagrant"ssh
    vagrant"destroy
    vagrant"up"web
    vagrant"ssh"web
    vagrant"destroy"web
    vagrant"up"db
    vagrant"ssh"db
    vagrant"destroy"db
    All VM’s
    Primary VM
    All VM’s

    View Slide

  107. And$
    on$that$
    bombshell$...

    View Slide

  108. vagrant'up
    #FTW

    View Slide

  109. View Slide