Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Automate your Infrastructure with Chef

Automate your Infrastructure with Chef

Talk given at ConFoo 2013 on February 28th, 2013.

Christian Joudrey

February 28, 2013
Tweet

More Decks by Christian Joudrey

Other Decks in Programming

Transcript

  1. Automate
    your Infrastructure
    with Chef

    View full-size slide

  2. cjoudrey  
    @

    View full-size slide

  3. c
    #
    #
    #
    #
    #
    #
    #
    in minutes

    View full-size slide

  4. #
    d
    #
    d
    c
    #
    #
    #
    #
    #
    d
    in minutes

    View full-size slide

  5. w
    Manual setup
    takes
    time

    View full-size slide

  6. #
    ruby 1.9.3
    #
    ruby 1.9.2
    !=
    and error-prone

    View full-size slide

  7. #
    ruby 1.9.3
    #
    ruby 1.9.2
    !=
    Oops!
    and error-prone

    View full-size slide

  8. What is ?!
    Chef

    View full-size slide

  9. 1
    Manage servers
    with
    ruby code

    View full-size slide

  10. instead of
    $ ssh root@app1
    Last login: Thu Feb 28
    ...
    # apt-get install nginx
    ...
    # vim /etc/nginx/nginx.conf
    ...
    # apt-get install ruby
    ...

    View full-size slide

  11. client server

    View full-size slide

  12. #
    node
    #
    node
    #
    node
    #
    chef server
    (server1 to server3.example.com)
    (chef.example.com)
    knife
    !
    (local machine)

    View full-size slide

  13. #
    node
    #
    node
    #
    node
    #
    chef server
    (server1 to server3.example.com)
    (chef.example.com)
    knife
    !
    (local machine)

    View full-size slide

  14. #
    node
    #
    node
    #
    node
    #
    chef server
    (server1 to server3.example.com)
    (chef.example.com)
    knife
    !
    (local machine)

    View full-size slide

  15. #
    node
    #
    node
    #
    node
    #
    chef server
    chef-client
    (server1 to server3.example.com)
    knife
    !
    (local machine)

    View full-size slide

  16. 2terminology
    Chef

    View full-size slide

  17. 2recipe
    Ruby file that contains Chef
    commands

    View full-size slide

  18. 2cookbook
    Collection of Chef recipes

    View full-size slide

  19. Getting started
    with
    Chef
    2

    View full-size slide

  20. git clone
    opscode/chef-repo
    https://github.com/opscode/chef-repo
    !

    View full-size slide

  21. !
    $ ls confoo
    ...
    cookbooks/
    data_bags/
    environments/
    roles/

    View full-size slide

  22. Install Chef on local
    machine
    !

    View full-size slide

  23. !
    gem install chef

    View full-size slide

  24. #
    Hosted* Chef server
    from Opscode
    * free up to 5 nodes

    View full-size slide

  25. Setup Knife on local
    machine
    !

    View full-size slide

  26. #
    node
    #
    node
    #
    node
    #
    chef server
    (server1 to server3.example.com)
    (chef.example.com)
    knife
    !
    (local machine)

    View full-size slide

  27. !
    $ ls confoo/.chef
    confoo-demo-validator.pem
    confoo-demo.pem
    knife.rb
    Copy files to REPO/.chef

    View full-size slide

  28. !
    $ cd confoo
    $ knife user list
    confoo-demo
    Test Knife

    View full-size slide

  29. 8
    Create your first
    cookbook
    $ cd confoo
    $ knife cookbook create
    nginx

    View full-size slide

  30. 8
    $ ls cookbooks/nginx
    ...
    attributes/
    providers/
    recipes/
    resources/
    templates/

    View full-size slide

  31. package "nginx"
    cookbooks/nginx/recipes/default.rb

    View full-size slide

  32. package installs using
    system’s package mgr

    View full-size slide

  33. cookbooks/nginx/recipes/default.rb
    package "nginx"
    service "nginx"

    View full-size slide

  34. service defines an
    available service

    View full-size slide

  35. cookbooks/nginx/recipes/default.rb
    package "nginx"
    service "nginx" do
    supports :status => true,
    :restart => true,
    :reload => true
    end

    View full-size slide

  36. cookbooks/nginx/recipes/default.rb
    package "nginx"
    service "nginx" do
    supports :status => true,
    :restart => true,
    :reload => true
    action [:enable, :start]
    end

    View full-size slide

  37. :enable start
    on server boot

    View full-size slide

  38. :start start
    when Chef runs

    View full-size slide

  39. 8
    Upload cookbook
    $ knife cookbook upload
    nginx
    Uploading nginx [0.1.0]

    View full-size slide

  40. Let’s test it
    on a node
    #

    View full-size slide

  41. !
    $ knife bootstrap \
    server1.example.com
    Bootstrap a node

    View full-size slide

  42. 2run list
    Ordered list of recipes and roles
    that get run on the node

    View full-size slide

  43. !
    $ knife node edit \
    server1.example.com
    Edit a node

    View full-size slide

  44. {
    "name": "server1.example.com",
    "run_list": [
    ]
    }

    View full-size slide

  45. {
    "name": "server1.example.com",
    "run_list": [
    "recipe[nginx::default]"
    ]
    }

    View full-size slide

  46. recipe[nginx::default]
    means default recipe
    of nginx cookbook

    View full-size slide

  47. $ ssh server1.example.com
    server1:~# chef-client
    Run Chef on the node
    #

    View full-size slide

  48. #
    Let’s configure
    nginx

    View full-size slide

  49. copy from server
    to nginx cookbook
    templates/default/nginx.conf.erb
    /etc/nginx/nginx.conf
    !

    View full-size slide

  50. cookbooks/nginx/recipes/default.rb
    package "nginx"
    service "nginx" do
    supports :status => true,
    :restart => true,
    :reload => true
    action [:enable, :start]
    end
    template "/etc/nginx/nginx.conf" do
    source "nginx.conf.erb"
    notifies :reload, "service[nginx]"
    end

    View full-size slide

  51. cookbooks/nginx/recipes/default.rb
    package "nginx"
    service "nginx" do
    supports :status => true,
    :restart => true,
    :reload => true
    action [:enable, :start]
    end
    template "/etc/nginx/nginx.conf" do
    source "nginx.conf.erb"
    notifies :reload, "service[nginx]"
    end

    View full-size slide

  52. cookbooks/nginx/recipes/default.rb
    package "nginx"
    service "nginx" do
    supports :status => true,
    :restart => true,
    :reload => true
    action [:enable, :start]
    end
    template "/etc/nginx/nginx.conf" do
    source "nginx.conf.erb"
    notifies :reload, "service[nginx]"
    end

    View full-size slide

  53. cookbooks/nginx/recipes/default.rb
    package "nginx"
    service "nginx" do
    supports :status => true,
    :restart => true,
    :reload => true
    action [:enable, :start]
    end
    template "/etc/nginx/nginx.conf" do
    source "nginx.conf.erb"
    notifies :reload, "service[nginx]"
    end

    View full-size slide

  54. !
    Upload the cookbook
    and run chef-client
    on node

    View full-size slide

  55. 2
    Chef is idempotent

    View full-size slide

  56. !
    What if we edit
    templates/default/nginx.conf.erb
    and run Chef

    View full-size slide

  57. Let’s run Chef
    one more time
    #

    View full-size slide

  58. nginx/templates/default/nginx.conf.erb
    user www-data;
    worker_processes 2;
    pid /var/run/nginx.pid;
    ...

    View full-size slide

  59. nginx/attributes/nginx.rb
    default['nginx']['worker_processes'] = 2

    View full-size slide

  60. nginx/templates/default/nginx.conf.erb
    user www-data;
    worker_processes <%= node['nginx']
    ['worker_processes'] %>;
    pid /var/run/nginx.pid;
    ...

    View full-size slide

  61. #
    Override for
    a specific node

    View full-size slide

  62. {
    "name": "server1.example.com",
    "run_list": [
    "recipe[nginx::default]"
    ]
    }

    View full-size slide

  63. {
    "name": "server1.example.com",
    "normal": {
    "nginx": {
    "worker_processes": 4
    },
    },
    "run_list": [
    "recipe[nginx::default]"
    ]
    }

    View full-size slide

  64. roles/app-server.rb
    name 'app-server'
    description 'app-server stuff'
    run_list(
    'recipe[nginx::default]'
    )
    override_attributes(
    'nginx' => {
    'worker_processes' => 2
    }
    )

    View full-size slide

  65. !
    $ knife role from file \
    app-server.rb
    Upload a role

    View full-size slide

  66. Apply the
    role on a node
    #

    View full-size slide

  67. {
    "name": "server1.example.com",
    "run_list": [
    "role[app-server]"
    ]
    }

    View full-size slide

  68. {
    "name": "server1.example.com",
    "run_list": [
    "role[base]",
    "role[app-server]"
    ]
    }

    View full-size slide

  69. 2
    Environments

    View full-size slide

  70. environments/production.rb
    name 'production'
    cookbook_versions 'nginx' => '= 0.1.0'

    View full-size slide

  71. {
    "name": "server1.example.com",
    "chef_environment": "production",
    "run_list": [
    "recipe[nginx::default]"
    ]
    }

    View full-size slide

  72. !
    Searching for nodes
    $ knife search node \
    role:app-server

    View full-size slide

  73. 8
    Searching can be
    done in recipes too!

    View full-size slide

  74. 8
    Searching can be
    done in recipes too!
    OMFG!

    View full-size slide

  75. backend app
    balance roundrobin
    server app1 10.10.0.1 check port 80
    server app2 10.10.0.2 check port 80
    server app3 10.10.0.3 check port 80

    View full-size slide

  76. nodes = search(
    :node,
    'role:app-server'
    )
    template "/etc/haproxy.conf" do
    source "haproxy.conf.erb"
    variables :nodes => nodes
    end

    View full-size slide

  77. backend www
    balance roundrobin
    <% @nodes.each do |n| %>
    server
    <%= n[:hostname] %>
    <%= n[:ipaddress] %>
    check port
    <% end %>

    View full-size slide

  78. Automation is
    important

    View full-size slide

  79. #
    staging/CI
    #
    production
    !
    development
    = =

    View full-size slide

  80. cjoudrey  
    @

    View full-size slide