Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Automate your Infrastructure with Chef

Automate your Infrastructure with Chef

Talk given at ConFoo 2013 on February 28th, 2013.

Christian Joudrey

February 28, 2013
Tweet

More Decks by Christian Joudrey

Other Decks in Programming

Transcript

  1. Automate
    your Infrastructure
    with Chef

    View Slide

  2. cjoudrey  
    @

    View Slide

  3. View Slide

  4. c
    #

    View Slide

  5. c
    #
    #
    #
    #
    #
    #
    #
    in minutes

    View Slide

  6. #
    d
    #
    d
    c
    #
    #
    #
    #
    #
    d
    in minutes

    View Slide

  7. w
    Manual setup
    takes
    time

    View Slide

  8. #
    ruby 1.9.3
    #
    ruby 1.9.2
    !=
    and error-prone

    View Slide

  9. #
    ruby 1.9.3
    #
    ruby 1.9.2
    !=
    Oops!
    and error-prone

    View Slide

  10. What is ?!
    Chef

    View Slide

  11. 1
    Manage servers
    with
    ruby code

    View Slide

  12. instead of
    $ ssh [email protected]
    Last login: Thu Feb 28
    ...
    # apt-get install nginx
    ...
    # vim /etc/nginx/nginx.conf
    ...
    # apt-get install ruby
    ...

    View Slide

  13. client server

    View Slide

  14. #
    node
    #
    node
    #
    node
    #
    chef server
    (server1 to server3.example.com)
    (chef.example.com)
    knife
    !
    (local machine)

    View Slide

  15. #
    node
    #
    node
    #
    node
    #
    chef server
    (server1 to server3.example.com)
    (chef.example.com)
    knife
    !
    (local machine)

    View Slide

  16. #
    node
    #
    node
    #
    node
    #
    chef server
    (server1 to server3.example.com)
    (chef.example.com)
    knife
    !
    (local machine)

    View Slide

  17. #
    node
    #
    node
    #
    node
    #
    chef server
    chef-client
    (server1 to server3.example.com)
    knife
    !
    (local machine)

    View Slide

  18. 2terminology
    Chef

    View Slide

  19. 2recipe
    Ruby file that contains Chef
    commands

    View Slide

  20. 2cookbook
    Collection of Chef recipes

    View Slide

  21. Getting started
    with
    Chef
    2

    View Slide

  22. git clone
    opscode/chef-repo
    https://github.com/opscode/chef-repo
    !

    View Slide

  23. !
    $ ls confoo
    ...
    cookbooks/
    data_bags/
    environments/
    roles/

    View Slide

  24. Install Chef on local
    machine
    !

    View Slide

  25. !
    gem install chef

    View Slide

  26. #
    Hosted* Chef server
    from Opscode
    * free up to 5 nodes

    View Slide

  27. #

    View Slide

  28. #

    View Slide

  29. Setup Knife on local
    machine
    !

    View Slide

  30. #
    node
    #
    node
    #
    node
    #
    chef server
    (server1 to server3.example.com)
    (chef.example.com)
    knife
    !
    (local machine)

    View Slide

  31. !
    $ ls confoo/.chef
    confoo-demo-validator.pem
    confoo-demo.pem
    knife.rb
    Copy files to REPO/.chef

    View Slide

  32. !
    $ cd confoo
    $ knife user list
    confoo-demo
    Test Knife

    View Slide

  33. 8
    Create your first
    cookbook
    $ cd confoo
    $ knife cookbook create
    nginx

    View Slide

  34. 8
    $ ls cookbooks/nginx
    ...
    attributes/
    providers/
    recipes/
    resources/
    templates/

    View Slide

  35. package "nginx"
    cookbooks/nginx/recipes/default.rb

    View Slide

  36. package installs using
    system’s package mgr

    View Slide

  37. cookbooks/nginx/recipes/default.rb
    package "nginx"
    service "nginx"

    View Slide

  38. service defines an
    available service

    View Slide

  39. cookbooks/nginx/recipes/default.rb
    package "nginx"
    service "nginx" do
    supports :status => true,
    :restart => true,
    :reload => true
    end

    View Slide

  40. cookbooks/nginx/recipes/default.rb
    package "nginx"
    service "nginx" do
    supports :status => true,
    :restart => true,
    :reload => true
    action [:enable, :start]
    end

    View Slide

  41. :enable start
    on server boot

    View Slide

  42. :start start
    when Chef runs

    View Slide

  43. 8
    Upload cookbook
    $ knife cookbook upload
    nginx
    Uploading nginx [0.1.0]

    View Slide

  44. Let’s test it
    on a node
    #

    View Slide

  45. !
    $ knife bootstrap \
    server1.example.com
    Bootstrap a node

    View Slide

  46. !

    View Slide

  47. 2run list
    Ordered list of recipes and roles
    that get run on the node

    View Slide

  48. !
    $ knife node edit \
    server1.example.com
    Edit a node

    View Slide

  49. {
    "name": "server1.example.com",
    "run_list": [
    ]
    }

    View Slide

  50. {
    "name": "server1.example.com",
    "run_list": [
    "recipe[nginx::default]"
    ]
    }

    View Slide

  51. recipe[nginx::default]
    means default recipe
    of nginx cookbook

    View Slide

  52. $ ssh server1.example.com
    server1:~# chef-client
    Run Chef on the node
    #

    View Slide

  53. #

    View Slide

  54. #

    View Slide

  55. #
    Let’s configure
    nginx

    View Slide

  56. copy from server
    to nginx cookbook
    templates/default/nginx.conf.erb
    /etc/nginx/nginx.conf
    !

    View Slide

  57. cookbooks/nginx/recipes/default.rb
    package "nginx"
    service "nginx" do
    supports :status => true,
    :restart => true,
    :reload => true
    action [:enable, :start]
    end
    template "/etc/nginx/nginx.conf" do
    source "nginx.conf.erb"
    notifies :reload, "service[nginx]"
    end

    View Slide

  58. cookbooks/nginx/recipes/default.rb
    package "nginx"
    service "nginx" do
    supports :status => true,
    :restart => true,
    :reload => true
    action [:enable, :start]
    end
    template "/etc/nginx/nginx.conf" do
    source "nginx.conf.erb"
    notifies :reload, "service[nginx]"
    end

    View Slide

  59. cookbooks/nginx/recipes/default.rb
    package "nginx"
    service "nginx" do
    supports :status => true,
    :restart => true,
    :reload => true
    action [:enable, :start]
    end
    template "/etc/nginx/nginx.conf" do
    source "nginx.conf.erb"
    notifies :reload, "service[nginx]"
    end

    View Slide

  60. cookbooks/nginx/recipes/default.rb
    package "nginx"
    service "nginx" do
    supports :status => true,
    :restart => true,
    :reload => true
    action [:enable, :start]
    end
    template "/etc/nginx/nginx.conf" do
    source "nginx.conf.erb"
    notifies :reload, "service[nginx]"
    end

    View Slide

  61. !
    Upload the cookbook
    and run chef-client
    on node

    View Slide

  62. #

    View Slide

  63. 2
    Chef is idempotent

    View Slide

  64. !
    What if we edit
    templates/default/nginx.conf.erb
    and run Chef

    View Slide

  65. #

    View Slide

  66. #

    View Slide

  67. #

    View Slide

  68. Let’s run Chef
    one more time
    #

    View Slide

  69. #

    View Slide

  70. 2Attributes

    View Slide

  71. nginx/templates/default/nginx.conf.erb
    user www-data;
    worker_processes 2;
    pid /var/run/nginx.pid;
    ...

    View Slide

  72. nginx/attributes/nginx.rb
    default['nginx']['worker_processes'] = 2

    View Slide

  73. nginx/templates/default/nginx.conf.erb
    user www-data;
    worker_processes <%= node['nginx']
    ['worker_processes'] %>;
    pid /var/run/nginx.pid;
    ...

    View Slide

  74. #
    Override for
    a specific node

    View Slide

  75. {
    "name": "server1.example.com",
    "run_list": [
    "recipe[nginx::default]"
    ]
    }

    View Slide

  76. {
    "name": "server1.example.com",
    "normal": {
    "nginx": {
    "worker_processes": 4
    },
    },
    "run_list": [
    "recipe[nginx::default]"
    ]
    }

    View Slide

  77. 2Roles

    View Slide

  78. roles/app-server.rb
    name 'app-server'
    description 'app-server stuff'
    run_list(
    'recipe[nginx::default]'
    )
    override_attributes(
    'nginx' => {
    'worker_processes' => 2
    }
    )

    View Slide

  79. !
    $ knife role from file \
    app-server.rb
    Upload a role

    View Slide

  80. Apply the
    role on a node
    #

    View Slide

  81. {
    "name": "server1.example.com",
    "run_list": [
    "role[app-server]"
    ]
    }

    View Slide

  82. #

    View Slide

  83. {
    "name": "server1.example.com",
    "run_list": [
    "role[base]",
    "role[app-server]"
    ]
    }

    View Slide

  84. 2
    Environments

    View Slide

  85. environments/production.rb
    name 'production'
    cookbook_versions 'nginx' => '= 0.1.0'

    View Slide

  86. {
    "name": "server1.example.com",
    "chef_environment": "production",
    "run_list": [
    "recipe[nginx::default]"
    ]
    }

    View Slide

  87. !
    Searching for nodes
    $ knife search node \
    role:app-server

    View Slide

  88. 2

    View Slide

  89. 8
    Searching can be
    done in recipes too!

    View Slide

  90. 8
    Searching can be
    done in recipes too!
    OMFG!

    View Slide

  91. backend app
    balance roundrobin
    server app1 10.10.0.1 check port 80
    server app2 10.10.0.2 check port 80
    server app3 10.10.0.3 check port 80

    View Slide

  92. nodes = search(
    :node,
    'role:app-server'
    )
    template "/etc/haproxy.conf" do
    source "haproxy.conf.erb"
    variables :nodes => nodes
    end

    View Slide

  93. backend www
    balance roundrobin
    <% @nodes.each do |n| %>
    server
    <%= n[:hostname] %>
    <%= n[:ipaddress] %>
    check port
    <% end %>

    View Slide

  94. 2Goodies

    View Slide

  95. View Slide

  96. View Slide

  97. View Slide

  98. View Slide

  99. View Slide

  100. Automation is
    important

    View Slide

  101. #
    staging/CI
    #
    production
    !
    development
    = =

    View Slide

  102. Thanks!

    View Slide

  103. cjoudrey  
    @

    View Slide