• List the components of Chef • Create a new cookbook • Use some primitives of Chef recipes • Follow some common workflows used by experienced Chef users/developers
course. • We will do some hands on exercises. • You should get a taste for automating with Chef. • You should have received instructions for prerequisites prior to this tutorial • Chef works and runs on Windows, but the exercises for this tutorial are Linux-based
key auth • Infrastructure data store such as node data • Search Service • Derivative Services? http://www.flickr.com/photos/core-materials/4419853626/sizes/o/in/photostream/
License, Version 2.0 • Hundreds of Individual and Corporate contributors. • Hundreds of cookbooks available from the community • http://community.opscode.com
• Treat like any other code base • Reconstruct business from code repository, data backup, and bare metal resources. http://www.flickr.com/photos/louisb/4555295187/
"apache2.conf.erb" owner "www-data" group "www-data" mode 00644 notifies :restart, "service[apache2]" end service "apache2" do supports :status => true, :restart => true action [:enable, :start] end Declarative abstraction to system resources
"apache2.conf.erb" owner "www-data" group "www-data" mode 00644 notifies :restart, "service[apache2]" end service "apache2" do supports :status => true, :restart => true action [:enable, :start] end
end template "/etc/apache2/apache2.conf" do source "apache2.conf.erb" owner "www-data" group "www-data" mode 00644 notifies :restart, "service[apache2]" end service "apache2" do supports :status => true, :restart => true action [:enable, :start] end
related components • Files, Templates, Libraries • A cookbook is responsible for configuring a single thing, e.g. • apache2 • postgresql • A recipe is responsible for a component, e.g. • api • server • client
exercises will be related to creating an "apache" cookbook. • The goals are to learn elements of Chef, not to learn Apache. We're going to do things the hard way, by typing in a lot of code. We have some gists with large sections of code already available to reduce what you need to type in certain places. • Errors and typos are good, as they will help students learn how to resolve errors.
whoami i-am-a-workstation This is an example of a command to run on your workstation user@hostname:~$ whoami i-am-a-chef-node This is an example of a command to run on your target node via SSH.
an empty chef-repo • Have a unique User created • Have an Organization created for use during training • Understand what Knife is • Have a working Knife configuration
Mac OS X / Linux % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 6515 100 6515 0 0 20600 0 --:--:-- --:--:-- --:--:-- 31172 Downloading Chef for ubuntu... Installing Chef Selecting previously unselected package chef. (Reading database ... 47446 files and directories currently installed.) Unpacking chef (from .../tmp.MqRJP6lz/chef__amd64.deb) ... Setting up chef (11.4.4-2.ubuntu.11.04) ... Thank you for installing Chef! Processing triggers for initramfs-tools ... update-initramfs: Generating /boot/initrd.img-3.2.0-48-virtual
clicking this • Unzip the zipfile - you’ll get a “chef- repo” • Put the “chef-repo” somewhere, e.g.: • C:\Users\you\chef-repo (Win) • /Users/you/chef-repo (Mac) • /home/you/chef-repo (Linux)
an interface between a local Chef repository and the Chef Server. Knife lets you manage: • Nodes • Cookbooks and recipes • Roles • Stores of JSON data (data bags), including encrypted data • Environments • Cloud resources, including provisioning • The installation of Chef on management workstations • Searching of indexed data on the Chef Server
“knife bootstrap” • Understand how knife bootstrap configures a node to use the Organization created in the previous section • Understand the basic configuration needed to run chef-client
to create a new cookbook • Understand what a recipe is • Understand how to use the package, service, and template resources • Know how to upload a cookbook to the Chef Server • Understand what a run list is, and how to set it • How to read the output of the chef-client run
"package" for Chef recipes • It contains all the recipes, files, templates, libraries, etc. required to configure a portion of your infrastructure • Typically cookbooks map 1:1 to a piece of software or functionality. • "our tomcat cookbook" • "our zlib cookbook" • "our security_policy cookbook"
the infrastructure • They're artifacts that have a name and a version • Metadata can also have information about the cookbook like its license and maintainer • Cookbooks can depend on other cookbooks, too
# # Copyright 2013, YOUR_COMPANY_NAME # # All rights reserved - Do Not Redistribute # package "apache2" do action :install end Exercise: Add a package resource to install Apache to the default recipe OPEN IN EDITOR: cookbooks/apache/recipes/default.rb
Redistribute # package "apache2" do action :install end service "apache2" do action [ :enable, :start ] end Exercise: Add a service resource to ensure the service is started and enabled at boot OPEN IN EDITOR: cookbooks/apache/recipes/default.rb
] end template "/var/www/index.html" do source "index.html.erb" mode "0644" end Add a template resource to write the home page OPEN IN EDITOR: cookbooks/apache/recipes/default.rb
resource we just wrote... • Is a template resource • Whose name is /var/www/index.html • With two parameters: • source of index.html.erb • mode of “0644”
• Has no action! • If you omit the action in Chef, we default to the most common positive action. In this case, it is the :create action. template "/var/www/index.html" do source "index.html.erb" mode "0644" end
apache # Recipe:: default # # Copyright 2013, YOUR_COMPANY_NAME # # All rights reserved - Do Not Redistribute # package "apache2" do action :install end service "apache2" do action [ :enable, :start ] end template "/var/www/index.html" do source "index.html.erb" mode "0644" end
to API endpoints • The verb indicates what we're doing with the cookbook • In this case we upload the cookbook to the server through the API. It's stored by the server... • We can show information about it too...
[sudo] password for opscode: Starting Chef Client, version 11.6.0 resolving cookbooks for run list: [] Synchronizing Cookbooks: Compiling Cookbooks... [2013-09-23T14:31:38+00:00] WARN: Node target1 has an empty run list. Converging 0 resources Chef Client finished, 0 resources updated
Starting Chef Client, version 11.6.0 resolving cookbooks for run list: ["apache"] Synchronizing Cookbooks: - apache Compiling Cookbooks... Converging 3 resources Recipe: apache::default * package[apache2] action install - install version 2.2.22-1ubuntu1 of package apache2 * service[apache2] action start (up to date) * service[apache2] action enable (up to date) * template[/var/www/index.html] action create - update content in file /var/www/index.html from 94850c to 463a6b --- /var/www/index.html 2013-09-23 14:34:24.378187238 +0000 +++ /tmp/chef-rendered-template20130923-3442-1gji5jy 2013-09-23 14:34:28.174187236 +0000 @@ -1,4 +1 @@ -<html><body><h1>It works!</h1> -<p>This is the default web page for this server.</p> -<p>The web server software is running but no content has been added, yet.</p> -</body></html> +<p>Hello, world</p> Chef Client finished, 2 resources updated
start (up to date) Why wasn't the service updated? • Chef takes idempotent actions on resources to ensure they are converged to the desired state • The apache2 package on Ubuntu automatically enables and starts the apache2 service in its post- install script • Thus, Chef didn't need to do anything
phases during "convergence" • Evaluate all the Ruby code in the recipe, looking for "Chef Resources" • Execute the providers' actions to put each resource in the declared state
Server stores node object data • It becomes searchable through the API with knife and with recipes • Some of the data comes from ohai, which takes an inventory of the system and emits JSON data • You can add data to the node through attributes in cookbooks, roles, directly on a node, etc
in your cookbooks' attributes files • Use "sane" defaults - no surprises • You can use attributes in roles to set new values • Roles take precedence over cookbook settings • When a value must be set to a certain value, use override, but use this sparingly • You can’t override ohai's (automatic) attributes!
are generic, arbitrary stores of information about the infrastructure. • Data Bag Items are JSON data • Our apache cookbook provides a good baseline • We'll drive site-specific virtual hosts with data bags
[:start, :enable] end execute "a2dissite default" do only_if do File.symlink?("/etc/apache2/sites-enabled/000-default") end notifies :restart, "service[apache2]" end template "/var/www/index.html" do Disable the default apache site
apache::vhosts recipe to manage the virtual hosts we created in data bag items • There's a number of new things to talk about in this recipe • We'll take this nice and slow :)
actions to be taken only if its argument returns false • The only_if parameter is the opposite of not_if - the actions are taken only if the arguments return true • Both not_if and only_if are part of Chef (resources), not part of Ruby
a string, or a Ruby block argument (do..end or {..}) • When the argument is a string, Chef evaluates it as a shell command to run. • When the argument is a Ruby block, Chef evaluates it as Ruby code to execute. • This is the equivalent to the code we wrote: not_if "test -L /etc/apache2/sites-enabled/#{site_name}"
source "index.html.erb" mode "0644" variables( :site_name => site_name, :port => site_data["port"] ) end end Index for each vhost https://gist.github.com/6673845
<p>My name is <%= node['hostname'] %></p> <p>We love <%= @site_name %></p> <p>Served from <%= node['ipaddress'] %>:<%= @port %></p> Update the index.html template
adding recipes directly to our single node's run list • But that’s not how infrastructure works - think about how you refer to servers • "It’s a web server" • "It’s a database server" • or, "It's a database-master server"... • "It’s a monitoring server"
encapsulate the run lists and attributes required for a server to "be" what you already think it is • In practice, roles make it easy to configure many nodes identically without repeating yourself each time • Roles are a first class API primitive on the Chef Server
creating roles • The best practice is that all of your roles live in the roles directory of your chef-repo • They can be created via the API and knife, but having them in source control gives you the history of changes
• Roles may have a description • Roles may have a run_list, just like a node • Roles may set node attributes • default_attributes • override_attributes http://docs.opscode.com/essentials_roles.html
flexibility • Set a "sane default" that will be used in a cookbook • Override it easily in a role (higher priority) • In all, there are 15 places where attributes come from (!!) • In practice, you'll use 2-3 most of the time. • The others are there when you need them. • http://docs.opscode.com/chef_overview_attributes.html
are not attributes • Data bags are a separate API end point • Data bags are not tied to a specific node or role in the infrastructure • Not even necessarily tied to anything, just data you want to store
• We can now search the Chef Server for all the "webserver" nodes • This is relevant for a variety of reasons • Load balancing several front ends (search from nginx, or haproxy for example) • Monitoring HTTP (search from nagios cookbook) • Graphing traffic (search from munin cookbook)
default (manages apache package and service) • vhosts (iterates over data bags and renders vhost configuration) • However, Opscode publishes an "apache2" cookbook that manages much much more, including all apache2 configuration, modules, sites Debian style w/ a2ensite/a2enmod, etc
an example • Directory structure follows the data bag API end point and "knife upload" makes it easy to upload everything • Create additional vhosts to see how this is dynamically expanded easily
the chef_server_url and the node_name values in ./chef/knife.rb • Get the validation client key from the Chef Server (this differs by implementation): • Open Source: •/etc/chef-server/chef-validator.pem • Hosted Chef / Private Chef • Download after creating an organization
different resources • Packages, files, services, users, symlinks, registry keys, and more • Each resource has one or more providers • Some resources have platform-specific providers (e.g., package, service, user, group) http://docs.opscode.com/resource.html
Ruby environment to be used in a Chef recipe • gem_package - install a RubyGem to be used by the system or an application http://docs.opscode.com/resource.html
common init systems available • Each platform has it's own provider • arch, debian, freebsd, gentoo, "init", insserv • invokercd, macosx (launchd), redhat, "simple" • solaris (SMF), upstart, windows http://docs.opscode.com/resource.html
that include new custom Chef Resources • apt (apt_repository) • aws (aws_ebs_volume, aws_elastic_ip, and more) • yum (yum_repository) • windows (windows_package, windows_feature, and more) • homebrew (homebrew_package) • runit (runit_service) • many more! http://docs.opscode.com/chef/lwrps_custom.html
nathenharvey
moulongzhang
you
saorimurooka
kazzpapa3
kworkdev
jacopen
takahiromatsui
estie
ota1022
gotok365
oracle4engineer
k_adachi_01
reverentgeek
smashingmag
sergeychernyshev
skipperchong
gr2m
inesmontani
aleyda
minecr
jcasabona
tenderlove
mkilby
![$ cd chef-repo Verify that knife is working [~/chef-repo]$](https://files.speakerdeck.com/presentations/98ffbba008530131a2491a4d92d43c23/slide_56.jpg){kind=link}
![service "apache2" do action [ :enable, :start ] end So](https://files.speakerdeck.com/presentations/98ffbba008530131a2491a4d92d43c23/slide_91.jpg){kind=link}
![service "apache2" do action [ :enable, :start ] end So](https://files.speakerdeck.com/presentations/98ffbba008530131a2491a4d92d43c23/slide_92.jpg){kind=link}
![service "apache2" do action [ :enable, :start ] end So](https://files.speakerdeck.com/presentations/98ffbba008530131a2491a4d92d43c23/slide_93.jpg){kind=link}
![service "apache2" do action [ :enable, :start ] end So](https://files.speakerdeck.com/presentations/98ffbba008530131a2491a4d92d43c23/slide_94.jpg){kind=link}
![$ knife node run list add target1 “recipe[apache]” Add apache](https://files.speakerdeck.com/presentations/98ffbba008530131a2491a4d92d43c23/slide_111.jpg){kind=link}
![* service[apache2] action enable (up to date) * service[apache2] action](https://files.speakerdeck.com/presentations/98ffbba008530131a2491a4d92d43c23/slide_113.jpg){kind=link}
![OPEN IN EDITOR: SAVE FILE! cookbooks/apache/attributes/default.rb default["apache"]["greeting"] = "World" Create](https://files.speakerdeck.com/presentations/98ffbba008530131a2491a4d92d43c23/slide_130.jpg){kind=link}
![OPEN IN EDITOR: SAVE FILE! cookbooks/apache/templates/default/index.html.erb <p>Hello, <%= node['apache']['greeting'] %>!</p>](https://files.speakerdeck.com/presentations/98ffbba008530131a2491a4d92d43c23/slide_132.jpg){kind=link}
![Checkpoint • We have a node attribute • node["apache"]["greeting"] •](https://files.speakerdeck.com/presentations/98ffbba008530131a2491a4d92d43c23/slide_135.jpg){kind=link}
![OPEN IN EDITOR: SAVE FILE! cookbooks/apache/templates/default/index.html.erb <p>Hello, <%= node['apache']['greeting'] %>!</p>](https://files.speakerdeck.com/presentations/98ffbba008530131a2491a4d92d43c23/slide_157.jpg){kind=link}
![$ knife node run list add target1 "recipe[apache::vhosts]" Update the](https://files.speakerdeck.com/presentations/98ffbba008530131a2491a4d92d43c23/slide_162.jpg){kind=link}
![$ knife node run list remove target1 "recipe[apache::vhosts]" Remove the](https://files.speakerdeck.com/presentations/98ffbba008530131a2491a4d92d43c23/slide_173.jpg){kind=link}
![$ knife node run list remove target1 "recipe[apache]" Remove the](https://files.speakerdeck.com/presentations/98ffbba008530131a2491a4d92d43c23/slide_174.jpg){kind=link}
![$ knife node run list add target1 "role[webserver]" Add the](https://files.speakerdeck.com/presentations/98ffbba008530131a2491a4d92d43c23/slide_175.jpg){kind=link}
![opscode@target1:~$ sudo chef-client Run chef-client * template[/srv/apache/bears/index.html] action create -](https://files.speakerdeck.com/presentations/98ffbba008530131a2491a4d92d43c23/slide_176.jpg){kind=link}
![Thank you! Questions & Answers [email protected] [email protected]](https://files.speakerdeck.com/presentations/98ffbba008530131a2491a4d92d43c23/slide_216.jpg){kind=link}