No content

Nathen •Technical Community Manager at Opscode •Co-host of the Food Fight Show Podcast •Meetup Organizer 

Opscode 

Chef is an automation platform for developers & systems engineers to continuously define, build, and manage infrastructure. CHEF USES: Recipes and Cookbooks that describe Infrastructure as Code. Chef enables people to easily build & manage complex & dynamic applications at massive scale • New model for describing infrastructure that promotes reuse • Programmatically provision and configure • Reconstruct business from code repository, data backup, and bare metal resources “ ” Chef 

http://www.flickr.com/photos/steffenz/337700069/ http://www.flickr.com/photos/kky/704056791/ Applications 

Infrastructure http://www.flickr.com/photos/sbh/462754460/ 

Collection of Resources http://www.flickr.com/photos/philliecasablanca/3354734116/ • Networking • Files • Directories • Symlinks • Mounts • Routes • Users • Groups • Tasks • Packages • Software • Services • Configuration • Other Stuff 

Acting in Concert http://www.flickr.com/photos/glowjangles/4081048126/ 

To Provide a Service http://www.flickr.com/photos/28309157@N08/3743455858/ 

And it Evolves http://www.flickr.com/photos/16339684@N00/2681435235/ 

Application Server See Node 

Application Server Application Database See Nodes 

Application Server Application Databases See Nodes Grow 

Application Servers Application Databases See Nodes Grow 

Application Servers Application Databases Load Balancer See Nodes Grow 

Application Servers Application Databases Load Balancers See Nodes Grow 

Application Servers Application Database Cache Load Balancers Application Databases See Nodes Grow 

Application Servers Application Database Cache Load Balancers Application Databases Tied Together with Configuration 

Application Servers Application Database Cache Load Balancers Floating IP? Application Databases Infrastructure is a Snowflake 

Load Balancers Application Servers NoSQL Database Slaves ApplicationCache Database Cache Database Evolving Complexity 

Configuration Management http://www.flickr.com/photos/philliecasablanca/3354734116/ 

New Requirements •A new developer joined the team, get her set-up with the application •Add New Relic monitoring to the application •Add a new module to the development site •Do not store the unencrypted admin password in the git repository 

Configuration Desperation http://www.flickr.com/photos/francoforeshock/5716969942/ 

• But you already guessed that, didn’t you? Chef Solves This Problem 

http://www.flickr.com/photos/louisb/4555295187/ • Programmatically provision and configure • Treat like any other code base • Reconstruct business from code repository, data backup, and bare metal resources. Chef is Infrastructure as Code 

http://www.flickr.com/photos/ssoosay/5126146763/ • Chef generates configurations directly on nodes from their run list • Reduce management complexity through abstraction • Store the configuration of your programs in version control Programs 

•Define Policy •Say what, not how •Pull not Push http://www.flickr.com/photos/bixentro/2591838509/ Declarative Interface to Resources 

That Looks Like This package "apache2" template "/etc/apache2/apache2.conf" do soucre "apache2.conf.erb" owner "root" group "root" mode "0644" variables(:allow_override => "All") notifies :reload, "service[apache2]" end service "apache2" do action [:enable,:start] supports :reload => true end 

• Recipes are collections of Resources • Cookbooks contain recipes, templates, files, custom resources, etc • Code re-use and modularity • Hundreds already on Community.opscode.com http://www.flickr.com/photos/shutterhacks/4474421855/ Recipes and Cookbooks 

http://www.flickr.com/photos/kathycsus/2686772625 • IP addresses • Hostnames • FQDNs • Search for nodes with Roles • Find configuration data Search 

pool_members = search("node","role:awesome_site) template "/etc/haproxy/haproxy.cfg" do source "haproxy-app_lb.cfg.erb" owner "root" group "root" mode "0644" variables :pool_members => pool_members.uniq notifies :restart, "service[haproxy]" end Pass Results To Templates 

# Set up application listeners here. listen application 0.0.0.0:80 balance roundrobin <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>: weight 1 maxconn 1 check <% end -%> <% if node["haproxy"]["enable_admin"] -%> listen admin 0.0.0.0:22002 mode http stats uri / <% end -%> Pass Results To Templates 

Drupal App Memcache MySQL Slaves MySQL Master Nagios Graphite So when this... 

Drupal App Memcache MySQL Slaves MySQL Master Nagios Graphite ...becomes this... 

Drupal App Memcache MySQL Slaves MySQL Master Nagios Graphite ...this can happen automatically 

Nagios Graphite Drupal App Memcache MySQL Slaves • Load balancer config • Nagios host ping • Nagios host ssh • Nagios host HTTP • Nagios host app health • Graphite CPU • Graphite Memory • Graphite Disk • Graphite SNMP • Memcache firewall • MySQL firewall • 11+ resource changes for 1 node addition Count the Resources 

Deploying Drupal with Chef 

Chef Components 

Our Workflow •Build Drupal locally •Deploy Drupal on EC2 •Iterate 

Local Development •Git repository •Virtual Machine •Chef •knife •Chef Server 

Git Repository $ mkdir -p ~/drupalcon/chef-repo $ cd ~/drupalcon/chef-repo $ git init 

Building Drupal Locally •Deploy in a local virtual machine managed by Vagrant, provisioned with Chef 

Vagrantfile Vagrant.configure("2") do |config| config.vm.hostname = "drupalcon-dev" config.vm.box = "opscode-ubuntu-12.04" config.vm.network :private_network, ip: "33.33.33.10" config.vm.network :forwarded_port, guest: 80, host: 8080 end 

Chef 

Chef Server 

Register VM with Chef Vagrant.configure("2") do |config| ... config.vm.provision :chef_client do |chef| chef.chef_server_url = "https://api.opscode.com/organizations/nhdrupalcon" chef.validation_key_path = ".chef/nhdrupalcon-validator.pem" chef.validation_client_name = "nhdrupalcon-validator" end end $ vagrant provision 

Chef Components 

Cookbooks •Download from community.opscode.com •Write our own •Upload to the Chef Server 

Community Site •950+ Cookbooks •Codify tribal knowledge •Reference 

Write our own Cookbook $ knife cookbook create awesome_site ** Creating cookbook awesome_site ** Creating README for cookbook: awesome_site ** Creating CHANGELOG for cookbook: awesome_site ** Creating metadata for cookbook: awesome_site 

Write our own Cookbook include_recipe "drupal" web_app "drupal" do template "drupal.conf.erb" docroot node['drupal']['dir'] server_name server_fqdn server_aliases node['fqdn'] end 

Data •Separate data from policy •Policy: Site has a document root •Data: /var/www/drupal •Policy: Apache has a Server Alias •Data: Fully-qualified domain name of the server web_app "drupal" do template "drupal.conf.erb" docroot node['drupal']['dir'] server_name server_fqdn server_aliases node['fqdn'] end 

Where does data come from? •Attributes •Data Bags •Encrypted Data Bags •Search 

Attributes •Specific details about a node •Defined by: •The state of the node •Cookbooks •Roles •Environments 

Ohai "languages": { "ruby": { }, "perl": { "version": "5.14.2", "archname": "x86_64-linux-gnu-thread-multi" }, "python": { "version": "2.7.3", "builddate": "Aug 1 2012, 05:14:39" }, "php": { "version": "5.3.10-1ubuntu3.6", "builddate": "(cli) (built: Mar" } }, "kernel": { "name": "Linux", "release": "3.2.0-32-virtual", "version": "#51-Ubuntu SMP Wed Sep 26 21:53:42 UTC 2012", "machine": "x86_64", "modules": { "isofs": { "size": "40257", "refcount": "0" }, "acpiphp": { "size": "24231", "refcount": "0" } }, "os": "GNU/Linux" }, "os": "linux", "os_version": "3.2.0-32-virtual", "ohai_time": 1369328621.3456137, "network": { "interfaces": { "lo": { "mtu": "16436", "flags": [ "LOOPBACK", "UP", "LOWER_UP" ], "encapsulation": "Loopback", "addresses": { "127.0.0.1": { "family": "inet", "prefixlen": "8", "netmask": "255.0.0.0", "scope": "Node" }, "::1": { "family": "inet6", "prefixlen": "128", "scope": "Node" } }, "state": "unknown" }, "eth0": { "type": "eth", "number": "0", "mtu": "1500", 

Attributes •Cookbooks, Roles, and Environments can also set attribute data •Extremely flexible mechanism for configuration •Cookbook: •Environment: default['drupal']['modules'] = ['views', 'webform'] default_attributes({ "drupal" => {"modules" => ["views","webform","token"]}}) 

Data Bags •Global Data •Stored in JSON •Accessible from the Chef Server •Can be encrypted { "id": "nharvey", "groups": ["sysadmin"], "uid": 2001, "shell": "/bin/bash", "comment": "Nathen Harvey ", "nagios": { "email": "[email protected]" } } 

Search pool_members = search("node","awesome_site) template "/etc/haproxy/haproxy.cfg" do source "haproxy-app_lb.cfg.erb" owner "root" group "root" mode "0644" variables :pool_members => pool_members.uniq notifies :restart, "service[haproxy]" end 

Upload our Cookbooks •We’re using •Cookbooks from the community site •A cookbook we wrote •Upload the cookbooks using knife $ knife cookbook upload -a 

Deploy Drupal 

Run List •Ordered list of roles or recipes to be run on a node •The run list represents the policy to be applied •chef-client •Executes on the node on a regular basis •Receives the run list from the Chef Server •Brings the node in-line with the policy 

Run List Server Server Server Server chef-server API chef-client “role[webserver]” node ntp client.rb openssh server.rb apache default.rb drupal default.rb chef-client “role[database]” node ntp client.rb openssh server.rb mysql server.rb 

Run chef-client config.vm.provision :chef_client do |chef| chef.chef_server_url = "https://api.opscode.com/organizations/nhdrupalcon" chef.validation_key_path = ".chef/nhdrupalcon-validator.pem" chef.validation_client_name = "nhdrupalcon-validator" chef.add_recipe "awesome_site" end $ vagrant provision 

Next Steps •Share the Vagrantfile with your teammates •Develop to a releasable state •Deploy to production 

Production Deploy •We’ll use Amazon’s EC2 for our initial deploy •...but we could deploy to any server •Using the same cookbooks as our development environment, deploy to EC2 $ knife ec2 server create -r "recipe[awesome_site]" -f m1.medium -I ami-641c8e0d -N ec2_drupal 

Current State •Local virtual machine for development running Drupal •“Production” instance on EC2 

Next Steps •Move Database to a different node •Add a Load Balancer and additional application servers •Add monitoring •Add log aggregation •etc. 

More about Chef 

Community 

Open Source •Apache 2 Software License •Continually growing number of contributors! •1400+ individuals, 200+ companies •Development repositories: •github.com/opscode •github.com/opscode-cookbooks •Regular code reviews via Google+ Hangout 

Get Started on Your Own •http://learnchef.com •http://docs.opscode.com •http://lists.opscode.com •#chef on Freenode IRC 

Wrap-up & Special Offer 

Infrastructure as Code •Repeatability •Automation •Agility •Scalability •Reassurance •Disaster Recovery •Happiness 

Thanks! •Promet Source •Will Milton •github.com/promet/drupal_cookbook •curl -L bit.ly/10McZSZ | bash; vagrant up •Marius Ducea •community.opscode.com/cookbooks/drupal 

Chef Training •Chef Introductory Workshop •Tomorrow in Corvallis •FREE to all DrupalCon Attendees •http://bit.ly/learnchef-drupalcon 

Thank You! •What Questions Do You Have? •@nathenharvey •Free Training Tomorrow (register by 9PM) •bit.ly/learnchef-drupalcon •Feedback on this session: portland2013.drupal.org/node/2963