Share
Embed
What's New in Chef 11 - Philly DevOps
Introduction to Chef and new features of version 11. Given at the Philly DevOps meetup group.
Nathen Harvey
February 27, 2013
Transcript
81 slides
-
Nathen Harvey • Technical Community Manager • Co-host Food Fight
Show • http://foodfightshow.org • @nathenharvey -
Chef is an automation platform for developers & systems engineers
to continuously define, build, and manage infrastructure. CHEF USES: Recipes and Cookbooks that describe Infrastructure as Code. Chef enables people to easily build & manage complex & dynamic applications at massive scale • New model for describing infrastructure that promotes reuse • Programmatically provision and configure • Reconstruct business from code repository, data backup, and bare metal resources “ ” Chef -
Collection of Resources http://www.flickr.com/photos/philliecasablanca/3354734116/ • Networking • Files • Directories
• Symlinks • Mounts • Routes • Users • Groups • Tasks • Packages • Software • Services • Configuration • Other Stuff -
Application Servers Application Database Cache Load Balancers Application Databases Tied
Together with Configuration -
Application Servers Application Database Cache Load Balancers Floating IP? Application
Databases Infrastructure is a Snowflake -
Load Balancers Application Servers NoSQL Database Slaves ApplicationCache Database Cache
Database Evolving Complexity -
Golden Images are not the answer • Gold is heavy
• Hard to transport • Hard to mold • Easy to lose configuration detail http://www.flickr.com/photos/garysoup/2977173063/ -
Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite •
Move SSH off port 22 • Lets put it on 2022 New Compliance Mandate! -
Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite •
edit /etc/ssh/sshd_config 1 2 3 4 5 6 6 Golden Image Updates -
Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite •
Delete, launch 1 2 3 4 5 6 7 8 9 10 11 12 • Repeat • Typically manually 12 Instance Replacements -
• Don’t break anything! • Bob just got fired =(
5 Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite 1 2 4 5 6 7 8 9 10 11 12 3 Done in Maintenance Windows -
Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite •
Invalid configs! Different IP Addresses? -
http://www.flickr.com/photos/louisb/4555295187/ • Programmatically provision and configure • Treat like any
other code base • Reconstruct business from code repository, data backup, and bare metal resources. Chef is Infrastructure as Code -
http://www.flickr.com/photos/ssoosay/5126146763/ • Chef generates configurations directly on nodes from their
run list • Reduce management complexity through abstraction • Store the configuration of your programs in version control Programs -
• Define Policy • Say what, not how • Pull
not Push Code Sample http://www.flickr.com/photos/bixentro/2591838509/ Declarative Interface to Resources -
package "ntp" do action :install end service "ntpd" do action
[:enable,:start] end template "/etc/ntpd.conf" do source "ntpd.conf.erb" owner "root" group "root" mode 0644 action :create variables(:time_server => “time.example.com”) notifies :restart, “service[ntpd]” end That looks like this -
Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite This
can happen automatically -
Nagios Graphite Jboss App Memcache Postgres Slaves • Load balancer
config • Nagios host ping • Nagios host ssh • Nagios host HTTP • Nagios host app health • Graphite CPU • Graphite Memory • Graphite Disk • Graphite SNMP • Memcache firewall • Postgres firewall • Postgres authZ config • 12+ resource changes for 1 node addition Count the resources -
Blueprint Your Infrastructure • Compute • Application • Storage •
Security • Network • Configuration Standards How Can Chef Help? Build Anything… • Provision compute resources in the Data Center and the Cloud • Infrastructure • Application Stacks • Big Data • HPC And Manage It Simply • Introduce continuous incremental change or total change. • Automatically reconfigure everything • Re-provision for disaster recovery • Fail-over to bare metal • Monitor for compliance • Cloud migrations become trivial Using 1,000’s of man-days of prior art! Discoverable and Searchable Infrastructure -
© Opscode, 2011 – Confidential – DO NOT DISTRIBUTE Chef
Provides a Model for Reuse That Works 42 800+ Cookbooks “Yesterday we started open sourcing some of our Opscode Chef work created at bestbuy.com; bit.ly/ yDV9Hl #Splunk #opschef #expectmor” -
What's New Since Chef 0.10? • Windows support • Librarian
• Spiceweasel • Foodcritic • Cookbooks in separate repositories • Private Chef • Food Fight podcast • knife plugins for openstack, hp, azure, gce • 0.10 renumbered to Chef 10 • full-stack client • why-run • output formatters • Test Kitchen • Solaris support • Chef Developer Summit(2!) • Berkshelf • ChefConf • Fauxhai • docs.opscode.com • Hangouts for reviews • nyan-cat output formatter -
What is NOT in Chef 11 • CouchDB • Ruby-based
chef-server-api • depsolver, gecode, treetop • Merb • OpenID support in Web UI • Migration tooling (coming soon) http://trainweb.org/carl/Pullman150/IMG_3380.jpg -
What’s NEW in Chef 11 • PostgreSQL • erchef •
nginx • Rails • bookshelf • omnibus-chef server -
What’s NEW in Chef 11 • chef-apply • partial search
• users with key pairs • partials in templates • knife-essentials -
nginx erchef solr (lucene) bookshelf PostgreSQL filesystem cookbook store filesystem
search index RabbitMQ chef-expander chef-webui Chef Server -
Erchef • Erlang • Concurrent, fault tolerant • Distributed systems
• Complete Chef Server API rewrite -
CPU Usage on Chef Server • Erchef + Ruby Chef
• 12 mo old • 3 CPU aggregate -
Database system Load Average CouchDB MySQL Of course, we all
know load average is not indicative of performance ;) -
Some other Erchef Features... • Nginx reverse proxy, https •
WebUI is Ruby on Rails • Bookshelf stores cookbooks, like S3 -
"Omnibus" Package • full-stack for the Chef server • RPMs,
DEBs, Build your own? • opscode.com/chef/install • chef-server-ctl • reconfigure • test • start/stop http://apod.nasa.gov/apod/astropix.html -
chef-apply • run a single recipe file • without modifying
the node's run_list • included with Chef, /usr/bin/chef-apply • chef-apply /path/to/recipe_file • chef-apply "content of a recipe file" • or from STDIN http://www.flickr.com/photos/albill/sets/72157628046395000/ -
Example % cat apache.rb package "apache2" file "/var/www/index.html" do content
"<h1>Hello, Philly DevOps!</h1>” end service "apache2" do action [:start, :enable] end % sudo chef-apply apache.rb -
partial search partial_search(:node, 'role:web', :keys => { 'name'
=> [ 'name' ], 'ip' => [ 'ipaddress' ], 'kernel_version' => [ 'kernel', 'version' ] } ).each do |result| puts result['name'] puts result['ip'] puts result['kernel_version'] end • instead of entire node, just the pieces you want • massive reduction in bandwidth and memory • (See also: http://ckbk.it/whitelist-node-attrs) -
users with keypairs • users can have key pairs, just
like clients • knife actions as a user instead of a client • post a public key when you create a user/client • "knife user" (11.2.0) http://www.flickr.com/photos/albill/sets/72157628046395000/ -
<?xml version='1.0' encoding='UTF-8'?> <server xmlns="urn:jboss:domain:1.2"> <extensions> <% if @infinispan -%>
<extension module="org.jboss.as.clustering.infinispan"/> <% end -%> <extension module="org.jboss.as.web"/> <% if @webservices -%> <extension module="org.jboss.as.webservices"/> <% end -%> <extension module="org.jboss.as.weld"/> </extensions> <% if @infinispan include_template "infinispan.xml.erb" end -%> <% if @webservices include_template "webservices.xml.erb" end -%> <socket-binding-group name="standard-sockets" > ...... </socket-binding-group> </server> • Thanks to Andrea Campi! partials in templates http://www.flickr.com/photos/modern_fred/2095565021/ -
knife-essentials • Treat local chef-repo and chef-server like a filesystem
• knife download • knife diff • knife show • knife upload cookbooks/apache2 • knife list -
Chef 11 Breaking Changes • http://docs.opscode.com/ breaking_changes_chef_11. html • chef-shell
(formerly shef) • no implicit node attributes (no more node['blah'] = 'foo') http://www.flickr.com/photos/modern_fred/2096352938/ -
Chef 11 Breaking Changes • attribute files may access role
and environment attributes • delayed notifications run after failed converge • encrypted data bag item format change • chef-client lock so safe from simultaneous runs -
Weird Syntax... -module(ranks). -export([fetch_title_and_rank/1]). -include_lib("xmerl/include/xmerl.hrl"). -define(BASE_URL, "http://webservices.amazon.com/onca/xml?" ++ "&ItemId="). fetch_title_and_rank(ISBN)
-> URL = amazon_url_for(ISBN), { ok, {_Status, _Headers, Body }} = http:request(URL), { Xml, _Rest } = xmerl_scan:string(Body), [ #xmlText{value=Rank} ] = xmerl_xpath:string("//SalesRank/text()", Xml), [ #xmlText{value=Title} ] = xmerl_xpath:string("//Title/text()", Xml), { Title, Rank }. amazon_url_for(ISBN) -> ?BASE_URL ++ ISBN. No :) http://pragdave.pragprog.com/pragdave/2007/04/a_first_erlang_.html -
Migration tools • Coming soon • But, it's an API...
• knife download, knife upload works for many use cases -
Food Fight Show • http://foodfightshow.org • The Podcast Where DevOps
Chef Do Battle • Regular updates about new Cookbooks, Knife-plugins, and more • Best Practices for working with Chef -
Chef Introductory Workshop • Save 10% with promo code “MEETUP”
• http://opscode.eventbrite.com • Get your tickets soon! -
Thanks ! • What questions do you have? • Nathen
Harvey • [email protected] • @nathenharvey