Upgrade to Pro — share decks privately, control downloads, hide ads and more …

What's New in Chef 11 - Philly DevOps

Nathen Harvey
February 27, 2013

What's New in Chef 11 - Philly DevOps

Introduction to Chef and new features of version 11. Given at the Philly DevOps meetup group.

Nathen Harvey

February 27, 2013

Transcript

81 slides
  1. Nathen Harvey • Technical Community Manager • Co-host Food Fight

    Show • http://foodfightshow.org • @nathenharvey
  2. Chef is an automation platform for developers & systems engineers

    to continuously define, build, and manage infrastructure. CHEF USES: Recipes and Cookbooks that describe Infrastructure as Code. Chef enables people to easily build & manage complex & dynamic applications at massive scale • New model for describing infrastructure that promotes reuse • Programmatically provision and configure • Reconstruct business from code repository, data backup, and bare metal resources “ ” Chef
  3. Collection of Resources http://www.flickr.com/photos/philliecasablanca/3354734116/ • Networking • Files • Directories

    • Symlinks • Mounts • Routes • Users • Groups • Tasks • Packages • Software • Services • Configuration • Other Stuff
  4. Golden Images are not the answer • Gold is heavy

    • Hard to transport • Hard to mold • Easy to lose configuration detail http://www.flickr.com/photos/garysoup/2977173063/
  5. Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite •

    Move SSH off port 22 • Lets put it on 2022 New Compliance Mandate!
  6. Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite •

    edit /etc/ssh/sshd_config 1 2 3 4 5 6 6 Golden Image Updates
  7. Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite •

    Delete, launch 1 2 3 4 5 6 7 8 9 10 11 12 • Repeat • Typically manually 12 Instance Replacements
  8. • Don’t break anything! • Bob just got fired =(

    5 Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite 1 2 4 5 6 7 8 9 10 11 12 3 Done in Maintenance Windows
  9. http://www.flickr.com/photos/louisb/4555295187/ • Programmatically provision and configure • Treat like any

    other code base • Reconstruct business from code repository, data backup, and bare metal resources. Chef is Infrastructure as Code
  10. http://www.flickr.com/photos/ssoosay/5126146763/ • Chef generates configurations directly on nodes from their

    run list • Reduce management complexity through abstraction • Store the configuration of your programs in version control Programs
  11. • Define Policy • Say what, not how • Pull

    not Push Code Sample http://www.flickr.com/photos/bixentro/2591838509/ Declarative Interface to Resources
  12. package "ntp" do action :install end service "ntpd" do action

    [:enable,:start] end template "/etc/ntpd.conf" do source "ntpd.conf.erb" owner "root" group "root" mode 0644 action :create variables(:time_server => “time.example.com”) notifies :restart, “service[ntpd]” end That looks like this
  13. Nagios Graphite Jboss App Memcache Postgres Slaves • Load balancer

    config • Nagios host ping • Nagios host ssh • Nagios host HTTP • Nagios host app health • Graphite CPU • Graphite Memory • Graphite Disk • Graphite SNMP • Memcache firewall • Postgres firewall • Postgres authZ config • 12+ resource changes for 1 node addition Count the resources
  14. Blueprint Your Infrastructure • Compute • Application • Storage •

    Security • Network • Configuration Standards How Can Chef Help? Build Anything… • Provision compute resources in the Data Center and the Cloud • Infrastructure • Application Stacks • Big Data • HPC And Manage It Simply • Introduce continuous incremental change or total change. • Automatically reconfigure everything • Re-provision for disaster recovery • Fail-over to bare metal • Monitor for compliance • Cloud migrations become trivial Using 1,000’s of man-days of prior art! Discoverable and Searchable Infrastructure
  15. © Opscode, 2011 – Confidential – DO NOT DISTRIBUTE Chef

    Provides a Model for Reuse That Works 42 800+ Cookbooks “Yesterday we started open sourcing some of our Opscode Chef work created at bestbuy.com; bit.ly/ yDV9Hl #Splunk #opschef #expectmor”
  16. What's New Since Chef 0.10? • Windows support • Librarian

    • Spiceweasel • Foodcritic • Cookbooks in separate repositories • Private Chef • Food Fight podcast • knife plugins for openstack, hp, azure, gce • 0.10 renumbered to Chef 10 • full-stack client • why-run • output formatters • Test Kitchen • Solaris support • Chef Developer Summit(2!) • Berkshelf • ChefConf • Fauxhai • docs.opscode.com • Hangouts for reviews • nyan-cat output formatter
  17. What is NOT in Chef 11 • CouchDB • Ruby-based

    chef-server-api • depsolver, gecode, treetop • Merb • OpenID support in Web UI • Migration tooling (coming soon) http://trainweb.org/carl/Pullman150/IMG_3380.jpg
  18. What’s NEW in Chef 11 • PostgreSQL • erchef •

    nginx • Rails • bookshelf • omnibus-chef server
  19. What’s NEW in Chef 11 • chef-apply • partial search

    • users with key pairs • partials in templates • knife-essentials
  20. nginx erchef solr (lucene) bookshelf PostgreSQL filesystem cookbook store filesystem

    search index RabbitMQ chef-expander chef-webui Chef Server
  21. CPU Usage on Chef Server • Erchef + Ruby Chef

    • 12 mo old • 3 CPU aggregate
  22. Database system Load Average CouchDB MySQL Of course, we all

    know load average is not indicative of performance ;)
  23. Some other Erchef Features... • Nginx reverse proxy, https •

    WebUI is Ruby on Rails • Bookshelf stores cookbooks, like S3
  24. "Omnibus" Package • full-stack for the Chef server • RPMs,

    DEBs, Build your own? • opscode.com/chef/install • chef-server-ctl • reconfigure • test • start/stop http://apod.nasa.gov/apod/astropix.html
  25. chef-apply • run a single recipe file • without modifying

    the node's run_list • included with Chef, /usr/bin/chef-apply • chef-apply /path/to/recipe_file • chef-apply "content of a recipe file" • or from STDIN http://www.flickr.com/photos/albill/sets/72157628046395000/
  26. Example % cat apache.rb package "apache2" file "/var/www/index.html" do content

    "<h1>Hello, Philly DevOps!</h1>” end service "apache2" do action [:start, :enable] end % sudo chef-apply apache.rb
  27. partial search partial_search(:node,  'role:web',      :keys  =>  {  'name'

     =>  [  'name'  ],                            'ip'      =>  [  'ipaddress'  ],                            'kernel_version'  =>  [  'kernel',  'version'  ]                        } ).each  do  |result|    puts  result['name']    puts  result['ip']    puts  result['kernel_version'] end • instead of entire node, just the pieces you want • massive reduction in bandwidth and memory • (See also: http://ckbk.it/whitelist-node-attrs)
  28. users with keypairs • users can have key pairs, just

    like clients • knife actions as a user instead of a client • post a public key when you create a user/client • "knife user" (11.2.0) http://www.flickr.com/photos/albill/sets/72157628046395000/
  29. <?xml version='1.0' encoding='UTF-8'?> <server xmlns="urn:jboss:domain:1.2"> <extensions> <% if @infinispan -%>

    <extension module="org.jboss.as.clustering.infinispan"/> <% end -%> <extension module="org.jboss.as.web"/> <% if @webservices -%> <extension module="org.jboss.as.webservices"/> <% end -%> <extension module="org.jboss.as.weld"/> </extensions> <% if @infinispan include_template "infinispan.xml.erb" end -%> <% if @webservices include_template "webservices.xml.erb" end -%> <socket-binding-group name="standard-sockets" > ...... </socket-binding-group> </server> • Thanks to Andrea Campi! partials in templates http://www.flickr.com/photos/modern_fred/2095565021/
  30. knife-essentials • Treat local chef-repo and chef-server like a filesystem

    • knife download • knife diff • knife show • knife upload cookbooks/apache2 • knife list
  31. Chef 11 Breaking Changes • http://docs.opscode.com/ breaking_changes_chef_11. html • chef-shell

    (formerly shef) • no implicit node attributes (no more node['blah'] = 'foo') http://www.flickr.com/photos/modern_fred/2096352938/
  32. Chef 11 Breaking Changes • attribute files may access role

    and environment attributes • delayed notifications run after failed converge • encrypted data bag item format change • chef-client lock so safe from simultaneous runs
  33. Weird Syntax... -module(ranks). -export([fetch_title_and_rank/1]). -include_lib("xmerl/include/xmerl.hrl"). -define(BASE_URL, "http://webservices.amazon.com/onca/xml?" ++ "&ItemId="). fetch_title_and_rank(ISBN)

    -> URL = amazon_url_for(ISBN), { ok, {_Status, _Headers, Body }} = http:request(URL), { Xml, _Rest } = xmerl_scan:string(Body), [ #xmlText{value=Rank} ] = xmerl_xpath:string("//SalesRank/text()", Xml), [ #xmlText{value=Title} ] = xmerl_xpath:string("//Title/text()", Xml), { Title, Rank }. amazon_url_for(ISBN) -> ?BASE_URL ++ ISBN. No :) http://pragdave.pragprog.com/pragdave/2007/04/a_first_erlang_.html
  34. Migration tools • Coming soon • But, it's an API...

    • knife download, knife upload works for many use cases
  35. Food Fight Show • http://foodfightshow.org • The Podcast Where DevOps

    Chef Do Battle • Regular updates about new Cookbooks, Knife-plugins, and more • Best Practices for working with Chef
  36. Chef Introductory Workshop • Save 10% with promo code “MEETUP”

    • http://opscode.eventbrite.com • Get your tickets soon!

More Decks by Nathen Harvey

Other Decks in Technology