Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Prepping the Kitchen

Ea72b50eef37ebe730c37d96c5b5dd51?s=47 someara
October 02, 2011

Prepping the Kitchen

Chef Concepts and Fundamental - Presented at Surge 2011

Ea72b50eef37ebe730c37d96c5b5dd51?s=128

someara

October 02, 2011
Tweet

Transcript

  1. None
  2. Prepping the Kitchen - Chef Concepts and Fundamentals someara@opscode.com www.opscode.com

  3. Overview • Infrastructure as code • Configuration Management Strategies •

    Chef
  4. Infrastructure as code

  5. Infrastructure "It is common to think in terms of individual

    machines rather than view an entire infrastructure as a combined whole" “A good infrastructure, whether departmental, divisional, or enterprise-wide, is a single loosely- coupled virtual machine, with hundreds or thousands of hard drives and CPU's.” -- Bootstrapping an Infrastructure USENIX LISA ’98 http://www.infrastructures.org/papers/bootstrap/bootstrap.html
  6. .... as code! • Programmatically provision and configure • Treat

    like any other code base • Reconstruct operations from code repository, data backup, and bare metal resources. http://www.flickr.com/photos/louisb/4555295187/
  7. • Infrastructure changes over time • Entropy • Changing business

    requirements Considerations http://www.flickr.com/photos/seatbelt67/502255276/
  8. Methodology http://www.flickr.com/photos/drachmann/327122302/

  9. Configuration Management Strategies

  10. Manual Configuration • Labor intensive • Error prone • Hard

    to reproduce • Unsustainable http://www.flickr.com/photos/pureimaginations/4805330106/
  11. Scripting • Typically very brittle • Throw away, one off

    scripts • grep sed awk perl • curl | bash http://www.flickr.com/photos/40389360@N00/2428706650/
  12. File Distribution • NFS mounts • rdist • scp-on-a-for-loop •

    rsync on cron http://www.flickr.com/photos/walkadog/4317655660
  13. This used to be awesome for i in `cat servers.txt`

    ; do scp ntp.conf root@$i:/etc/ ntpd.conf ; done for i in `cat servers.txt` ; do ssh root@$i /etc/init.d/ntpd restart ; done for i in `cat servers.txt` ; do ssh root@$i chkconfig ntpd on ; done • ^ does not scale http://www.flickr.com/photos/alexerde/3479006495
  14. Declarative Syntax http://www.flickr.com/photos/bixentro/2591838509/ • Define policy • Say what, not

    how • Abstract interface to resources • Enables some interesting behavior
  15. Declarative Tools • LCFG • CFEngine • BCFG2 • Puppet

    • Chef Declarative Syntax
  16. Declarative Syntax • You’ll hear this a lot • Property

    of declarative interface • Eliminates brittleness of scripting • Identity function: f(x)=x package "ntp" do action :install end Idempotence service "ntpd" do action [:enable,:start] end cookbook_file "/etc/ntp.conf" do source "ntp.conf" owner "root" group "root" mode 0644 action :create notifies :restart, “service[ntpd]” end
  17. Declarative Syntax package "ntp" do action :install end Idempotence service

    "ntpd" do action [:enable,:start] end cookbook_file "/etc/ntp.conf" do source "ntp.conf" owner "root" group "root" mode 0644 action :create notifies :restart, “service[ntpd]” end while true do end • You’ll hear this a lot • Property of declarative interface • Eliminates brittleness of scripting • Identity function: f(x)=x • Safe to repeat
  18. Declarative Syntax • Agents “converge” a system to desired state

    • Repetition inches closer to desired state • It eventually gets there • SCIENCE! Convergence http://www.flickr.com/photos/tolomea/4852616645/
  19. Declarative Syntax Convergence package "ntp" do action :install ignore_failure true

    end service "ntpd" do action [:enable,:start] ignore_failure true end cookbook_file "/etc/ntp.conf" do source "ntp.conf" owner "root" group "root" mode 0644 action :create notifies :restart, “service[ntpd]” ignore_failure true end • Agents “converge” a system to desired state • Repetition inches closer to desired state • It eventually gets there • SCIENCE!
  20. Declarative Syntax Convergence • Fights entropy, unauthorized changes, and gingivitis

    • Update function inputs to deal with changing requirements # echo “boom” > /etc/ntp.conf ; \ chef-client $ grep server /etc/ntp.conf | head -n 1 us.pool.ntp.org $ ps -e | grep ntp 1799 ? 00:00:00 ntpd # /etc/init.d/ntpd stop ; chef-client ps -e | grep ntp 1822 ? 00:00:00 ntpd
  21. • Often made by hand (still!?) • Stop that. •

    Generate them based on database content • Infrastructures evolve Config Generation http://www.flickr.com/photos/jabella/4753170413/
  22. Application See Node

  23. Application Application Database See Nodes

  24. Application App Databases See Nodes Grow

  25. App Servers App Databases See Nodes Grow

  26. App LB App Servers App Databases See Nodes Grow

  27. App LBs App Servers App Databases See Nodes Grow

  28. App LBs App Servers App DB Cache App DBs See

    Nodes Grow
  29. App LBs App Servers App DB Cache App DBs Stitched

    together with configs
  30. App LBs App Servers App DB Cache App DBs Stitched

    together with configs Floating IP?
  31. App LBs App Servers NoSQL DB slaves Cache DB Cache

    DBs Complexity increases quickly
  32. Complexity increases very quickly DC1 DC3 DC2

  33. Generate configs • Centralized generation • Version control! • Distribute

    with packages, Chef, git, whatever. http://www.flickr.com/photos/ssoosay/5126146763/
  34. Generate configs • Local generation directly on nodes • Reduces

    management complexity • No need to distribute • Version control the programs instead http://www.flickr.com/photos/ssoosay/5126146763/
  35. Chef

  36. • Declarative interface to resources • Database of nodes and

    their roles • Grab remote configs • Generate configs locally All That Stuff
  37. • Data Driven Infrastructure • Use APIs to obtain data

    • chef-server, SQL, anything. • Feed resources parameters • IPs, FQDNs, memory sizes, • Templates, package, firewall rules and more!
  38. Architecture • Code Repository • Chef Server • Chef Clients

    • Data Bags • Recipes and Cookbooks • Roles and Run Lists http://www.flickr.com/photos/boedker/3871267007
  39. Code Repository • Version control • Development workflows • Sharing

    is Caring
  40. Server Server Server Server chef-server Cookbook Knife Role Data Bag

    Knife Knife RESTful API Cookbook Cookbook Chef Server • Upload from laptop with knife
  41. Chef Clients Server Server Server Server chef-server RESTful API •

    Clients are API users • Read • Write • Search chef-client chef-client chef-client chef-client chef-client Knife Knife
  42. Chef Clients Server Server Server Server chef-server RESTful API chef-client

    • Clients are API users • Public keys on server • Private keys local to machines chef-client chef-client chef-client chef-client Knife Knife someara.pem jtimberman.pem node5.fqdn.pem someara.pub jtimberman.pub node5.fqdn.pub
  43. Run Lists Server Server Server Server chef-server API chef-client Ohai!

    Give me recipe[ntp::client] node ntp client.rb
  44. Run Lists Server Server Server Server chef-server API chef-client Ohai!

    Give me “ntp::client”, “openssh::server” node ntp client.rb openssh server.rb
  45. Run Lists Server Server Server Server chef-server API chef-client Ohai!

    Give me “recipe[ntp::client]”, “recipe[openssh::server]”, “recipe[apache]”, “recipe[php]” node ntp client.rb openssh server.rb apache default.rb php default.rb
  46. Roles Role Recipe Recipe Recipe Role Role Recipe Recipe Recipe

    Role Recipe Server Server Server Server chef-server API Knife
  47. Server Server Server Server chef-server API chef-client Ohai! Give me

    “role[base]”, “role[webserver]” node ntp client.rb openssh server.rb apache default.rb php default.rb Roles
  48. Server Server Server Server chef-server API chef-client “role[webserver]” node ntp

    client.rb openssh server.rb apache default.rb php default.rb Roles chef-client “role[database]” node ntp client.rb openssh server.rb mysql server.rb
  49. Bootstrapping nodes • Get chef-client installed • Write run list

    to a file • “Press go” http://www.flickr.com/photos/liftarn/1447521121/
  50. • knife ec2 server create -r ‘role [webserver]’ • knife

    bootstrap 10.9.8.7 -r ‘role[webserver]’ • Cobbler Bootstrapping nodes http://www.flickr.com/photos/hakonjarl/4010080214/
  51. Bootstrapping nodes • Ohai generates a JSON attributes list •

    Run list and attributes are combined into a Node object • Can be viewed and searched through API { "kernel": { "machine": "x86_64", "name": "Darwin", "os": "Darwin", "version": "Darwin Kernel Version 10.4.0: Fri Apr 23 18:28:53 PDT 2010; root:xnu-1504.7.4~1/RELEASE_I386", "release": "10.4.0" }, "platform_version": "10.6.4", "platform": "mac_os_x", "platform_build": "10F569", "domain": "local", "os": "darwin", "current_user": "mray", "ohai_time": 1278602661.60043, "os_version": "10.4.0", "uptime": "18 days 17 hours 49 minutes 18 seconds", "ipaddress": "10.13.37.116", "hostname": "morbo", "fqdn": "morbomorbo.local", "uptime_seconds": 1619358 }
  52. Bootstrapping nodes http://www.flickr.com/photos/architopher/457885721 • Run list is requested • Cookbooks

    downloaded • Recipes executed • Node saved to chef-server
  53. Cookbooks and Recipes • Cookbooks contain recipes • And everything

    they need to work • Templates, files, custom resources, etc http://www.flickr.com/photos/shutterhacks/4474421855/
  54. $ tree -a cookbooks/haproxy/ ᵓᴷᴷ README.md ᵓᴷᴷ attributes ᴹ ᵋᴷᴷ

    default.rb ᵓᴷᴷ metadata.rb ᵓᴷᴷ recipes ᴹ ᵓᴷᴷ app_lb.rb ᴹ ᵋᴷᴷ default.rb ᵋᴷᴷ templates ᵋᴷᴷ default ᵓᴷᴷ haproxy-app_lb.cfg.erb ᵓᴷᴷ haproxy-default.erb ᵋᴷᴷ haproxy.cfg.erb • Cookbooks contain recipes • And everything they need to work • Templates, files, custom resources, etc Cookbooks
  55. package "haproxy" do action :install end template "/etc/default/haproxy" do source

    "haproxy-default.erb" owner "root" group "root" mode 0644 notifies :restart, "service[haproxy]" end service "haproxy" do action [:enable, :start] end • Recipes contain lists of resources Recipes
  56. Resources

  57. package "apache2" do version "2.2.11-2ubuntu2.6" action :install end template "/etc/apache2/apache2.conf"

    do source "apache2.conf.erb" owner "root" group "root" mode 0644 action :create end Resources
  58. • Have a type package "apache2" do version "2.2.11-2ubuntu2.6" action

    :install end template "/etc/apache2/apache2.conf" do source "apache2.conf.erb" owner "root" group "root" mode 0644 action :create end Resources
  59. • Have a type • Have a name package "apache2"

    do version "2.2.11-2ubuntu2.6" action :install end template "/etc/apache2/apache2.conf" do source "apache2.conf.erb" owner "root" group "root" mode 0644 action :create end Resources
  60. • Have a type • Have a name • Have

    parameters package "apache2" do version "2.2.11-2ubuntu2.6" action :install end template "/etc/apache2/apache2.conf" do source "apache2.conf.erb" owner "root" group "root" mode 0644 action :create end Resources
  61. • Have a type • Have a name • Have

    parameters • Take action to put the resource in the declared state package "apache2" do version "2.2.11-2ubuntu2.6" action :install end template "/etc/apache2/apache2.conf" do source "apache2.conf.erb" owner "root" group "root" mode 0644 action :create end Resources
  62. Searching http://www.flickr.com/photos/fotos_medem/3399096196/

  63. Searching • All object in Chef server are indexed by

    Solr http://www.flickr.com/photos/fotos_medem/3399096196/
  64. Searching • All object in Chef server are indexed by

    Solr • Can search through the API http://www.flickr.com/photos/fotos_medem/3399096196/
  65. Searching • All object in Chef server are indexed by

    Solr • Can search through the API • From knife and in recipes http://www.flickr.com/photos/fotos_medem/3399096196/
  66. Searching • All object in Chef server are indexed by

    Solr • Can search through the API • From knife and in recipes • Returns an array of JSON Node objects http://www.flickr.com/photos/fotos_medem/3399096196/
  67. webservers = search("node", "role:webserver”) knife search node role:webserver Systems Integration

  68. pool_members = search("node","role:webserver”) template "/etc/haproxy/haproxy.cfg" do source "haproxy-app_lb.cfg.erb" owner "root"

    group "root" mode 0644 variables :pool_members => pool_members.uniq notifies :restart, "service[haproxy]" end Pass results into Templates
  69. pool_members = search("node","role:webserver”) template "/etc/haproxy/haproxy.cfg" do source "haproxy-app_lb.cfg.erb" owner "root"

    group "root" mode 0644 variables :pool_members => pool_members.uniq notifies :restart, "service[haproxy]" end Pass results into Templates
  70. # Set up application listeners here. listen application 0.0.0.0:80 balance

    roundrobin <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%> <% if node["haproxy"]["enable_admin"] -%> listen admin 0.0.0.0:22002 mode http stats uri / <% end -%> Pass results into Templates
  71. Change • Various ways • Add or remove a node

    to the infrastructure • Run chef-client
  72. $ grep servers /etc/haproxy/haproxy.cfg servers node2.mylan 10.9.8.10 servers node3.mylan 10.9.8.11

    $ knife ec2 server create -r ‘webserver’ $ knife ec2 server create -r ‘webserver’ $ knife ssh ‘role:webserver’ chef-client $ grep servers /etc/haproxy/haproxy.cfg servers node2.mylan 10.9.8.10 servers node3.mylan 10.9.8.11 servers node4.mylan 10.9.8.12 servers node5.mylan 10.9.8.13 Run chef-client
  73. Change Inputs • Edit recipes • Edit run lists •

    chef-client http://www.flickr.com/photos/dhutchman/128541987
  74. Out of slides! http://www.flickr.com/photos/calonyr11/2630312566/

  75. Questions? sales@opscode.com www.opscode.com