Prepping the Kitchen

Chef Concepts and Fundamental - Presented at Surge 2011

Transcript

2. Prepping the Kitchen - Chef Concepts and Fundamentals someara@opscode.com www.opscode.com
3. Overview • Infrastructure as code • Configuration Management Strategies • Chef
4. Infrastructure as code
5. Infrastructure "It is common to think in terms of individual machines rather than view an entire infrastructure as a combined whole" “A good infrastructure, whether departmental, divisional, or enterprise-wide, is a single loosely- coupled virtual machine, with hundreds or thousands of hard drives and CPU's.” -- Bootstrapping an Infrastructure USENIX LISA ’98 http://www.infrastructures.org/papers/bootstrap/bootstrap.html
6. .... as code! • Programmatically provision and configure • Treat like any other code base • Reconstruct operations from code repository, data backup, and bare metal resources. http://www.flickr.com/photos/louisb/4555295187/
7. • Infrastructure changes over time • Entropy • Changing business requirements Considerations http://www.flickr.com/photos/seatbelt67/502255276/
8. Methodology http://www.flickr.com/photos/drachmann/327122302/
9. Configuration Management Strategies
10. Manual Configuration • Labor intensive • Error prone • Hard to reproduce • Unsustainable http://www.flickr.com/photos/pureimaginations/4805330106/
11. Scripting • Typically very brittle • Throw away, one off scripts • grep sed awk perl • curl | bash http://www.flickr.com/photos/40389360@N00/2428706650/
12. File Distribution • NFS mounts • rdist • scp-on-a-for-loop • rsync on cron http://www.flickr.com/photos/walkadog/4317655660
13. This used to be awesome for i in `cat servers.txt` ; do scp ntp.conf root@$i:/etc/ ntpd.conf ; done for i in `cat servers.txt` ; do ssh root@$i /etc/init.d/ntpd restart ; done for i in `cat servers.txt` ; do ssh root@$i chkconfig ntpd on ; done • ^ does not scale http://www.flickr.com/photos/alexerde/3479006495
14. Declarative Syntax http://www.flickr.com/photos/bixentro/2591838509/ • Define policy • Say what, not how • Abstract interface to resources • Enables some interesting behavior
15. Declarative Tools • LCFG • CFEngine • BCFG2 • Puppet • Chef Declarative Syntax
16. Declarative Syntax • You’ll hear this a lot • Property of declarative interface • Eliminates brittleness of scripting • Identity function: f(x)=x package "ntp" do action :install end Idempotence service "ntpd" do action [:enable,:start] end cookbook_file "/etc/ntp.conf" do source "ntp.conf" owner "root" group "root" mode 0644 action :create notifies :restart, “service[ntpd]” end
17. Declarative Syntax package "ntp" do action :install end Idempotence service "ntpd" do action [:enable,:start] end cookbook_file "/etc/ntp.conf" do source "ntp.conf" owner "root" group "root" mode 0644 action :create notifies :restart, “service[ntpd]” end while true do end • You’ll hear this a lot • Property of declarative interface • Eliminates brittleness of scripting • Identity function: f(x)=x • Safe to repeat
18. Declarative Syntax • Agents “converge” a system to desired state • Repetition inches closer to desired state • It eventually gets there • SCIENCE! Convergence http://www.flickr.com/photos/tolomea/4852616645/
19. Declarative Syntax Convergence package "ntp" do action :install ignore_failure true end service "ntpd" do action [:enable,:start] ignore_failure true end cookbook_file "/etc/ntp.conf" do source "ntp.conf" owner "root" group "root" mode 0644 action :create notifies :restart, “service[ntpd]” ignore_failure true end • Agents “converge” a system to desired state • Repetition inches closer to desired state • It eventually gets there • SCIENCE!
20. Declarative Syntax Convergence • Fights entropy, unauthorized changes, and gingivitis • Update function inputs to deal with changing requirements # echo “boom” > /etc/ntp.conf ; \ chef-client $ grep server /etc/ntp.conf | head -n 1 us.pool.ntp.org $ ps -e | grep ntp 1799 ? 00:00:00 ntpd # /etc/init.d/ntpd stop ; chef-client ps -e | grep ntp 1822 ? 00:00:00 ntpd
21. • Often made by hand (still!?) • Stop that. • Generate them based on database content • Infrastructures evolve Config Generation http://www.flickr.com/photos/jabella/4753170413/
22. Application See Node
23. Application Application Database See Nodes
24. Application App Databases See Nodes Grow
25. App Servers App Databases See Nodes Grow
26. App LB App Servers App Databases See Nodes Grow
27. App LBs App Servers App Databases See Nodes Grow
28. App LBs App Servers App DB Cache App DBs See Nodes Grow
29. App LBs App Servers App DB Cache App DBs Stitched together with configs
30. App LBs App Servers App DB Cache App DBs Stitched together with configs Floating IP?
31. App LBs App Servers NoSQL DB slaves Cache DB Cache DBs Complexity increases quickly
32. Complexity increases very quickly DC1 DC3 DC2
33. Generate configs • Centralized generation • Version control! • Distribute with packages, Chef, git, whatever. http://www.flickr.com/photos/ssoosay/5126146763/
34. Generate configs • Local generation directly on nodes • Reduces management complexity • No need to distribute • Version control the programs instead http://www.flickr.com/photos/ssoosay/5126146763/
35. Chef
36. • Declarative interface to resources • Database of nodes and their roles • Grab remote configs • Generate configs locally All That Stuff
37. • Data Driven Infrastructure • Use APIs to obtain data • chef-server, SQL, anything. • Feed resources parameters • IPs, FQDNs, memory sizes, • Templates, package, firewall rules and more!
38. Architecture • Code Repository • Chef Server • Chef Clients • Data Bags • Recipes and Cookbooks • Roles and Run Lists http://www.flickr.com/photos/boedker/3871267007
39. Code Repository • Version control • Development workflows • Sharing is Caring
40. Server Server Server Server chef-server Cookbook Knife Role Data Bag Knife Knife RESTful API Cookbook Cookbook Chef Server • Upload from laptop with knife
41. Chef Clients Server Server Server Server chef-server RESTful API • Clients are API users • Read • Write • Search chef-client chef-client chef-client chef-client chef-client Knife Knife
42. Chef Clients Server Server Server Server chef-server RESTful API chef-client • Clients are API users • Public keys on server • Private keys local to machines chef-client chef-client chef-client chef-client Knife Knife someara.pem jtimberman.pem node5.fqdn.pem someara.pub jtimberman.pub node5.fqdn.pub
43. Run Lists Server Server Server Server chef-server API chef-client Ohai! Give me recipe[ntp::client] node ntp client.rb
44. Run Lists Server Server Server Server chef-server API chef-client Ohai! Give me “ntp::client”, “openssh::server” node ntp client.rb openssh server.rb
45. Run Lists Server Server Server Server chef-server API chef-client Ohai! Give me “recipe[ntp::client]”, “recipe[openssh::server]”, “recipe[apache]”, “recipe[php]” node ntp client.rb openssh server.rb apache default.rb php default.rb
46. Roles Role Recipe Recipe Recipe Role Role Recipe Recipe Recipe Role Recipe Server Server Server Server chef-server API Knife
47. Server Server Server Server chef-server API chef-client Ohai! Give me “role[base]”, “role[webserver]” node ntp client.rb openssh server.rb apache default.rb php default.rb Roles
48. Server Server Server Server chef-server API chef-client “role[webserver]” node ntp client.rb openssh server.rb apache default.rb php default.rb Roles chef-client “role[database]” node ntp client.rb openssh server.rb mysql server.rb
49. Bootstrapping nodes • Get chef-client installed • Write run list to a file • “Press go” http://www.flickr.com/photos/liftarn/1447521121/
50. • knife ec2 server create -r ‘role [webserver]’ • knife bootstrap 10.9.8.7 -r ‘role[webserver]’ • Cobbler Bootstrapping nodes http://www.flickr.com/photos/hakonjarl/4010080214/
51. Bootstrapping nodes • Ohai generates a JSON attributes list • Run list and attributes are combined into a Node object • Can be viewed and searched through API { "kernel": { "machine": "x86_64", "name": "Darwin", "os": "Darwin", "version": "Darwin Kernel Version 10.4.0: Fri Apr 23 18:28:53 PDT 2010; root:xnu-1504.7.4~1/RELEASE_I386", "release": "10.4.0" }, "platform_version": "10.6.4", "platform": "mac_os_x", "platform_build": "10F569", "domain": "local", "os": "darwin", "current_user": "mray", "ohai_time": 1278602661.60043, "os_version": "10.4.0", "uptime": "18 days 17 hours 49 minutes 18 seconds", "ipaddress": "10.13.37.116", "hostname": "morbo", "fqdn": "morbomorbo.local", "uptime_seconds": 1619358 }
52. Bootstrapping nodes http://www.flickr.com/photos/architopher/457885721 • Run list is requested • Cookbooks downloaded • Recipes executed • Node saved to chef-server
53. Cookbooks and Recipes • Cookbooks contain recipes • And everything they need to work • Templates, files, custom resources, etc http://www.flickr.com/photos/shutterhacks/4474421855/
54. $ tree -a cookbooks/haproxy/ ᵓᴷᴷ README.md ᵓᴷᴷ attributes ᴹ ᵋᴷᴷ default.rb ᵓᴷᴷ metadata.rb ᵓᴷᴷ recipes ᴹ ᵓᴷᴷ app_lb.rb ᴹ ᵋᴷᴷ default.rb ᵋᴷᴷ templates ᵋᴷᴷ default ᵓᴷᴷ haproxy-app_lb.cfg.erb ᵓᴷᴷ haproxy-default.erb ᵋᴷᴷ haproxy.cfg.erb • Cookbooks contain recipes • And everything they need to work • Templates, files, custom resources, etc Cookbooks
55. package "haproxy" do action :install end template "/etc/default/haproxy" do source "haproxy-default.erb" owner "root" group "root" mode 0644 notifies :restart, "service[haproxy]" end service "haproxy" do action [:enable, :start] end • Recipes contain lists of resources Recipes
56. Resources
57. package "apache2" do version "2.2.11-2ubuntu2.6" action :install end template "/etc/apache2/apache2.conf" do source "apache2.conf.erb" owner "root" group "root" mode 0644 action :create end Resources
58. • Have a type package "apache2" do version "2.2.11-2ubuntu2.6" action :install end template "/etc/apache2/apache2.conf" do source "apache2.conf.erb" owner "root" group "root" mode 0644 action :create end Resources
59. • Have a type • Have a name package "apache2" do version "2.2.11-2ubuntu2.6" action :install end template "/etc/apache2/apache2.conf" do source "apache2.conf.erb" owner "root" group "root" mode 0644 action :create end Resources
60. • Have a type • Have a name • Have parameters package "apache2" do version "2.2.11-2ubuntu2.6" action :install end template "/etc/apache2/apache2.conf" do source "apache2.conf.erb" owner "root" group "root" mode 0644 action :create end Resources
61. • Have a type • Have a name • Have parameters • Take action to put the resource in the declared state package "apache2" do version "2.2.11-2ubuntu2.6" action :install end template "/etc/apache2/apache2.conf" do source "apache2.conf.erb" owner "root" group "root" mode 0644 action :create end Resources
62. Searching http://www.flickr.com/photos/fotos_medem/3399096196/
63. Searching • All object in Chef server are indexed by Solr http://www.flickr.com/photos/fotos_medem/3399096196/
64. Searching • All object in Chef server are indexed by Solr • Can search through the API http://www.flickr.com/photos/fotos_medem/3399096196/
65. Searching • All object in Chef server are indexed by Solr • Can search through the API • From knife and in recipes http://www.flickr.com/photos/fotos_medem/3399096196/
66. Searching • All object in Chef server are indexed by Solr • Can search through the API • From knife and in recipes • Returns an array of JSON Node objects http://www.flickr.com/photos/fotos_medem/3399096196/
67. webservers = search("node", "role:webserver”) knife search node role:webserver Systems Integration
68. pool_members = search("node","role:webserver”) template "/etc/haproxy/haproxy.cfg" do source "haproxy-app_lb.cfg.erb" owner "root" group "root" mode 0644 variables :pool_members => pool_members.uniq notifies :restart, "service[haproxy]" end Pass results into Templates
69. pool_members = search("node","role:webserver”) template "/etc/haproxy/haproxy.cfg" do source "haproxy-app_lb.cfg.erb" owner "root" group "root" mode 0644 variables :pool_members => pool_members.uniq notifies :restart, "service[haproxy]" end Pass results into Templates
70. # Set up application listeners here. listen application 0.0.0.0:80 balance roundrobin <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%> <% if node["haproxy"]["enable_admin"] -%> listen admin 0.0.0.0:22002 mode http stats uri / <% end -%> Pass results into Templates
71. Change • Various ways • Add or remove a node to the infrastructure • Run chef-client
72. $ grep servers /etc/haproxy/haproxy.cfg servers node2.mylan 10.9.8.10 servers node3.mylan 10.9.8.11 $ knife ec2 server create -r ‘webserver’ $ knife ec2 server create -r ‘webserver’ $ knife ssh ‘role:webserver’ chef-client $ grep servers /etc/haproxy/haproxy.cfg servers node2.mylan 10.9.8.10 servers node3.mylan 10.9.8.11 servers node4.mylan 10.9.8.12 servers node5.mylan 10.9.8.13 Run chef-client
73. Change Inputs • Edit recipes • Edit run lists • chef-client http://www.flickr.com/photos/dhutchman/128541987
74. Out of slides! http://www.flickr.com/photos/calonyr11/2630312566/
75. Questions? sales@opscode.com www.opscode.com