Share
Embed
Intro to Chef - Charm City Linux
A brief introduction to Chef presented at the Charm City Linux group.
Nathen Harvey
May 13, 2013
Transcript
68 slides
-
Nathen • Technical Community Manager, Opscode • Co-host of Food
Fight Show podcast • Co-organizer • DevOpsDC • MongoDC • Co-moderator - Ruby Hangout (usually) • Formerly - Web Operations at CustomInk -
Chef is an automation platform for developers & systems engineers
to continuously define, build, and manage infrastructure. CHEF USES: Recipes and Cookbooks that describe Infrastructure as Code. Chef enables people to easily build & manage complex & dynamic applications at massive scale • New model for describing infrastructure that promotes reuse • Programmatically provision and configure • Reconstruct business from code repository, data backup, and bare metal resources “ ” Chef -
Evolving towards an automation framework • Just build it •
Keep notes in server.txt • Move notes to the wiki • Custom scripts (in scm?!) • Build from template / Golden Images • Automation framework -
See Nodes Grow Application Servers Application Database Cache Load Balancers
Application Databases -
Tied together with Config Application Servers Application Database Cache Load
Balancers Application Databases -
Infrastructure is a Snowflake Application Servers Application Database Cache Load
Balancers Floating IP? Application Databases -
Evolving Complexity Load Balancers Application Servers NoSQL Database Slaves ApplicationCache
Database Cache Database -
Golden Images are not the answer • Gold is heavy
• Hard to transport • Hard to mold • Easy to lose configuration detail http://www.flickr.com/photos/garysoup/2977173063/ -
Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite •
Move SSH off port 22 • Lets put it on 2022 New Compliance Mandate! -
Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite •
edit /etc/ssh/sshd_config 1 2 3 4 5 6 6 Golden Image Updates -
Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite •
Delete, launch 1 2 3 4 5 6 7 8 9 10 11 12 • Repeat • Typically manually 12 Instance Replacements -
• Don’t break anything! • Bob just got fired =(
5 Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite 1 2 4 5 6 7 8 9 10 11 12 3 Done in Maintenance Windows -
Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite •
Invalid configs! Different IP Addresses? -
Chef is Infrastructure as Code http://www.flickr.com/photos/louisb/4555295187/ • Programmatically provision and
configure • Treat like any other code base • Reconstruct business from code repository, data backup, and bare metal resources. -
• Chef-Client generates configurations directly on nodes from their run
list • Reduce management complexity through abstraction • Store the configuration of your programs in version control http://www.flickr.com/photos/ssoosay/5126146763/ Nodes -
Collections of Resources • Networking • Files • Directories •
Symlinks • Mounts • Routes • Users • Groups • Tasks • Packages • Software • Services • Configurations • Other Stuff http://www.flickr.com/photos/stevekeys/3123167585/ -
Declarative Interface to Resources • Define policy • Say what,
not how • Pull not Push http://www.flickr.com/photos/bixentro/2591838509/ -
Ruby! extra_packages = case node['platform'] when "ubuntu","debian" %w{ ruby1.8 ruby1.8-dev
rdoc1.8 ri1.8 libopenssl-ruby } end extra_packages.each do |pkg| package pkg do action :install end end -
Recipes and Cookbooks • Recipes are collections of Resources •
Cookbooks contain recipes, templates, files, custom resources, etc • Code re-use and modularity • Hundreds already on Community.opscode.com http://www.flickr.com/photos/shutterhacks/4474421855/ -
http://www.flickr.com/photos/kathycsus/2686772625 • IP addresses • Hostnames • FQDNs • Search
for nodes with Roles • Find configuration data Search -
pool_members = search("node","role:webserver”) template "/etc/haproxy/haproxy.cfg" do source "haproxy-app_lb.cfg.erb" owner "root"
group "root" mode 0644 variables :pool_members => pool_members.uniq notifies :restart, "service[haproxy]" end Pass Results to Templates -
# Set up application listeners here. listen application 0.0.0.0:80 balance
roundrobin <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%> <% if node["haproxy"]["enable_admin"] -%> listen admin 0.0.0.0:22002 mode http stats uri / <% end -%> Pass Results to Templates -
Jboss App Memcache Postgres Slaves Postgres Master So when this
Nagios Graphite -
Nagios Graphite Count the resources Jboss App Memcache Postgres Slaves
• Load balancer config • Nagios host ping • Nagios host ssh • Nagios host HTTP • Nagios host app health • Graphite CPU • Graphite Memory • Graphite Disk • Graphite SNMP • Memcache firewall • Postgres firewall • Postgres authZ config • 12+ resource changes for 1 node addition -
Build anything • Simple internal applications • Complex external applications
• Workstations • Hadoop clusters • IaaS infrastructure • PaaS infrastructure • SaaS applications • Storage systems • You name it http://www.flickr.com/photos/hyku/245010680/ -
And manage it simply http://www.flickr.com/photos/helico/404640681/ • Automatically reconfigure everything •
Linux, Windows, Unixes, BSDs • Load balancers • Metrics collection systems • Monitoring systems • Cloud migrations become trivial -
knife with the Chef Server • knife node • create/delete/edit
• list • knife cookbook ... • knife role ... • knife environment ... -
knife search • What operating systems are running? • What
version of ruby is running? • How much memory do you have on each machine? -
"memory": { "swap": { "cached": "0kB", "total": "4128760kB", "free": "4128760kB"
}, "total": "2055676kB", "free": "1646524kB", "buffers": "35032kB", "cached": "210276kB", "active": "125336kB", "inactive": "142884kB", "dirty": "8kB", "writeback": "0kB", "anon_pages": "22976kB", "mapped": "8416kB", "slab": "121512kB", "slab_reclaimable": "41148kB", "slab_unreclaim": "80364kB", "page_tables": "1784kB", "nfs_unstable": "0kB", "bounce": "0kB", "commit_limit": "5156596kB", "committed_as": "74980kB", "vmalloc_total": "34359738367kB", "vmalloc_used": "274512kB", "vmalloc_chunk": "34359449936kB" }, Ohai! "block_device": { "ram0": { "size": "32768", "removable": "0" }, "ram1": { "size": "32768", "removable": "0" }, "ram2": { "size": "32768", "removable": "0" }, "hostname": "server-1", "fqdn": "server-1.example.com", "domain": "example.com", "network": { "interfaces": { "eth0": { "type": "eth", "number": "0", "encapsulation": "Ethernet", "addresses": { "00:0C:29:43:26:C5": { "family": "lladdr" }, "192.168.177.138": { "family": "inet", "broadcast": "192.168.177.255", "netmask": "255.255.255.0" }, "fe80::20c:29ff:fe43:26c5": { "family": "inet6", "prefixlen": "64", "scope": "Link" } }, -
knife search • knife search “*:*” -a platform • knife
search “*:*” -a languages.ruby.version • knife search “*:*” -a memory.total -
knife bootstrap • knife bootstrap SERVER -r 'role[webserver]' -i ~/.ssh/
id_rsa • SSH to the machine given existing credentials • Install the Chef Client • Register with the Chef Server • Run the initial Run List • Now managed with Chef! -
knife ec2 • $ knife ec2 • Available ec2 subcommands:
(for details, knife SUB-COMMAND --help) • ** EC2 COMMANDS ** • knife ec2 flavor list (options) • knife ec2 instance data (options) • knife ec2 server create (options) • knife ec2 server delete SERVER [SERVER] (options) • knife ec2 server list (options) • $ knife ec2 server create -S keypair -i ~/.ssh/id_rsa -x ubuntu -I ami-4721882e -f m1.small -r 'role[webserver]' -
knife openstack • $ knife openstack • Available openstack subcommands:
(for details, knife SUB- COMMAND --help) • ** OPENSTACK COMMANDS ** • knife openstack flavor list (options) • knife openstack image list (options) • knife openstack server create (options) • knife openstack server delete SERVER [SERVER] (options) • knife openstack server list (options) • $ knife openstack server create -S keypair -i ~/.ssh/id_rsa • -x ubuntu -I 1231 -f standard.small -r 'role[webserver]' -
Chef for Infrastructure Portability • knife ec2 • knife rackspace
• knife hp • knife google • knife azure • knife cloudstack • knife openstack • knife vsphere • ... and many others -
Testing Tools • knife cookbook test - Verify ruby syntax
• Foodcritic - Cookbook linter • Chefspec - Unit testing recipes • Fauxhai - Mock all the things • Minitest Chef Handler - post-converge tests • Why-run - Best guess -
Moar Testing Tools • Vagrant - Local development and testing
• Test Kitchen - Cross-platform testing • Cucumber Chef - acceptance & integration testing -
The Chef Community • Apache License, Version 2.0 • 1200+
Individual contributors • 200+ Corporate contributors • Google, HP, Dell, Rackspace, VMware, Joyent, Calxeda, Heroku, SUSE and many more • 800+ cookbooks • http://community.opscode.com -
Food Fight Show • http://foodfightshow.org • The Podcast Where DevOps
Chef Do Battle • Regular updates about new Cookbooks, Knife- plugins, and more • Best Practices for working with Chef -
Food Fight Show • The Phoenix Project • elastic search
• LWRPs, Libraries, and Definitions • Zookeeper • Adaptive Fault Detection • What’s New in Chef 11 • ...and many more!