Share
Embed
Achieving Infrastructure Portability with Chef
Chef presentation from Build A Cloud Day at SCaLE 11x
Nathen Harvey
February 22, 2013
Transcript
60 slides
-
Achieving Infrastructure Portability with Chef ScALE 11x Nathen Harvey [email protected]
IRC/Twitter/GitHub: nathenharvey www.opscode.com -
• Instant infrastructure • Unlimited capacity • Autoscaling • No
commitment • Immediate replacement Why the Cloud? -
• Each cloud defines themselves against Amazon • Entry into
the market is easier • Not a lot of price competition • Feature parity is growing Cloud Differentiation -
See Nodes Grow Application Servers Application Database Cache Load Balancers
Application Databases -
Tied together with Config Application Servers Application Database Cache Load
Balancers Application Databases -
Infrastructure is a Snowflake Application Servers Application Database Cache Load
Balancers Floating IP? Application Databases -
Evolving Complexity Load Balancers Application Servers NoSQL Database Slaves ApplicationCache
Database Cache Database -
Chef is Infrastructure as Code http://www.flickr.com/photos/louisb/4555295187/ • Programmatically provision and
configure • Treat like any other code base • Reconstruct business from code repository, data backup, and bare metal resources. -
• Chef-Client generates configurations directly on nodes from their run
list • Reduce management complexity through abstraction • Store the configuration of your programs in version control http://www.flickr.com/photos/ssoosay/5126146763/ Nodes -
Collections of Resources • Networking • Files • Directories •
Symlinks • Mounts • Routes • Users • Groups • Tasks • Packages • Software • Services • Configurations • Other Stuff http://www.flickr.com/photos/stevekeys/3123167585/ -
Declarative Interface to Resources • Define policy • Say what,
not how • Pull not Push http://www.flickr.com/photos/bixentro/2591838509/ -
Ruby! extra_packages = case node['platform'] when "ubuntu","debian" %w{ ruby1.8 ruby1.8-dev
rdoc1.8 ri1.8 libopenssl-ruby } end extra_packages.each do |pkg| package pkg do action :install end end -
Recipes and Cookbooks • Recipes are collections of Resources •
Cookbooks contain recipes, templates, files, custom resources, etc • Code re-use and modularity • Hundreds already on Community.opscode.com http://www.flickr.com/photos/shutterhacks/4474421855/ -
http://www.flickr.com/photos/kathycsus/2686772625 • IP addresses • Hostnames • FQDNs • Search
for nodes with Roles • Find configuration data Search -
pool_members = search("node","role:webserver”) template "/etc/haproxy/haproxy.cfg" do source "haproxy-app_lb.cfg.erb" owner "root"
group "root" mode 0644 variables :pool_members => pool_members.uniq notifies :restart, "service[haproxy]" end Pass Results to Templates -
# Set up application listeners here. listen application 0.0.0.0:80 balance
roundrobin <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%> <% if node["haproxy"]["enable_admin"] -%> listen admin 0.0.0.0:22002 mode http stats uri / <% end -%> Pass Results to Templates -
Jboss App Memcache Postgres Slaves Postgres Master So when this
Nagios Graphite -
Nagios Graphite Count the resources Jboss App Memcache Postgres Slaves
• Load balancer config • Nagios host ping • Nagios host ssh • Nagios host HTTP • Nagios host app health • Graphite CPU • Graphite Memory • Graphite Disk • Graphite SNMP • Memcache firewall • Postgres firewall • Postgres authZ config • 12+ resource changes for 1 node addition -
Build anything • Simple internal applications • Complex external applications
• Workstations • Hadoop clusters • IaaS infrastructure • PaaS infrastructure • SaaS applications • Storage systems • You name it http://www.flickr.com/photos/hyku/245010680/ -
And manage it simply http://www.flickr.com/photos/helico/404640681/ • Automatically reconfigure everything •
Linux, Windows, Unixes, BSDs • Load balancers • Metrics collection systems • Monitoring systems • Cloud migrations become trivial -
knife with the Chef Server • knife node • create/delete/edit
• list • knife cookbook ... • knife role ... • knife environment ... -
knife bootstrap knife bootstrap SERVER -r 'role[webserver]' -i ~/.ssh/id_rsa •
SSH to the machine given existing credentials • Install the Chef Client • Register with the Chef Server • Run the initial Run List • Now managed with Chef! -
knife ec2 $ knife ec2 Available ec2 subcommands: (for details,
knife SUB-COMMAND -- help) ** EC2 COMMANDS ** knife ec2 flavor list (options) knife ec2 instance data (options) knife ec2 server create (options) knife ec2 server delete SERVER [SERVER] (options) knife ec2 server list (options) $ knife ec2 server create -S keypair -i ~/.ssh/id_rsa -x ubuntu -I ami-4721882e -f m1.small -r 'role[webserver]' -
knife openstack $ knife openstack Available openstack subcommands: (for details,
knife SUB- COMMAND --help) ** OPENSTACK COMMANDS ** knife openstack flavor list (options) knife openstack image list (options) knife openstack server create (options) knife openstack server delete SERVER [SERVER] (options) knife openstack server list (options) $ knife openstack server create -S keypair -i ~/.ssh/id_rsa -x ubuntu -I 1231 -f standard.small -r 'role[webserver]' -
Chef for Infrastructure Portability • knife ec2 • knife rackspace
• knife hp • knife google • knife azure • knife cloudstack • knife openstack • knife vsphere • ... and many others -
The Chef Community • Apache License, Version 2.0 • 850+
Individual contributors • 150+ Corporate contributors • HP, Dell, Rackspace, VMware, Joyent, Calxeda, Heroku, SUSE and many more • 550+ cookbooks • http://community.opscode.com -
• Vagrant • VMware • CloudStack • Eucalyptus • OpenStack
• bare metal • AWS • Rackspace • HP • Google • Azure • many others Desktop, Virtualization, Private & Public Clouds -
• Vagrant • VMware • CloudStack • Eucalyptus • OpenStack
• bare metal Desktop, Virtualization, Private & Public Clouds • AWS • Rackspace • HP • Google • Azure • many others -
Tale of the Tape • Artur Bergman, CEO at Fastly
• It's All About Speed • http://youtu.be/qRnTejOMbZU • Jason Stowe, CEO at Cycle Computing • CycleCloud + Chef = 50,000-core Utility Supercomputer for Science • http://youtu.be/cEaQB6e7G0Q -
Chef and Abstractions • Resources and Providers • Cookbooks (may)
normalize deployment • Knife treats APIs the same • Chef strives to not be opinionated -
Environments • Lock down versions of cookbooks • stable releases
vs. development • Enforce attributes for deployment • Ports, addresses, etc. • Different run lists based on environment • Debugging enabled? -
Environments • Use the same infrastructure code for wherever you
deploy • Development, QA, Pre-Production, Prod • Role-based Access Controls to restrict the promotion of deployment code -
TL;DR • Every infrastructure is a unique snowflake • Understand
the costs associated with the features of your platform(s) of choice. • Chef enables Infrastructure Portability • "Data Gravity" is the primary concern -
Further Resources • http://opscode.com/ • http://community.opscode.com/ • http://docs.opscode.com • http://wiki.opscode.com/
• http://lists.opscode.com • http://youtube.com/user/Opscode -
Food Fight Show • http://foodfightshow.org • The Podcast Where DevOps
Chef Do Battle • Regular updates about new Cookbooks, Knife-plugins, and more • Best Practices for working with Chef -
More Training in LA • SOCAL-CHEF saves you $200 •
http://opscode.eventbrite.com/