Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Achieving Infrastructure Portability with Chef

Nathen Harvey
February 22, 2013

Achieving Infrastructure Portability with Chef

Chef presentation from Build A Cloud Day at SCaLE 11x

Nathen Harvey

February 22, 2013
Tweet

More Decks by Nathen Harvey

Other Decks in Technology

Transcript

  1. • Instant infrastructure • Unlimited capacity • Autoscaling • No

    commitment • Immediate replacement Why the Cloud?
  2. • Each cloud defines themselves against Amazon • Entry into

    the market is easier • Not a lot of price competition • Feature parity is growing Cloud Differentiation
  3. Chef is Infrastructure as Code http://www.flickr.com/photos/louisb/4555295187/ • Programmatically provision and

    configure • Treat like any other code base • Reconstruct business from code repository, data backup, and bare metal resources.
  4. • Chef-Client generates configurations directly on nodes from their run

    list • Reduce management complexity through abstraction • Store the configuration of your programs in version control http://www.flickr.com/photos/ssoosay/5126146763/ Nodes
  5. Collections of Resources • Networking • Files • Directories •

    Symlinks • Mounts • Routes • Users • Groups • Tasks • Packages • Software • Services • Configurations • Other Stuff http://www.flickr.com/photos/stevekeys/3123167585/
  6. Declarative Interface to Resources • Define policy • Say what,

    not how • Pull not Push http://www.flickr.com/photos/bixentro/2591838509/
  7. Ruby! extra_packages = case node['platform'] when "ubuntu","debian" %w{ ruby1.8 ruby1.8-dev

    rdoc1.8 ri1.8 libopenssl-ruby } end extra_packages.each do |pkg| package pkg do action :install end end
  8. Recipes and Cookbooks • Recipes are collections of Resources •

    Cookbooks contain recipes, templates, files, custom resources, etc • Code re-use and modularity • Hundreds already on Community.opscode.com http://www.flickr.com/photos/shutterhacks/4474421855/
  9. pool_members = search("node","role:webserver”) template "/etc/haproxy/haproxy.cfg" do source "haproxy-app_lb.cfg.erb" owner "root"

    group "root" mode 0644 variables :pool_members => pool_members.uniq notifies :restart, "service[haproxy]" end Pass Results to Templates
  10. # Set up application listeners here. listen application 0.0.0.0:80 balance

    roundrobin <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%> <% if node["haproxy"]["enable_admin"] -%> listen admin 0.0.0.0:22002 mode http stats uri / <% end -%> Pass Results to Templates
  11. Nagios Graphite Count the resources Jboss App Memcache Postgres Slaves

    • Load balancer config • Nagios host ping • Nagios host ssh • Nagios host HTTP • Nagios host app health • Graphite CPU • Graphite Memory • Graphite Disk • Graphite SNMP • Memcache firewall • Postgres firewall • Postgres authZ config • 12+ resource changes for 1 node addition
  12. Build anything • Simple internal applications • Complex external applications

    • Workstations • Hadoop clusters • IaaS infrastructure • PaaS infrastructure • SaaS applications • Storage systems • You name it http://www.flickr.com/photos/hyku/245010680/
  13. And manage it simply http://www.flickr.com/photos/helico/404640681/ • Automatically reconfigure everything •

    Linux, Windows, Unixes, BSDs • Load balancers • Metrics collection systems • Monitoring systems • Cloud migrations become trivial
  14. knife with the Chef Server • knife node • create/delete/edit

    • list • knife cookbook ... • knife role ... • knife environment ...
  15. knife bootstrap knife bootstrap SERVER -r 'role[webserver]' -i ~/.ssh/id_rsa •

    SSH to the machine given existing credentials • Install the Chef Client • Register with the Chef Server • Run the initial Run List • Now managed with Chef!
  16. knife ec2 $ knife ec2 Available ec2 subcommands: (for details,

    knife SUB-COMMAND -- help) ** EC2 COMMANDS ** knife ec2 flavor list (options) knife ec2 instance data (options) knife ec2 server create (options) knife ec2 server delete SERVER [SERVER] (options) knife ec2 server list (options) $ knife ec2 server create -S keypair -i ~/.ssh/id_rsa -x ubuntu -I ami-4721882e -f m1.small -r 'role[webserver]'
  17. knife openstack $ knife openstack Available openstack subcommands: (for details,

    knife SUB- COMMAND --help) ** OPENSTACK COMMANDS ** knife openstack flavor list (options) knife openstack image list (options) knife openstack server create (options) knife openstack server delete SERVER [SERVER] (options) knife openstack server list (options) $ knife openstack server create -S keypair -i ~/.ssh/id_rsa -x ubuntu -I 1231 -f standard.small -r 'role[webserver]'
  18. Chef for Infrastructure Portability • knife ec2 • knife rackspace

    • knife hp • knife google • knife azure • knife cloudstack • knife openstack • knife vsphere • ... and many others
  19. The Chef Community • Apache License, Version 2.0 • 850+

    Individual contributors • 150+ Corporate contributors • HP, Dell, Rackspace, VMware, Joyent, Calxeda, Heroku, SUSE and many more • 550+ cookbooks • http://community.opscode.com
  20. • Vagrant • VMware • CloudStack • Eucalyptus • OpenStack

    • bare metal • AWS • Rackspace • HP • Google • Azure • many others Desktop, Virtualization, Private & Public Clouds
  21. • Vagrant • VMware • CloudStack • Eucalyptus • OpenStack

    • bare metal Desktop, Virtualization, Private & Public Clouds • AWS • Rackspace • HP • Google • Azure • many others
  22. Tale of the Tape • Artur Bergman, CEO at Fastly

    • It's All About Speed • http://youtu.be/qRnTejOMbZU • Jason Stowe, CEO at Cycle Computing • CycleCloud + Chef = 50,000-core Utility Supercomputer for Science • http://youtu.be/cEaQB6e7G0Q
  23. Chef and Abstractions • Resources and Providers • Cookbooks (may)

    normalize deployment • Knife treats APIs the same • Chef strives to not be opinionated
  24. Environments • Lock down versions of cookbooks • stable releases

    vs. development • Enforce attributes for deployment • Ports, addresses, etc. • Different run lists based on environment • Debugging enabled?
  25. Environments • Use the same infrastructure code for wherever you

    deploy • Development, QA, Pre-Production, Prod • Role-based Access Controls to restrict the promotion of deployment code
  26. TL;DR • Every infrastructure is a unique snowflake • Understand

    the costs associated with the features of your platform(s) of choice. • Chef enables Infrastructure Portability • "Data Gravity" is the primary concern
  27. Food Fight Show • http://foodfightshow.org • The Podcast Where DevOps

    Chef Do Battle • Regular updates about new Cookbooks, Knife-plugins, and more • Best Practices for working with Chef