Slide 1

Slide 1 text

Introduction to Chef [email protected] www.opscode.com @someara

Slide 2

Slide 2 text

No content

Slide 3

Slide 3 text

• U has a cloud • Now what? http://www.flickr.com/photos/ian_munroe/4758240536/ Congratulations!!!

Slide 4

Slide 4 text

APIs are awesome • You can provision compute resources in seconds • You can provision storage resources in seconds • That’s cool. http://www.flickr.com/photos/jdhancock/3634246981/

Slide 5

Slide 5 text

Chef can help with that • knife ec2 server create • knife rackspace server create • knife terremark server create • knife voxel server create • knife gandi server create • knife cloudstack server create • knife vsphere server create • knife eucalyptus server create • knife openstack server create http://www.flickr.com/photos/kyz/3122499444/

Slide 6

Slide 6 text

But then what? http://www.flickr.com/photos/doctorow/2698336843

Slide 7

Slide 7 text

You need to configure them

Slide 8

Slide 8 text

http://www.flickr.com/photos/sbh/462754460/ Infrastructure

Slide 9

Slide 9 text

Collection of Resources • Nodes • Networking • Files • Directories • Symlinks • Mounts • Routes • Users • Groups • Tasks • Packages • Software • Services • Configurations • Stuff http://www.flickr.com/photos/stevekeys/3123167585/

Slide 10

Slide 10 text

Acting in concert http://www.flickr.com/photos/glowjangles/4081048126/

Slide 11

Slide 11 text

http://www.flickr.com/photos/28309157@N08/3743455858/ To provide a Service

Slide 12

Slide 12 text

http://www.flickr.com/photos/16339684@N00/2681435235/ And it evolves

Slide 13

Slide 13 text

Application See Node

Slide 14

Slide 14 text

Application Application Database See Nodes

Slide 15

Slide 15 text

Application App Databases See Nodes Grow

Slide 16

Slide 16 text

App Servers App Databases See Nodes Grow

Slide 17

Slide 17 text

App LB App Servers App Databases See Nodes Grow

Slide 18

Slide 18 text

App LBs App Servers App Databases See Nodes Grow

Slide 19

Slide 19 text

App LBs App Servers App DB Cache App DBs See Nodes Grow

Slide 20

Slide 20 text

App LBs App Servers App DB Cache App DBs Stitched together with configs

Slide 21

Slide 21 text

App LB App Servers App DB Cache App DBs Floating IP? Your Infrastructure is a snow flake

Slide 22

Slide 22 text

App LBs App Servers NoSQL DB slaves Cache DB Cache DBs Complexity increases quickly

Slide 23

Slide 23 text

Complexity increases very quickly DC1 DC3 DC2

Slide 24

Slide 24 text

Configuration Management http://www.flickr.com/photos/philliecasablanca/3354734116/

Slide 25

Slide 25 text

Golden Images are not the answer • Gold is heavy • Hard to transport • Hard to mold • Easy to lose configuration detail http://www.flickr.com/photos/garysoup/2977173063/

Slide 26

Slide 26 text

Jboss App Memcache Postgres Slaves Postgres Master Typical Boring Infrastructure Nagios Graphite

Slide 27

Slide 27 text

Jboss App Memcache Postgres Slaves Postgres Master New Compliance Mandate Nagios Graphite • Move SSH off port 22 • Lets put it on 2022

Slide 28

Slide 28 text

Jboss App Memcache Postgres Slaves Postgres Master 6 Golden Image Updates Nagios Graphite • edit /etc/ssh/sshd_config 1 2 3 4 5 6

Slide 29

Slide 29 text

Jboss App Memcache Postgres Slaves Postgres Master 12 Instance Replacements Nagios Graphite • Delete, launch 1 2 3 4 5 6 7 8 9 10 11 12 • Repeat • Typically manually

Slide 30

Slide 30 text

Done in Maintenance Windows • Don’t break anything! • Bob just got fired =( 5 Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite 1 2 4 5 6 7 8 9 10 11 12 3

Slide 31

Slide 31 text

Jboss App Memcache Postgres Slaves Postgres Master Different IP Addresses? Nagios Graphite • Invalid configs!

Slide 32

Slide 32 text

http://www.flickr.com/photos/francoforeshock/5716969942/ Configuration Desperation

Slide 33

Slide 33 text

Chef Solves This Problem • But you already guessed that, didn’t you?

Slide 34

Slide 34 text

• Generate configurations directly on nodes • Reduce management complexity • Version control the programs http://www.flickr.com/photos/ssoosay/5126146763/ Programs!

Slide 35

Slide 35 text

Declarative Interface to Resources • Define policy • Say what, not how • Pull not Push http://www.flickr.com/photos/bixentro/2591838509/

Slide 36

Slide 36 text

Chef is Infrastructure as Code http://www.flickr.com/photos/louisb/4555295187/ • Programmatically provision and configure • Treat like any other code base • Reconstruct business from code repository, data backup, and bare metal resources.

Slide 37

Slide 37 text

package "ntp" do action :install end service "ntpd" do action [:enable,:start] end template "/etc/ntpd.conf" do source "ntpd.conf.erb" owner "root" group "root" mode 0644 action :create variables(:time_server => “time.example.com”) notifies :restart, “service[ntpd]” end That looks like this

Slide 38

Slide 38 text

package "net-snmp" do action :install end service "snmpd" do action [:enable,:start] end template "/etc/snmpd.conf" do source "snmpd.conf.erb" owner "root" group "root" mode 0644 action :create variables(:community_string => “not_public”) notifies :restart, “service[snmpd]” end Or this

Slide 39

Slide 39 text

Recipes and Cookbooks • Recipes are collections of Resources • Cookbooks contain recipes, templates, files, custom resources, etc • Code re-use and modularity http://www.flickr.com/photos/shutterhacks/4474421855/

Slide 40

Slide 40 text

Run Lists Server Server Server Server chef-server API chef-client recipe[ntp::client] node ntp client.rb

Slide 41

Slide 41 text

Run Lists Server Server Server Server chef-server API chef-client “ntp::client”, “openssh::server” node ntp client.rb openssh server.rb

Slide 42

Slide 42 text

Run Lists Server Server Server Server chef-server API chef-client “recipe[ntp::client]”, “recipe[openssh::server]”, “recipe[apache]”, “recipe[php]” node ntp client.rb openssh server.rb apache default.rb php default.rb

Slide 43

Slide 43 text

Roles Role Recipe Recipe Recipe Role Role Recipe Recipe Recipe Role Recipe Server Server Server Server chef-server API Knife

Slide 44

Slide 44 text

Run Lists Server Server Server Server chef-server API chef-client “recipe[ntp::client]”, “recipe[openssh::server]”, “recipe[apache]”, “recipe[php]” node ntp client.rb openssh server.rb apache default.rb php default.rb

Slide 45

Slide 45 text

Server Server Server Server chef-server API chef-client “role[base]”, “role[webserver]” node ntp client.rb openssh server.rb apache default.rb php default.rb Roles

Slide 46

Slide 46 text

Server Server Server Server chef-server API chef-client “role[webserver]” node ntp client.rb openssh server.rb apache default.rb php default.rb Roles chef-client “role[database]” node ntp client.rb openssh server.rb mysql server.rb

Slide 47

Slide 47 text

http://www.flickr.com/photos/kathycsus/2686772625 Search • IP addresses • Hostnames • FQDNs • Search for nodes with Roles • Find configuration data

Slide 48

Slide 48 text

pool_members = search("node","role:webserver”) template "/etc/haproxy/haproxy.cfg" do source "haproxy-app_lb.cfg.erb" owner "root" group "root" mode 0644 variables :pool_members => pool_members.uniq notifies :restart, "service[haproxy]" end Pass results into Templates

Slide 49

Slide 49 text

# Set up application listeners here. listen application 0.0.0.0:80 balance roundrobin <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%> <% if node["haproxy"]["enable_admin"] -%> listen admin 0.0.0.0:22002 mode http stats uri / <% end -%> Pass results into Templates

Slide 50

Slide 50 text

Jboss App Memcache Postgres Slaves Postgres Master So when this Nagios Graphite

Slide 51

Slide 51 text

Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite Becomes this

Slide 52

Slide 52 text

Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite This can happen automatically

Slide 53

Slide 53 text

Nagios Graphite Count the resources Jboss App Memcache Postgres Slaves • Load balancer config • Nagios host ping • Nagios host ssh • Nagios host HTTP • Nagios host app health • Graphite CPU • Graphite Memory • Graphite Disk • Graphite SNMP • Memcache firewall • Postgres firewall Postgres authZ config • 12+ resource changes for 1 node addition

Slide 54

Slide 54 text

http://www.flickr.com/photos/evelynishere/2798236471/ CLONING CANNOT COPE WITH THIS • Chef can.

Slide 55

Slide 55 text

Build anything • Simple internal applications • Complex internal applications • Workstations • Hadoop clusters • IaaS applications • PaaS applications • SaaS applications • Storage systems • You name it http://www.flickr.com/photos/hyku/245010680/

Slide 56

Slide 56 text

And manage it simply http://www.flickr.com/photos/helico/404640681/ • Automatically reconfigure everything • Load balancers • Metrics collection systems • Monitoring systems • Whatever • Cloud migrations become trivial

Slide 57

Slide 57 text

Questions? [email protected] www.opscode.com