Share
Embed
Getting Started with Chef for Rails
Presentation given at the Northern Virginia Ruby Users Group.
Overview of Chef
Building a project in Chef
Provision a Server for Your Rails App
Additional resources
Nathen Harvey
January 29, 2013
Transcript
98 slides
-
The Joy of Cooking: Whip up a Rails Environment with
Chef Nathen Harvey @nathenharvey Northern Virginia Ruby Users Group -
Nathen • Technical Community Manager, Opscode • Co-organizer • DevOpsDC
• MongoDC • Formerly - Web Operations at CustomInk -
Agenda • Overview of Chef • Building a project in
Chef • Provision a Server for Your Rails App • Additional resources • Obligatory Shameless Plugs • Please Ask Questions Along the Way -
Evolution of Configuration Management • Just build it • Keep
notes in server.txt • Move notes to the wiki • Custom scripts (in scm?!) • Configuration Management framework -
When should I use a CM Framework? • After you
outgrow Heroku -
Collection of Resources • Nodes • Networking • Files •
Directories • Symlinks • Mounts • Routes • Users • Groups • Tasks • Packages • Software • Services • Configurations • Stuff http://www.flickr.com/photos/stevekeys/3123167585/ -
App LBs App Servers App DB Cache App DBs See
Nodes Grow -
App LBs App Servers App DB Cache App DBs Stitched
together with configs -
App LB App Servers App DB Cache App DBs Floating
IP? Your Infrastructure is a snow flake -
App LBs App Servers NoSQL DB slaves Cache DB Cache
DBs Complexity increases quickly -
Golden Images are not the answer • Gold is heavy
• Hard to transport • Hard to mold • Easy to lose configuration detail http://www.flickr.com/photos/garysoup/2977173063/ -
Rails App Memcache Postgres Slaves Postgres Master New Compliance Mandate
Nagios Graphite • Move SSH off port 22 • Lets put it on 2022 -
Rails App Memcache Postgres Slaves Postgres Master 6 Golden Image
Updates Nagios Graphite • edit /etc/ssh/sshd_config 1 2 3 4 5 6 -
Rails App Memcache Postgres Slaves Postgres Master 12 Instance Replacements
Nagios Graphite • Delete, launch 1 2 3 4 5 6 7 8 9 10 11 12 • Repeat • Typically manually -
Done in Maintenance Windows • Don’t break anything! • Bob
just got fired =( 5 Rails App Memcache Postgres Slaves Postgres Master Nagios Graphite 1 2 4 5 6 7 8 9 10 11 12 3 -
Rails App Memcache Postgres Slaves Postgres Master Different IP Addresses?
Nagios Graphite • Invalid configs! -
• Generate configurations directly on nodes • Reduce management complexity
• Version control the programs http://www.flickr.com/photos/ssoosay/5126146763/ Programs! -
Declarative Interface to Resources • Define policy • Say what,
not how • Pull not Push http://www.flickr.com/photos/bixentro/2591838509/ -
Chef is Infrastructure as Code http://www.flickr.com/photos/louisb/4555295187/ • Programmatically provision and
configure • Treat like any other code base • Reconstruct business from code repository, data backup, and bare metal resources. -
package "ntp" do action :install end service "ntpd" do action
[:enable,:start] end template "/etc/ntpd.conf" do source "ntpd.conf.erb" owner "root" group "root" mode 0644 action :create variables(:time_server => “time.example.com”) notifies :restart, “service[ntpd]” end That looks like this -
Rails App Memcache Postgres Slaves Postgres Master So when this
Nagios Graphite -
Rails App Memcache Postgres Slaves Postgres Master Nagios Graphite This
can happen automatically -
Nagios Graphite Count the resources Rails App Memcache Postgres Slaves
• Load balancer config • Nagios host ping • Nagios host ssh • Nagios host HTTP • Nagios host app health • Graphite CPU • Graphite Memory • Graphite Disk • Graphite SNMP • Memcache firewall • Postgres firewall • Postgres authZ config • 12+ resource changes for 1 node addition -
Building a Project in Chef OK, so how do I
build something with Chef? -
Building a Chef Project • First, come up with your
policy / specification • Abstract the resources in your spec -
Resources package "tmux" do action :install end directory "/u/apps/awesome" do
owner "apache" group "apache" action :create recursive true end -
Building a Chef Project • First, come up with your
policy / specification • Abstract the resources in your spec • Write recipes -
Recipe include_recipe "app_user" app_name = node["app_name"] app_user = node["app_user"] app_group
= node["app_group"] %w(releases shared).each do |dir| directory "/u/apps/#{app_name}/#{dir}" do mode "0755" owner app_user group app_group recursive true end end -
Building a Chef Project • First, come up with your
policy / specification • Abstract the resources in your spec • Write recipes • Package recipes in cookbooks -
Cookbooks |-- ldirectord | |-- README.md | |-- attributes |
| `-- default.rb | |-- metadata.rb | |-- recipes | | `-- default.rb | `-- templates | `-- default | `-- site.cf.erb -
Cookbooks |-- monit | |-- README.md | |-- attributes |
| `-- default.rb | |-- files | | `-- ubuntu | | `-- monit.default | |-- metadata.rb | |-- recipes | | `-- default.rb | `-- templates | `-- default | `-- monitrc.erb -
Recipes and Cookbooks • Recipes are collections of Resources •
Cookbooks contain recipes, templates, files, custom resources, etc • Code re-use and modularity http://www.flickr.com/photos/shutterhacks/4474421855/ -
Building a Chef Project • First, come up with your
policy / specification • Abstract the resources in your spec • Write recipes • Package recipes in cookbooks • Apply recipes to nodes -
Nodes • Representation of a host • runs the Chef
client • has attributes • has a list of recipes to be applied -
Building a Chef Project • First, come up with your
policy / specification • Abstract the resources in your spec • Write recipes • Package recipes in cookbooks • Apply recipes to nodes • Group things into roles -
Roles • mechanism for easily composing sets of functionality •
have attributes and a list of recipes to be applied -
Run Lists Server Server Server Server chef-server API chef-client recipe[rails_app]
node rails_app default.rb -
Run Lists Server Server Server Server chef-server API chef-client “role[base]”,
“role[my_awesome_app]” node ntp default.rb sudo default.rb rails_app default.rb -
Server Server Server Server chef-server API chef-client “role[webserver]” node ntp
client.rb openssh server.rb apache default.rb rails_app default.rb Roles chef-client “role[database]” node ntp client.rb openssh server.rb postgresql server.rb -
Roles name "base" description "Base of all nodes" default_attributes( "newrelic"
=> { "license_key" => "cbb1f5..." } ) run_list( "recipe[base_config]", "recipe[users]", "recipe[groups]", "recipe[sudo]" ) -
Building a Chef Project • First, come up with your
policy / specification • Abstract the resources in your spec • Write recipes • Package recipes in cookbooks • Apply recipes to nodes • Group things into roles -
© Opscode, 2011 – Confidential – DO NOT DISTRIBUTE Chef
Provides a Model for Reuse That Works 64 700+ Cookbooks “Yesterday we started open sourcing some of our Opscode Chef work created at bestbuy.com; bit.ly/ yDV9Hl #Splunk #opschef #expectmor” -
Knife • Chef’s secret weapon • Command-line interface for interacting
with • Chef Server API • Local Chef Repository • Community Site • and more... -
Download Community Cookbooks $ knife cookbook site download passenger_apache2 $
knife cookbook site download mysql -
Create a Cookbook $ knife cookbook create rails_app ** Creating
cookbook rails_app ** Creating README for cookbook: rails_app ** Creating CHANGELOG for cookbook: rails_app ** Creating metadata for cookbook: rails_app -
Application Recipe • Set-up some directories %w(releases shared shared/system shared/pids
shared/logs shared/config).each do |dir| directory "#{deploy_to}/#{app_name}/#{dir}" do action :create owner app_user group app_group mode "0664" recursive true end end -
Application Recipe • Configure Apache / Passenger web_app app_name do
docroot "#{deploy_to}/current/public" server_name "#{app_name}.#{node["domain"]}" server_aliases [ app_name, "localhost", node["hostname"] ] rails_env "production" end -
node[“domain”] • cookbook • default[“domain”] = “localhost” • environments •
test •“domain” : “test.awesomeco.com” • production •“domain” : “awesomeco.com" -
node[“hostname”] • The node knows it’s own hostname • Not
something we need to set in our code -
"memory": { "swap": { "cached": "0kB", "total": "4128760kB", "free": "4128760kB"
}, "total": "2055676kB", "free": "1646524kB", "buffers": "35032kB", "cached": "210276kB", "active": "125336kB", "inactive": "142884kB", "dirty": "8kB", "writeback": "0kB", "anon_pages": "22976kB", "mapped": "8416kB", "slab": "121512kB", "slab_reclaimable": "41148kB", "slab_unreclaim": "80364kB", "page_tables": "1784kB", "nfs_unstable": "0kB", "bounce": "0kB", "commit_limit": "5156596kB", "committed_as": "74980kB", "vmalloc_total": "34359738367kB", "vmalloc_used": "274512kB", "vmalloc_chunk": "34359449936kB" }, Ohai! "block_device": { "ram0": { "size": "32768", "removable": "0" }, "ram1": { "size": "32768", "removable": "0" }, "ram2": { "size": "32768", "removable": "0" }, "hostname": "server-1", "fqdn": "server-1.example.com", "domain": "example.com", "network": { "interfaces": { "eth0": { "type": "eth", "number": "0", "encapsulation": "Ethernet", "addresses": { "00:0C:29:43:26:C5": { "family": "lladdr" }, "192.168.177.138": { "family": "inet", "broadcast": "192.168.177.255", "netmask": "255.255.255.0" }, "fe80::20c:29ff:fe43:26c5": { "family": "inet6", "prefixlen": "64", "scope": "Link" } }, -
Database Recipe mysql_connection_info = { :host => "localhost", :username =>
'root', :password => node['mysql']['server_root_password'] } mysql_database app_name do connection mysql_connection_info action :create end -
Database Recipe mysql_database_user node["database"]["user"] do connection mysql_connection_info password node["database"]["pw"] database_name
node["database"]["name"] host "%" action :grant end -
Create Roles • mechanism for easily composing sets of functionality
• have attributes and a list of recipes to be applied • Define as Ruby or JSON • base-ubuntu • rails-web • rails-database -
Role as Ruby name "base-ubuntu" description "all Ubuntu servers" run_list(
"recipe[apt]", "recipe[build-essential]" ) -
Role as JSON { "name": "rails-web", "description": "Web Server for
our Rails App", "default_attributes": { }, "run_list": [ "recipe[rails_app::web]" ] } -
Run Chef Client Server Server Server Server chef-server API chef-client
“role[base-ubuntu]”, “role[rails-web]” node apt default.rb build-essential default.rb rails-app web.rb -
Review • Server provisioned and communicating with Chef Server API
• Apache & Passenger installed with a default configuration • MySQL installed and running -
Deploying Your App • Pull - Use Chef • Push
- Use Capistrano -
Capistrano + Chef webservers = [] web_query = Chef::Search::Query.new web_query.search(:node,
'role:dcrug_web') do |h| webservers << h["fqdn"] end role :web, *webservers -
But wait, there’s more! • Encrypted Data Bags • Environments
• Lightweight Resources and Providers • Exception and Report Handlers -
Want More? • community.opscode.com • wiki.opscode.com • docs.opscode.com • #chef
on irc.freenode.net -
Food Fight Show • The podcast where DevOps Chefs Do
Battle • foodfightshow.org -
Get Involved Locally! • meetup.com/DevOpsDC • meetup.com/Arlington-Ruby • Feb 13
• zero-to-Rails Application -
Learn More • Use discount code ‘MEETUP’ to save 10%!
• opscode.eventbrite.com -
Find Me • @nathenharvey • http://nathenharvey.com • [email protected] • Thank
You! • What Questions Do You Have?