$30 off During Our Annual Pro Sale. View Details »

Getting Started with Chef for Rails

Getting Started with Chef for Rails

Presentation given at the Northern Virginia Ruby Users Group.

Overview of Chef
Building a project in Chef
Provision a Server for Your Rails App
Additional resources

Nathen Harvey

January 29, 2013
Tweet

More Decks by Nathen Harvey

Other Decks in Technology

Transcript

  1. The Joy of Cooking: Whip up a Rails Environment with

    Chef Nathen Harvey @nathenharvey Northern Virginia Ruby Users Group
  2. Nathen • Technical Community Manager, Opscode • Co-organizer • DevOpsDC

    • MongoDC • Formerly - Web Operations at CustomInk
  3. Agenda • Overview of Chef • Building a project in

    Chef • Provision a Server for Your Rails App • Additional resources • Obligatory Shameless Plugs • Please Ask Questions Along the Way
  4. Overview of Chef What is this thing again?

  5. Evolution of Configuration Management • Just build it • Keep

    notes in server.txt • Move notes to the wiki • Custom scripts (in scm?!) • Configuration Management framework
  6. When should I use a CM Framework? • After you

    outgrow Heroku
  7. Chef

  8. Applications http://www.flickr.com/photos/steffenz/337700069/ http://www.flickr.com/photos/kky/704056791/

  9. http://www.flickr.com/photos/sbh/462754460/ Infrastructure

  10. Collection of Resources • Nodes • Networking • Files •

    Directories • Symlinks • Mounts • Routes • Users • Groups • Tasks • Packages • Software • Services • Configurations • Stuff http://www.flickr.com/photos/stevekeys/3123167585/
  11. Acting in concert http://www.flickr.com/photos/glowjangles/4081048126/

  12. http://www.flickr.com/photos/28309157@N08/3743455858/ To provide a Service

  13. http://www.flickr.com/photos/16339684@N00/2681435235/ And it evolves

  14. Application See Node

  15. Application Application Database See Nodes

  16. Application App Databases See Nodes Grow

  17. App Servers App Databases See Nodes Grow

  18. App LB App Servers App Databases See Nodes Grow

  19. App LBs App Servers App Databases See Nodes Grow

  20. App LBs App Servers App DB Cache App DBs See

    Nodes Grow
  21. App LBs App Servers App DB Cache App DBs Stitched

    together with configs
  22. App LB App Servers App DB Cache App DBs Floating

    IP? Your Infrastructure is a snow flake
  23. App LBs App Servers NoSQL DB slaves Cache DB Cache

    DBs Complexity increases quickly
  24. Complexity increases very quickly DC1 DC3 DC2

  25. Configuration Management http://www.flickr.com/photos/philliecasablanca/3354734116/

  26. Rails App Memcache Postgres Slaves Postgres Master Typical Boring Infrastructure

    Nagios Graphite
  27. Golden Images are not the answer • Gold is heavy

    • Hard to transport • Hard to mold • Easy to lose configuration detail http://www.flickr.com/photos/garysoup/2977173063/
  28. Rails App Memcache Postgres Slaves Postgres Master New Compliance Mandate

    Nagios Graphite • Move SSH off port 22 • Lets put it on 2022
  29. Rails App Memcache Postgres Slaves Postgres Master 6 Golden Image

    Updates Nagios Graphite • edit /etc/ssh/sshd_config 1 2 3 4 5 6
  30. Rails App Memcache Postgres Slaves Postgres Master 12 Instance Replacements

    Nagios Graphite • Delete, launch 1 2 3 4 5 6 7 8 9 10 11 12 • Repeat • Typically manually
  31. Done in Maintenance Windows • Don’t break anything! • Bob

    just got fired =( 5 Rails App Memcache Postgres Slaves Postgres Master Nagios Graphite 1 2 4 5 6 7 8 9 10 11 12 3
  32. Rails App Memcache Postgres Slaves Postgres Master Different IP Addresses?

    Nagios Graphite • Invalid configs!
  33. http://www.flickr.com/photos/francoforeshock/5716969942/ Configuration Desperation

  34. Chef Solves This Problem • But you already guessed that,

    didn’t you?
  35. • Generate configurations directly on nodes • Reduce management complexity

    • Version control the programs http://www.flickr.com/photos/ssoosay/5126146763/ Programs!
  36. Declarative Interface to Resources • Define policy • Say what,

    not how • Pull not Push http://www.flickr.com/photos/bixentro/2591838509/
  37. Chef is Infrastructure as Code http://www.flickr.com/photos/louisb/4555295187/ • Programmatically provision and

    configure • Treat like any other code base • Reconstruct business from code repository, data backup, and bare metal resources.
  38. package "ntp" do action :install end service "ntpd" do action

    [:enable,:start] end template "/etc/ntpd.conf" do source "ntpd.conf.erb" owner "root" group "root" mode 0644 action :create variables(:time_server => “time.example.com”) notifies :restart, “service[ntpd]” end That looks like this
  39. Rails App Memcache Postgres Slaves Postgres Master So when this

    Nagios Graphite
  40. Rails App Memcache Postgres Slaves Postgres Master Nagios Graphite Becomes

    this
  41. Rails App Memcache Postgres Slaves Postgres Master Nagios Graphite This

    can happen automatically
  42. Nagios Graphite Count the resources Rails App Memcache Postgres Slaves

    • Load balancer config • Nagios host ping • Nagios host ssh • Nagios host HTTP • Nagios host app health • Graphite CPU • Graphite Memory • Graphite Disk • Graphite SNMP • Memcache firewall • Postgres firewall • Postgres authZ config • 12+ resource changes for 1 node addition
  43. Building a Project in Chef OK, so how do I

    build something with Chef?
  44. Building a Chef Project • First, come up with your

    policy / specification • Abstract the resources in your spec
  45. Resources package "tmux" do action :install end directory "/u/apps/awesome" do

    owner "apache" group "apache" action :create recursive true end
  46. Building a Chef Project • First, come up with your

    policy / specification • Abstract the resources in your spec • Write recipes
  47. Recipe include_recipe "app_user" app_name = node["app_name"] app_user = node["app_user"] app_group

    = node["app_group"] %w(releases shared).each do |dir| directory "/u/apps/#{app_name}/#{dir}" do mode "0755" owner app_user group app_group recursive true end end
  48. Building a Chef Project • First, come up with your

    policy / specification • Abstract the resources in your spec • Write recipes • Package recipes in cookbooks
  49. Cookbooks |-- ldirectord | |-- README.md | |-- attributes |

    | `-- default.rb | |-- metadata.rb | |-- recipes | | `-- default.rb | `-- templates | `-- default | `-- site.cf.erb
  50. Cookbooks |-- monit | |-- README.md | |-- attributes |

    | `-- default.rb | |-- files | | `-- ubuntu | | `-- monit.default | |-- metadata.rb | |-- recipes | | `-- default.rb | `-- templates | `-- default | `-- monitrc.erb
  51. Recipes and Cookbooks • Recipes are collections of Resources •

    Cookbooks contain recipes, templates, files, custom resources, etc • Code re-use and modularity http://www.flickr.com/photos/shutterhacks/4474421855/
  52. Building a Chef Project • First, come up with your

    policy / specification • Abstract the resources in your spec • Write recipes • Package recipes in cookbooks • Apply recipes to nodes
  53. Nodes • Representation of a host • runs the Chef

    client • has attributes • has a list of recipes to be applied
  54. Building a Chef Project • First, come up with your

    policy / specification • Abstract the resources in your spec • Write recipes • Package recipes in cookbooks • Apply recipes to nodes • Group things into roles
  55. Roles • mechanism for easily composing sets of functionality •

    have attributes and a list of recipes to be applied
  56. Run Lists Server Server Server Server chef-server API chef-client recipe[rails_app]

    node rails_app default.rb
  57. Run Lists Server Server Server Server chef-server API chef-client “role[base]”,

    “role[my_awesome_app]” node ntp default.rb sudo default.rb rails_app default.rb
  58. Server Server Server Server chef-server API chef-client “role[webserver]” node ntp

    client.rb openssh server.rb apache default.rb rails_app default.rb Roles chef-client “role[database]” node ntp client.rb openssh server.rb postgresql server.rb
  59. Roles name "base" description "Base of all nodes" default_attributes( "newrelic"

    => { "license_key" => "cbb1f5..." } ) run_list( "recipe[base_config]", "recipe[users]", "recipe[groups]", "recipe[sudo]" )
  60. Building a Chef Project • First, come up with your

    policy / specification • Abstract the resources in your spec • Write recipes • Package recipes in cookbooks • Apply recipes to nodes • Group things into roles
  61. Provision Rails with Chef Let’s do this!

  62. Landscape of Chef-managed Infrastructure

  63. Rails Application • Apache • Passenger • PostgreSQL • Rails

    Application
  64. © Opscode, 2011 – Confidential – DO NOT DISTRIBUTE Chef

    Provides a Model for Reuse That Works 64 700+ Cookbooks “Yesterday we started open sourcing some of our Opscode Chef work created at bestbuy.com; bit.ly/ yDV9Hl #Splunk #opschef #expectmor”
  65. Knife • Chef’s secret weapon • Command-line interface for interacting

    with • Chef Server API • Local Chef Repository • Community Site • and more...
  66. Download Community Cookbooks $ knife cookbook site download passenger_apache2 $

    knife cookbook site download mysql
  67. Create a Cookbook $ knife cookbook create rails_app ** Creating

    cookbook rails_app ** Creating README for cookbook: rails_app ** Creating CHANGELOG for cookbook: rails_app ** Creating metadata for cookbook: rails_app
  68. Write Our Rails App’s Recipes • Application Recipe • Database

    Recipe
  69. Application Recipe • Set-up some directories %w(releases shared shared/system shared/pids

    shared/logs shared/config).each do |dir| directory "#{deploy_to}/#{app_name}/#{dir}" do action :create owner app_user group app_group mode "0664" recursive true end end
  70. Application Recipe • Configure Apache / Passenger web_app app_name do

    docroot "#{deploy_to}/current/public" server_name "#{app_name}.#{node["domain"]}" server_aliases [ app_name, "localhost", node["hostname"] ] rails_env "production" end
  71. Attributes

  72. node[“domain”] • cookbook • default[“domain”] = “localhost” • environments •

    test •“domain” : “test.awesomeco.com” • production •“domain” : “awesomeco.com"
  73. node[“hostname”] • The node knows it’s own hostname • Not

    something we need to set in our code
  74. "memory": { "swap": { "cached": "0kB", "total": "4128760kB", "free": "4128760kB"

    }, "total": "2055676kB", "free": "1646524kB", "buffers": "35032kB", "cached": "210276kB", "active": "125336kB", "inactive": "142884kB", "dirty": "8kB", "writeback": "0kB", "anon_pages": "22976kB", "mapped": "8416kB", "slab": "121512kB", "slab_reclaimable": "41148kB", "slab_unreclaim": "80364kB", "page_tables": "1784kB", "nfs_unstable": "0kB", "bounce": "0kB", "commit_limit": "5156596kB", "committed_as": "74980kB", "vmalloc_total": "34359738367kB", "vmalloc_used": "274512kB", "vmalloc_chunk": "34359449936kB" }, Ohai! "block_device": { "ram0": { "size": "32768", "removable": "0" }, "ram1": { "size": "32768", "removable": "0" }, "ram2": { "size": "32768", "removable": "0" }, "hostname": "server-1", "fqdn": "server-1.example.com", "domain": "example.com", "network": { "interfaces": { "eth0": { "type": "eth", "number": "0", "encapsulation": "Ethernet", "addresses": { "00:0C:29:43:26:C5": { "family": "lladdr" }, "192.168.177.138": { "family": "inet", "broadcast": "192.168.177.255", "netmask": "255.255.255.0" }, "fe80::20c:29ff:fe43:26c5": { "family": "inet6", "prefixlen": "64", "scope": "Link" } },
  75. Attributes

  76. Database Recipe mysql_connection_info = { :host => "localhost", :username =>

    'root', :password => node['mysql']['server_root_password'] } mysql_database app_name do connection mysql_connection_info action :create end
  77. Database Recipe mysql_database_user node["database"]["user"] do connection mysql_connection_info password node["database"]["pw"] database_name

    node["database"]["name"] host "%" action :grant end
  78. Upload Cookbooks to Chef Server

  79. Create Roles • mechanism for easily composing sets of functionality

    • have attributes and a list of recipes to be applied • Define as Ruby or JSON • base-ubuntu • rails-web • rails-database
  80. Role as Ruby name "base-ubuntu" description "all Ubuntu servers" run_list(

    "recipe[apt]", "recipe[build-essential]" )
  81. Role as JSON { "name": "rails-web", "description": "Web Server for

    our Rails App", "default_attributes": { }, "run_list": [ "recipe[rails_app::web]" ] }
  82. Run Chef Client Server Server Server Server chef-server API chef-client

    “role[base-ubuntu]”, “role[rails-web]” node apt default.rb build-essential default.rb rails-app web.rb
  83. Running the Chef Client • Automatically • cron • daemon

  84. knife ssh $ knife ssh "roles:rails-web" "sudo chef- client"

  85. knife ssh $ knife ssh "roles:rails-web" "sudo chef-client"

  86. Review • Server provisioned and communicating with Chef Server API

    • Apache & Passenger installed with a default configuration • MySQL installed and running
  87. Deploying Your App • Pull - Use Chef • Push

    - Use Capistrano
  88. Capistrano role :web, "web01","web02","web03"

  89. Capistrano + Chef webservers = [] web_query = Chef::Search::Query.new web_query.search(:node,

    'role:dcrug_web') do |h| webservers << h["fqdn"] end role :web, *webservers
  90. But wait, there’s more! • Encrypted Data Bags • Environments

    • Lightweight Resources and Providers • Exception and Report Handlers
  91. Additional Resources Where to go next

  92. Want More? • community.opscode.com • wiki.opscode.com • docs.opscode.com • #chef

    on irc.freenode.net
  93. Food Fight Show • The podcast where DevOps Chefs Do

    Battle • foodfightshow.org
  94. Get Involved Locally! • meetup.com/DevOpsDC • meetup.com/Arlington-Ruby • Feb 13

    • zero-to-Rails Application
  95. Learning Chef Series

  96. Learn More • Use discount code ‘MEETUP’ to save 10%!

    • opscode.eventbrite.com
  97. #ChefConf 2013

  98. Find Me • @nathenharvey • http://nathenharvey.com • nharvey@opscode.com • Thank

    You! • What Questions Do You Have?