$30 off During Our Annual Pro Sale. View Details »

The Joy of Cooking: Deploying Drupal with Chef

The Joy of Cooking: Deploying Drupal with Chef

An introduction to Chef and information on deploying Drupal with Chef. This presentation was originally given at DrupalCon 2013.

Nathen Harvey

May 23, 2013
Tweet

More Decks by Nathen Harvey

Other Decks in Technology

Transcript

  1. None
  2. Nathen •Technical Community Manager at Opscode •Co-host of the Food

    Fight Show Podcast •Meetup Organizer
  3. Opscode

  4. Chef is an automation platform for developers & systems engineers

    to continuously define, build, and manage infrastructure. CHEF USES: Recipes and Cookbooks that describe Infrastructure as Code. Chef enables people to easily build & manage complex & dynamic applications at massive scale • New model for describing infrastructure that promotes reuse • Programmatically provision and configure • Reconstruct business from code repository, data backup, and bare metal resources “ ” Chef
  5. http://www.flickr.com/photos/steffenz/337700069/ http://www.flickr.com/photos/kky/704056791/ Applications

  6. Infrastructure http://www.flickr.com/photos/sbh/462754460/

  7. Collection of Resources http://www.flickr.com/photos/philliecasablanca/3354734116/ • Networking • Files • Directories

    • Symlinks • Mounts • Routes • Users • Groups • Tasks • Packages • Software • Services • Configuration • Other Stuff
  8. Acting in Concert http://www.flickr.com/photos/glowjangles/4081048126/

  9. To Provide a Service http://www.flickr.com/photos/28309157@N08/3743455858/

  10. And it Evolves http://www.flickr.com/photos/16339684@N00/2681435235/

  11. Application Server See Node

  12. Application Server Application Database See Nodes

  13. Application Server Application Databases See Nodes Grow

  14. Application Servers Application Databases See Nodes Grow

  15. Application Servers Application Databases Load Balancer See Nodes Grow

  16. Application Servers Application Databases Load Balancers See Nodes Grow

  17. Application Servers Application Database Cache Load Balancers Application Databases See

    Nodes Grow
  18. Application Servers Application Database Cache Load Balancers Application Databases Tied

    Together with Configuration
  19. Application Servers Application Database Cache Load Balancers Floating IP? Application

    Databases Infrastructure is a Snowflake
  20. Load Balancers Application Servers NoSQL Database Slaves ApplicationCache Database Cache

    Database Evolving Complexity
  21. Configuration Management http://www.flickr.com/photos/philliecasablanca/3354734116/

  22. New Requirements •A new developer joined the team, get her

    set-up with the application •Add New Relic monitoring to the application •Add a new module to the development site •Do not store the unencrypted admin password in the git repository
  23. Configuration Desperation http://www.flickr.com/photos/francoforeshock/5716969942/

  24. • But you already guessed that, didn’t you? Chef Solves

    This Problem
  25. http://www.flickr.com/photos/louisb/4555295187/ • Programmatically provision and configure • Treat like any

    other code base • Reconstruct business from code repository, data backup, and bare metal resources. Chef is Infrastructure as Code
  26. http://www.flickr.com/photos/ssoosay/5126146763/ • Chef generates configurations directly on nodes from their

    run list • Reduce management complexity through abstraction • Store the configuration of your programs in version control Programs
  27. •Define Policy •Say what, not how •Pull not Push http://www.flickr.com/photos/bixentro/2591838509/

    Declarative Interface to Resources
  28. That Looks Like This package "apache2" template "/etc/apache2/apache2.conf" do soucre

    "apache2.conf.erb" owner "root" group "root" mode "0644" variables(:allow_override => "All") notifies :reload, "service[apache2]" end service "apache2" do action [:enable,:start] supports :reload => true end
  29. • Recipes are collections of Resources • Cookbooks contain recipes,

    templates, files, custom resources, etc • Code re-use and modularity • Hundreds already on Community.opscode.com http://www.flickr.com/photos/shutterhacks/4474421855/ Recipes and Cookbooks
  30. http://www.flickr.com/photos/kathycsus/2686772625 • IP addresses • Hostnames • FQDNs • Search

    for nodes with Roles • Find configuration data Search
  31. pool_members = search("node","role:awesome_site) template "/etc/haproxy/haproxy.cfg" do source "haproxy-app_lb.cfg.erb" owner "root"

    group "root" mode "0644" variables :pool_members => pool_members.uniq notifies :restart, "service[haproxy]" end Pass Results To Templates
  32. # Set up application listeners here. listen application 0.0.0.0:80 balance

    roundrobin <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>: weight 1 maxconn 1 check <% end -%> <% if node["haproxy"]["enable_admin"] -%> listen admin 0.0.0.0:22002 mode http stats uri / <% end -%> Pass Results To Templates
  33. Drupal App Memcache MySQL Slaves MySQL Master Nagios Graphite So

    when this...
  34. Drupal App Memcache MySQL Slaves MySQL Master Nagios Graphite ...becomes

    this...
  35. Drupal App Memcache MySQL Slaves MySQL Master Nagios Graphite ...this

    can happen automatically
  36. Nagios Graphite Drupal App Memcache MySQL Slaves • Load balancer

    config • Nagios host ping • Nagios host ssh • Nagios host HTTP • Nagios host app health • Graphite CPU • Graphite Memory • Graphite Disk • Graphite SNMP • Memcache firewall • MySQL firewall • 11+ resource changes for 1 node addition Count the Resources
  37. Deploying Drupal with Chef

  38. Chef Components

  39. Our Workflow •Build Drupal locally •Deploy Drupal on EC2 •Iterate

  40. Local Development •Git repository •Virtual Machine •Chef •knife •Chef Server

  41. Git Repository $ mkdir -p ~/drupalcon/chef-repo $ cd ~/drupalcon/chef-repo $

    git init
  42. Building Drupal Locally •Deploy in a local virtual machine managed

    by Vagrant, provisioned with Chef
  43. Vagrantfile Vagrant.configure("2") do |config| config.vm.hostname = "drupalcon-dev" config.vm.box = "opscode-ubuntu-12.04"

    config.vm.network :private_network, ip: "33.33.33.10" config.vm.network :forwarded_port, guest: 80, host: 8080 end
  44. Chef

  45. Chef Server

  46. Register VM with Chef Vagrant.configure("2") do |config| ... config.vm.provision :chef_client

    do |chef| chef.chef_server_url = "https://api.opscode.com/organizations/nhdrupalcon" chef.validation_key_path = ".chef/nhdrupalcon-validator.pem" chef.validation_client_name = "nhdrupalcon-validator" end end $ vagrant provision
  47. Chef Components

  48. Cookbooks •Download from community.opscode.com •Write our own •Upload to the

    Chef Server
  49. Community Site •950+ Cookbooks •Codify tribal knowledge •Reference

  50. Write our own Cookbook $ knife cookbook create awesome_site **

    Creating cookbook awesome_site ** Creating README for cookbook: awesome_site ** Creating CHANGELOG for cookbook: awesome_site ** Creating metadata for cookbook: awesome_site
  51. Write our own Cookbook include_recipe "drupal" web_app "drupal" do template

    "drupal.conf.erb" docroot node['drupal']['dir'] server_name server_fqdn server_aliases node['fqdn'] end
  52. Data •Separate data from policy •Policy: Site has a document

    root •Data: /var/www/drupal •Policy: Apache has a Server Alias •Data: Fully-qualified domain name of the server web_app "drupal" do template "drupal.conf.erb" docroot node['drupal']['dir'] server_name server_fqdn server_aliases node['fqdn'] end
  53. Where does data come from? •Attributes •Data Bags •Encrypted Data

    Bags •Search
  54. Attributes •Specific details about a node •Defined by: •The state

    of the node •Cookbooks •Roles •Environments
  55. Ohai "languages": { "ruby": { }, "perl": { "version": "5.14.2",

    "archname": "x86_64-linux-gnu-thread-multi" }, "python": { "version": "2.7.3", "builddate": "Aug 1 2012, 05:14:39" }, "php": { "version": "5.3.10-1ubuntu3.6", "builddate": "(cli) (built: Mar" } }, "kernel": { "name": "Linux", "release": "3.2.0-32-virtual", "version": "#51-Ubuntu SMP Wed Sep 26 21:53:42 UTC 2012", "machine": "x86_64", "modules": { "isofs": { "size": "40257", "refcount": "0" }, "acpiphp": { "size": "24231", "refcount": "0" } }, "os": "GNU/Linux" }, "os": "linux", "os_version": "3.2.0-32-virtual", "ohai_time": 1369328621.3456137, "network": { "interfaces": { "lo": { "mtu": "16436", "flags": [ "LOOPBACK", "UP", "LOWER_UP" ], "encapsulation": "Loopback", "addresses": { "127.0.0.1": { "family": "inet", "prefixlen": "8", "netmask": "255.0.0.0", "scope": "Node" }, "::1": { "family": "inet6", "prefixlen": "128", "scope": "Node" } }, "state": "unknown" }, "eth0": { "type": "eth", "number": "0", "mtu": "1500",
  56. Attributes •Cookbooks, Roles, and Environments can also set attribute data

    •Extremely flexible mechanism for configuration •Cookbook: •Environment: default['drupal']['modules'] = ['views', 'webform'] default_attributes({ "drupal" => {"modules" => ["views","webform","token"]}})
  57. Data Bags •Global Data •Stored in JSON •Accessible from the

    Chef Server •Can be encrypted { "id": "nharvey", "groups": ["sysadmin"], "uid": 2001, "shell": "/bin/bash", "comment": "Nathen Harvey <nharvey@opscode.com>", "nagios": { "email": "you@example.com" } }
  58. Search pool_members = search("node","awesome_site) template "/etc/haproxy/haproxy.cfg" do source "haproxy-app_lb.cfg.erb" owner

    "root" group "root" mode "0644" variables :pool_members => pool_members.uniq notifies :restart, "service[haproxy]" end
  59. Upload our Cookbooks •We’re using •Cookbooks from the community site

    •A cookbook we wrote •Upload the cookbooks using knife $ knife cookbook upload -a
  60. Deploy Drupal

  61. Run List •Ordered list of roles or recipes to be

    run on a node •The run list represents the policy to be applied •chef-client •Executes on the node on a regular basis •Receives the run list from the Chef Server •Brings the node in-line with the policy
  62. Run List Server Server Server Server chef-server API chef-client “role[webserver]”

    node ntp client.rb openssh server.rb apache default.rb drupal default.rb chef-client “role[database]” node ntp client.rb openssh server.rb mysql server.rb
  63. Run chef-client config.vm.provision :chef_client do |chef| chef.chef_server_url = "https://api.opscode.com/organizations/nhdrupalcon" chef.validation_key_path

    = ".chef/nhdrupalcon-validator.pem" chef.validation_client_name = "nhdrupalcon-validator" chef.add_recipe "awesome_site" end $ vagrant provision
  64. Next Steps •Share the Vagrantfile with your teammates •Develop to

    a releasable state •Deploy to production
  65. Production Deploy •We’ll use Amazon’s EC2 for our initial deploy

    •...but we could deploy to any server •Using the same cookbooks as our development environment, deploy to EC2 $ knife ec2 server create -r "recipe[awesome_site]" -f m1.medium -I ami-641c8e0d -N ec2_drupal
  66. Current State •Local virtual machine for development running Drupal •“Production”

    instance on EC2
  67. Next Steps •Move Database to a different node •Add a

    Load Balancer and additional application servers •Add monitoring •Add log aggregation •etc.
  68. More about Chef

  69. Community

  70. Open Source •Apache 2 Software License •Continually growing number of

    contributors! •1400+ individuals, 200+ companies •Development repositories: •github.com/opscode •github.com/opscode-cookbooks •Regular code reviews via Google+ Hangout
  71. Get Started on Your Own •http://learnchef.com •http://docs.opscode.com •http://lists.opscode.com •#chef on

    Freenode IRC
  72. Wrap-up & Special Offer

  73. Infrastructure as Code •Repeatability •Automation •Agility •Scalability •Reassurance •Disaster Recovery

    •Happiness
  74. Thanks! •Promet Source •Will Milton •github.com/promet/drupal_cookbook •curl -L bit.ly/10McZSZ |

    bash; vagrant up •Marius Ducea •community.opscode.com/cookbooks/drupal
  75. Chef Training •Chef Introductory Workshop •Tomorrow in Corvallis •FREE to

    all DrupalCon Attendees •http://bit.ly/learnchef-drupalcon
  76. Thank You! •What Questions Do You Have? •@nathenharvey •Free Training

    Tomorrow (register by 9PM) •bit.ly/learnchef-drupalcon •Feedback on this session: portland2013.drupal.org/node/2963