Servers So Easy A Caveman Can Do It

OLF 2012 Talk

Transcript

1. Servers So Easy A Caveman Can Do It Christopher H. Laco » claco@chrislaco.com » @claco 1 1 Wednesday, May 1, 13
2. 2 Legal Disclaimer Ohio Revised Code 314159 Section (c), Subsection (a), Paragraph (t) states in part: “...any person or persons presenting computer related content to a group [audience] is required to show at a minimum of at least one (1) domestic feline [cat] picture during previously stated presentation.” 2 Wednesday, May 1, 13
3. 3 Playing Cat Banjo 3 Wednesday, May 1, 13
4. Shout out to our sponsors 4 opscode.com vagrantup.com ohiolinux.org 4 Wednesday, May 1, 13
5. Your humble speaker Reformed Music Major Turned Nerd H.A.H.S. Hosting At Home Syndrome S.I.B.D. Servers In Basement Disease Full Stack Nerd Hardware/Software/Network/”DevOps” 5 5 Wednesday, May 1, 13
6. “Back in my day...” 6 6 Wednesday, May 1, 13
7. “Back in my day...” 7 7 Wednesday, May 1, 13
8. “Back in my day...” 8 8 Wednesday, May 1, 13
9. “Back in my day...” 9 9 Wednesday, May 1, 13
10. “Back in my day...” 10 10 Wednesday, May 1, 13
11. “Back in my day...” 11 “And we liked it...” “WE LOVED IT!” 11 Wednesday, May 1, 13
12. Aggravations Your First Day. Computer? Configured? Configuration required the “server guy” MBP setup is different than servers Testing locally is different than testing upstream “Works on my machine” is dangerous Troubleshooting production is risky/difficult Upgrades introduce risk 12 12 Wednesday, May 1, 13
13. Aggravations continued... Changing deployment logic is troublesome Disaster recovery is costly/long Adding capacity takes time Nothing is repeatable Managing multiple servers is tedious 13 13 Wednesday, May 1, 13
14. Goals Get app instance on Day #1 Any Engineer/QA can spin up machines Same configuration everywhere Test outside of the MBP bubble Reproduce production problems out of band Test OS/Software upgrades easily Tune production deployment without production 14 14 Wednesday, May 1, 13
15. Goals continued... Recover from server failures quickly Add more servers when traffic increases Make setup repeatable Manage servers in bulk 15 15 Wednesday, May 1, 13
16. Step 3: Deploy Step 2: Configure Step 1: Provision 16 Three Steps To A New Server 16 Wednesday, May 1, 13
17. Fast Moving Train! 17 http://chrislaco.com/devops-toolbox/ 17 Wednesday, May 1, 13
18. 18 Provisioning a New Server 18 Wednesday, May 1, 13
19. Where Do We Put The Server? 19 Amazon AWS / EC2 Rackspace RackCloud SoftLayer, SliceHost, Linode, TerraHost, OpenStack, Eucalyptus Local VirtualBox Install Existing Servers Managed Host / Service Provider 19 Wednesday, May 1, 13
20. What Needs Provisioned? 20 Operating System + root access Install just enough to run configure / deploy steps later Install Ruby / Ohai / Chef / RubyShadow / Bundler Remove Future Roadblocks in Configuration / Deployment SSH Config: Disable Require TTY, Env Keep PATH Disable SeLinux (KickStart Bug! / Chef Recipes / Apache) Configure $PATH: environment, bashrc, profile, etc LD PATH: ldconfig (bundler deployment cache issues) 20 Wednesday, May 1, 13
21. How Do We Do It? 21 VirtualBox 4.2.1 veewee - Creates “Boxes” - https://github.com/jedi4ever/veewee vagrant - Manages Boxes / Instances - http://vagrantup.com/ EC2 / RackCloud - https://github.com/opscode/ knife / knife-ec2 - Manages AWS Instances knife / knife-rackspace - Manages RackCloud Instances Managed / Existing Servers ssh / sudo / su 21 Wednesday, May 1, 13
22. VirtualBox Provisioning 22 Install VirtualBox 4.2.1 Install Ruby Gems $ gem install veewee vagrant Define/Customize a new machine image $ vagrant basebox define MyServer CentOS-5.6-x86_64-netboot Edit KickStart Config - ks.cfg Edit Post Install Script - postinstall.sh 22 Wednesday, May 1, 13
23. 23 23 Wednesday, May 1, 13
24. 24 24 Wednesday, May 1, 13
25. 25 25 Wednesday, May 1, 13
26. 26 26 Wednesday, May 1, 13
27. VirtualBox Provisioning cont... 27 Install VirtualBox 4.2.1 4.1.22* (Guest Additions!) Install Ruby Gems $ gem install veewee vagrant Define/Customize a new machine image $ vagrant basebox define MyServer CentOS-5.6-x86_64-netboot Build the machine image $ vagrant basebox build MyServer $ vagrant basebox export MyServer 27 Wednesday, May 1, 13
28. 28 28 Wednesday, May 1, 13
29. 29 29 Wednesday, May 1, 13
30. 30 30 Wednesday, May 1, 13
31. EC2/RackCloud Provisioning 31 Install Ruby Gems $ gem install chef knife-ec2 knife-rackspace Configure API Keys in ~/.chef/knife.rb knife[:aws_access_key_id] = "Your AWS Access Key ID" knife[:aws_secret_access_key] = "Your AWS Access Key" Customize the server image $ vim ~/.chef/bootstrap/centos56.rb 31 Wednesday, May 1, 13
32. 32 32 Wednesday, May 1, 13
33. EC2/RC Provisioning cont... 33 Install Ruby Gems $ gem install chef knife-ec2 knife-rackspace Configure API Keys in ~/.chef/knife.rb knife[:aws_access_key_id] = "Your AWS Access Key ID" knife[:aws_secret_access_key] = "Your AWS Access Key" Customize the server image $ vim ~/.chef/bootstrap/centos56.rb Create the machine image $ knife ec2 create -I ami-0a59bb63 -d centos56 ... 33 Wednesday, May 1, 13
34. Login To Your New Server 34 VirtualBox / Vagrant $ vagrant ssh [ssh vagrant@localhost -p 2222] EC2 $ ssh root@ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com -i ec2-group-key.pem Rackspace $ ssh root@xxx-xxx-xxx-xxx.staticip.rackspace.com Questions? 34 Wednesday, May 1, 13
35. 35 Configuring Your New Server 35 Wednesday, May 1, 13
36. What Is Chef? Configuration management for “Nodes” or servers It is a “Cookbook” full of configuration “Recipes” plus “Data Bags” Install “build” user. Set password. Configure ssh key. Configure github access. Cookbooks, Recipes, Data Bags stored upstream on OpsCode server Client downloads recipes and runs them on each server Configure things differently by “Environment”: development, staging, production Configure “Roles” or groups of recipes: app, db, caching, services, etc Manage Cookbooks, Recipes, Roles and Nodes from command line 36 36 Wednesday, May 1, 13
37. What Does A Data Bag Do? 37 { "id": "build", "uid": 1000, "gid": 1000, "comment": "Build User", "shell": "/bin/bash", "password": "$1$31Pf4SgRy$edFhgUyhUBDE3%eUSD4rmk1", "ssh_keys": "ssh-rsa AAAABC2TbS43DAAABD4ER3DH4WT....default", "sudoers": "ALL=(ALL) ALL" } 37 Wednesday, May 1, 13
38. What Does A Recipe Do? 38 home_dir = "/home/#{u['id']}" group u['id'] do gid u['gid'] end user u['id'] do uid u['uid'] gid u['gid'] shell u['shell'] password u['password'] home home_dir end template "#{home_dir}/.ssh/authorized_keys" do source "authorized_keys.erb" owner u['id'] group u['gid'] || u['id'] mode "0600" variables :ssh_keys => u['ssh_keys'] end 38 Wednesday, May 1, 13
39. What Does A Role Do? 39 name "app" description "App role for all web servers." run_list "role[base]", # <-- Creates users from databag/recipe "recipe[apache2]", "recipe[apache2::mod_ssl]", "recipe[mysql::client]", "recipe[passenger_apache2]", "recipe[passenger_apache2::mod_rails]", "recipe[sphinx]", "recipe[company::ssl]", "recipe[company::mainsite]", "recipe[company::mobilesite]" 39 Wednesday, May 1, 13
40. What Does An Environment Do? 40 name "development" default_attributes "company" => { "mainsite" => { "virtual_host" => "localhost" } } name "staging" default_attributes "company" => { "mainsite" => { "virtual_host" => "staging.company.com" } } name "production" default_attributes "company" => { "mainsite" => { "virtual_host" => "www.company.com" } } 40 Wednesday, May 1, 13
41. Manage Everything Via Terminal 41 $ knife node list app1, services2, staging-db, vagrant-userx-mainsite, .... $ knife cookbook list apache, xml, xslt, imagemagic, company::users, mysql, .... $ knife search node "chef_environment:production AND role:services" Node Name: services1 FQDN: services1.company.com IP: 172.16.2.3 $ knife ssh “name:app*” “sudo chef-client -r ‘recipe[users]’” app1.company.com Running Chef client...... app2.company.com Running Chef client...... 41 Wednesday, May 1, 13
42. Upload Everything to OpsCode 42 $ knife environment from file ./environments.rb $ knife data bad from file Users ./build.json $ knife role from file ./base.rb ./web.rb $ knife cookbook upload --all 42 Wednesday, May 1, 13
43. Running Chef 43 VirtualBox / Vagrant chef-client automatically runs after vagrant up vagrant provision to manually reconfigure the server EC2 / Rackspace chef-client automatically runs after knife create bootstrap sudo chef-client to manually reconfigure the server Managed Servers sudo chef-client to manually configure the servers Automate from afar! knife ssh “name:mynode” “sudo chef-client” 43 Wednesday, May 1, 13
44. What About My Mac! 44 Install Using Homebrew! - https://github.com/mathie/chef-homebrew Install Using Dmg! - https://github.com/opscode/cookbooks/tree/master/dmg dmg_package "Google Chrome" do dmg_name "googlechrome" source "https://dl-ssl.google.com/.../GGRM/googlechrome.dmg" checksum "7daa2dc5c46d9bfb14f1d7ff4b33884325e5e63e6..." action :install end dmg_package "Virtualbox" do source "http://dlc.sun.com.edge...VirtualBox...-OSX.dmg" type "mpkg" end 44 Wednesday, May 1, 13
45. 45 Deploy Your Application 45 Wednesday, May 1, 13
46. Configure Capistrano 46 Create a multistage environment configuration (multistage plugin or inline tasks) $ gem install capistrano $ cd MyApp $ Capify . $ vim config/deploy.rb 46 Wednesday, May 1, 13
47. Override Defaults Per Destination 47 role :web, "localhost" role :db, "localhost", :primary => true task :vagrant do set :port, 2222 end task :ec2 do # same as :rackspace role :web, ENV['address'] role :db, ENV['address'], :primary => true ssh_options[:keys] = “~/.ssh/your-default-key-pair.pem” end task :production do role :web, "app1.company.com", "app2.company.com” role :db, "app1.company.com", :primary => true ssh_options[:keys] = “~/.ssh/your-default-key-pair.pem” end 47 Wednesday, May 1, 13
48. Deploy Your Application 48 $ cap <environment> <action> branch=value address=value rails_env=environment $ cap ec2 deploy:initial address=xxx.xxx.xxx.xxx branch=mybranch $ cap rackspace deploy:initial address=xxx.xxx.xxx.xxx branch=mybranch $ cap vagrant deploy:initial [address=localhost] branch=mybranch $ cap production deploy:update [branch=master] $ cap deploy:update branch=mybranch (uses defaults) 48 Wednesday, May 1, 13
49. Capistrano Deploy Tasks 49 deploy:initial deploy:setup, deploy:update, db:setup, sphinx:reindex, starling:restart, workling:restart, deploy:restart Run on fresh instances: vagrant, ec2, rackspace. db:setup is disabled for production environment. deploy:web:enable / deploy:web:disable Also does cluster:put / cluster:pull deploy:tests:environment, deploy:test:connections. Anything! 49 Wednesday, May 1, 13
50. Setup Project Contents 50 .chef - Preconfigured to talk to OpsCode, EC2, Rackspace! chef - Company Cookbooks, Recipes, Roles, Environments, DataBags vagrant/definitions - Preconfigured CentOS 5.6 x64 Machine! vagrant/instances/mainsite - Preconfigured Vagrant Site Instance! cd vagrant/instances/mainsite; vagrant up; cap vagrant deploy:initial 50 Wednesday, May 1, 13
51. Gitify Your Work. Clone And Go 51 $ git clone git@github.com:Company/company_setup.git $ cd company_setup $ bundle install $ rake ec2/rackspace instance:create $ cd vagrant/instances/mainsite && vagrant up $ cd company_mainsite $ cap ec2/rackspace deploy:initial address=xxx.xxx.xxx.xxx branch=gerbilsauce 51 Wednesday, May 1, 13
52. 52 Meeting Our Goals 52 Wednesday, May 1, 13
53. Goals Revisited New Guy/Gal gets instance on Day #1 Any Engineer/QA can spin up machines Same configuration everywhere Add more servers when traffic increases Test OS/Software upgrades easily Tune production deployment without production Reproduce production problems out of band Recover from server failures quickly 53 53 Wednesday, May 1, 13
54. Goals Revisited continued... Test outside of the MBP bubble Make setup repeatable Manage servers in bulk 54 54 Wednesday, May 1, 13
55. This Is “Easy”? 55 Sorry. I lied. Servers are difficult. This is an infrastructure investment. Make knowledge transfer possible. Make the difficult possible so you can work on something else. 55 Wednesday, May 1, 13
56. I Forgot The Last Step... 56 56 Wednesday, May 1, 13
57. Step 4: Drink Beers! 57 57 Wednesday, May 1, 13
58. http://startupbus.com/ 58 58 Wednesday, May 1, 13
59. Thanks for participating! 59 Slides: http://speakerdeck.com/u/claco Email: claco@chrislaco.com Twitter: @claco 59 Wednesday, May 1, 13