RailsConf 2012 - Stack Smashing (Cornflower Blue)

RailsConf 2012 - Stack Smashing (Cornflower Blue)

D1a58c46532900ba65fd439e64527ef4?s=128

David Czarnecki

April 25, 2012
Tweet

Transcript

  1. 14.

    SSO, Profile Service Community Pro Circuit, Live Experience Store Photo

    Tool, Carousel Tool, League Tool Entitlements, Redemption Starcraft Arena MLG.tv Progamer, Pro Stats
  2. 33.

    Gemfile group :assets do gem 'sass-rails', '~> 3.2.3' gem 'coffee-rails',

    '~> 3.2.1' gem 'compass', '= 0.12.alpha.4' gem 'uglifier', '~> 1.0.3' end gem ‘jquery-rails’
  3. 34.

    application.rb if defined?(Bundler) # If you precompile assets before deploying

    to production, use this line Bundler.require(*Rails.groups(:assets => %w(development test))) # If you want your assets lazily compiled in production, use this line # Bundler.require(:default, :assets, Rails.env) end ... # Enable the asset pipeline config.assets.enabled = true
  4. 35.

    production.rb # Compress assets and add digests config.assets.compress = true

    config.assets.js_compressor = Uglifier.new(:copyright => false) if defined?(Uglifier) config.assets.digest = true # Precompile additional assets (application.js, application.css, and all non-JS/CSS are already added) config.assets.precompile += %w( home.css home.js admin.css admin.js custom-application.js ) config.assets.precompile += [/plugins\/jquery\.ui\.selectmenu\.(css|js) $/, /plugins\/jquery\.gwfselect\.(css|js)$/]
  5. 36.

    terminal $ mkdir app/assets $ mv public/images/ app/assets/ $ mv

    public/javascripts/ app/assets/ $ mv public/stylesheets/ app/assets/
  6. 42.

    sv-rails-run.erb #!/bin/bash exec 2>&1 <% unicorn_command = @options[:unicorn_command] || 'unicorn_rails'

    -%> test -f /var/rails/.rvm/scripts/rvm || exit 1 exec /usr/bin/sudo -u rails -i <<END export HOME=/var/rails source /var/rails/.rvm/scripts/rvm || exit 1 cd <%= @options[:root] %> || exit 1 exec bundle exec <%= unicorn_command %> -c config/unicorn.rb - E <%= @options[:environment] %> END
  7. 43.

    unicorn.rb rails_env = ENV['RAILS_ENV'] || 'production' worker_processes (rails_env == 'production'

    ? 4 : 1) preload_app true # Restart any workers that haven't responded in 30 seconds timeout 30 # Listen on a Unix data socket case rails_env when 'production' || 'staging' listen "/var/rails/application/tmp/sockets/ #{rails_env}.sock", :backlog => 2048 else listen "#{`pwd`.strip}/tmp/sockets/#{rails_env}.sock" end
  8. 49.

    database.yml production: adapter: mysql2 host: machine-name reconnect: true pool: 5

    database: appname_production username: secret password: sup3rs3cr3t encoding: utf8
  9. 51.

    database.yml production: adapter: mysql2 host: machine-name reconnect: true pool: 5

    database: appname_production username: secret password: sup3rs3cr3t encoding: utf8
  10. 55.

    database.yml production: adapter: mysql2 host: mysql.yourcompany.int reconnect: true pool: 5

    database: appname_production username: secret password: sup3rs3cr3t encoding: utf8
  11. 64.

    Individual application ... 'application' => { :root => '/var/rails/application/current', :environment

    => 'production', :queues => {'application_queue' => 4, 'application_mailer' => 1, 'application_checkin_expiration' => 1}, :queue_intervals => {}, :resque_log => '/var/rails/application/shared/ log/resque.log' }, ...
  12. 65.

    node[:ruby][:sites].each_pair do |site, opts| runit_service site do owner 'rails' group

    'rails' template_name 'rails' options opts end if opts[:resque_scheduler] == `hostname`.strip runit_service "resque-scheduler-#{site}" do owner 'rails' group 'rails' template_name 'resque-scheduler' options opts end end if opts[:queues] opts[:queues].each do |queue, workers| options_for_template = opts.dup options_for_template[:queue] = queue 1.upto(workers) do |index| runit_service "resque-#{site}-#{queue}-worker-#{index}" do owner 'rails' group 'rails' template_name 'resque' options options_for_template end end end end end
  13. 66.

    sv-resque-run.erb #!/bin/bash exec 2>&1 test -f /var/rails/.rvm/scripts/rvm || exit 1

    exec /usr/bin/sudo -u rails -i <<END export HOME=/var/rails source /var/rails/.rvm/scripts/rvm || exit 1 cd <%= @options[:root] %> || exit 1 RAILS_ENV=<%= @options[:environment] %> QUEUES=<%= @options[:queue] %> INTERVAL=<%= @options[:queue_intervals] [@options[:queue]] || '5' %> exec bundle exec rake environment resque:work >><%= @options[:resque_log] %> 2>&1 END
  14. 69.
  15. 70.

    deploy.rb set :application, 'some-mlg-app' set :application_server, 'unicorn' require 'capistrano/agora/base' load

    'deploy' if respond_to?(:namespace) require 'capistrano/agora/airbrake' require 'capistrano/agora/assets' require 'capistrano/agora/rvm' require 'capistrano/agora/hipchat' set :hipchat_room_name, 'Some MLG Application' require 'capistrano/agora/logging' require 'capistrano/agora/resque' require 'capistrano/agora/symlinks' require 'capistrano/agora/sv' require 'capistrano/agora/unicorn' set :resque_queues, { 'some-mlg-app.retrieve_stuff' => 4, 'some-mlg-app.update_stuff' => 1, 'some-mlg-app.email_stuff' => 1 } set :asset_directory, 'public/players' before 'deploy:restart', 'deploy:assets:precompile_with_skip'
  16. 73.
  17. 76.

    base.rb Capistrano::Configuration.instance.load do default_run_options[:pty] = true ssh_options[:forward_agent] = true set

    :scm, :git set :deploy_via, :remote_cache set :keep_releases, 7 set :use_sudo, false set :branch, fetch(:branch, "master") unless exists?(:branch) set :gateway, "#{fetch(:user, `whoami`.strip)}@your.dmz.com" unless exists?(:gateway) set :repository, "git@github.com:agoragames/#{application}.git" unless exists?(:repository) set :deploy_to, "/var/rails/#{application}" unless exists?(:deploy_to) set :shared_nfs_dir, "/var/shared/rails/#{application}" end
  18. 77.

    assets.rb Capistrano::Configuration.instance.load do set :asset_directory, 'public/assets' set :assets_dependencies, %w(app/assets vendor/assets

    Gemfile.lock config/routes.rb) namespace :deploy do namespace :assets do task :precompile_with_skip, :roles => :web, :except => { :no_release => true } do from = source.next_revision(current_revision) if capture("cd #{latest_release} && #{source.local.log(previous_revision, current_revision)} #{assets_dependencies.join(' ')} | wc -l").to_i > 0 run "cd #{fetch(:current_path)} && bundle exec rake assets:precompile RAILS_ENV=#{rails_env}" else logger.info "Skipping asset pre-compilation because there were no asset changes. Copying assets from #{previous_release}." run "cp -R #{previous_release}/#{asset_directory} #{latest_release}/ #{asset_directory}" end end end end end
  19. 78.

    resque.rb Capistrano::Configuration.instance.load do namespace :resque do desc <<-DESC Restart the

    Resque workers for an application after a deploy or deploy:migrations DESC task :restart_workers, :roles => :app, :except => { :no_release => true } do if exists?(:resque_queues) fetch(:resque_queues).each do |queue_name, worker_count| 1.upto(worker_count) do |worker_index| run "sv restart resque-#{fetch(:application)}-#{queue_name}-worker- #{worker_index}" end end else logger.info('You must define the :resque_queues variable for the resque:restart_workers task to work') end end end after "deploy", "resque:restart_workers" after "deploy:migrations", "resque:restart_workers" end
  20. 79.

    unicorn.rb require 'capistrano/agora/helpers' Capistrano::Configuration.instance.load do namespace :unicorn do desc 'Increase

    number of unicorn workers' task :increase_workers, :roles => :app do num_workers = fetch(:num_workers, 1) unicorn_hosts = fetch(:unicorn_hosts, ['host1', 'host2']) unicorn_hosts.each do |host| worker_process_id = capture("cat /etc/sv/#{fetch(:application)}/supervise/pid", :hosts => host).chomp 1.upto(num_workers.to_i) do run("kill -TTIN #{worker_process_id}", :hosts => host) end end end desc 'Decrease number of unicorn workers' task :decrease_workers, :roles => :app do num_workers = fetch(:num_workers, 1) unicorn_hosts = fetch(:unicorn_hosts, ['host1', 'host2']) unicorn_hosts.each do |host| worker_process_id = capture("cat /etc/sv/#{fetch(:application)}/supervise/pid", :hosts => host).chomp 1.upto(num_workers.to_i) do run("kill -TTOU #{worker_process_id}", :hosts => host) end end end end end
  21. 87.
  22. 95.

    command_steps.rb def retry_times(times) begin yield rescue case times -= 1

    when 0 raise else retry end end end When /^I go to "(https?:\/\/[^"]*)"$/ do |uri| begin @host = uri.host retry_times(3) { http(uri) } rescue Errno::ECONNREFUSED @output = 'connection refused' @status = 127 rescue Timeout::Error @output = 'execution expired' @status = 127 end end
  23. 96.

    command_steps.rb Then /^the (?:output|response) should (?:include|contain) "([^"]*)"$/ do |string| string

    = eval("\"#{string}\"") assert @output.include?(string), "expected to find \"#{string}\" in the output from #{@host}, but did not" end Then /^the (?:output|response) should not (?:include|contain) "([^"]*)"$/ do |string| string = eval("\"#{string}\"") assert !@output.include?(string), "expected not to find \"#{string}\" in the output from #{@host}, but did" end Then /^the (?:output|response) should (?:include|contain) "([^"]*X[^"]*)", where X is less than (\d+)$/ do |string, value| string = eval("\"#{string}\"") regex = Regexp.new(string.sub('X', '(\d+)')) assert @output =~ regex x = $1.to_i assert x < value, "expected \"#{x}\" to be less than \"#{value}\" in the output from #{@host}" end
  24. 97.

    dns_steps.rb When /^I do a DNS lookup for ([\w\-\.]+)$/ do

    |name| @host = name begin @alias = Socket.gethostbyname(name).first rescue SocketError @alias = nil end end Then /^it should point (?:at|to) ([\w\-\.]+)$/ do |name| assert_equal name, @alias, "expected #{@host} to CNAME to #{@alias}, but it didn't" end
  25. 98.

    file_steps.rb Before do @stats = [] end When /^I stat

    (\S+)$/ do |glob| Dir[glob].each do |path| @stats << File.stat(path) end end Then /^there should be at least (\d+) files?$/ do |count| assert @stats.length >= count end Then /^the most recently modified file should be less than (\d+) (\w+)s? old$/ do |count, unit| assert @stats.collect { |stat| stat.mtime }.max > time_ago(count, unit) end
  26. 99.

    ping_steps.rb When /^I ping ([\w\-\.]+)$/ do |host| @host = host

    body = nil IO.popen(['/bin/ping', '-c', '1', '-n', host]) { |io| body = io.read } if $?.to_i == 0 body =~ /^rtt min\/avg\/max\/mdev = (\d\.\d{3}+)\/(\d\.\d{3}+)\/(\d\.\d{3}+)\/(\d\.\d{3}+) ms$/ @status = true @response = $2.to_f else @status = false @response = 0.0 end end Then /^I receive a response$/ do assert @status, "did not receive a response from #{@host}" end Then /^I receive a response within (\d+)(?: ?ms| milliseconds)$/ do |ms| assert @status, "did not receive a response from #{@host}" assert @response < ms, "response time from #{@host} of #{@response} was slower than #{ms} milliseconds" end
  27. 103.

    Feature: Applications @critical Scenario: Ensure the company site is available

    When I go to "http://www.yourcompany.com/" Then the response should either be a 200 or a 302 to "http://www.yourcompany.com/whateva"
  28. 105.

    Feature: MongoDB In order to support our environment And avoid

    lost data resulting from system failure or incompetence As a responsible system administrator I want to ensure that our MongoDB databases are running as expected And that we have good slaves of the masters (well, secondaries to our primaries...) And that our snapshotted backups are up to date @critical Scenario Outline: Ensure MongoD is running When I SSH to <host> and run `pgrep mongod` Then it should exit successfully Examples: | host | | mongo-primary.int | | mongo-secondary.int | | mongo-secondary.int |
  29. 107.

    Feature: Redis @critical Scenario Outline: Ensure a Redis host is

    up and running the expected version When I open a socket to <host>:<port> and send "INFO\r\n" Then the output should include "redis_version:<version>" Examples: | host | port | version | | redis.int | 6379 | 2.4.2 | | redis-staging.int | 6379 | 2.4.2 |
  30. 125.

    PostBuildScript #!/bin/bash if [ "$GIT_BRANCH" == "origin/HEAD" ] || [

    "$GIT_BRANCH" == 'master' ]; then curl "http://continuous-integration.com/job/project-deploy/build" else echo "$GIT_BRANCH, not deploying" fi
  31. 128.

    Build #!/bin/bash source "$HOME/.rvm/scripts/rvm" [[ -s ".rvmrc" ]] && source

    .rvmrc bundle install && bundle exec cap production deploy:migrations -S user=deploy -S branch=$GIT_BRANCH
  32. 130.
  33. 132.
  34. 133.

    DRY

  35. 134.
  36. 135.
  37. 136.