Saving your Rails applications.

Hector Bustillos @hecbuma

RailsRumble 2015

contributron.r15 .railsrumble.co m/

… the Talk

A walk in the park… A walk in the park…

RoR in production should be a walk in the park.

so le WHY?

Community

Deploy

Issues time!

My Advices

Arch Diagram

Started GET "/health_check/all.json" for 127.0.0.1 at 2015-11-23 12:48:21 +0200

$ curl -vS 127.0.0.1:3000/health_check/all.json < HTTP/1.1 200 OK  < X-Frame-Options:
SAMEORIGIN  < X-XSS-Protection: 1; mode=block  < X-Content-Type-Options: nosniff  < X-UA-Compatible: chrome=1  < Content-Type: text/plain; charset=utf-8  < Cache-Control: no-store, must-revalidate, private, max-age=0  < Set-Cookie: visitor=cc640bf1-fefa-44f2-8970-dd107cccbd5b; path=/; expires=Thu, 23 Nov 2017 10:38:36 -0000  < Set-Cookie: visit=98f1b480-7e5b-4ae8-9a2d-0737cbe09d75; path=/; expires=Mon, 23 Nov 2015 11:08:36 -0000  < Set-Cookie: __profilin=p%3Dt; path=/  < Set-Cookie: __profilin=p%3Dt; path=/  < Set-Cookie: __profilin=p%3Dt; path=/  < X-Request-Id: 2cd1f1ad-a6e2-4b2c-aa3d-12c3528cf91a  < X-Runtime: 0.221707  < X-MiniProfiler-Ids: ["3if05lhmkns9848fxlti","oiuuqu4s66zq6dwubb5c","tjt197wj7fdpjgzy0 ep","ad7cmq3srah9gsh7sd3","duuis0gvv6nav9lfc11c"]  < Connection: close  < Server: thin  <  * Closing connection 0  {"healthy":true,"message":"success"}%

gem 'health_check'

def index  checks = params[:checks] || 'standard'  begin  errors =
HealthCheck::Utils.process_checks(checks)  rescue Exception => e  errors = e.message  end   if errors.blank?  obj = { :healthy => true, :message => HealthCheck.success }  respond_to do |format|  format.html { render :text => HealthCheck.success, :content_type => 'text/plain' }  format.json { render :xml => obj.to_json }  format.xml { render :xml => obj.to_xml }  format.any { render :text => HealthCheck.success, :content_type => 'text/plain' }  end  else  msg = "health_check failed: #{errors}"  obj = { :healthy => false, :message => msg }  respond_to do |format|  format.html { render :text => msg, :status => HealthCheck.http_status_for_error_text, :content_type => 'text/plain' }  format.json { render :xml => obj.to_json, :status => HealthCheck.http_status_for_error_object}  format.xml { render :xml => obj.to_xml, :status => HealthCheck.http_status_for_error_object }  format.any { render :text => msg, :status => HealthCheck.http_status_for_error_text, :content_type => 'text/plain' }  end  # Log a single line as some uptime checkers only record that it failed, not the text returned  if logger  silence_level, logger.level = logger.level, @old_logger_level if @old_logger_level  logger.info msg  logger.level = silence_level if @old_logger_level  end  end  end

Monitoring

Child process Child process Child process EC2 Instance

gem "eye", :github => "kostya/eye"

Eye.config do  logger "/data/spree/shared/log/eye.log"  end    Eye.application("spree") do |app|   
group "services" do    process :unicorn do  working_dir "/data/spree/current"  pid_file "/data/spree/shared/pids/unicorn.pid"  stdall "/data/spree/shared/log/eye-unicorn.log"  start_command "bundle exec unicorn_rails -c /data/spree/ shared/config/unicorn.rb -E production -D"  stop_command "kill -QUIT "  restart_command "kill -USR2 {PID}" # for rolling restarts    monitor_children do  restart_command 'kill -QUIT {PID}' # for this child process  check :memory, every: 10.seconds, below: 350.megabytes, :times => [3,5] # 350mb at least 3 out of the last 5 times checked  end    end  end  end

APP_ROOT = "/data/spree"    worker_processes 13  working_directory APP_ROOT + "/current" 
  timeout 180    listen APP_ROOT + "/shared/sockets/unicorn.sock", :backlog => 1024    pid APP_ROOT + "/shared/pids/unicorn.pid"    stderr_path APP_ROOT + "/shared/log/unicorn.stderr.log"    preload_app true    before_exec do |server|  ENV["BUNDLE_GEMFILE"] = APP_ROOT + "/current/Gemfile"  end    before_fork do |server, worker|  defined?(ActiveRecord::Base) and ActiveRecord::Base.connection.disconnect!    # kills old children after zero downtime deploy  old_pid = APP_ROOT + "/shared/pids/unicorn.pid.oldbin"  if File.exists?(old_pid) && server.pid != old_pid  begin  sig = (worker.nr + 1) >= server.worker_processes ? :QUIT : :TTOU  Process.kill(sig, File.read(old_pid).to_i)  rescue Errno::ENOENT, Errno::ESRCH  end  end    #sleep 1  end    after_fork do |server, worker|  defined?(ActiveRecord::Base) and ActiveRecord::Base.establish_connection    defined?(Rails) and Rails.cache.respond_to?(:reconnect) and Rails.cache.reconnect  end

Proﬁling

Error Notiﬁcations

on Memory Keep an eye

RoR footprint

$ ls -lh db/ total 238192  -rw-r--r-- 1 hecbuma staff
84M Mar 3 2015 GeoIP2-City.mmdb  -rw-r--r-- 1 hecbuma staff 32M Nov 23 12:25 GeoLite2-City.mmdb  -rw-r--r-- 1 hecbuma staff 90K Nov 23 12:25 schema.rb  -rw-r--r-- 1 hecbuma staff 65K Mar 4 2015 seed.sql  -rw-r--r-- 1 hecbuma staff 31K Nov 23 12:25 seeds.rb  -rw-r--r-- 1 hecbuma staff 193K Nov 23 12:25 structure.sql

9,254,006

CREATE TABLE `maxmind_ipv4_addresses` (  ìd` int(11) NOT NULL AUTO_INCREMENT,  `network`
varchar(255) DEFAULT NULL,  `geoname_id` int(11) DEFAULT NULL,  `registered_country_geoname_id` int(11) DEFAULT NULL,  `represented_country_geoname_id` int(11) DEFAULT NULL,  ìs_anonymous_proxy` tinyint(1) DEFAULT NULL,  ìs_satellite_provider` tinyint(1) DEFAULT NULL,  `postal_code` varchar(255) DEFAULT NULL,  `latitude` float DEFAULT NULL,  `longitude` float DEFAULT NULL,  `begin_ip_num` bigint(20) DEFAULT NULL,  ènd_ip_num` bigint(20) DEFAULT NULL,  PRIMARY KEY (ìd`),  KEY ìndex_maxmind_ipv4_addresses_on_end_ip_num` (ènd_ip_num`),  KEY ìndex_maxmind_ipv4_addresses_on_begin_ip_num` (`begin_ip_num`)  ) ENGINE=InnoDB AUTO_INCREMENT=9254006 DEFAULT CHARSET=latin1;  /*!40101 SET character_set_client = @saved_cs_client */;

mysql> LOAD DATA LOCAL INFILE 'file.csv' INTO TABLE t1  FIELDS
TERMINATED BY ',' LINES TERMINATED BY '\n'  (@col1,@col2,@col3,@col4) set name=@col4,id=@col2;

ip_address = IPAddr.new(address).to_i joins(:location).where("begin_ip_num <= #{ip_address}")  .order('begin_ip_num DESC')

Memory leaks

memory: ruby 2.64 GB

gem 'oink'   gem 'bullet'  gem 'rack-mini-profiler', require: false  gem
'ruby-prof'  gem 'stackprof'

def shop_index  @tracking_service.track("checkout_funnel", {user_id: spree_current_user,  funnel_step: 'shop_view',  previous_url: session[:previous_url]}) if
@tracking_service    @is_left_nav = 1  @bits = @bit_taxons    variantAssociations = [ :prices, :default_price, :option_values]    @productIncludes = [  { :master => variantAssociations },  { :classifications => :taxon },  {:variant_images => {:taggings => :tag} }  ]  #@kits = Spree::Product.kits_in_shop  @kits = Spree::Product.includes([ {:classifications => :taxon}, {:variant_images => {:taggings => :tag} } ]).where(:spree_taxons => {:name => "Kits"}, inactive: false).reject{|p| p.in_taxon?("Hidden") || p.in_taxon?('Retired')}  @collections = Spree::Product.active.is_active.collections_only  @accessories = Spree::Product.is_active.active.accessories_only  @cloud = Spree::Product.find_by_slug 'cloudbit'  # @expansion_packs = Spree::Product.is_active.active.expansion_only  @expansion_packs = Spree::Product.includes([ {:classifications => :taxon}, {:variant_images => {:taggings => :tag} } ]).where(:spree_taxons => {:name => "Expansion Packs"}, inactive: false)    @project_packs = Spree::Product.is_active.active.project_only  @bundles = Spree::Product.is_active.active.bundles_only  @product_taxons = Spree::Product.product_taxons_new    ...     end

def shop_index  @tracking_service.track("checkout_funnel", {user_id: spree_current_user,  funnel_step: 'shop_view',  previous_url: session[:previous_url]}) if
@tracking_service    @is_left_nav = 1  @bits = @bit_taxons    @productIncludes = [:taxons, :variants, :variant_images]    all_products = Spree::Product.select('spree_products.*, spree_taxons.name as taxon_name').includes(@productIncludes).  joins(:taxons).where( spree_taxons: { name: ['Kits',  'Expansion Packs',  'bundles',  'Project Packs', 'accessories'] }).is_active.active.group_by{|ele| ele.taxons.first.name }        @kits = all_products['Kits']  @accessories = all_products['Accessories']  @expansion_packs = all_products['Expansion Packs']  @project_packs = all_products['Project Packs']  @bundles = []  ...    end

gem 'bullet' 

/kits/base-kit  N+1 Query detected  Spree::ProductProperty => [:property]  Add to your
finder: :includes => [:property]  N+1 Query method call stack  /Users/hecbuma/Projects/little_bits/app/models/spree/product_decorator.rb: 287:in `block in get_property'  /Users/hecbuma/Projects/little_bits/app/models/spree/product_decorator.rb: 287:in `get_property'  /Users/hecbuma/Projects/little_bits/app/helpers/price_helper.rb:87:in `lb_price_nohtml'  /Users/hecbuma/Projects/little_bits/app/helpers/price_helper.rb:5:in `lb_price'  /Users/hecbuma/Projects/little_bits/app/views/spree/products/ _product_intro.html.erb:15:in `_5844de58232a26f585dafe25ceb554ca'  /Users/hecbuma/Projects/little_bits/app/views/spree/products/_kit.html.erb: 6:in `_3fb56cc986f9d742d01520ca5dd8ea58'  /Users/hecbuma/Projects/little_bits/app/views/spree/products/show.html.erb:2:in `block in _92b057a2f06e5acf89bbace9581f9c9b'  /Users/hecbuma/Projects/little_bits/app/helpers/little_bits_cache_helper.rb: 15:in `block in lb_cache'  /Users/hecbuma/Projects/little_bits/app/helpers/little_bits_cache_helper.rb: 14:in `lb_cache'  /Users/hecbuma/Projects/little_bits/app/views/spree/products/show.html.erb:1:in `_92b057a2f06e5acf89bbace9581f9c9b'  /Users/hecbuma/Projects/little_bits/app/controllers/ application_controller.rb:100:in `profile' 

An Extra

ActiveRecord::RecordNotFound - Couldn't find User with 'id'=64

Saving your Rails applications.

Saving your Rails applications.

More Decks by Hector Bustillos

Other Decks in Programming

Featured

Transcript