Rails Application Visibility

Transcript

1. Operational Visibility for Rails Applications Zach Taylor
2. Today’s sponsor:
3. Rails logging basics # config/application.rb 
 config.logger = Logger.new(STDOUT) config.log_level = :warn logger.debug "Person attributes hash: #{@person.attributes.inspect}" logger.info "Processing the request..." logger.fatal "Terminating application, raised unrecoverable error!!!"
4. Rails logging basics Started GET "/posts/new" for 127.0.0.1 at 2019-01-13 15:45:17 -0600 (0.4ms) SELECT "schema_migrations"."version" FROM "schema_migrations" ORDER BY "schema_migrations"."version" ASC ↳ /Users/zach/.rvm/gems/ruby-2.6.0@monitoring/gems/activerecord-5.2.2/lib/active_record/log_subscriber.rb:98 Processing by PostsController#new as HTML Rendering posts/new.html.erb within layouts/application Rendered posts/_form.html.erb (25.9ms) Rendered posts/new.html.erb within layouts/application (28.0ms) Completed 200 OK in 290ms (Views: 266.6ms | ActiveRecord: 0.4ms)
5. Rails logging basics tail -f log/production.log
6. Rails logging upgrade # config/application.rb config.log_tags = [:request_id]
7. Rails logging upgrade # config/application.rb config.log_tags = [:request_id] [14560c14-f724-42d7-9fb9-6722d79c71d4] Started GET "/posts/new" for 127.0.0.1 at 2019-01-13 15:48:46 -0600 [14560c14-f724-42d7-9fb9-6722d79c71d4] (0.4ms) SELECT "schema_migrations"."version" FROM "schema_migrations" ORDER BY "schema_migrations"."version" ASC [14560c14-f724-42d7-9fb9-6722d79c71d4] ↳ /Users/zach/.rvm/gems/ruby-2.6.0@monitoring/gems/activerecord-5.2.2/lib/ active_record/log_subscriber.rb:98 [14560c14-f724-42d7-9fb9-6722d79c71d4] Processing by PostsController#new as HTML [14560c14-f724-42d7-9fb9-6722d79c71d4] Rendering posts/new.html.erb within layouts/application [14560c14-f724-42d7-9fb9-6722d79c71d4] Rendered posts/_form.html.erb (26.3ms) [14560c14-f724-42d7-9fb9-6722d79c71d4] Rendered posts/new.html.erb within layouts/application (29.1ms) [14560c14-f724-42d7-9fb9-6722d79c71d4] Completed 200 OK in 264ms (Views: 253.5ms | ActiveRecord: 0.4ms)
8. Rails logging upgrade # config/application.rb config.log_tags = [:request_id, :domain] [a7556d4f-b447-4570-a9a8-3886ab312ab5] [localhost] Started GET "/posts/new" for 127.0.0.1 at 2019-01-13 16:00:05 -0600 [a7556d4f-b447-4570-a9a8-3886ab312ab5] [localhost] (0.3ms) SELECT "schema_migrations"."version" FROM "schema_migrations" ORDER BY "schema_migrations"."version" ASC [a7556d4f-b447-4570-a9a8-3886ab312ab5] [localhost] ↳ /Users/zach/.rvm/gems/ruby-2.6.0@monitoring/gems/ activerecord-5.2.2/lib/active_record/log_subscriber.rb:98 [a7556d4f-b447-4570-a9a8-3886ab312ab5] [localhost] Processing by PostsController#new as HTML [a7556d4f-b447-4570-a9a8-3886ab312ab5] [localhost] Rendering posts/new.html.erb within layouts/application [a7556d4f-b447-4570-a9a8-3886ab312ab5] [localhost] Rendered posts/_form.html.erb (17.9ms) [a7556d4f-b447-4570-a9a8-3886ab312ab5] [localhost] Rendered posts/new.html.erb within layouts/application (20.1ms) [a7556d4f-b447-4570-a9a8-3886ab312ab5] [localhost] Completed 200 OK in 274ms (Views: 260.7ms | ActiveRecord: 0.6ms)
9. Rails logging upgrade # config/application.rb config.log_tags = [:request_id, :domain, :remote_ip, :headers, :fullpath, :request_method, :query_ parameters] [f7546385-492f-4c4a-90e7-f72ea9e7a2aa] [localhost] [127.0.0.1] [#<ActionDispatch::Http::Headers:0x0007f9ccfb18d60>] [/posts/new?meow=yes] [GET] [{“meow”=>"yes"}]
10. Rails logging upgrade # app/controllers/posts_controller.rb logger.tagged("TEST") { logger.info “Something happened" } [107113c0-da89-4b75-9b7d-60649e4de5f9] [TEST] Something happened
11. Rails log aggregation gem "lograge" # config/application.rb config.lograge.enabled = true method=GET path=/posts/new format=html controller=PostsController action=new status=200 duration=253.35 view=240.05 db=0.58 [046b826e-61b4-4973-89f1-59247099824a] method=GET path=/posts/new format=html controller=PostsController action=new status=200 duration=222.03 view=210.56 db=0.61
12. Rails log aggregation # config/application.rb config.lograge.enabled = true config.lograge.formatter = Lograge::Formatters::Json.new config.lograge.custom_options = lambda do |event| { source: ["my_ruby_app"], params: event.payload[:params] } end {“method":"GET","path":"/posts/ new","format":"html","controller":"PostsController","action":"new","status": 200,"duration":250.86,"view":237.2,"db":0.9,"source":["my_ruby_app"],"params": {"meow":"yes","controller":"posts","action":"new"}}
13. Rails log aggregation
14. Log aggregation tools and services • Datadog https://www.datadoghq.com • Papertrail https://papertrailapp.com • Loggly https://www.loggly.com • LogDNA https://logdna.com • Splunk https://www.splunk.com • Sumologic https://www.sumologic.com • Timber https://timber.io • Greylog • Elasticsearch • AWS/Google Cloud
15. Exception tracking • Rack middleware
16. Exception tracking def index begin foo.bar rescue => e Rails.logger.error { "#{e.message} #{e.backtrace.join("\n")}" } Rollbar.report_exception(e) nil end end
17. Exception tracking services • Sentry https://sentry.io • Airbrake https://airbrake.io • Rollbar https://rollbar.com • Raygun https://raygun.com • Bugsnag https://www.bugsnag.com • Honeybadger https://www.honeybadger.io
18. ActiveSupport::Notifications ActiveSupport::Notifications.instrument "my.custom.event", this: :data do # do your custom stuff here end ActiveSupport::Notifications.subscribe "my.custom.event" do |name, started, finished, unique_id, data| puts data.inspect # {:this=>:data} end ActiveSupport::Notifications.subscribe "process_action.action_controller" do |*args| event = ActiveSupport::Notifications::Event.new *args event.name # => "process_action.action_controller" event.duration # => 10 (in milliseconds) event.payload # => {:extra=>information} Rails.logger.info "#{event} Received!" end https://guides.rubyonrails.org/active_support_instrumentation.html
19. ActiveSupport::Notifications https://guides.rubyonrails.org/active_support_instrumentation.html
20. Custom Metrics
21. Application Performance Monitoring • Could take all these ActiveSupport notifications and log them, send them to a system running StatsD, Prometheus, InfluxDB, etc. • Services that do this for you, and handle the information
22. Application Performance Monitoring • New Relic https://newrelic.com • Datadog https://www.datadoghq.com • AppSignal https://appsignal.com • Scout https://scoutapp.com • Skylight https://www.skylight.io • Librato https://www.librato.com • AppDynamics https://www.appdynamics.com • AppOptics https://www.appoptics.com • AWS/GCP
23. External monitoring • Processor usage • Number of threads • Number of processes • Memory usage • Disk usage • Network traffic • Request response time • Requests per second (nginx/apache) • Request statuses (2xx/4xx/5xx) • Supporting services (postgres/mysql/ redis) • Page load/render time
24. External monitoring (advanced/uncommon) • Queue size (sidekiq/mq/etc) • Bucket usage (s3/gcs) • Memory page faults • Process context switches
25. https://medium.com/@copyconstruct/logs-and-metrics-6d34d3026e38