Fullstack monitoring Monitor any layer of your Ruby application Dmitry Salahutdinov Summer Saint P Rubyconf 2021 

Save people time for something that matters! Is a leading e-grocery company in Russia 

Me on GitHub 

Why you do need Ruby-app monitoring? • To know how application performs • Ensure application works as expected (SRE) • What limit of capacity is has • To prevent degradations • Investigate degradations (when happen) • To evaluate results of optimisations 

Yabeda framework Tooling extended extendable framework! 

Yabeda DSL 

Yabeda adapters 

Yabeda adapters Developer friendly: “get up and running” or “run example & explore” 

Common Ruby-application Is complicated and has multi-layer stack 

1. Puma 2. Rails 3. Rack / Rack-attack 4. Sidekiq 5. Kafka 6. External Services/HTTP requests 7. Non-standard metrics usage (memory profiling) We will cover 

Puma monitoring With yabeda-puma-plugin ruby gem 

Metrics 

Capacity 

Capacity gap 

Lack of capacity alert 

Application Busy metric 

No content

Rails monitoring With yabeda-rails ruby gem 

Yabeda::Rails dashbord 

Slowest queries 

SLO: serve 95% of requests within 250ms 

APDEX score 

APDEX (by endpoint): Yabeda version 

Error rate 

Rack monitoring 

Rack custom logger 

Rack::Attack monitoring With yabeda-rack-attack ruby gem 

Rack::Attack monitoring 

Throttled requests 

- rule can works incorrect - periodically call outrange? - bad requests produces workload Why to monitor abusive requests? 

Tracked requests Research the number of specific requests 

Sidekiq monitoring With yabeda-sidekiq ruby gem 

Sidekiq dashbord 

Latency - critical jobs SLO - any other jobs default SLO max(sidekiq_queue_latency{queue=”sms”}) by (queue) > 300 

Job Error rate (by queue) 

max (sidekiq_running_job_runtime) by (queue) > 300 Job execution time 

- prevent capacity lack - investigate bottlenecks Why to monitor Sidekick? 

Kafka client monitoring With yabeda-kafka ruby gem 

Kafka metrics With ruby-kafka 

Kafka metrics With yabeda-kafka ruby gem MVP 

Kafka metrics 

External HTTP request monitoring With yabeda-http_requests ruby gem 

Why to monitor - how good HTTP requests are made - how many times they consume - what is the distribution of HTTP response statuses 

Metrics 

Log memory profiling (research) With memory_profiler gem 

No content

Results 

Why to monitor? - determine hungry jobs - Its parameters (you can log them) 

Research execution time (research) 

Server side rendering execution time 

Product metrics (for developers) 

Metrics: - yabeda-rb/yabeda-puma-plugin - Built-in metrics for rack-attack monitoring out of the box! - yabeda-rb/yabeda-rails - Yabeda plugin to collect basic metrics for Rails applications - yabeda-rb/yabeda-graphql - Measure and understand how good your GraphQL-Ruby application works - yabeda-rb/yabeda-sidekiq -Yabeda plugin for complete monitoring of Sidekiq - yabeda-rb/yabeda-faktory - Built-in metrics for monitoring Faktory worker for Ruby out of the box - yabeda-rb/yabeda-http_requests - monitor how many outgoing HTTP calls your application does (uses Sniffer) - dsalahutdinov/yabeda-kafka - metrics for ruby-kafka Kafka client - dsalahutdinov/yabeda-rack_attack - Built-in metrics for rack-attack monitoring out of the box! - efigence/yabeda-grape - Metrics for monitoring Grape endpoints - Placewise/yabeda-gruf - metrics for gRPC Ruby Framework - ianks/yabeda-gc - Built in metrics for monitoring the Ruby garbage collector. Exporters: - yabeda-rb/yabeda-prometheus - Adapter to expose metrics collected by Yabeda plugins to Prometheus - yabeda-rb/yabeda-prometheus-mmap - Yabeda Prometheus exporter based on prometheus-client-mmap Adapters: - yabeda-rb/yabeda-datadog - Yabeda Datadog adapter. Collect and send custom metrics from Ruby apps to Datadog. - yabeda-rb/yabeda-newrelic - NewRelic adapter for Yabeda monitoring framework - asusikov/yabeda-statsd - Yabeda adapter for easy exporting collected custom metrics from your application to the Statsd server. Existing tools 

Non-yet-existing tools 

Why to invent new bicycle? - Flexibility (g.e. the APDEX in context of endpoint) - Newrelic does not logs specific metrics and tools (g.e. sidekiq latency, Kafka client) - Yabeda can complement NewRelic (with specific metrics & NewRelic adapter) - Modularity (use what you need) - Cost/Maintenance (self-hosted vs SaaS) - Yabeda is the research tool - Better developer experience - SRE-ready Why not just use Newrelic? 

Thank you 🧡