Scaling Rails: The Journey to 200M Notifications

In the talk "Scaling Rails: The Journey to 200M Notifications", Gustavo Araujo will share his experience scaling a Ruby on Rails application to send millions of notifications, covering challenges, trade-offs, and technical decisions involved in turning an MVP into a robust and scalable system.

Transcript

1. SCALING RAILS The Journey to 200M Notifications
2. 󰞦 Software Engineer @ CloudWalk gustavoaraujo.dev garaujodev garaujodev Gustavo Araujo
3. 󰞦 Software Engineer @ CloudWalk 🎯 Focused on Ruby on Rails and Elixir gustavoaraujo.dev garaujodev garaujodev Gustavo Araujo
4. 󰞦 Software Engineer @ CloudWalk 🎯 Focused on Ruby on Rails and Elixir 💜 Passionate about code quality, observability, and performance gustavoaraujo.dev garaujodev garaujodev Gustavo Araujo
5. 󰞦 Software Engineer @ CloudWalk 🎯 Focused on Ruby on Rails and Elixir 💜 Passionate about code quality, observability, and performance 󰜼 Sharing knowledge through talks and technical content gustavoaraujo.dev garaujodev garaujodev Gustavo Araujo
6. Application Challenges ➡ High-Volume Notifications (1B last year)
7. Application Challenges ➡ High-Volume Notifications (1B last year) ➡ Multi-channel (e.g., WhatsApp, SMS, Email, Push)
8. Application Challenges ➡ High-Volume Notifications (1B last year) ➡ Several Database Writes ➡ Multi-channel (e.g., WhatsApp, SMS, Email, Push)
9. Application Challenges ➡ High-Volume Notifications (1B last year) ➡ Several Database Writes ➡ Multi-channel (e.g., WhatsApp, SMS, Email, Push) ➡ Unpredictable workloads
10. The challenges we faced, and the key lessons learned
11. CONCEPT OF SCALE
12. CONCEPT OF SCALE
13. HOW WE STARTED?
14. CHOOSE YOUR WEBSERVER
15. ⚙Concurrency Model ⚡ Performance ✅ Best Use Case ❌ Limitations WebRick Single-threaded + single-process Very low Development only Not production-ready; removed in Rails 6+ Ruby Webservers
16. ⚙Concurrency Model ⚡ Performance ✅ Best Use Case ❌ Limitations WebRick Single-threaded + single-process Very low Development only Not production-ready; removed in Rails 6+ Passenger Multi-threaded + multi-process (hybrid) Medium to High Easy-to-deploy production apps Advanced features require paid license Ruby Webservers
17. ⚙Concurrency Model ⚡ Performance ✅ Best Use Case ❌ Limitations WebRick Single-threaded + single-process Very low Development only Not production-ready; removed in Rails 6+ Passenger Multi-threaded + multi-process (hybrid) Medium to High Easy-to-deploy production apps Advanced features require paid license Unicorn Multi-process (no threads) High for CPU-bound apps Apps requiring process-level isolation No concurrency per worker; not ideal for I/O-bound apps Ruby Webservers
18. ⚙Concurrency Model ⚡ Performance ✅ Best Use Case ❌ Limitations WebRick Single-threaded + single-process Very low Development only Not production-ready; removed in Rails 6+ Passenger Multi-threaded + multi-process (hybrid) Medium to High Easy-to-deploy production apps Advanced features require paid license Unicorn Multi-process (no threads) High for CPU-bound apps Apps requiring process-level isolation No concurrency per worker; not ideal for I/O-bound apps Puma Multi-threaded + multi-process (clustered) Highly configurable — scales with threads and workers Modern Rails apps needing concurrent I/O handling Requires tuning (threads/workers + DB pool alignment) Ruby Webservers
19. Puma ✅ ❌ Clustered Mode Best for Multi-core CPUs, CPU-bound applications Higher memory usage since each worker is a separate process
20. Puma ✅ ❌ Clustered Mode Best for Multi-core CPUs, CPU-bound applications Higher memory usage since each worker is a separate process Single Mode Best for I/O-bound apps, lower memory usage Limitation: Single process, may not fully utilize multi-core CPUs
21. How Many Workers? Scenario Recommendation 🖥 App runs on VM/bare metal Use 1 worker per available CPU core
22. How Many Workers? Scenario Recommendation 🖥 App runs on VM/bare metal Use 1 worker per available CPU core 📦 App runs in Docker/Kubernetes Respect container CPU limit (cpu_limit = 2) → set workers = 2
23. How Many Workers? Scenario Recommendation 🖥 App runs on VM/bare metal Use 1 worker per available CPU core 📦 App runs in Docker/Kubernetes Respect container CPU limit (cpu_limit = 2) → set workers = 2 🧠 App has high memory usage Use fewer workers, increase threads
24. How Many Workers? Scenario Recommendation 🖥 App runs on VM/bare metal Use 1 worker per available CPU core 📦 App runs in Docker/Kubernetes Respect container CPU limit (cpu_limit = 2) → set workers = 2 🧠 App has high memory usage Use fewer workers, increase threads 🧪 Unclear limits or mixed load Start with 2–4 workers and benchmark
25. How Many Threads? Factor Guideline 🔄 I/O-bound app Use more threads (e.g. 16–32) to handle concurrency
26. How Many Threads? Factor Guideline 🔄 I/O-bound app Use more threads (e.g. 16–32) to handle concurrency 🧮 CPU-bound app Fewer threads (e.g. 4–8) to avoid contention
27. How Many Threads? Factor Guideline 🔄 I/O-bound app Use more threads (e.g. 16–32) to handle concurrency 🧮 CPU-bound app Fewer threads (e.g. 4–8) to avoid contention 🧵 Low traffic env (dev/staging) Use something like threads 1, 4
28. DEPLOYMENT STRATEGIES
29. Platform as a Service (PaaS) Example: Heroku, fly.io ✅ Easy to get started and Built-in autoscaling
30. Platform as a Service (PaaS) Example: Heroku, fly.io ✅ Easy to get started and Built-in autoscaling ❌ Cost to scale and limited control over infrastructure for advanced tuning
31. Infrastructure as a Service (IaaS) Example: AWS, GCP, Azure ✅ Full control (Instances, Auto Scaling, Load Balancers)
32. Infrastructure as a Service (IaaS) Example: AWS, GCP, Azure ✅ Full control (Instances, Auto Scaling, Load Balancers) ❌ High availability requires manual setup, fixed resource provisioning
33. Container Orchestration Example: Kubernetes (can run on AWS, GCP, Azure, or On-Premise) ✅ Portability and fine-grained control over resources(CPU, memory, limits per pod)
34. Container Orchestration Example: Kubernetes (can run on AWS, GCP, Azure, or On-Premise) ✅ Portability and fine-grained control over resources(CPU, memory, limits per pod) ❌ Steep learning curve — complex concepts (pods, services, volumes)
35. Kubernetes Scaling Strategies ↔ HPA (Horizontal Pod Autoscaler) → reacts to CPU/memory usage
36. Kubernetes Scaling Strategies ↔ HPA (Horizontal Pod Autoscaler) → reacts to CPU/memory usage ↕ VPA (Vertical Pod Autoscaler) → adjusts resource requests for individual pods
37. Kubernetes Scaling Strategies ↔ HPA (Horizontal Pod Autoscaler) → reacts to CPU/memory usage ↕ VPA (Vertical Pod Autoscaler) → adjusts resource requests for individual pods 🧱 Cluster Autoscaler → adds/removes nodes to accommodate workload
38. Horizontal Pod Autoscaling (HPA)
39. Kubernetes Event-driven Autoscaling (KEDA)
40. WHAT WE LEARNED
41. Common Pitfalls 🧩 Issue 🛠 Cause ⚠ Impact DB Pool Mismatch THREADS > pool in database.yml Connection timeouts and ActiveRecord errors
42. Common Pitfalls 🧩 Issue 🛠 Cause ⚠ Impact DB Pool Mismatch THREADS > pool in database.yml Connection timeouts and ActiveRecord errors Excessive Thread Count Threads set too high without real concurrency need Increased memory usage, thread contention, and degraded app performance
43. Common Pitfalls 🧩 Issue 🛠 Cause ⚠ Impact DB Pool Mismatch THREADS > pool in database.yml Connection timeouts and ActiveRecord errors Excessive Thread Count Threads set too high without real concurrency need Increased memory usage, thread contention, and degraded app performance Resource Overcommit Too many workers × threads for available CPU/memory Application instability due to memory exhaustion (OOM) or CPU throttling
44. Database Insights ➡ Read Replicas
45. Database Insights ➡ Read Replicas ➡ Add and maintain proper indexes
46. Database Insights ➡ Read Replicas ➡ Add and maintain proper indexes ➡ Optimize slow queries
47. Database Insights ➡ Read Replicas ➡ Add and maintain proper indexes ➡ Optimize slow queries ➡ Use partitioning
48. Database Insights ➡ Read Replicas ➡ Add and maintain proper indexes ➡ Optimize slow queries ➡ Use partitioning ➡ Consider cache strategies
49. Pro Tips Use background jobs Offload heavy work to async queues
50. Fail fast, retry smart Use retries with backoff (Sidekiq, Shoryuken, etc.) to avoid overload loops Pro Tips
51. Use observability tools Datadog, Sentry, NewRelic, AppSignal, Skylight… Pro Tips
52. Thank you! gustavoaraujo.dev garaujodev garaujodev We are hiring! cloudwalk.io/jobs
53. References https://www.speedshop.co/2015/07/29/scaling-ruby-apps-to-1000-rpm.html https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/ https://keda.sh/docs/2.16/concepts/ https://kubernetes.io/docs/concepts/workloads/autoscaling/