Truly Madly Deeply Parallel Ruby Web Applications

My slides of talk at http://rubyconfindia.org/2014/

Traditional Ruby application architecture involves multiple processes to achieve parallelism. This talk covers below topics.
1. Process Parallelism is not a very efficient way of scaling.
2. "Safe for Parallelism" is more important than being just "Threadsafe".
3. JRuby or Rubinius

1. Truly Madly Deeply Parallel Ruby (Web)* Applications @harikrishnan83 * Thanks @headius
2. How many of you are building web applications?
3. How many of you have more than 1 user using it at a time?
4. How many of you use scaling out as the way hit scale?
5. How many of you are scaling up to hit scale?
6. What is a parallel environment?
7. Parallelism
8. Concurrency Thread A - load Thread B - load Thread A - increment Thread B - increment Thread A - save Thread B - save i = 0 i = 0 i = 1 i = 1
9. Ruby web application deployment
10. Process Parallelism Reverse Proxy Unicorn Processes Database Layer
11. On my current project
12. We only use EC2 small instances
13. Because it is very hard to utilize a high spec machine Process Context Switch is Expensive
14. Today... • We have 4 small EC2 instances • 2 Puma processes run on each • Handles about 100,000 requests per hour • And this is our Private alpha
15. We need to... • Handle about 1 million requests per hour • Which means about 40-45 EC2 small instances
16. This is not trivial • Costs a lot of money • Lot of time required to maintain these boxes • Being elastic will become very important • Cost also in terms of more Devops time
17. In general
18. It is easier to baby sit few boxes
19. Than a lot!
20. Ideally, we would like to both scale up and scale out
21. i.e. we want to achieve the same throughput with, say, just 5 large instances
22. Enter thread based parallelism
23. Why were we not doing this till now?
24. Threads are hard* They share memory and mutate things They share memory and mutate things * - Supposedly
25. And there is the ubiquitous issue ‘Thread Safety’
26. Before we go there, first lets look at some code
28. The real question is...
29. Are you “Safe for Parallelization”
30. Understanding this will take you a long way in “getting parallel”
31. Things to remember while moving to threaded parallelism
32. #1 - Always identify the shared resources
33. Shared Resource • Objects • DB rows • Caches • Log files!
34. #2 - Bank on thread safe libraries
35. Libraries • Data structures • JSON, XML parsing, HTTP clients etc • Generally, auditing all the gems you use for thread safety is a good idea
36. If you only use thread safe libraries are you ‘safe for parallelism’?
37. Rails is thread safe right? Why is everyone concerned about thread safety in the first place?
38. #3 - If two libraries are thread safe, code that uses both of them need not be
39. Rails thread safety model • Instantiate everything for every request • No shared state (global objects) • Different from, say, Java (single servlet object per container, IOC with singletons etc.)
40. #4 - Try and stick to Rails’ way of handling requests
41. Are you ‘Safe for parallelism’ if you follow these steps?
42. Well, it depends...
43. Validating, say, through a green bar is very hard.
44. Always give yourself some time to stabilize. The move is definitely not overnight!
45. Speaking of the move, move where?
46. Since Rubinius is mostly MRI like, its simpler
47. I personally love JRuby more because of my JVM background
48. Lots of good things have been spoken about JRuby
49. Some gotchas based on my experience
50. JRuby impacts Developers • The JRuby startup time (mostly because of the JVM startup time) can sometimes kill red- green cycle time • Sometimes, you should be OK with stooping down to Java code to figure out why something is not working
51. JRuby impacts OPs • You no longer have a ruby app in prod, its a Java app • GC tuning, Process monitoring, Profiling etc. are very different on a JVM
52. Thread Parallelism Reverse Proxy Puma Instance Database Layer Threads
53. Thank you! @harikrishnan83