Threads & JRuby: the simple alternative to evented

Although event-driven solutions for handling concurrency are getting a lot of attention, they are not without compromise. The JVM provides a robust, battle-tested threading model, along with powerful concurrency libraries that make it easy to solve a wide-variety of problems that need a concurrent solution. By using JRuby, developers can get the advantages of dynamic languages like JavaScript, but wield the full power of the JVM, all while producing clean, comprehensible, testable code.

This talk will relate a real-world problem faced by LivingSocial, and its solution, to a more general set of guidelines, tools, and techniques that any developer can use to deliver a highly performant, concurrent system that is easy to understand.

Transcript

1. Threads & JRuby the simple alternative to evented david.copeland@livingsocial.com @davetron5000 www.naildrivin5.com www.awesomecommandlineapps.com Thursday, July 19, 12
2. { :me => { :twitter => “@davetron5000”, :author => “Build Awesome Command-Line Applications in Ruby”, } } Thursday, July 19, 12
3. Thursday, July 19, 12
4. { :me => { :twitter => “@davetron5000”, :author => “Build Awesome Command Line Applications in Ruby”, :techlead => “LivingSocial”, } } Thursday, July 19, 12
5. First Assignment Thursday, July 19, 12
6. Command Line App Write a Thursday, July 19, 12
7. Command Line App Write a To charge credit cards faster Thursday, July 19, 12
8. NO NEW HARDWARE Thursday, July 19, 12
9. Thursday, July 19, 12
10. THREADS! Thursday, July 19, 12
11. Are we maximizing our resources? Thursday, July 19, 12
12. Are we maximizing our resources? Thursday, July 19, 12
13. What problem are we solving? Thursday, July 19, 12
14. Do some I/O Compute Stuff Thursday, July 19, 12
15. Do some I/O Thursday, July 19, 12
16. Do some I/O Can I do I/O? Read/Write Block YES NO Thursday, July 19, 12
17. Do some I/O Block Thursday, July 19, 12
18. Block Thursday, July 19, 12
19. Let someone else work Block Thursday, July 19, 12
20. Maximize Resources Block Thursday, July 19, 12
21. Thursday, July 19, 12
22. Will our code be Thursday, July 19, 12
23. Will our code be easy to write? Thursday, July 19, 12
24. easy to understand? Will our code be easy to write? Thursday, July 19, 12
25. easy to understand? easy to test? Will our code be easy to write? Thursday, July 19, 12
26. Are we maximizing our resources? Are we maximizing our resources? Thursday, July 19, 12
27. Run lots of processes Thursday, July 19, 12
28. Thursday, July 19, 12
29. fork { # your code } Thursday, July 19, 12
30. fork { # your code } not available on JVM or in JavaScript Thursday, July 19, 12
31. Thursday, July 19, 12
32. for i in {1..10} do ./your_app done Thursday, July 19, 12
33. easy to understand Thursday, July 19, 12
34. doesn’t maximize resources Thursday, July 19, 12
35. Parent Thursday, July 19, 12
36. Parent Parent’s Memory Thursday, July 19, 12
37. Parent Parent’s Memory Child fork { Parent’s Memory (copy of) Thursday, July 19, 12
38. Event-based I/O Thursday, July 19, 12
39. var http = require('http'); http.createServer(function (req, res) { res.writeHead(200, {'Content-Type': 'text/plain'}); res.end('Hello World\n'); }).listen(1337, '127.0.0.1'); Thursday, July 19, 12
40. // bring in special classes and setup eventedThing.whenIOIsReady(function(results) { // do something with results }).startEventLoop(); Thursday, July 19, 12
41. Need to do I/O again? Thursday, July 19, 12
42. // bring in special classes and setup eventedThing.whenIOIsReady(function(results) { // do something with results otherEventedThing.whenMoreIOIsReady(function(moreResults) { // do something with more I/O }); }).startEventLoop(); Thursday, July 19, 12
43. // bring in special classes and setup eventedThing.whenIOIsReady(function(results) { // do something with results otherEventedThing.whenMoreIOIsReady(function(moreResults) { // do something with more I/O evenMoreEventedThings.whenYetMorIOIsReady(function(moreResults) { // do even more stuff with I/O }); }); }).startEventLoop(); Thursday, July 19, 12
44. public class EchoServer { public static void main(String[] args) throws Exception { ServerBootstrap bootstrap = new ServerBootstrap( new NioServerSocketChannelFactory( Executors.newCachedThreadPool(), Executors.newCachedThreadPool())); bootstrap.setPipelineFactory(new ChannelPipelineFactory() { public ChannelPipeline getPipeline() throws Exception { return Channels.pipeline(new EchoServerHandler()); } }); bootstrap.bind(new InetSocketAddress(8080)); } } Thursday, July 19, 12
45. public class EchoServer { public static void main(String[] args) throws Exception { ServerBootstrap bootstrap = new ServerBootstrap( new NioServerSocketChannelFactory( Executors.newCachedThreadPool(), Executors.newCachedThreadPool())); bootstrap.setPipelineFactory(new ChannelPipelineFactory() { public ChannelPipeline getPipeline() throws Exception { return Channels.pipeline(new EchoServerHandler()); } }); bootstrap.bind(new InetSocketAddress(8080)); } } Thursday, July 19, 12
46. public class EchoServerHandler extends SimpleChannelUpstreamHandler { @Override public void messageReceived(ChannelHandlerContext ctx, MessageEvent e) { e.getChannel().write(e.getMessage()); } @Override public void exceptionCaught(ChannelHandlerContext ctx, ExceptionEvent e) { e.getChannel().close(); } } Thursday, July 19, 12
47. NOT easy to understand Thursday, July 19, 12
48. kinda maximizes resources Thursday, July 19, 12
49. Entire call chain must be evented Thursday, July 19, 12
50. Not true parallelism Thursday, July 19, 12
51. Not true parallelism unless you run lots of processes Thursday, July 19, 12
52. NOT easy to understand kinda maximizes resources Thursday, July 19, 12
53. Threads Thursday, July 19, 12
54. Thursday, July 19, 12
55. Thread.new { # your code } Thursday, July 19, 12
56. fork { # your code } Thursday, July 19, 12
57. Thread.new { # your code } Thursday, July 19, 12
58. new Thread(new Runnable() { public void run() { // your code } }).start(); Thursday, July 19, 12
59. Need to do I/O? Thursday, July 19, 12
60. Need to do I/O? Not a problem Thursday, July 19, 12
61. Need to do I/O THREE TIMES? Thursday, July 19, 12
62. Need to do I/O THREE TIMES? Not a problem Thursday, July 19, 12
63. What if I need to calculate π to the 2,345,123rd decimal place? Thursday, July 19, 12
64. NOT A PROBLEM Thursday, July 19, 12
65. Threads achieve true parallelism Thursday, July 19, 12
66. easy to understand Thursday, July 19, 12
67. maximizes resources Thursday, July 19, 12
68. JRuby Thursday, July 19, 12
69. Ruby runtime… Thursday, July 19, 12
70. Ruby runtime… …running on the JVM Thursday, July 19, 12
71. new Thread(new Runnable() { public void run() { List<Person> people = queryPeople(); for (Person p: people) { if (p.getAge() < 18) { // do one thing for minors } else { // do another thing for adults } } } }).start(); Thursday, July 19, 12
72. Thread.new { query_people.each do |p| if p.get_age < 18 # do one thing for minors else # do another thing for adults end end } Thursday, July 19, 12
73. Thread.new { query_people.each do |p| if p.get_age < 18 # do one thing for minors else # do another thing for adults end end } Thursday, July 19, 12
74. Thread.new { query_people.each do |p| if p.get_age < 18 # do one thing for minors else # do another thing for adults end end } Thursday, July 19, 12
75. Fast, powerful dynamic language… Thursday, July 19, 12
76. Fast, powerful dynamic language… …full power of the JVM Thursday, July 19, 12
77. Other Rubies? MRI 1.8 Green Threads No Parallelism MRI 1.9 Real Threads No Parallelism Rubinius Real Threads, no GIL True Parallelism Thursday, July 19, 12
78. Other Rubies? MRI 1.8 Green Threads No Parallelism MRI 1.9 Real Threads No Parallelism Rubinius Real Threads, no GIL True Parallelism Thursday, July 19, 12
79. So, you’ve decided to use Threads… Thursday, July 19, 12
80. You need to know four things Thursday, July 19, 12
81. Thursday, July 19, 12
82. Start & Manage Thursday, July 19, 12
83. Start & Manage (Dealing with) Shared State Thursday, July 19, 12
84. Start & Manage (Dealing with) Shared State Using Third Party Libraries Thursday, July 19, 12
85. Start & Manage (Dealing with) Shared State Using Third Party Libraries Context Switching Thursday, July 19, 12
86. Start & Manage Threads Thursday, July 19, 12
87. Chaos Thursday, July 19, 12
88. Thread.new { # your code } Thread.new { # moar code } # etc Thursday, July 19, 12
89. Hand-Roll Thursday, July 19, 12
90. threads = [] threads << Thread.new { # your code } threads << Thread.new { # moar code } # etc threads.each(&:join) # All threads have completed exit 0 Thursday, July 19, 12
91. java.util.concurrent Thursday, July 19, 12
92. java_import java.util.concurrent service = Executors.new_fixed_thread_pool(100) service.execute { # your code } service.execute { # some other code } service.shutdown service.await_termination(10,SECONDS) service.shutdown_now Thursday, July 19, 12
93. java_import java.util.concurrent service = Executors.new_fixed_thread_pool(100) service.execute { # your code } service.execute { # some other code } service.shutdown service.await_termination(10,SECONDS) service.shutdown_now Thursday, July 19, 12
94. java_import java.util.concurrent service = Executors.new_fixed_thread_pool(100) service.execute { # your code } service.execute { # some other code } service.shutdown service.await_termination(10,SECONDS) service.shutdown_now Thursday, July 19, 12
95. java_import java.util.concurrent service = Executors.new_fixed_thread_pool(100) service.execute { # your code } service.execute { # some other code } service.shutdown service.await_termination(10,SECONDS) service.shutdown_now Thursday, July 19, 12
96. java_import java.util.concurrent service = Executors.new_fixed_thread_pool(100) service.execute { # your code } service.execute { # some other code } service.shutdown service.await_termination(10,SECONDS) service.shutdown_now Thursday, July 19, 12
97. java_import java.util.concurrent service = Executors.new_fixed_thread_pool(100) service.execute { # your code } service.execute { # some other code } service.shutdown service.await_termination(10,SECONDS) service.shutdown_now Thursday, July 19, 12
98. service = Executors.new_fixed_thread_pool(10) tcp_server = TCPServer.new("127.0.0.1",8080) Signal.trap('SIGINT') { service.shutdown } loop do { s = tcp_server.accept service.execute { s.puts calculate_pi() s.close } break if service.is_shutdown } service.await_termination(10,TimeUnit.SECONDS) service.shutdown_now Thursday, July 19, 12
99. So much more Thursday, July 19, 12
100. ScheduledExecutorService So much more Thursday, July 19, 12
101. ScheduledExecutorService So much more ThreadFactory Thursday, July 19, 12
102. Read the javadocs ScheduledExecutorService So much more ThreadFactory Thursday, July 19, 12
103. Shared State Thursday, July 19, 12
104. results = [] Thread.new { results << some_result() } Thread.new { results << other_result() } Thursday, July 19, 12
105. mutex = Mutex.new results = [] Thread.new { mutex.synchronize { results << some_result() } } Thread.new { mutex.synchronize { results << other_result() } } Thursday, July 19, 12
106. Avoid explicit locking/sync Thursday, July 19, 12
107. results = ConcurrentLinkedQueue.new Thread.new { results.offer(some_result()) } Thread.new { results.offer(some_result()) } Thursday, July 19, 12
108. So much more Thursday, July 19, 12
109. ConcurrentHashMap So much more Thursday, July 19, 12
110. ConcurrentHashMap So much more AtomicReference Thursday, July 19, 12
111. Read the javadocs ConcurrentHashMap So much more AtomicReference Thursday, July 19, 12
112. Third Party Libraries Thursday, July 19, 12
113. Are they thread safe? Thursday, July 19, 12
114. Probably Thursday, July 19, 12
115. Probably …but to be sure Thursday, July 19, 12
116. variables Thursday, July 19, 12
117. variables Global Thursday, July 19, 12
118. variables Global Class Thursday, July 19, 12
119. Most are fine Thursday, July 19, 12
120. Context Switching Thursday, July 19, 12
121. Amdahl’s Law Thursday, July 19, 12
122. Performance Gains Increase in # of Threads Speedup Thursday, July 19, 12
123. Performance Gains with Context Switching Increase in # of Threads Speedup Cost of Context Switching Takes Over Thursday, July 19, 12
124. Thread Pools Thursday, July 19, 12
125. Your Code Your Code Your Code Your Code Thread Thread Thread Thread Pool Your Code Your Code Your Code Your Code Thursday, July 19, 12
126. Your Code Your Code Your Code Your Code Thread Thread Thread Thread Pool Your Code Your Code Your Code Your Code Thursday, July 19, 12
127. Your Code Your Code Your Code Your Code Thread Thread Thread Thread Pool Your Code Your Code Your Code Thursday, July 19, 12
128. Thread.new { # your code } Thursday, July 19, 12
129. service.execute { # your code } Thursday, July 19, 12
130. Not usually a concern Thursday, July 19, 12
131. Always use Threads, right? Thursday, July 19, 12
132. Thread UNsafe Libraries Thursday, July 19, 12
133. Evented Thursday, July 19, 12
134. Cannot use JRuby/JVM Thursday, July 19, 12
135. Evented Thursday, July 19, 12
136. Otherwise, Threads will simplify your code and maximize your resources Thursday, July 19, 12
137. Ask “why shouldn’t I use threads?” Thursday, July 19, 12
138. Exercises Thursday, July 19, 12
139. Echo Server •Listen on a port •Respond to each request in a new Thread •Extra Credit: Record stats on requests in a shared data structure Thursday, July 19, 12
140. Connection Pool •Allow N clients to access X shared instances of, say, Redis (where N > X) •Clients “check out” a connection and get exclusive access •Clients “check in” when done •Instances get re-used Thursday, July 19, 12
141. THANKS! david.copeland@livingsocial.com @davetron5000 www.naildrivin5.com www.awesomecommandlineapps.com Thursday, July 19, 12
142. THANKS! david.copeland@livingsocial.com @davetron5000 www.naildrivin5.com www.awesomecommandlineapps.com We’re Hiring! Thursday, July 19, 12