Concurrency and Distributed Systems in JRuby

Presentation given av JRubyConf EU/Eurucamp 2012

Transcript

1. CONCURRENCY AND DISTRIBUTED SYSTEMS IN JRUBY @iconara
2. speakerdeck.com/u/iconara (real time!)
3. Theo / @iconara
4. Chief Architect at
5. big data with Ruby since 2009
6. MAKING THIS
7. INTO THIS
8. big data with Ruby since 2009
9. APRIL 2011 stuck
10. MAY 2011 let’s try JRuby
11. JUNE 2011 wrapping Java libraries for fun and for profit
12. HOTBUNNIES natural Ruby API for RabbitMQ
13. JULY 2011 threads are better than processes
14. JRUBY
15. JRUBY
16. J.U.C (java.util.concurrent)
17. SHORTCUT Java Concurrency in Practice
18. EXECUTORS configurable thread pools, a better abstraction than using Thread directly
19. thread_pool = Executors.new_fixed_thread_pool(16) all_the_things.each do |thing| thread_pool.submit do crunch_crunch_crunch(thing) end end
20. BLOCKING QUEUES easy producer/consumer patterns
21. J.U.C.ATOMIC optimistic locking primitives, compare-and-swap, thread safe non-blocking counters
22. MORE AWESOMES ConcurrentHashMap, CountDownLatch, ForkJoinPool, Google Guava, LMAX distruptor
23. TL; DR use j.u.c and avoid mutable state and you’re safe, mostly
24. enough rope THREADS
25. AUGUST 2011 threads are a mess, let’s try actors
26. ACTORS a higher level abstraction of concurrency
27. AKKA concurrency library for Scala, famous actor implementation
28. MIKKA Akka wrapper for JRuby
29. DO ALL THE THINGS Programming Concurrency on the JVM
30. AUGUST 2011 async systems with no back pressure mechanism will die of memory starvation
31. STATE AGGREGATION INPUT PERSISTENCE OUTPUT TIMER
32. STATE AGGREGATION INPUT PERSISTENCE OUTPUT TIMER !
33. AUGUST 2011 actors are cool, but not really worth it when all the arrows in your flow chart point in the same direction
34. DIVIDE DIVIDE DIVIDE run multiple independent, synchronous workers, tune to the number of CPUs & IO wait
35. SEPTEMBER 2011 blocking all the way down, back pressure all the way up
36. AUGUST 2011 (short rant about MongoDB’s global write lock)
37. NEW SHINY TOYS for building new shiny products
38. CASSANDRA distributed database, mostly magic
39. CASSANDRA very verbose drivers ColumnFamilyUpdater<String, String> updater = template.createUpdater("a key"); updater.setString("domain", "www.datastax.com"); updater.setLong("time", System.currentTimeMillis()); try { template.update(updater); } catch (HectorException e) { // hurgh }
40. EURYDICE JRuby nails it new_data = {:domain => 'www.datastax.com', :time => Time.now.to_i} column_family.update('a key', new_data)
41. STORM stream processing framework written in Clojure
42. STORM (it does the arrows, you do the blobs)
43. REDSTORM Storm for JRuby
44. DON’T DO THIS AT HOME KIDS! the key to building concurrent and distributed systems is to separate what you want to do from the concurrent- and distributed-ness, and let someone else take care of that YOUR CODE
45. JRUBY IS A SLIPPERY SLOPE TO JBOSS let there be no doubt about it
46. SCALING IS HARD go to Srdan’s talk tomorrow
47. KTHXBAI @iconara github.com/iconara architecturalatrocities.com burtcorp.com