Upgrade to Pro — share decks privately, control downloads, hide ads and more …

RubyConf.ph 2014 - ZOMGSCALE! With Celluloid & JRuby

RubyConf.ph 2014 - ZOMGSCALE! With Celluloid & JRuby

146e52d49d361f85c0945487452fc6a0?s=128

Ben Lovell

March 29, 2014
Tweet

Transcript

  1. zomgscale! with Celluloid and JRuby Ben Lovell

  2. benlovell _j

  3. 113581334398839860922 !

  4. None
  5. None
  6. None
  7. “I HAVE KILLED, AND I WILL KILL AGAIN”

  8. None
  9. None
  10. None
  11. None
  12. None
  13. None
  14. None
  15. None
  16. None
  17. None
  18. None
  19. None
  20. None
  21. None
  22. None
  23. None
  24. None
  25. None
  26. HEY LADIES

  27. ❤️

  28. None
  29. None
  30. zomgscale! with Celluloid and JRuby Ben Lovell

  31. Moore’s Law

  32. None
  33. I don’t like bungee jumping... ! ...but I do like

    skiing Roger Moore
  34. None
  35. every few years CPU clocks have doubled

  36. but recently this growth has stalled

  37. cores ++++++++

  38. the free lunch is over Herb Sutter

  39. harness the POWER in those cores

  40. ?

  41. concurrency parallelism

  42. so there’s a difference?

  43. concurrent!

  44. parallel?

  45. None
  46. processes or threads

  47. processes memory constraints communication x cores == x processes?

  48. processes what about fork(2) and CoW friendly GC ?

  49. not CoW friendly

  50. threads shared state locks and lock granularity race conditions can

    be hard to reason about
  51. zomg! I love multithreaded code - NOBODY EVER

  52. None
  53. None
  54. so what’s up with MRI?

  55. well, nothing but...

  56. GIL

  57. some things the GIL is responsible for...

  58. San Francisco Bay oil spillage

  59. ...maritime disasters

  60. THANKS, GIL!

  61. GIL

  62. MRI not so bad if you’re I/O bound

  63. MRI but what about computation?

  64. meh. thread-level parallelism is available right now! ...just not with

    MRI
  65. None
  66. rubinius 2.0.0 due for release 2042 OLD SLIDE ALERT!

  67. so now that we have truly parallel threads is the

    problem solved?
  68. ! ! rules of threading

  69. Don’t do it!

  70. If you must do it don’t share data across threads

  71. If you must share data across threads don’t share mutable

    data
  72. If you must share mutable data across threads synchronise access

    to this data
  73. don’t communicate by sharing memory... ! ...share memory by communicating

    go
  74. None
  75. painless multithreaded programming for ruby

  76. Tony Arcieri Tim Carey-Smith Ben Langfeld @bascule @halorgium @benlangfeld The

    Maintainers
  77. None
  78. None
  79. a concurrent object oriented programming framework which lets you build

    multithreaded programs out of concurrent objects just as easily as you build sequential programs out of regular objects
  80. None
  81. based upon the actor model

  82. actor model first proposed way back in 1970

  83. actor model actors are isolated within lightweight processes

  84. actor model actors possess identity

  85. actor model absolutely no shared state

  86. actor model actors don’t need to compete for locks

  87. actor model are sent messages asynchronously

  88. actor model messages are buffered by a mailbox

  89. actor model the actor works off each message sequentially

  90. actor model has implementations in many languages

  91. None
  92. None
  93. None
  94. None
  95. celluloid actors automatically synchronize state

  96. 1 class Actor! 2 attr_reader :counter! 3 ! 4 def

    initialize! 5 @counter = 0! 6 @mutex = Mutex.new! 7 end! 8 ! 9 def increment! 10 @mutex.synchronize do! 11 @counter += 1! 12 end! 13 end! 14 end!
  97. with celluloid the same example...

  98. 1 require "celluloid"! 2 ! 3 class Actor! 4 include

    Celluloid! 5 attr_reader :counter! 6 ! 7 def initialize! 8 @counter = 0! 9 end! 10 ! 11 def increment! 12 @counter += 1! 13 end! 14 end!
  99. None
  100. celluloid actors are active objects living within threads

  101. 1 require "celluloid"! 2 ! 3 class Actor! 4 include

    Celluloid! 5 end! 6 ! 7 actor = Actor.new! 8 actor.inspect! 9 #=> <Celluloid::ActorProxy(Actor:0x3feaecbb38e0)>! 10 ! 11 Thread.main! 12 #=> <Thread:0x007f86290b8ce8 run>! 13 ! 14 actor.thread! 15 #=> <Thread:0x007f862ad27a78 sleep>!
  102. 1 module Celluloid! 2 module ClassMethods! 3 # Create a

    new actor! 4 def new(*args, &block)! 5 proxy = Actor.new(allocate, actor_options).proxy! 6 proxy._send_(:initialize, *args, &block)! 7 proxy! 8 end! 9 #...! 10 end! 11 #...! 12 end!
  103. celluloid actors messages you send are buffered via the actor’s

    mailbox...
  104. celluloid actors ... until the actor is ready to act

    upon them
  105. ______________! < ETOOMANYACTS >! --------------! \ ^__^! \ (oo)\_______! (__)\

    )\/\! ||----w |! || ||! !
  106. celluloid actors no pattern matching just regular messages

  107. celluloid actors poll their mailbox in a message loop

  108. 1 class Actor! 2 # Wrap the given subject with

    an Actor! 3 def initialize(subject, options = {})! 4 @subject = subject! 5 @mailbox = options[:mailbox] || Mailbox.new! 6 @running = true! 7 ! 8 @thread = ThreadHandle.new(:actor) do! 9 setup_thread! 10 run! 11 end! 12 #...! 13 end! 14 #...! 15 end!
  109. 1 class Actor! 2 def run! 3 #...! 4 while

    @running! 5 if message = @mailbox.receive(timeout_interval)! 6 handle_message message! 7 else! 8 # No message indicates a timeout! 9 @timers.fire! 10 @receivers.fire_timers! 11 end! 12 end! 13 #...! 14 shutdown! 15 end! 16 end!
  110. celluloid actors act upon messages sequentially

  111. what about ordering? no guarantees

  112. celluloid actors dispatch calls within fibers

  113. fibers? cooperative lightweight user space some gotchas...

  114. celluloid actors can dispatch synchronously

  115. 1 require "celluloid"! 2 ! 3 class Actor! 4 include

    Celluloid! 5 ! 6 def compute_all_the_things! 7 sleep 2! 8 puts "42"! 9 end! 10 end! 11 ! 12 actor = Actor.new! 13 actor.compute_all_the_things! 14 puts "done!"! ! #=> 42! #=> done!! blocking
  116. celluloid actors can dispatch asynchronously

  117. 1 require "celluloid"! 2 ! 3 class Actor! 4 include

    Celluloid! 5 ! 6 def compute_all_the_things! 7 sleep 2! 8 puts "42"! 9 end! 10 end! 11 ! 12 actor = Actor.new! 13 actor.async.compute_all_the_things! 14 puts "done!"! 15 ! 16 #=> done!! 17 #=> 42! returns immediately
  118. celluloid actors can perform tasks in futures

  119. 1 require "celluloid"! 2 ! 3 class Actor! 4 include

    Celluloid! 5 ! 6 def compute_all_the_things! 7 sleep 2! 8 "42"! 9 end! 10 end! 11 ! 12 actor = Actor.new! 13 future = actor.future.compute_all_the_things! 14 puts "done!"! 15 puts future.value! 16 ! 17 #=> done!! 18 #=> 42! returns immediately blocks until a value is yielded
  120. celluloid actors are accessible by reference or name

  121. 1 require "celluloid"! 2 ! 3 class Actor! 4 include

    Celluloid! 5 ! 6 def compute_all_the_things! 7 sleep 2! 8 puts "42"! 9 end! 10 end! 11 ! 12 actor = Actor.new! 13 Celluloid::Actor[:foo] = actor! 14 ! 15 actor.inspect! 16 #=> <Celluloid::ActorProxy(Actor:0x3feb3ec11308)>! 17 Celluloid::Actor[:foo].inspect! 18 #=> <Celluloid::ActorProxy(Actor:0x3feb3ec11308)>!
  122. celluloid actors are fault tolerant ... let it crash!

  123. 1 require "celluloid/autostart"! 2 ! 3 class Actor! 4 include

    Celluloid! 5 ! 6 def compute_all_the_things! 7 puts "42"! 8 end! 9 ! 10 def zomg_crash! 11 raise "derp!"! 12 end! 13 end! 14 ! 15 supervisor = Actor.supervise_as :foo! 16 ! 17 begin! 18 Celluloid::Actor[:foo].zomg_crash! 19 rescue! 20 puts "whoops"! 21 end! 22 ! 23 Celluloid::Actor[:foo].compute_all_the_things! 24 ! 25 #=> whoops! 26 #=> 42! crash the actor fresh actor take care of me!
  124. celluloid actors can be arranged as pooled workers

  125. 1 require "celluloid"! 2 ! 3 class Actor! 4 include

    Celluloid! 5 ! 6 def compute_all_the_things! 7 sleep 1! 8 puts "42"! 9 end! 10 end! 11 ! 12 pool = Actor.pool! 13 ! 14 4.times { pool.compute_all_the_things }! 15 ! 16 #=> 42! 17 #=> 42 and so on...! size*cores load up the workers
  126. there’s more timers links supervision groups pub/sub conditions

  127. that low hanging fruit? yeah, about that...

  128. but there is one tip! blocking I/O... don’t

  129. None
  130. an event-driven IO system for building fast, scalable network applications

    that integrate directly with celluloid actors
  131. None
  132. unlike certain other evented I/O systems which limit you to

    a single event loop per process Celluloid::IO lets you make as many actors as you want system resources permitting
  133. None
  134. None
  135. a distributed extension to celluloid which provides distributed and concurrent

    objects for ruby that are both robust and fault-tolerant
  136. None
  137. a fast non-blocking and evented web server. Thanks to celluloid,

    Reel works great for multithreaded applications and provides traditional multithreaded blocking IO support too.
  138. EXPERIMENTAL or broken as it is known outside of OSS

  139. to summarise...

  140. the future of ruby concurrency and parallelism?

  141. thanks! @benlovell ?