Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Live Long and Prosper with Celluloid & JRuby

Live Long and Prosper with Celluloid & JRuby

In this talk you'll learn about the Celluloid ecosystem. We'll take a glance at:

Celluloid - the OOP & Actor model based concurrency framework
Celluloid::IO - an evented I/O framework
DCell - distributed Actors for Celluloid
Reel - the Celluloid::IO powered web server

I'll cover the Actor model - Celluloid's inspiration for their approach to concurrent objects. How we can best achieve concurrency without mind-bending synchronisation, mutexes, deadlocks and all the bad stuff, in idiomatic and simple to grasp Ruby.

We'll finish up by briefly covering JRuby and why it's my preferred Ruby VM for concurrency and true parallelism with Celluloid.

This talk was given at JRubyconf EU - 2013.

146e52d49d361f85c0945487452fc6a0?s=128

Ben Lovell

August 15, 2013
Tweet

Transcript

  1. LIVE LONG AND PROSPER WITH CELLULOID AND JRUBY @benlovell

  2. @benlovell

  3. @benlovell

  4. None
  5. WE GOT THREADS! REAL THREADS! NO GIL THREADS!

  6. WE GOT THREADS! REAL THREADS! NO GIL THREADS!

  7. WE GOT THREADS! REAL THREADS! NO GIL THREADS!

  8. WE GOT THREADS! BUT HOW DO WE USE THEM EFFECTIVELY?

  9. LOCKS SYNCHRONISATION SHARED STATE NON-DETERMINISM UGHHHH!

  10. INSANITY: DOING THE SAME THING OVER AND OVER AGAIN AND

    EXPECTING DIFFERENT RESULTS
  11. THREADING: DOING THE SAME THING OVER AND OVER AGAIN AND

    ACHIEVING DIFFERENT RESULTS
  12. THE RULES OF THREADING

  13. NUMBER 1...

  14. DON’T DO IT!

  15. QUESTION: HOW DO SOFTWARE ENGINEERS DEAL WITH HARD PROBLEMS? ?

  16. ANSWER: 1. BEER 2. STACKOVERFLOW ?

  17. ANSWER: WE ABSTRACT THEM AWAY ?

  18. QUESTION: HOW DO WE ABSTRACT AWAY CONCURRENCY PRIMITIVES THREADS AND

    OTHER MIND BENDING SHIT... ?
  19. QUESTION: ... WHILE STILL ENJOYING THE BEAUTIFUL AND IDIOMATIC RUBY

    WHICH WE ARE ACCUSTOMED TO? ?
  20. None
  21. @bascule @halorgium @benlangfeld LOVINGLY MAINTAINED BY

  22. A concurrent object oriented programming framework for Ruby which lets

    you build multithreaded programs out of concurrent objects just as easily as you build sequential programs out of regular objects “
  23. None
  24.   ACTOR MODEL A LONG ESTABLISHED PATTERN WITH MANY

    IMPLEMENTATIONS
  25.   ACTORS LIGHTWEIGHT PROCESSES THAT POSSESS IDENTITY AND COMMUNICATE

    BY MESSAGING
  26.   ACTORS MAY CREATE OTHER ACTORS BUFFER MESSAGES IN

    A MAILBOX PROCESS MESSAGES SERIALLY
  27.   ACTORS BY VIRTUE NEVER COMPETE FOR LOCKS!

  28. None
  29. None
  30.  CELLULOID ACTORS AUTOMATICALLY SYNCHRONISE ACCESS TO STATE THEY ENCAPSULATE

  31.  1 class Actor 2 attr_reader :counter 3 4 def

    initialize 5 @counter = 0 6 @mutex = Mutex.new 7 end 8 9 def increment 10 @mutex.synchronize do 11 @counter += 1 12 end 13 end 14 end
  32.  WITH CELLULOID...

  33.  1 require "celluloid" 2 3 class Actor 4 include

    Celluloid 5 attr_reader :counter 6 7 def initialize 8 @counter = 0 9 end 10 11 def increment 12 @counter += 1 13 end 14 end
  34.  CELLULOID ACTORS ARE ACTIVE OBJECTS ISOLATED WITHIN THREADS

  35.  1 require "celluloid" 2 3 class Actor 4 include

    Celluloid 5 end 6 7 actor = Actor.new 8 actor.inspect 9 #=> <Celluloid::ActorProxy(Actor:0x3feaecbb38e0)> 10 11 Thread.main 12 #=> <Thread:0x007f86290b8ce8 run> 13 14 actor.thread 15 #=> <Thread:0x007f862ad27a78 sleep>
  36.  CELLULOID ACTORS MESSAGES ARE BUFFERED IN THE ACTOR’S MAILBOX

  37.  CELLULOID ACTORS MESSAGES ARE PROCESSED SEQUENTIALLY FROM YOUR ACTOR’S

    MAILBOX
  38.  CELLULOID ACTORS NO PATTERN MATCHING FOR BEHAVIOURS JUST REGULAR

    METHODS
  39.  CELLULOID ACTORS DISPATCH TASKS IN FIBERS COOPERATIVE LIGHTWEIGHT USER

    SPACE
  40.  CELLULOID ACTORS CAN DISPATCH SYNCHRONOUSLY

  41.  1 require "celluloid" 2 3 class Actor 4 include

    Celluloid 5 6 def compute_all_the_things 7 sleep 2 8 puts "42" 9 end 10 end 11 12 actor = Actor.new 13 actor.compute_all_the_things 14 puts "done!" #=> 42 #=> done! blocking
  42.  CELLULOID ACTORS CAN DISPATCH ASYNCHRONOUSLY

  43.  1 require "celluloid" 2 3 class Actor 4 include

    Celluloid 5 6 def compute_all_the_things 7 sleep 2 8 puts "42" 9 end 10 end 11 12 actor = Actor.new 13 actor.async.compute_all_the_things 14 puts "done!" 15 16 #=> done! 17 #=> 42 non-blocking
  44.  CELLULOID ACTORS CAN DISPATCH IN FUTURES

  45.  1 require "celluloid" 2 3 class Actor 4 include

    Celluloid 5 6 def compute_all_the_things 7 sleep 2 8 "42" 9 end 10 end 11 12 actor = Actor.new 13 future = actor.future.compute_all_the_things 14 puts "done!" 15 puts future.value 16 17 #=> done! 18 #=> 42 returns immediately blocks until a value is yielded
  46.  CELLULOID ACTORS ARE FAULT TOLERANT LET IT CRASH!

  47.  1 require "celluloid/autostart" 2 3 class Actor 4 include

    Celluloid 5 6 def compute_all_the_things 7 puts "42" 8 end 9 10 def zomg_crash 11 raise "derp!" 12 end 13 end 14 15 supervisor = Actor.supervise_as :foo 16 17 begin 18 Celluloid::Actor[:foo].zomg_crash 19 rescue 20 puts "whoops" 21 end 22 23 Celluloid::Actor[:foo].compute_all_the_things 24 25 #=> whoops 26 #=> 42 crash the actor fresh actor supervise the actor
  48.  CELLULOID ACTORS CAN BE ARRANGED IN POOLS OF WORKERS

  49.  1 require "celluloid" 2 3 class Actor 4 include

    Celluloid 5 6 def compute_all_the_things 7 sleep 1 8 puts "42" 9 end 10 end 11 12 pool = Actor.pool 13 14 4.times { pool.compute_all_the_things } 15 16 #=> 42 17 #=> 42 and so on... size == cores load up the workers
  50.  THERE’S MORE... TIMERS LINKING SUPERVISION GROUPS PUB/SUB CONDITIONS

  51.  CELLULOID OO AND CONCURRENCY SYNC, ASYNC AND FUTURES NO

    DEADLOCKS FAULT TOLERANT SIMPLE TO REASON ABOUT
  52. None
  53. An event-driven IO system for building fast, scalable network applications

    that integrate directly with celluloid actors “
  54.  CELLULOID IO FAST, EVENTED IO BUT WITHOUT CALLBACKS

  55. None
  56.  CELLULOID IO BUILT UPON THE HIGH PERFORMANCE SELECTOR NIO4R

  57.  CELLULOID IO MULTIPLEXES IO ASYNCHRONOUSLY BUT PROVIDES A SYNCHRONOUS

    API
  58.  CELLULOID IO WHY BOTHER? BLOCK ON IO? THE ACTOR

    BLOCKS
  59.  CELLULOID IO PROVIDES DUCK TYPES FOR THE STDLIB SOCKETS

  60. None
  61.  CELLULOID IO SHOULD NOT BE THE DEFAULT APPROACH FOR

    IO...
  62.  CELLULOID IO IDEAL FOR LARGE NUMBERS OF MOSTLY IDLE

    CONNECTIONS
  63. None
  64. A distributed extension to celluloid which provides distributed and concurrent

    objects for ruby that are both robust and fault-tolerant “
  65.  DCELL BROKERLESS MESSAGING TRANSPORT VIA ZEROMQ

  66.  DCELL SIMPLE REGISTRATION SIMPLE DISCOVERY

  67.  1 require 'dcell' 2 3 DCell.start :id => "itchy",

    :addr => "tcp://127.0.0.1:9001" 4 5 class Itchy 6 include Celluloid 7 #... 8 end 9 10 Itchy.supervise_as :itchy start a node make your actor available
  68.  1 require 'dcell' 2 3 DCell.start :id => "scratchy",

    :addr => "tcp://127.0.0.1:9002" 4 itchy_node = DCell::Node["itchy"] 5 itchy_node[:itchy].fight! start a node discover itchy node itchy cell
  69.  DCELL SO MUCH MORE... BUT NOT PRODUCTION READY

  70. None
  71. A fast non-blocking and evented web server. Thanks to celluloid,

    Reel works great for multithreaded applications and provides traditional multithreaded blocking IO support too. “
  72.  REEL NON-BLOCKING OR BLOCKING VIA ACTOR PER CONNECTION

  73.  REEL EXPOSES STANDARD REQUESTS OR WEBSOCKETS

  74.  REEL PROVIDES A “BARE METAL” API

  75. None
  76.  1 require 'reel' 2 3 class MyServer < Reel::Server

    4 def initialize(host = "127.0.0.1", port = 3000) 5 super(host, port, &method(:on_connection)) 6 end 7 8 def on_connection(connection) 9 while request = connection.request 10 if request.websocket? 11 handle_websocket(request) 12 break 13 else 14 handle_request(request) 15 end 16 end 17 end 18 19 def handle_request(request) 20 request.respond :ok, "Hello, world!" 21 end 22 23 def handle_websocket(sock) 24 sock << "Hello everyone out there in WebSocket land!" 25 sock.close 26 end 27 end 28 29 MyServer.run
  77.  REEL PROVIDES AN EXPERIMENTAL RACK ADAPTER

  78. None
  79. @halorgium DERP!

  80. ?