Live Long and Prosper with Celluloid & JRuby

In this talk you'll learn about the Celluloid ecosystem. We'll take a glance at:

Celluloid - the OOP & Actor model based concurrency framework
Celluloid::IO - an evented I/O framework
DCell - distributed Actors for Celluloid
Reel - the Celluloid::IO powered web server

I'll cover the Actor model - Celluloid's inspiration for their approach to concurrent objects. How we can best achieve concurrency without mind-bending synchronisation, mutexes, deadlocks and all the bad stuff, in idiomatic and simple to grasp Ruby.

We'll finish up by briefly covering JRuby and why it's my preferred Ruby VM for concurrency and true parallelism with Celluloid.

This talk was given at JRubyconf EU - 2013.

Transcript

1. LIVE LONG AND PROSPER WITH CELLULOID AND JRUBY @benlovell
2. @benlovell
3. @benlovell
5. WE GOT THREADS! REAL THREADS! NO GIL THREADS!
6. WE GOT THREADS! REAL THREADS! NO GIL THREADS!
7. WE GOT THREADS! REAL THREADS! NO GIL THREADS!
8. WE GOT THREADS! BUT HOW DO WE USE THEM EFFECTIVELY?
9. LOCKS SYNCHRONISATION SHARED STATE NON-DETERMINISM UGHHHH!
10. INSANITY: DOING THE SAME THING OVER AND OVER AGAIN AND EXPECTING DIFFERENT RESULTS
11. THREADING: DOING THE SAME THING OVER AND OVER AGAIN AND ACHIEVING DIFFERENT RESULTS
12. THE RULES OF THREADING
13. NUMBER 1...
14. DON’T DO IT!
15. QUESTION: HOW DO SOFTWARE ENGINEERS DEAL WITH HARD PROBLEMS? ?
16. ANSWER: 1. BEER 2. STACKOVERFLOW ?
17. ANSWER: WE ABSTRACT THEM AWAY ?
18. QUESTION: HOW DO WE ABSTRACT AWAY CONCURRENCY PRIMITIVES THREADS AND OTHER MIND BENDING SHIT... ?
19. QUESTION: ... WHILE STILL ENJOYING THE BEAUTIFUL AND IDIOMATIC RUBY WHICH WE ARE ACCUSTOMED TO? ?
21. @bascule @halorgium @benlangfeld LOVINGLY MAINTAINED BY
22. A concurrent object oriented programming framework for Ruby which lets you build multithreaded programs out of concurrent objects just as easily as you build sequential programs out of regular objects “
24.   ACTOR MODEL A LONG ESTABLISHED PATTERN WITH MANY IMPLEMENTATIONS
25.   ACTORS LIGHTWEIGHT PROCESSES THAT POSSESS IDENTITY AND COMMUNICATE BY MESSAGING
26.   ACTORS MAY CREATE OTHER ACTORS BUFFER MESSAGES IN A MAILBOX PROCESS MESSAGES SERIALLY
27.   ACTORS BY VIRTUE NEVER COMPETE FOR LOCKS!
30.  CELLULOID ACTORS AUTOMATICALLY SYNCHRONISE ACCESS TO STATE THEY ENCAPSULATE
31.  1 class Actor 2 attr_reader :counter 3 4 def initialize 5 @counter = 0 6 @mutex = Mutex.new 7 end 8 9 def increment 10 @mutex.synchronize do 11 @counter += 1 12 end 13 end 14 end
32.  WITH CELLULOID...
33.  1 require "celluloid" 2 3 class Actor 4 include Celluloid 5 attr_reader :counter 6 7 def initialize 8 @counter = 0 9 end 10 11 def increment 12 @counter += 1 13 end 14 end
34.  CELLULOID ACTORS ARE ACTIVE OBJECTS ISOLATED WITHIN THREADS
35.  1 require "celluloid" 2 3 class Actor 4 include Celluloid 5 end 6 7 actor = Actor.new 8 actor.inspect 9 #=> <Celluloid::ActorProxy(Actor:0x3feaecbb38e0)> 10 11 Thread.main 12 #=> <Thread:0x007f86290b8ce8 run> 13 14 actor.thread 15 #=> <Thread:0x007f862ad27a78 sleep>
36.  CELLULOID ACTORS MESSAGES ARE BUFFERED IN THE ACTOR’S MAILBOX
37.  CELLULOID ACTORS MESSAGES ARE PROCESSED SEQUENTIALLY FROM YOUR ACTOR’S MAILBOX
38.  CELLULOID ACTORS NO PATTERN MATCHING FOR BEHAVIOURS JUST REGULAR METHODS
39.  CELLULOID ACTORS DISPATCH TASKS IN FIBERS COOPERATIVE LIGHTWEIGHT USER SPACE
40.  CELLULOID ACTORS CAN DISPATCH SYNCHRONOUSLY
41.  1 require "celluloid" 2 3 class Actor 4 include Celluloid 5 6 def compute_all_the_things 7 sleep 2 8 puts "42" 9 end 10 end 11 12 actor = Actor.new 13 actor.compute_all_the_things 14 puts "done!" #=> 42 #=> done! blocking
42.  CELLULOID ACTORS CAN DISPATCH ASYNCHRONOUSLY
43.  1 require "celluloid" 2 3 class Actor 4 include Celluloid 5 6 def compute_all_the_things 7 sleep 2 8 puts "42" 9 end 10 end 11 12 actor = Actor.new 13 actor.async.compute_all_the_things 14 puts "done!" 15 16 #=> done! 17 #=> 42 non-blocking
44.  CELLULOID ACTORS CAN DISPATCH IN FUTURES
45.  1 require "celluloid" 2 3 class Actor 4 include Celluloid 5 6 def compute_all_the_things 7 sleep 2 8 "42" 9 end 10 end 11 12 actor = Actor.new 13 future = actor.future.compute_all_the_things 14 puts "done!" 15 puts future.value 16 17 #=> done! 18 #=> 42 returns immediately blocks until a value is yielded
46.  CELLULOID ACTORS ARE FAULT TOLERANT LET IT CRASH!
47.  1 require "celluloid/autostart" 2 3 class Actor 4 include Celluloid 5 6 def compute_all_the_things 7 puts "42" 8 end 9 10 def zomg_crash 11 raise "derp!" 12 end 13 end 14 15 supervisor = Actor.supervise_as :foo 16 17 begin 18 Celluloid::Actor[:foo].zomg_crash 19 rescue 20 puts "whoops" 21 end 22 23 Celluloid::Actor[:foo].compute_all_the_things 24 25 #=> whoops 26 #=> 42 crash the actor fresh actor supervise the actor
48.  CELLULOID ACTORS CAN BE ARRANGED IN POOLS OF WORKERS
49.  1 require "celluloid" 2 3 class Actor 4 include Celluloid 5 6 def compute_all_the_things 7 sleep 1 8 puts "42" 9 end 10 end 11 12 pool = Actor.pool 13 14 4.times { pool.compute_all_the_things } 15 16 #=> 42 17 #=> 42 and so on... size == cores load up the workers
50.  THERE’S MORE... TIMERS LINKING SUPERVISION GROUPS PUB/SUB CONDITIONS
51.  CELLULOID OO AND CONCURRENCY SYNC, ASYNC AND FUTURES NO DEADLOCKS FAULT TOLERANT SIMPLE TO REASON ABOUT
53. An event-driven IO system for building fast, scalable network applications that integrate directly with celluloid actors “
54.  CELLULOID IO FAST, EVENTED IO BUT WITHOUT CALLBACKS
56.  CELLULOID IO BUILT UPON THE HIGH PERFORMANCE SELECTOR NIO4R
57.  CELLULOID IO MULTIPLEXES IO ASYNCHRONOUSLY BUT PROVIDES A SYNCHRONOUS API
58.  CELLULOID IO WHY BOTHER? BLOCK ON IO? THE ACTOR BLOCKS
59.  CELLULOID IO PROVIDES DUCK TYPES FOR THE STDLIB SOCKETS
61.  CELLULOID IO SHOULD NOT BE THE DEFAULT APPROACH FOR IO...
62.  CELLULOID IO IDEAL FOR LARGE NUMBERS OF MOSTLY IDLE CONNECTIONS
64. A distributed extension to celluloid which provides distributed and concurrent objects for ruby that are both robust and fault-tolerant “
65.  DCELL BROKERLESS MESSAGING TRANSPORT VIA ZEROMQ
66.  DCELL SIMPLE REGISTRATION SIMPLE DISCOVERY
67.  1 require 'dcell' 2 3 DCell.start :id => "itchy", :addr => "tcp://127.0.0.1:9001" 4 5 class Itchy 6 include Celluloid 7 #... 8 end 9 10 Itchy.supervise_as :itchy start a node make your actor available
68.  1 require 'dcell' 2 3 DCell.start :id => "scratchy", :addr => "tcp://127.0.0.1:9002" 4 itchy_node = DCell::Node["itchy"] 5 itchy_node[:itchy].fight! start a node discover itchy node itchy cell
69.  DCELL SO MUCH MORE... BUT NOT PRODUCTION READY
71. A fast non-blocking and evented web server. Thanks to celluloid, Reel works great for multithreaded applications and provides traditional multithreaded blocking IO support too. “
72.  REEL NON-BLOCKING OR BLOCKING VIA ACTOR PER CONNECTION
73.  REEL EXPOSES STANDARD REQUESTS OR WEBSOCKETS
74.  REEL PROVIDES A “BARE METAL” API
76.  1 require 'reel' 2 3 class MyServer < Reel::Server 4 def initialize(host = "127.0.0.1", port = 3000) 5 super(host, port, &method(:on_connection)) 6 end 7 8 def on_connection(connection) 9 while request = connection.request 10 if request.websocket? 11 handle_websocket(request) 12 break 13 else 14 handle_request(request) 15 end 16 end 17 end 18 19 def handle_request(request) 20 request.respond :ok, "Hello, world!" 21 end 22 23 def handle_websocket(sock) 24 sock << "Hello everyone out there in WebSocket land!" 25 sock.close 26 end 27 end 28 29 MyServer.run
77.  REEL PROVIDES AN EXPERIMENTAL RACK ADAPTER
79. @halorgium DERP!
80. ?