Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
RubyConf.ph 2014 - ZOMGSCALE! With Celluloid & ...
Search
Ben Lovell
March 29, 2014
Programming
0
350
RubyConf.ph 2014 - ZOMGSCALE! With Celluloid & JRuby
Ben Lovell
March 29, 2014
Tweet
Share
More Decks by Ben Lovell
See All by Ben Lovell
FOSS like a BOSS!
benlovell
0
250
RESCUE SQUAD: Rails Edition - Ancient City Ruby 2015
benlovell
0
230
1UP! Level up your development career
benlovell
1
370
Rescue Squad: Rails Edition - Rails Israel 2014
benlovell
2
310
Fast, testable and sane JSON-APIs with Rails-API et al
benlovell
3
450
Fast, testable and sane APIs - RubyC.eu Kiev 2014
benlovell
3
290
Fast, testable and sane APIs - Ancient City Ruby 2014
benlovell
2
300
Gophers! Go, google's open source language - WXG Guildford 2013
benlovell
1
410
ZOMGscale! with Celluloid & JRuby - RubyShift 2013 Kiev
benlovell
5
730
Other Decks in Programming
See All in Programming
Amazon Q CLI開発で学んだAIコーディングツールの使い方
licux
3
170
なぜ今、Terraformの本を書いたのか? - 著者陣に聞く!『Terraformではじめる実践IaC』登壇資料
fufuhu
3
390
ZeroETLで始めるDynamoDBとS3の連携
afooooil
0
150
バイブコーディング超えてバイブデプロイ〜CloudflareMCPで実現する、未来のアプリケーションデリバリー〜
azukiazusa1
3
780
Flutterと Vibe Coding で個人開発!
hyshu
1
220
iOS開発スターターキットの作り方
akidon0000
0
230
構造化・自動化・ガードレール - Vibe Coding実践記 -
tonegawa07
0
170
抽象化という思考のツール - 理解と活用 - / Abstraction-as-a-Tool-for-Thinking
shin1x1
1
930
AI Ramen Fight
yusukebe
0
120
SQLアンチパターン第2版 データベースプログラミングで陥りがちな失敗とその対策 / Intro to SQL Antipatterns 2nd
twada
PRO
36
11k
AIに安心して任せるためにTypeScriptで一意な型を作ろう
arfes0e2b3c
0
330
それ CLI フレームワークがなくてもできるよ / Building CLI Tools Without Frameworks
orgachem
PRO
17
3.6k
Featured
See All Featured
A designer walks into a library…
pauljervisheath
207
24k
BBQ
matthewcrist
89
9.8k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
49
2.9k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
251
21k
Embracing the Ebb and Flow
colly
86
4.8k
Connecting the Dots Between Site Speed, User Experience & Your Business [WebExpo 2025]
tammyeverts
8
420
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
34
3.1k
Building a Modern Day E-commerce SEO Strategy
aleyda
43
7.4k
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
marcduiker
30
2.2k
Raft: Consensus for Rubyists
vanstee
140
7k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
507
140k
Unsuck your backbone
ammeep
671
58k
Transcript
zomgscale! with Celluloid and JRuby Ben Lovell
benlovell _j
113581334398839860922 !
None
None
None
“I HAVE KILLED, AND I WILL KILL AGAIN”
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
HEY LADIES
❤️
None
None
zomgscale! with Celluloid and JRuby Ben Lovell
Moore’s Law
None
I don’t like bungee jumping... ! ...but I do like
skiing Roger Moore
None
every few years CPU clocks have doubled
but recently this growth has stalled
cores ++++++++
the free lunch is over Herb Sutter
harness the POWER in those cores
?
concurrency parallelism
so there’s a difference?
concurrent!
parallel?
None
processes or threads
processes memory constraints communication x cores == x processes?
processes what about fork(2) and CoW friendly GC ?
not CoW friendly
threads shared state locks and lock granularity race conditions can
be hard to reason about
zomg! I love multithreaded code - NOBODY EVER
None
None
so what’s up with MRI?
well, nothing but...
GIL
some things the GIL is responsible for...
San Francisco Bay oil spillage
...maritime disasters
THANKS, GIL!
GIL
MRI not so bad if you’re I/O bound
MRI but what about computation?
meh. thread-level parallelism is available right now! ...just not with
MRI
None
rubinius 2.0.0 due for release 2042 OLD SLIDE ALERT!
so now that we have truly parallel threads is the
problem solved?
! ! rules of threading
Don’t do it!
If you must do it don’t share data across threads
If you must share data across threads don’t share mutable
data
If you must share mutable data across threads synchronise access
to this data
don’t communicate by sharing memory... ! ...share memory by communicating
go
None
painless multithreaded programming for ruby
Tony Arcieri Tim Carey-Smith Ben Langfeld @bascule @halorgium @benlangfeld The
Maintainers
None
None
a concurrent object oriented programming framework which lets you build
multithreaded programs out of concurrent objects just as easily as you build sequential programs out of regular objects
None
based upon the actor model
actor model first proposed way back in 1970
actor model actors are isolated within lightweight processes
actor model actors possess identity
actor model absolutely no shared state
actor model actors don’t need to compete for locks
actor model are sent messages asynchronously
actor model messages are buffered by a mailbox
actor model the actor works off each message sequentially
actor model has implementations in many languages
None
None
None
None
celluloid actors automatically synchronize state
1 class Actor! 2 attr_reader :counter! 3 ! 4 def
initialize! 5 @counter = 0! 6 @mutex = Mutex.new! 7 end! 8 ! 9 def increment! 10 @mutex.synchronize do! 11 @counter += 1! 12 end! 13 end! 14 end!
with celluloid the same example...
1 require "celluloid"! 2 ! 3 class Actor! 4 include
Celluloid! 5 attr_reader :counter! 6 ! 7 def initialize! 8 @counter = 0! 9 end! 10 ! 11 def increment! 12 @counter += 1! 13 end! 14 end!
None
celluloid actors are active objects living within threads
1 require "celluloid"! 2 ! 3 class Actor! 4 include
Celluloid! 5 end! 6 ! 7 actor = Actor.new! 8 actor.inspect! 9 #=> <Celluloid::ActorProxy(Actor:0x3feaecbb38e0)>! 10 ! 11 Thread.main! 12 #=> <Thread:0x007f86290b8ce8 run>! 13 ! 14 actor.thread! 15 #=> <Thread:0x007f862ad27a78 sleep>!
1 module Celluloid! 2 module ClassMethods! 3 # Create a
new actor! 4 def new(*args, &block)! 5 proxy = Actor.new(allocate, actor_options).proxy! 6 proxy._send_(:initialize, *args, &block)! 7 proxy! 8 end! 9 #...! 10 end! 11 #...! 12 end!
celluloid actors messages you send are buffered via the actor’s
mailbox...
celluloid actors ... until the actor is ready to act
upon them
______________! < ETOOMANYACTS >! --------------! \ ^__^! \ (oo)\_______! (__)\
)\/\! ||----w |! || ||! !
celluloid actors no pattern matching just regular messages
celluloid actors poll their mailbox in a message loop
1 class Actor! 2 # Wrap the given subject with
an Actor! 3 def initialize(subject, options = {})! 4 @subject = subject! 5 @mailbox = options[:mailbox] || Mailbox.new! 6 @running = true! 7 ! 8 @thread = ThreadHandle.new(:actor) do! 9 setup_thread! 10 run! 11 end! 12 #...! 13 end! 14 #...! 15 end!
1 class Actor! 2 def run! 3 #...! 4 while
@running! 5 if message = @mailbox.receive(timeout_interval)! 6 handle_message message! 7 else! 8 # No message indicates a timeout! 9 @timers.fire! 10 @receivers.fire_timers! 11 end! 12 end! 13 #...! 14 shutdown! 15 end! 16 end!
celluloid actors act upon messages sequentially
what about ordering? no guarantees
celluloid actors dispatch calls within fibers
fibers? cooperative lightweight user space some gotchas...
celluloid actors can dispatch synchronously
1 require "celluloid"! 2 ! 3 class Actor! 4 include
Celluloid! 5 ! 6 def compute_all_the_things! 7 sleep 2! 8 puts "42"! 9 end! 10 end! 11 ! 12 actor = Actor.new! 13 actor.compute_all_the_things! 14 puts "done!"! ! #=> 42! #=> done!! blocking
celluloid actors can dispatch asynchronously
1 require "celluloid"! 2 ! 3 class Actor! 4 include
Celluloid! 5 ! 6 def compute_all_the_things! 7 sleep 2! 8 puts "42"! 9 end! 10 end! 11 ! 12 actor = Actor.new! 13 actor.async.compute_all_the_things! 14 puts "done!"! 15 ! 16 #=> done!! 17 #=> 42! returns immediately
celluloid actors can perform tasks in futures
1 require "celluloid"! 2 ! 3 class Actor! 4 include
Celluloid! 5 ! 6 def compute_all_the_things! 7 sleep 2! 8 "42"! 9 end! 10 end! 11 ! 12 actor = Actor.new! 13 future = actor.future.compute_all_the_things! 14 puts "done!"! 15 puts future.value! 16 ! 17 #=> done!! 18 #=> 42! returns immediately blocks until a value is yielded
celluloid actors are accessible by reference or name
1 require "celluloid"! 2 ! 3 class Actor! 4 include
Celluloid! 5 ! 6 def compute_all_the_things! 7 sleep 2! 8 puts "42"! 9 end! 10 end! 11 ! 12 actor = Actor.new! 13 Celluloid::Actor[:foo] = actor! 14 ! 15 actor.inspect! 16 #=> <Celluloid::ActorProxy(Actor:0x3feb3ec11308)>! 17 Celluloid::Actor[:foo].inspect! 18 #=> <Celluloid::ActorProxy(Actor:0x3feb3ec11308)>!
celluloid actors are fault tolerant ... let it crash!
1 require "celluloid/autostart"! 2 ! 3 class Actor! 4 include
Celluloid! 5 ! 6 def compute_all_the_things! 7 puts "42"! 8 end! 9 ! 10 def zomg_crash! 11 raise "derp!"! 12 end! 13 end! 14 ! 15 supervisor = Actor.supervise_as :foo! 16 ! 17 begin! 18 Celluloid::Actor[:foo].zomg_crash! 19 rescue! 20 puts "whoops"! 21 end! 22 ! 23 Celluloid::Actor[:foo].compute_all_the_things! 24 ! 25 #=> whoops! 26 #=> 42! crash the actor fresh actor take care of me!
celluloid actors can be arranged as pooled workers
1 require "celluloid"! 2 ! 3 class Actor! 4 include
Celluloid! 5 ! 6 def compute_all_the_things! 7 sleep 1! 8 puts "42"! 9 end! 10 end! 11 ! 12 pool = Actor.pool! 13 ! 14 4.times { pool.compute_all_the_things }! 15 ! 16 #=> 42! 17 #=> 42 and so on...! size*cores load up the workers
there’s more timers links supervision groups pub/sub conditions
that low hanging fruit? yeah, about that...
but there is one tip! blocking I/O... don’t
None
an event-driven IO system for building fast, scalable network applications
that integrate directly with celluloid actors
None
unlike certain other evented I/O systems which limit you to
a single event loop per process Celluloid::IO lets you make as many actors as you want system resources permitting
None
None
a distributed extension to celluloid which provides distributed and concurrent
objects for ruby that are both robust and fault-tolerant
None
a fast non-blocking and evented web server. Thanks to celluloid,
Reel works great for multithreaded applications and provides traditional multithreaded blocking IO support too.
EXPERIMENTAL or broken as it is known outside of OSS
to summarise...
the future of ruby concurrency and parallelism?
thanks! @benlovell ?