Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Railsconf 2009 Rabbitmq
Search
hungryblank
April 24, 2012
Programming
1
140
Railsconf 2009 Rabbitmq
hungryblank
April 24, 2012
Tweet
Share
More Decks by hungryblank
See All by hungryblank
Rails Underground 2009 RabbitMQ
hungryblank
3
770
Other Decks in Programming
See All in Programming
pnpm に provenance のダウングレード を検出する PR を出してみた
ryo_manba
1
150
TFLintカスタムプラグインで始める Terraformコード品質管理
bells17
2
360
Six and a half ridiculous things to do with Quarkus
hollycummins
0
210
The Past, Present, and Future of Enterprise Java
ivargrimstad
0
600
iOSでSVG画像を扱う
kishikawakatsumi
0
160
AI Agent 時代的開發者生存指南
eddie
4
2.1k
TransformerからMCPまで(現代AIを理解するための羅針盤)
mickey_kubo
7
4.8k
NixOS + Kubernetesで構築する自宅サーバーのすべて
ichi_h3
0
1.2k
Ktorで簡単AIアプリケーション
tsukakei
0
100
Catch Up: Go Style Guide Update
andpad
0
250
Go言語の特性を活かした公式MCP SDKの設計
hond0413
1
490
bootcamp2025_バックエンド研修_WebAPIサーバ作成.pdf
geniee_inc
0
120
Featured
See All Featured
Music & Morning Musume
bryan
46
6.9k
Optimizing for Happiness
mojombo
379
70k
Being A Developer After 40
akosma
91
590k
The Straight Up "How To Draw Better" Workshop
denniskardys
238
140k
Docker and Python
trallard
46
3.6k
Typedesign – Prime Four
hannesfritz
42
2.8k
BBQ
matthewcrist
89
9.8k
Keith and Marios Guide to Fast Websites
keithpitt
411
23k
The Pragmatic Product Professional
lauravandoore
36
7k
The MySQL Ecosystem @ GitHub 2015
samlambert
251
13k
A Tale of Four Properties
chriscoyier
161
23k
The Language of Interfaces
destraynor
162
25k
Transcript
%w(map reduce).first A Tale About Rabbits, Latency and Slim Crontabs
Paolo Negri thanks to: www.autoscout24.de
Summary:
Map http://www.matthiasdittrich.com/projekte/dliste/visualisations/index.html
http://www.flickr.com/photos/myxi/448253580 rabbitMQ
crontab diet http://www.flickr.com/photos/tim_norris/2600843131/
Map Reduce “Programming model for processing and generating large data
sets” (Google)
Map Reduce "Map" step the master node takes the input,
chops it up into smaller sub-problems, and distributes those to worker nodes. (Wikipedia)
The problem Invoicing our clients
Is it as simple as... clients.map do |client| client.invoice end
No! •distributed •concurrent Because the process is:
Problems: •How many nodes? •How many workers? •Distribution mechanism to
feed the workers?
What about queuing? • the master node takes the input,
chops it up into smaller sub-problems, and publishes them in a queue • workers independently consume the content of the queue
Here comes • RabbitMQ is an implementation of AMQP, the
emerging standard for high performance enterprise messaging • It’s opensource • Can be used to manage queues • Written in Erlang
Erlang? • Erlang is a general-purpose concurrent programming language designed
by Ericsson • distributed • fault tolerant • soft real time • high availability
Install it •sudo apt-get install rabbitmq •sudo gem install tmm1-amqp
Do it - master node
Use it - worker node
What and where RabbitMQ (Erlang) TCP/IP Master (ruby) Worker (ruby)
Worker (ruby)
Get for free • Decoupling master/worker • Workers take care
of feeding themselves • Flexible number of workers
Get for free • RabbitMQ can be clustered • Support
of message acknowledgement • Queues can be persisted on disk (at a price) • low latency
Queue • Is an actual entity • has a name
• can be inspected and managed
EventMachine
EventMachine • Non blocking IO and lightweight concurrency • eliminate
the complexities of high- performance threaded network programming Is an implementation of Reactor Pattern
EventMachine
EventMachine amqp gem is built on EventMachine => you’re in
a context where you can leverage concurrent programming
EM - Deferrables
EM - Deferrables “The Deferrable pattern allows you to specify
any number of Ruby code blocks that will be executed at some future time when the status of the Deferrable object changes “
EM - Deferrables
EM - Deferrables
Deferrables ClientStat Arrears without deferrables with deferrables Time ClientStat Arrears
Achieved so far • Easy distribution of tasks • Architecture
that supports arbitrary number of workers (and masters) • Concurrency within the single worker
More rabbits Analogy with email system
Multicasting - producer
Multicasting - consumer
Multicasting Exchange msg A Queue1 Queue3 Queue2 Cons1 Cons2 Cons3
Publisher
Multicasting Exchange Queue1 Queue3 Queue2 msg A msg A msg
A Cons1 Cons2 Cons3 Publisher
Not only queues then • communication across hosts, heterogeneous systems
• low latency • clustering Use messages distribution to build the nervous system of your app
Where to start? crontab -l 5 * * * *
bin/do_the_quick_thing.rb 0 2 * * * bin/do_the_scary_thing.rb
Cron • Simple • Reliable • No maintenance • Status
is not explicit • Locking? • Shot and forget
Queue • Distributed easily • Reliable • Can be inspected
• Add/decrease workers •Makes you think! • Adds more complexity
On github - Projects • eventmachine/eventmachine • tmm1/amqp • macournoyer/thin
• famoseagle/carrot • celldee/bunny • ezmobius/nanite
Q&A ?
Thanks! Paolo Negri / hungryblank