Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Railsconf 2009 Rabbitmq
Search
hungryblank
April 24, 2012
Programming
1
140
Railsconf 2009 Rabbitmq
hungryblank
April 24, 2012
Tweet
Share
More Decks by hungryblank
See All by hungryblank
Rails Underground 2009 RabbitMQ
hungryblank
3
770
Other Decks in Programming
See All in Programming
組込みだけじゃない!TinyGo で始める無料クラウド開発入門
otakakot
0
270
Cursorハンズオン実践!
eltociear
2
1.1k
NixOS + Kubernetesで構築する自宅サーバーのすべて
ichi_h3
0
840
Claude CodeによるAI駆動開発の実践 〜そこから見えてきたこれからのプログラミング〜
iriikeita
0
240
After go func(): Goroutines Through a Beginner’s Eye
97vaibhav
0
390
Introducing ReActionView: A new ActionView-Compatible ERB Engine @ Kaigi on Rails 2025, Tokyo, Japan
marcoroth
3
1k
高度なUI/UXこそHotwireで作ろう Kaigi on Rails 2025
naofumi
4
4.1k
はじめてのDSPy - 言語モデルを『プロンプト』ではなく『プログラミング』するための仕組み
masahiro_nishimi
2
480
Serena MCPのすすめ
wadakatu
4
1k
Foundation Modelsを実装日本語学習アプリを作ってみた!
hypebeans
0
110
Go言語はstack overflowの夢を見るか?
logica0419
0
330
AI Coding Meetup #3 - 導入セッション / ai-coding-meetup-3
izumin5210
0
3.3k
Featured
See All Featured
Rebuilding a faster, lazier Slack
samanthasiow
84
9.2k
Intergalactic Javascript Robots from Outer Space
tanoku
273
27k
VelocityConf: Rendering Performance Case Studies
addyosmani
332
24k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
37
2.6k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
162
15k
Gamification - CAS2011
davidbonilla
81
5.5k
GraphQLの誤解/rethinking-graphql
sonatard
73
11k
Performance Is Good for Brains [We Love Speed 2024]
tammyeverts
12
1.2k
Designing Dashboards & Data Visualisations in Web Apps
destraynor
231
53k
Unsuck your backbone
ammeep
671
58k
Java REST API Framework Comparison - PWX 2021
mraible
34
8.9k
YesSQL, Process and Tooling at Scale
rocio
173
14k
Transcript
%w(map reduce).first A Tale About Rabbits, Latency and Slim Crontabs
Paolo Negri thanks to: www.autoscout24.de
Summary:
Map http://www.matthiasdittrich.com/projekte/dliste/visualisations/index.html
http://www.flickr.com/photos/myxi/448253580 rabbitMQ
crontab diet http://www.flickr.com/photos/tim_norris/2600843131/
Map Reduce “Programming model for processing and generating large data
sets” (Google)
Map Reduce "Map" step the master node takes the input,
chops it up into smaller sub-problems, and distributes those to worker nodes. (Wikipedia)
The problem Invoicing our clients
Is it as simple as... clients.map do |client| client.invoice end
No! •distributed •concurrent Because the process is:
Problems: •How many nodes? •How many workers? •Distribution mechanism to
feed the workers?
What about queuing? • the master node takes the input,
chops it up into smaller sub-problems, and publishes them in a queue • workers independently consume the content of the queue
Here comes • RabbitMQ is an implementation of AMQP, the
emerging standard for high performance enterprise messaging • It’s opensource • Can be used to manage queues • Written in Erlang
Erlang? • Erlang is a general-purpose concurrent programming language designed
by Ericsson • distributed • fault tolerant • soft real time • high availability
Install it •sudo apt-get install rabbitmq •sudo gem install tmm1-amqp
Do it - master node
Use it - worker node
What and where RabbitMQ (Erlang) TCP/IP Master (ruby) Worker (ruby)
Worker (ruby)
Get for free • Decoupling master/worker • Workers take care
of feeding themselves • Flexible number of workers
Get for free • RabbitMQ can be clustered • Support
of message acknowledgement • Queues can be persisted on disk (at a price) • low latency
Queue • Is an actual entity • has a name
• can be inspected and managed
EventMachine
EventMachine • Non blocking IO and lightweight concurrency • eliminate
the complexities of high- performance threaded network programming Is an implementation of Reactor Pattern
EventMachine
EventMachine amqp gem is built on EventMachine => you’re in
a context where you can leverage concurrent programming
EM - Deferrables
EM - Deferrables “The Deferrable pattern allows you to specify
any number of Ruby code blocks that will be executed at some future time when the status of the Deferrable object changes “
EM - Deferrables
EM - Deferrables
Deferrables ClientStat Arrears without deferrables with deferrables Time ClientStat Arrears
Achieved so far • Easy distribution of tasks • Architecture
that supports arbitrary number of workers (and masters) • Concurrency within the single worker
More rabbits Analogy with email system
Multicasting - producer
Multicasting - consumer
Multicasting Exchange msg A Queue1 Queue3 Queue2 Cons1 Cons2 Cons3
Publisher
Multicasting Exchange Queue1 Queue3 Queue2 msg A msg A msg
A Cons1 Cons2 Cons3 Publisher
Not only queues then • communication across hosts, heterogeneous systems
• low latency • clustering Use messages distribution to build the nervous system of your app
Where to start? crontab -l 5 * * * *
bin/do_the_quick_thing.rb 0 2 * * * bin/do_the_scary_thing.rb
Cron • Simple • Reliable • No maintenance • Status
is not explicit • Locking? • Shot and forget
Queue • Distributed easily • Reliable • Can be inspected
• Add/decrease workers •Makes you think! • Adds more complexity
On github - Projects • eventmachine/eventmachine • tmm1/amqp • macournoyer/thin
• famoseagle/carrot • celldee/bunny • ezmobius/nanite
Q&A ?
Thanks! Paolo Negri / hungryblank