Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Sidekiq
Search
Tymon Tobolski
July 01, 2015
Programming
1
120
Sidekiq
Tymon Tobolski
July 01, 2015
Tweet
Share
More Decks by Tymon Tobolski
See All by Tymon Tobolski
Only possible with Elixir - ubots Case Study
teamon
0
180
Fun with Elixir Macros
teamon
1
380
Elixir GenStage & Flow
teamon
2
840
Elixir - Bydgoszcz Web Development Meetup
teamon
2
590
Git - Monterail style
teamon
1
130
Rails Assets wroc_love.rb
teamon
1
630
Angular replacements for jQuery-based libraries
teamon
1
260
Angular replacements for jQuery-based libraries
teamon
2
290
Rails Assets LRUG
teamon
0
7.2k
Other Decks in Programming
See All in Programming
サイコロで理解する統計的仮説検定の考え方
tatamiya
4
950
Amazon SQSコンシューマー疎結合への旅 - 出張! #DevelopersIO IT技術ブログの中の人が語る勉強会 #3
quiver
0
270
効率化に挑戦してみたらモバイル開発が少し快適になった話
ryunakayama
0
130
DMMプラットフォームがTiDB Cloudを採用した背景
pospome
9
4.1k
検証も兼ねて個人開発でHonoとかと向き合った話
hanetsuki
1
1.1k
Apache Hive 4 on Treasure Data
ryukobayashi
0
340
Random\Randomizer クラスで日常のあれこれを解決しよう! / Random\Randomizer class solves familiar trouble
cocoeyes02
0
250
Hanami and htmx
bkuhlmann
0
210
使ってみよう Azure AI Document Intelligence
kosmosebi
2
320
PHPの次期バージョンはこの時期どうなっているのか - Internalsの開発体制について - PHPカンファレンス小田原
youkidearitai
PRO
1
190
Elm Form Validation
bkuhlmann
0
510
VS Code をプロダクトにどう取り込むか
onomax
1
370
Featured
See All Featured
Practical Orchestrator
shlominoach
182
9.7k
Agile that works and the tools we love
rasmusluckow
325
20k
Art, The Web, and Tiny UX
lynnandtonic
289
19k
Done Done
chrislema
178
15k
Documentation Writing (for coders)
carmenintech
60
3.9k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
244
20k
Rebuilding a faster, lazier Slack
samanthasiow
73
8.2k
Fireside Chat
paigeccino
21
2.6k
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
322
20k
KATA
mclloyd
15
12k
Side Projects
sachag
451
41k
Reflections from 52 weeks, 52 projects
jeffersonlam
345
19k
Transcript
SIDEKIQ BACKGROUND JOBS IN RUBY © Tymon Tobolski, 2015
WHAT & WHY © Tymon Tobolski, 2015
class CommentsController < ApplicationController def create @comment = @post.comments.create!(comment_params) Notifications.notify_commenters(@comment)
head :ok end end © Tymon Tobolski, 2015
class Notifications def self.notify_commenters(comment) comment.post.comments_authors.each do |user| send_email_about_new_comment(user, comment) end
end end © Tymon Tobolski, 2015
SYNCHRONOUS DELIVERY class CommentsController < ApplicationController def create @comment =
@post.comments.create!(comment_params) # T = 10ms Notifications.notify_commenters(@comment) # T = 100.000ms, boom, timeout! head :ok end end class Notifications def self.notify_commenters(comment) comment.post.comments_authors.each do |user| send_email_about_new_comment(user, comment) # T += 1000ms end end end © Tymon Tobolski, 2015
ASYNCHRONOUS DELIVERY class CommentsController < ApplicationController def create @comment =
@post.comments.create!(comment_params) # T = 10ms NotifyCommentersWorker.perform_async(@comment.id) # T = 11ms, \o/ head :ok end end class Notifications def self.notify_commenters(comment) comment.post.comments_authors.each do |user| send_email_about_new_comment(user, comment) # T += 1000ms still, but we don't care anymore end end end © Tymon Tobolski, 2015
SIDEKIQ WORKER # app/workers/notify_commenters_worker.rb class NotifyCommentersWorker include Sidekiq::Worker def perform(comment_id)
comment = Comment.find(comment_id) Notifications.notify_commenters(comment) end end © Tymon Tobolski, 2015
MAGIC? © Tymon Tobolski, 2015
REDIS © Tymon Tobolski, 2015
REDIS.IO Redis is an open source, BSD licensed, advanced key-value
cache and store. It is often referred to as a data structure server since keys can contain strings, hashes, lists, sets, sorted sets, bitmaps and hyperloglogs, blah blah blah ... © Tymon Tobolski, 2015
REDIS.IO Redis is an open source, BSD licensed, advanced key-value
cache and store. It is often referred to as a data structure server since keys can contain strings, hashes, lists, sets, sorted sets, bitmaps and hyperloglogs, blah blah blah ... © Tymon Tobolski, 2015
LIST IN REDIS WORLD ... is really a list and
an array and a stack and a queue © Tymon Tobolski, 2015
QUEUE FIFO - FIRST IN, FIRST OUT ACTION QUEUE STATUS
--------------------------- [] PUSH 1 [1] PUSH 2 [2,1] PUSH 3 [3,2,1] POP [3,2] PUSH 4 [4,3,2] POP [4,3] POP [4] © Tymon Tobolski, 2015
QUEUE FIFO - FIRST IN, FIRST OUT ACTION mylist CONTENT
-------------------------------- (nil) LPUSH mylist 1 [1] LPUSH mylist 2 [2,1] LPUSH mylist 3 [3,2,1] RPOP mylist [3,2] LPUSH mylist 4 [4,3,2] RPOP mylist [4,3] RPOP mylist [4] © Tymon Tobolski, 2015
FLOW © Tymon Tobolski, 2015
RULES Sidekiq Best Practices © Tymon Tobolski, 2015
ALWAYS USE IDS BAD: NotifyCommentersWorker.perform_async(@comment) GOOD: NotifyCommentersWorker.perform_async(@comment.id) Job arguments MUST
be serializable to JSON def perform(comment_id) comment = Comment.find(comment_id) # ... end © Tymon Tobolski, 2015
IDEMPOTENT AND TRANSACTIONAL JOBS Sidekiq will execute your job at
least once. def perform(...) # ... User.where(id: id1).update_all(["balance = balance + ?", amount]) User.where(id: id2).update_all(["balance = balance - ?", amount]) # ... end © Tymon Tobolski, 2015
IDEMPOTENT AND TRANSACTIONAL JOBS Sidekiq will execute your job at
least once. def perform(...) # ... ActiveRecord.transation do User.where(id: id1).update_all(["balance = balance + ?", amount]) User.where(id: id2).update_all(["balance = balance - ?", amount]) end # ... end © Tymon Tobolski, 2015
CONCURRENCY & PARALLELISM def process(...) User.find_each do |user| send_email_to(user) end
end © Tymon Tobolski, 2015
CONCURRENCY & PARALLELISM def process(...) User.select(:id).find_each do |user| SendEmailWorker.perform_async(user.id) end
end © Tymon Tobolski, 2015
FOREMAN / HEROKU # Procfile web: bin/rails server -p $PORT
worker: bin/sidekiq © Tymon Tobolski, 2015
SIDEKIQ WEB © Tymon Tobolski, 2015
SIDEKIQ WEB # Gemfile gem 'sinatra', :require => nil #
config/routes.rb require 'sidekiq/web' authenticate :user, lambda { |u| u.admin? } do mount Sidekiq::Web => '/sidekiq' end © Tymon Tobolski, 2015
QUESTIONS? © Tymon Tobolski, 2015