Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Running Jobs at Scale
Search
Kir Shatrov
June 16, 2018
Programming
1
180
Running Jobs at Scale
My talk from GORUCO 2018 in New York.
Kir Shatrov
June 16, 2018
Tweet
Share
More Decks by Kir Shatrov
See All by Kir Shatrov
Operating Rails in Kubernetes
kirs
3
390
RailsClub 2016
kirs
2
300
Performance regressions in Ruby on Rails Core
kirs
0
190
Building a toolkit to detect performance regressions in Ruby on Rails core
kirs
3
5.1k
Развертывание веб-приложений и фреймворк Capistrano
kirs
1
260
Capistrano 3
kirs
4
2.5k
Other Decks in Programming
See All in Programming
RubyKaigiでおいしいピクルスになろう! / Become a Delicious Pickle at RubyKaigi!
unikounio
0
5.6k
自作HTTPルーターから新しいServeMuxへ
bmf_san
3
1.2k
DatadogでAPI毎のSQL発行数を可視化してN+1を改善した話
nealle
0
490
Ruby Function Composition
bkuhlmann
1
370
Project Loom 概要と Scala における有用性 @ ScalaMatsuri2024
th0rikosh1
0
230
ライブラリをパブリッシュせずにすばやく試す
exoego
2
120
来週から実践できる! LayerXのバクラク事業部で行われている Go 関連の読書会のご紹介 / Bakuraku Go Reading Group
upamune
5
330
Make CPU #3 dalance
dalance
1
440
Pure GoでアニメーションGIFのリサイズを実装する
logica0419
0
230
詳解 "Fixing For Loops in Go 1.22" 自作linterをgolangci-lintへコントリビュートした話
qualiarts
7
680
AIで生成したものをAIでチェックしてる話
ippey
0
170
DevTools と デバッグ と 私
kozy4324
1
510
Featured
See All Featured
Making the Leap to Tech Lead
cromwellryan
126
8.6k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
139
42k
YesSQL, Process and Tooling at Scale
rocio
165
14k
The Cost Of JavaScript in 2023
addyosmani
24
4.1k
5 minutes of I Can Smell Your CMS
philhawksworth
199
19k
Typedesign – Prime Four
hannesfritz
36
2.2k
Music & Morning Musume
bryan
42
5.7k
Building Better People: How to give real-time feedback that sticks.
wjessup
356
18k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
353
28k
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
kevinliebholz
10
3.6k
A Modern Web Designer's Workflow
chriscoyier
689
190k
Six Lessons from altMBA
skipperchong
22
3.1k
Transcript
Running Jobs at Scale Kir Shatrov GORUCO 2018, @kirshatrov
GORUCO 2018, @kirshatrov
GORUCO 2018, @kirshatrov
GORUCO 2018, @kirshatrov
GORUCO 2018, @kirshatrov
Jobs GORUCO 2018, @kirshatrov
class ExampleJob < ActiveJob::Base def perform ... end end GORUCO
2018, @kirshatrov
class ExampleJob < ActiveJob::Base def perform Product.all.find_each do |product| product.sync_and_refresh
end end end GORUCO 2018, @kirshatrov
class ExampleJob < ActiveJob::Base def perform Product.all.find_each do |product| product.sync_and_refresh
end end end minutes? hours? days? GORUCO 2018, @kirshatrov
Long-running jobs GORUCO 2018, @kirshatrov
Long-running jobs — Deploys and termination GORUCO 2018, @kirshatrov
Long-running jobs — Deploys and termination — Abort and re-enqueue
— Progress lost GORUCO 2018, @kirshatrov
GORUCO 2018, @kirshatrov
Long-running jobs — Deploys and termination — Abort and re-enqueue
— Progress lost — Job may never complete GORUCO 2018, @kirshatrov
Long-running jobs — Deploys and termination — Abort and re-enqueue
— Progress lost — Job may never complete — Capacity and worker starvation GORUCO 2018, @kirshatrov
Long-running jobs — Deploys and termination — Abort and re-enqueue
— Progress lost — Job may never complete — Capacity and worker starvation — Cloud ☁ GORUCO 2018, @kirshatrov
Why is it taking long? Because it iterates over a
long collection. GORUCO 2018, @kirshatrov
What if jobs were interruptible and resumable? GORUCO 2018, @kirshatrov
Split the job definition 1. Collection to process 2. Work
to be done GORUCO 2018, @kirshatrov
Split the job definition 1. Collection to process ≫ Product.all
2. Work to be done GORUCO 2018, @kirshatrov
Split the job definition 1. Collection to process ≫ Product.all
2. Work to be done ≫ product.sync_and_refresh GORUCO 2018, @kirshatrov
class ExampleJob < ActiveJob::Base include Iteration def collection Product.all end
def each_iteration(product) product.sync_and_refresh end end GORUCO 2018, @kirshatrov
— def perform — collection — each_iteration GORUCO 2018, @kirshatrov
Product.all cursor: 1 GORUCO 2018, @kirshatrov
Product.all cursor: 2 GORUCO 2018, @kirshatrov
Product.all cursor: 3 GORUCO 2018, @kirshatrov
Product.all cursor: 4 GORUCO 2018, @kirshatrov
Product.all cursor: 5 GORUCO 2018, @kirshatrov
Product.all cursor: 450123 GORUCO 2018, @kirshatrov
class WhateverJob < ActiveJob::Base include Iteration def collection Enumerator.new do
|enum| 3.times do |n| enum << n end end end def each_iteration(n) # do something three times! end end GORUCO 2018, @kirshatrov
Endless possibilities — Interrupt and resume at any moment —
Progress tracking — Parallel computations — Throttling by default GORUCO 2018, @kirshatrov
Benefits for the infrastructure — Keep supporting long-running jobs —
Success for Cloud runtime — Make scale invisible for developers — Opportunities to save money with short-living instances in Cloud GORUCO 2018, @kirshatrov
Thank you! @kirshatrov GORUCO 2018, @kirshatrov