Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Running Jobs at Scale
Search
Kir Shatrov
June 16, 2018
Programming
0
180
Running Jobs at Scale
My talk from GORUCO 2018 in New York.
Kir Shatrov
June 16, 2018
Tweet
Share
More Decks by Kir Shatrov
See All by Kir Shatrov
Operating Rails in Kubernetes
kirs
2
390
RailsClub 2016
kirs
2
300
Performance regressions in Ruby on Rails Core
kirs
0
190
Building a toolkit to detect performance regressions in Ruby on Rails core
kirs
3
5.1k
Развертывание веб-приложений и фреймворк Capistrano
kirs
1
260
Capistrano 3
kirs
4
2.5k
Other Decks in Programming
See All in Programming
HonoのRPCで真の型安全が欲しかった
kosei28
1
160
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
1
130
Jetpack Composeとデザインシステム
rmakiyama
0
240
Using "modern" Ruby to build a better, faster Homebrew
mikemcquaid
2
300
仕様と実装で学ぶOpenTelemetry
drumato
2
1.2k
Slackワークフローで感謝を伝える機能/WiFi 自動接続/Figma to React Component/障害レポート君 Team3@NOT A HOTEL
nakaohiroshi
0
240
TypeScriptとGraphQLで実現する 型安全なAPI実装 / TSKaigi 2024
hokaccha
5
2.9k
株式会社ゼネテック
genetec
0
130
Upgrading Legacy to the Latest PHP Version
afilina
PRO
0
100
The test code generator using static analysis and LLM
mikik0
1
300
RailsConf 2024: Riffing on Rails: sketch your way to better designed code
kaspth
1
220
Open standards for building event-driven applications in the cloud
meteatamel
0
240
Featured
See All Featured
Principles of Awesome APIs and How to Build Them.
keavy
121
16k
Java REST API Framework Comparison - PWX 2021
mraible
PRO
18
7k
The Brand Is Dead. Long Live the Brand.
mthomps
49
31k
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
84
45k
Build The Right Thing And Hit Your Dates
maggiecrowley
25
2k
Build your cross-platform service in a week with App Engine
jlugia
226
17k
Art, The Web, and Tiny UX
lynnandtonic
290
19k
Code Review Best Practice
trishagee
56
15k
jQuery: Nuts, Bolts and Bling
dougneiner
60
7.2k
Optimizing for Happiness
mojombo
371
69k
A Philosophy of Restraint
colly
197
16k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
20
1.8k
Transcript
Running Jobs at Scale Kir Shatrov GORUCO 2018, @kirshatrov
GORUCO 2018, @kirshatrov
GORUCO 2018, @kirshatrov
GORUCO 2018, @kirshatrov
GORUCO 2018, @kirshatrov
Jobs GORUCO 2018, @kirshatrov
class ExampleJob < ActiveJob::Base def perform ... end end GORUCO
2018, @kirshatrov
class ExampleJob < ActiveJob::Base def perform Product.all.find_each do |product| product.sync_and_refresh
end end end GORUCO 2018, @kirshatrov
class ExampleJob < ActiveJob::Base def perform Product.all.find_each do |product| product.sync_and_refresh
end end end minutes? hours? days? GORUCO 2018, @kirshatrov
Long-running jobs GORUCO 2018, @kirshatrov
Long-running jobs — Deploys and termination GORUCO 2018, @kirshatrov
Long-running jobs — Deploys and termination — Abort and re-enqueue
— Progress lost GORUCO 2018, @kirshatrov
GORUCO 2018, @kirshatrov
Long-running jobs — Deploys and termination — Abort and re-enqueue
— Progress lost — Job may never complete GORUCO 2018, @kirshatrov
Long-running jobs — Deploys and termination — Abort and re-enqueue
— Progress lost — Job may never complete — Capacity and worker starvation GORUCO 2018, @kirshatrov
Long-running jobs — Deploys and termination — Abort and re-enqueue
— Progress lost — Job may never complete — Capacity and worker starvation — Cloud ☁ GORUCO 2018, @kirshatrov
Why is it taking long? Because it iterates over a
long collection. GORUCO 2018, @kirshatrov
What if jobs were interruptible and resumable? GORUCO 2018, @kirshatrov
Split the job definition 1. Collection to process 2. Work
to be done GORUCO 2018, @kirshatrov
Split the job definition 1. Collection to process ≫ Product.all
2. Work to be done GORUCO 2018, @kirshatrov
Split the job definition 1. Collection to process ≫ Product.all
2. Work to be done ≫ product.sync_and_refresh GORUCO 2018, @kirshatrov
class ExampleJob < ActiveJob::Base include Iteration def collection Product.all end
def each_iteration(product) product.sync_and_refresh end end GORUCO 2018, @kirshatrov
— def perform — collection — each_iteration GORUCO 2018, @kirshatrov
Product.all cursor: 1 GORUCO 2018, @kirshatrov
Product.all cursor: 2 GORUCO 2018, @kirshatrov
Product.all cursor: 3 GORUCO 2018, @kirshatrov
Product.all cursor: 4 GORUCO 2018, @kirshatrov
Product.all cursor: 5 GORUCO 2018, @kirshatrov
Product.all cursor: 450123 GORUCO 2018, @kirshatrov
class WhateverJob < ActiveJob::Base include Iteration def collection Enumerator.new do
|enum| 3.times do |n| enum << n end end end def each_iteration(n) # do something three times! end end GORUCO 2018, @kirshatrov
Endless possibilities — Interrupt and resume at any moment —
Progress tracking — Parallel computations — Throttling by default GORUCO 2018, @kirshatrov
Benefits for the infrastructure — Keep supporting long-running jobs —
Success for Cloud runtime — Make scale invisible for developers — Opportunities to save money with short-living instances in Cloud GORUCO 2018, @kirshatrov
Thank you! @kirshatrov GORUCO 2018, @kirshatrov