Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Running Jobs at Scale
Search
Kir Shatrov
June 16, 2018
Programming
1
190
Running Jobs at Scale
My talk from GORUCO 2018 in New York.
Kir Shatrov
June 16, 2018
Tweet
Share
More Decks by Kir Shatrov
See All by Kir Shatrov
Operating Rails in Kubernetes
kirs
3
400
RailsClub 2016
kirs
2
300
Performance regressions in Ruby on Rails Core
kirs
0
190
Building a toolkit to detect performance regressions in Ruby on Rails core
kirs
3
5.2k
Развертывание веб-приложений и фреймворк Capistrano
kirs
1
260
Capistrano 3
kirs
4
2.6k
Other Decks in Programming
See All in Programming
CSC307 Lecture 13
javiergs
PRO
0
150
AHC035解説
terryu16
0
710
日付と正規化
megmogmog1965
0
140
継続的な活動で築く地方エンジニアの道
myamashii
2
350
Cloudflare Workers x AWS Lambdaの組み合わせユースケース / Cloudflare Workers x AWS Lambda Combination Use Case
seike460
PRO
2
310
最古の関数型言語「Lisp」ことはじめ / lisp_in_kamiyama
uhooi
1
190
CSC307 Lecture 12
javiergs
PRO
0
220
はしめてのプログラミングとロボット制御
watawatavoltage
0
290
Folding Cheat Sheet #7
philipschwarz
PRO
0
150
APIのない大学ログインWebサービスをWKWebViewとJavaScriptでアプリ化した話
akidon0000
1
330
さきがけから振り返るアーキテクチャ刷新 / Reflecting on the Architectural Renewal from the Vanguard
nrslib
2
770
企業向け生成AIアプリの 開発から得られた知見
takaakikakei
0
310
Featured
See All Featured
Build The Right Thing And Hit Your Dates
maggiecrowley
28
2.2k
A Modern Web Designer's Workflow
chriscoyier
689
190k
Teambox: Starting and Learning
jrom
130
8.6k
A Tale of Four Properties
chriscoyier
155
22k
Infographics Made Easy
chrislema
238
18k
What the flash - Photography Introduction
edds
65
11k
Ruby is Unlike a Banana
tanoku
96
10k
How To Stay Up To Date on Web Technology
chriscoyier
784
250k
The Straight Up "How To Draw Better" Workshop
denniskardys
229
130k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
353
29k
Designing Experiences People Love
moore
136
23k
Optimizing for Happiness
mojombo
373
69k
Transcript
Running Jobs at Scale Kir Shatrov GORUCO 2018, @kirshatrov
GORUCO 2018, @kirshatrov
GORUCO 2018, @kirshatrov
GORUCO 2018, @kirshatrov
GORUCO 2018, @kirshatrov
Jobs GORUCO 2018, @kirshatrov
class ExampleJob < ActiveJob::Base def perform ... end end GORUCO
2018, @kirshatrov
class ExampleJob < ActiveJob::Base def perform Product.all.find_each do |product| product.sync_and_refresh
end end end GORUCO 2018, @kirshatrov
class ExampleJob < ActiveJob::Base def perform Product.all.find_each do |product| product.sync_and_refresh
end end end minutes? hours? days? GORUCO 2018, @kirshatrov
Long-running jobs GORUCO 2018, @kirshatrov
Long-running jobs — Deploys and termination GORUCO 2018, @kirshatrov
Long-running jobs — Deploys and termination — Abort and re-enqueue
— Progress lost GORUCO 2018, @kirshatrov
GORUCO 2018, @kirshatrov
Long-running jobs — Deploys and termination — Abort and re-enqueue
— Progress lost — Job may never complete GORUCO 2018, @kirshatrov
Long-running jobs — Deploys and termination — Abort and re-enqueue
— Progress lost — Job may never complete — Capacity and worker starvation GORUCO 2018, @kirshatrov
Long-running jobs — Deploys and termination — Abort and re-enqueue
— Progress lost — Job may never complete — Capacity and worker starvation — Cloud ☁ GORUCO 2018, @kirshatrov
Why is it taking long? Because it iterates over a
long collection. GORUCO 2018, @kirshatrov
What if jobs were interruptible and resumable? GORUCO 2018, @kirshatrov
Split the job definition 1. Collection to process 2. Work
to be done GORUCO 2018, @kirshatrov
Split the job definition 1. Collection to process ≫ Product.all
2. Work to be done GORUCO 2018, @kirshatrov
Split the job definition 1. Collection to process ≫ Product.all
2. Work to be done ≫ product.sync_and_refresh GORUCO 2018, @kirshatrov
class ExampleJob < ActiveJob::Base include Iteration def collection Product.all end
def each_iteration(product) product.sync_and_refresh end end GORUCO 2018, @kirshatrov
— def perform — collection — each_iteration GORUCO 2018, @kirshatrov
Product.all cursor: 1 GORUCO 2018, @kirshatrov
Product.all cursor: 2 GORUCO 2018, @kirshatrov
Product.all cursor: 3 GORUCO 2018, @kirshatrov
Product.all cursor: 4 GORUCO 2018, @kirshatrov
Product.all cursor: 5 GORUCO 2018, @kirshatrov
Product.all cursor: 450123 GORUCO 2018, @kirshatrov
class WhateverJob < ActiveJob::Base include Iteration def collection Enumerator.new do
|enum| 3.times do |n| enum << n end end end def each_iteration(n) # do something three times! end end GORUCO 2018, @kirshatrov
Endless possibilities — Interrupt and resume at any moment —
Progress tracking — Parallel computations — Throttling by default GORUCO 2018, @kirshatrov
Benefits for the infrastructure — Keep supporting long-running jobs —
Success for Cloud runtime — Make scale invisible for developers — Opportunities to save money with short-living instances in Cloud GORUCO 2018, @kirshatrov
Thank you! @kirshatrov GORUCO 2018, @kirshatrov