Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up for free
Running Jobs at Scale
Kir Shatrov
June 16, 2018
Programming
0
170
Running Jobs at Scale
My talk from GORUCO 2018 in New York.
Kir Shatrov
June 16, 2018
Tweet
Share
More Decks by Kir Shatrov
See All by Kir Shatrov
Operating Rails in Kubernetes
kirs
2
350
RailsClub 2016
kirs
2
280
Performance regressions in Ruby on Rails Core
kirs
0
180
Building a toolkit to detect performance regressions in Ruby on Rails core
kirs
3
4.3k
Развертывание веб-приложений и фреймворк Capistrano
kirs
1
240
Capistrano 3
kirs
4
2.3k
Other Decks in Programming
See All in Programming
Meet Swift Regex
usamik26
0
370
What's new in Android development tools まとめ
mkeeda
0
390
Jetpack Composeでの画面遷移
iwata_n
0
190
Deep Dive Into Google Zanzibar and its Concepts for Authorization Scenarios
dschenkelman
1
140
社用PCのdotfiles管理 / dotfiles-in-company
yammerjp
0
140
Angular-basierte Micro Frontends mit Module Federation @API Summit
manfredsteyer
PRO
0
110
Scrum Fest Osaka 2022/5年で200人になったスタートアップの アジャイル開発の歴史とリアル
atamaplus
1
960
"What's new in Swift"の要約 / swift_5_7_summary
uhooi
1
340
BASE BANKチームの技術選定と歴史 / how to decide technology selection for startup
budougumi0617
0
1.4k
Get Ready for Jakarta EE 10
ivargrimstad
0
1.1k
Terraform Plan/Apply結果の自動通知
ymmy02
0
280
Springin‘でみんなもクリエイターに!
ueponx
0
220
Featured
See All Featured
Design by the Numbers
sachag
271
17k
Principles of Awesome APIs and How to Build Them.
keavy
113
15k
The Illustrated Children's Guide to Kubernetes
chrisshort
15
36k
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
315
19k
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
181
15k
How to Ace a Technical Interview
jacobian
265
21k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
237
19k
Bash Introduction
62gerente
597
210k
Designing on Purpose - Digital PM Summit 2013
jponch
106
5.6k
How GitHub Uses GitHub to Build GitHub
holman
465
280k
Why You Should Never Use an ORM
jnunemaker
PRO
47
7.6k
What's in a price? How to price your products and services
michaelherold
229
9.4k
Transcript
Running Jobs at Scale Kir Shatrov GORUCO 2018, @kirshatrov
GORUCO 2018, @kirshatrov
GORUCO 2018, @kirshatrov
GORUCO 2018, @kirshatrov
GORUCO 2018, @kirshatrov
Jobs GORUCO 2018, @kirshatrov
class ExampleJob < ActiveJob::Base def perform ... end end GORUCO
2018, @kirshatrov
class ExampleJob < ActiveJob::Base def perform Product.all.find_each do |product| product.sync_and_refresh
end end end GORUCO 2018, @kirshatrov
class ExampleJob < ActiveJob::Base def perform Product.all.find_each do |product| product.sync_and_refresh
end end end minutes? hours? days? GORUCO 2018, @kirshatrov
Long-running jobs GORUCO 2018, @kirshatrov
Long-running jobs — Deploys and termination GORUCO 2018, @kirshatrov
Long-running jobs — Deploys and termination — Abort and re-enqueue
— Progress lost GORUCO 2018, @kirshatrov
GORUCO 2018, @kirshatrov
Long-running jobs — Deploys and termination — Abort and re-enqueue
— Progress lost — Job may never complete GORUCO 2018, @kirshatrov
Long-running jobs — Deploys and termination — Abort and re-enqueue
— Progress lost — Job may never complete — Capacity and worker starvation GORUCO 2018, @kirshatrov
Long-running jobs — Deploys and termination — Abort and re-enqueue
— Progress lost — Job may never complete — Capacity and worker starvation — Cloud ☁ GORUCO 2018, @kirshatrov
Why is it taking long? Because it iterates over a
long collection. GORUCO 2018, @kirshatrov
What if jobs were interruptible and resumable? GORUCO 2018, @kirshatrov
Split the job definition 1. Collection to process 2. Work
to be done GORUCO 2018, @kirshatrov
Split the job definition 1. Collection to process ≫ Product.all
2. Work to be done GORUCO 2018, @kirshatrov
Split the job definition 1. Collection to process ≫ Product.all
2. Work to be done ≫ product.sync_and_refresh GORUCO 2018, @kirshatrov
class ExampleJob < ActiveJob::Base include Iteration def collection Product.all end
def each_iteration(product) product.sync_and_refresh end end GORUCO 2018, @kirshatrov
— def perform — collection — each_iteration GORUCO 2018, @kirshatrov
Product.all cursor: 1 GORUCO 2018, @kirshatrov
Product.all cursor: 2 GORUCO 2018, @kirshatrov
Product.all cursor: 3 GORUCO 2018, @kirshatrov
Product.all cursor: 4 GORUCO 2018, @kirshatrov
Product.all cursor: 5 GORUCO 2018, @kirshatrov
Product.all cursor: 450123 GORUCO 2018, @kirshatrov
class WhateverJob < ActiveJob::Base include Iteration def collection Enumerator.new do
|enum| 3.times do |n| enum << n end end end def each_iteration(n) # do something three times! end end GORUCO 2018, @kirshatrov
Endless possibilities — Interrupt and resume at any moment —
Progress tracking — Parallel computations — Throttling by default GORUCO 2018, @kirshatrov
Benefits for the infrastructure — Keep supporting long-running jobs —
Success for Cloud runtime — Make scale invisible for developers — Opportunities to save money with short-living instances in Cloud GORUCO 2018, @kirshatrov
Thank you! @kirshatrov GORUCO 2018, @kirshatrov