Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Running Jobs at Scale
Search
Kir Shatrov
June 16, 2018
Programming
1
190
Running Jobs at Scale
My talk from GORUCO 2018 in New York.
Kir Shatrov
June 16, 2018
Tweet
Share
More Decks by Kir Shatrov
See All by Kir Shatrov
Operating Rails in Kubernetes
kirs
3
400
RailsClub 2016
kirs
2
300
Performance regressions in Ruby on Rails Core
kirs
0
200
Building a toolkit to detect performance regressions in Ruby on Rails core
kirs
3
5.2k
Развертывание веб-приложений и фреймворк Capistrano
kirs
1
260
Capistrano 3
kirs
4
2.6k
Other Decks in Programming
See All in Programming
TypeScriptとGraphQLを活用した変化に強いプロダクト作り
bicstone
4
570
Hono・Prisma・AWSでGeoなAPI開発
nokonoko1203
5
610
めざせ!WKWebViewマスター! / WKWebView Master
marcy731
3
480
What we keep in mind when migrating from Serverless Framework to AWS CDK and AWS SAM
kasacchiful
1
130
Go Code Generation at newmo / 2024-08-27 #newmo_layerx_go
genkey6
0
520
Wallet API, Verifier APIで実現するIDカード on iPhoneの世界
shitamori1272
1
320
connect-go で面倒くささと戦う / 2024-08-27 #newmo_layerx_go
izumin5210
2
590
Using Livebook to build and deploy internal tools @ ElixirConf 2024
hugobarauna
0
210
実践 Advanced CallKit 〜快適な通話の実現に向けて〜
mot_techtalk
3
110
Rustではじめる負荷試験
skanehira
5
1.1k
労務ドメインを快適に開発する方法 / How to Comfortably Develop in the Labor Domain
yuki21
1
250
Rubyのobject_id
qnighy
6
1.3k
Featured
See All Featured
ReactJS: Keep Simple. Everything can be a component!
pedronauck
662
120k
Building a Modern Day E-commerce SEO Strategy
aleyda
35
6.8k
ParisWeb 2013: Learning to Love: Crash Course in Emotional UX Design
dotmariusz
109
6.9k
Intergalactic Javascript Robots from Outer Space
tanoku
268
26k
Music & Morning Musume
bryan
46
6k
Art, The Web, and Tiny UX
lynnandtonic
294
20k
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
190
16k
A better future with KSS
kneath
235
17k
GitHub's CSS Performance
jonrohan
1029
450k
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
103
47k
Fantastic passwords and where to find them - at NoRuKo
philnash
47
2.7k
Automating Front-end Workflow
addyosmani
1365
200k
Transcript
Running Jobs at Scale Kir Shatrov GORUCO 2018, @kirshatrov
GORUCO 2018, @kirshatrov
GORUCO 2018, @kirshatrov
GORUCO 2018, @kirshatrov
GORUCO 2018, @kirshatrov
Jobs GORUCO 2018, @kirshatrov
class ExampleJob < ActiveJob::Base def perform ... end end GORUCO
2018, @kirshatrov
class ExampleJob < ActiveJob::Base def perform Product.all.find_each do |product| product.sync_and_refresh
end end end GORUCO 2018, @kirshatrov
class ExampleJob < ActiveJob::Base def perform Product.all.find_each do |product| product.sync_and_refresh
end end end minutes? hours? days? GORUCO 2018, @kirshatrov
Long-running jobs GORUCO 2018, @kirshatrov
Long-running jobs — Deploys and termination GORUCO 2018, @kirshatrov
Long-running jobs — Deploys and termination — Abort and re-enqueue
— Progress lost GORUCO 2018, @kirshatrov
GORUCO 2018, @kirshatrov
Long-running jobs — Deploys and termination — Abort and re-enqueue
— Progress lost — Job may never complete GORUCO 2018, @kirshatrov
Long-running jobs — Deploys and termination — Abort and re-enqueue
— Progress lost — Job may never complete — Capacity and worker starvation GORUCO 2018, @kirshatrov
Long-running jobs — Deploys and termination — Abort and re-enqueue
— Progress lost — Job may never complete — Capacity and worker starvation — Cloud ☁ GORUCO 2018, @kirshatrov
Why is it taking long? Because it iterates over a
long collection. GORUCO 2018, @kirshatrov
What if jobs were interruptible and resumable? GORUCO 2018, @kirshatrov
Split the job definition 1. Collection to process 2. Work
to be done GORUCO 2018, @kirshatrov
Split the job definition 1. Collection to process ≫ Product.all
2. Work to be done GORUCO 2018, @kirshatrov
Split the job definition 1. Collection to process ≫ Product.all
2. Work to be done ≫ product.sync_and_refresh GORUCO 2018, @kirshatrov
class ExampleJob < ActiveJob::Base include Iteration def collection Product.all end
def each_iteration(product) product.sync_and_refresh end end GORUCO 2018, @kirshatrov
— def perform — collection — each_iteration GORUCO 2018, @kirshatrov
Product.all cursor: 1 GORUCO 2018, @kirshatrov
Product.all cursor: 2 GORUCO 2018, @kirshatrov
Product.all cursor: 3 GORUCO 2018, @kirshatrov
Product.all cursor: 4 GORUCO 2018, @kirshatrov
Product.all cursor: 5 GORUCO 2018, @kirshatrov
Product.all cursor: 450123 GORUCO 2018, @kirshatrov
class WhateverJob < ActiveJob::Base include Iteration def collection Enumerator.new do
|enum| 3.times do |n| enum << n end end end def each_iteration(n) # do something three times! end end GORUCO 2018, @kirshatrov
Endless possibilities — Interrupt and resume at any moment —
Progress tracking — Parallel computations — Throttling by default GORUCO 2018, @kirshatrov
Benefits for the infrastructure — Keep supporting long-running jobs —
Success for Cloud runtime — Make scale invisible for developers — Opportunities to save money with short-living instances in Cloud GORUCO 2018, @kirshatrov
Thank you! @kirshatrov GORUCO 2018, @kirshatrov