Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
background job patterns
Search
Matt Yoho
May 23, 2012
Programming
2
490
background job patterns
Matt Yoho
May 23, 2012
Tweet
Share
More Decks by Matt Yoho
See All by Matt Yoho
Working with Git and GitHub
mattyoho
4
300
Qualitative Analysis of Algorithms
mattyoho
5
410
Metaprogramming Ruby
mattyoho
19
670
Exploiting The Resource Idiom
mattyoho
2
390
testing revisited - outside-in
mattyoho
1
110
shared_auth.pdf
mattyoho
0
2.3k
Request-Response cycle
mattyoho
5
180
Intro to Rest
mattyoho
6
580
Other Decks in Programming
See All in Programming
ファインディの テックブログ爆誕までの軌跡
starfish719
1
780
[JAWS-UG横浜 #79] re:Invent 2024 の DB アップデートは Multi-Region!
maroon1st
0
130
Compose でデザインと実装の差異を減らすための取り組み
oidy
1
240
DevinとCursorから学ぶAIエージェントメモリーの設計とMoatの考え方
itarutomy
1
500
ecspresso, ecschedule, lambroll を PipeCDプラグインとして動かしてみた (プロトタイプ) / Running ecspresso, ecschedule, and lambroll as PipeCD Plugins (prototype)
tkikuc
2
2.3k
Оптимизируем производительность блока Казначейство
lamodatech
0
990
[Fin-JAWS 第38回 ~re:Invent 2024 金融re:Cap~]FaultInjectionServiceアップデート@pre:Invent2024
shintaro_fukatsu
0
340
ErdMap: Thinking about a map for Rails applications
makicamel
1
1.1k
2024年のkintone API振り返りと2025年 / kintone API look back in 2024
tasshi
0
170
traP の部内 ISUCON とそれを支えるポータル / PISCON Portal
ikura_hamu
0
230
ペアーズでの、Langfuseを中心とした評価ドリブンなリリースサイクルのご紹介
fukubaka0825
1
200
Scaling your build logic
antalmonori
1
150
Featured
See All Featured
Agile that works and the tools we love
rasmusluckow
328
21k
Embracing the Ebb and Flow
colly
84
4.5k
Being A Developer After 40
akosma
89
590k
How to Ace a Technical Interview
jacobian
276
23k
The Myth of the Modular Monolith - Day 2 Keynote - Rails World 2024
eileencodes
20
2.4k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
30
2.1k
The Straight Up "How To Draw Better" Workshop
denniskardys
232
140k
Bash Introduction
62gerente
610
210k
GraphQLとの向き合い方2022年版
quramy
44
13k
GitHub's CSS Performance
jonrohan
1030
460k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
44
9.4k
VelocityConf: Rendering Performance Case Studies
addyosmani
327
24k
Transcript
Approaches to Background Jobs Wednesday, May 23, 12
Architectures Wednesday, May 23, 12
“one soul, two bodies” Wednesday, May 23, 12
Wednesday, May 23, 12
prenuptial agreement Wednesday, May 23, 12
Wednesday, May 23, 12
Types of Jobs Wednesday, May 23, 12
Run-once (Resque, DelayedJob) Wednesday, May 23, 12
Event occurs, job enqueued Worker instantiate job model Load AR
object(s) model Create AR object(s) Hit API job {id, token} job goes away Wednesday, May 23, 12
Recurring jobs (Recursive queuing) Wednesday, May 23, 12
Event occurs, job enqueued Worker instantiate job model Load AR
object(s) model Create AR object(s) Hit API job {id, token} Wait 90s re-enqueue self Wednesday, May 23, 12
Scheduled jobs (resque-scheduler, cron) Wednesday, May 23, 12
Event occurs, job scheduled Worker instantiate job model Load AR
object(s) model Create AR object(s) Hit API job {id, token} job goes away 90s enqueue job Wednesday, May 23, 12
CancelAbandonedOrders: cron: "*/5 * * * *" clear_leaderboards_contributors: cron: "30
6 * * 1" class: ClearLeaderboards queue: low args: contributors description: "Reset the weekly leaders" Wednesday, May 23, 12
Approach can work, but is a bit kludgy Wednesday, May
23, 12
Difficult or impossible to adjust intervals programmatically (dynamically) Wednesday, May
23, 12
May want to respond to traffic, load, rate-limiting, by adding
or subtracting workers, changing interval Wednesday, May 23, 12
Are we meeting the SLA? Wednesday, May 23, 12
Service Level Agreement Wednesday, May 23, 12
Service Level Agreement “At least every 5 minutes” Wednesday, May
23, 12
Hypothetical: SLA: 30 seconds Mean job time: 5 seconds Wednesday,
May 23, 12
Increased complexity and decision making around when to run “jobs”
Wednesday, May 23, 12
Sounds like an app Wednesday, May 23, 12
models, views, controllers, jobs, client libs Wednesday, May 23, 12
models, views, controllers, jobs, client libs jobs, client libraries Wednesday,
May 23, 12
models, views, controllers jobs, client libraries Wednesday, May 23, 12
models, views, controllers jobs, client libraries Wednesday, May 23, 12
models, views, controllers jobs, client libraries message queue Wednesday, May
23, 12
models, views, controllers jobs, client libraries message queue Emits event
messages Wednesday, May 23, 12
models, views, controllers jobs, client libraries message queue m “User
123 linked twitter account with token ‘abc’” Wednesday, May 23, 12
models, views, controllers jobs, client libraries message queue m “User
123 linked twitter account with token ‘abc’” Wednesday, May 23, 12
models, views, controllers jobs, client libraries message queue m Consumes
messages, acts Wednesday, May 23, 12
models, views, controllers jobs, client libraries message queue m Consumes
messages, acts Wednesday, May 23, 12
models, views, controllers jobs, client libraries message queue m “Every
30 seconds, I’ll hit the twitter api with token ‘abc’ and then the feed engine api for user 123” Wednesday, May 23, 12
Wednesday, May 23, 12
pros: increased flexibility & control horizontal scaling smaller apps fewer
specs per app cons: burden of complexity on you Wednesday, May 23, 12