Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
background job patterns
Search
Matt Yoho
May 23, 2012
Programming
2
500
background job patterns
Matt Yoho
May 23, 2012
Tweet
Share
More Decks by Matt Yoho
See All by Matt Yoho
Working with Git and GitHub
mattyoho
4
300
Qualitative Analysis of Algorithms
mattyoho
5
420
Metaprogramming Ruby
mattyoho
19
680
Exploiting The Resource Idiom
mattyoho
2
390
testing revisited - outside-in
mattyoho
1
120
shared_auth.pdf
mattyoho
0
2.3k
Request-Response cycle
mattyoho
5
180
Intro to Rest
mattyoho
6
580
Other Decks in Programming
See All in Programming
React は次の10年を生き残れるか:3つのトレンドから考える
oukayuka
41
16k
副作用と戦う PHP リファクタリング ─ ドメインイベントでビジネスロジックを解きほぐす
kajitack
3
500
バイブスあるコーディングで ~PHP~ 便利ツールをつくるプラクティス
uzulla
1
300
LLMは麻雀を知らなすぎるから俺が教育してやる
po3rin
2
1.5k
MySQL9でベクトルカラム登場!PHP×AWSでのAI/類似検索はこう変わる
suguruooki
1
260
抽象化という思考のツール - 理解と活用 - / Abstraction-as-a-Tool-for-Thinking
shin1x1
1
900
What's new in Adaptive Android development
fornewid
0
130
DynamoDBは怖くない!〜テーブル設計の勘所とテスト戦略〜
hyamazaki
0
100
階層化自動テストで開発に機動力を
ickx
1
450
コーディングエージェント概観(2025/07)
itsuki_t88
1
470
Comparing decimals in Swift Testing
417_72ki
0
140
知って得する@cloudflare_vite-pluginのあれこれ
chimame
1
130
Featured
See All Featured
Fireside Chat
paigeccino
37
3.6k
Done Done
chrislema
185
16k
Adopting Sorbet at Scale
ufuk
77
9.5k
Docker and Python
trallard
45
3.5k
A Tale of Four Properties
chriscoyier
160
23k
Building Flexible Design Systems
yeseniaperezcruz
328
39k
10 Git Anti Patterns You Should be Aware of
lemiorhan
PRO
656
60k
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
126
53k
Visualization
eitanlees
146
16k
Designing Dashboards & Data Visualisations in Web Apps
destraynor
231
53k
How GitHub (no longer) Works
holman
314
140k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
229
22k
Transcript
Approaches to Background Jobs Wednesday, May 23, 12
Architectures Wednesday, May 23, 12
“one soul, two bodies” Wednesday, May 23, 12
Wednesday, May 23, 12
prenuptial agreement Wednesday, May 23, 12
Wednesday, May 23, 12
Types of Jobs Wednesday, May 23, 12
Run-once (Resque, DelayedJob) Wednesday, May 23, 12
Event occurs, job enqueued Worker instantiate job model Load AR
object(s) model Create AR object(s) Hit API job {id, token} job goes away Wednesday, May 23, 12
Recurring jobs (Recursive queuing) Wednesday, May 23, 12
Event occurs, job enqueued Worker instantiate job model Load AR
object(s) model Create AR object(s) Hit API job {id, token} Wait 90s re-enqueue self Wednesday, May 23, 12
Scheduled jobs (resque-scheduler, cron) Wednesday, May 23, 12
Event occurs, job scheduled Worker instantiate job model Load AR
object(s) model Create AR object(s) Hit API job {id, token} job goes away 90s enqueue job Wednesday, May 23, 12
CancelAbandonedOrders: cron: "*/5 * * * *" clear_leaderboards_contributors: cron: "30
6 * * 1" class: ClearLeaderboards queue: low args: contributors description: "Reset the weekly leaders" Wednesday, May 23, 12
Approach can work, but is a bit kludgy Wednesday, May
23, 12
Difficult or impossible to adjust intervals programmatically (dynamically) Wednesday, May
23, 12
May want to respond to traffic, load, rate-limiting, by adding
or subtracting workers, changing interval Wednesday, May 23, 12
Are we meeting the SLA? Wednesday, May 23, 12
Service Level Agreement Wednesday, May 23, 12
Service Level Agreement “At least every 5 minutes” Wednesday, May
23, 12
Hypothetical: SLA: 30 seconds Mean job time: 5 seconds Wednesday,
May 23, 12
Increased complexity and decision making around when to run “jobs”
Wednesday, May 23, 12
Sounds like an app Wednesday, May 23, 12
models, views, controllers, jobs, client libs Wednesday, May 23, 12
models, views, controllers, jobs, client libs jobs, client libraries Wednesday,
May 23, 12
models, views, controllers jobs, client libraries Wednesday, May 23, 12
models, views, controllers jobs, client libraries Wednesday, May 23, 12
models, views, controllers jobs, client libraries message queue Wednesday, May
23, 12
models, views, controllers jobs, client libraries message queue Emits event
messages Wednesday, May 23, 12
models, views, controllers jobs, client libraries message queue m “User
123 linked twitter account with token ‘abc’” Wednesday, May 23, 12
models, views, controllers jobs, client libraries message queue m “User
123 linked twitter account with token ‘abc’” Wednesday, May 23, 12
models, views, controllers jobs, client libraries message queue m Consumes
messages, acts Wednesday, May 23, 12
models, views, controllers jobs, client libraries message queue m Consumes
messages, acts Wednesday, May 23, 12
models, views, controllers jobs, client libraries message queue m “Every
30 seconds, I’ll hit the twitter api with token ‘abc’ and then the feed engine api for user 123” Wednesday, May 23, 12
Wednesday, May 23, 12
pros: increased flexibility & control horizontal scaling smaller apps fewer
specs per app cons: burden of complexity on you Wednesday, May 23, 12