Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Fair by design: orchestrating background jobs i...

Sponsored · Your Podcast. Everywhere. Effortlessly. Share. Educate. Inspire. Entertain. You do you. We'll handle the rest. →

Fair by design: orchestrating background jobs inย Ruby

Are you treating your users fairly? They could be stuck in the queue while a greedy user monopolizes resources. And you might not even know it! In this post, youโ€™ll see if itโ€™s time for you to take background job prioritization seriouslyโ€”and how to make it fair for all users.

Alexander Baygeldin

July 16, 2026

Transcript

105 slides
  1. ๐Ÿง‘๐Ÿณ ๐Ÿง‘๐Ÿณ ๐Ÿง‘๐Ÿณ ๐Ÿ• ๐Ÿ• ๐Ÿง‘๐Ÿณ ๐Ÿง‘๐Ÿณ ๐Ÿงข ๐Ÿงข ๐Ÿง‘๐Ÿณ

    ๐Ÿงข ๐Ÿง‘๐Ÿณ ๐Ÿงข ๐Ÿง‘๐Ÿณ ๐Ÿงข ๐Ÿง‘๐Ÿณ ๐Ÿงข ๐Ÿง‘๐Ÿณ ๐Ÿงข ๐Ÿง‘๐Ÿณ ๐Ÿงข ๐Ÿง‘๐Ÿณ ๐Ÿงข ๐Ÿง‘๐Ÿณ ๐Ÿงข part-time chefs
  2. ๐Ÿง‘๐Ÿณ ๐Ÿง‘๐Ÿณ ๐Ÿง‘๐Ÿณ ๐Ÿ• ๐Ÿ• ๐Ÿง‘๐Ÿณ ๐Ÿง‘๐Ÿณ ๐Ÿงข ๐Ÿงข ๐Ÿง‘๐Ÿณ

    ๐Ÿงข ๐Ÿง‘๐Ÿณ ๐Ÿงข ๐Ÿง‘๐Ÿณ ๐Ÿงข ๐Ÿง‘๐Ÿณ ๐Ÿงข ๐Ÿง‘๐Ÿณ ๐Ÿงข ๐Ÿง‘๐Ÿณ ๐Ÿงข ๐Ÿง‘๐Ÿณ ๐Ÿงข ๐Ÿง‘๐Ÿณ ๐Ÿงข part-time chefs At some point improving Quality of Service at more operational cost leads to diminishing returns
  3. G = n โˆ‘ i=1 n โˆ‘ j=1 xi โˆ’

    xj 2n2 ยฏ x = n โˆ‘ i=1 n โˆ‘ j=1 xi โˆ’ xj 2n n โˆ‘ i=1 xi ๐’ฅ (x1 , x2 , โ€ฆ, xn ) = (โˆ‘n i=1 xi )2 n โ‹… โˆ‘n i=1 xi 2 = x2 x2 = 1 1 + ฬ‚ cv 2 Jain's index? Gini index? Some other guy's index?
  4. How do we know if we're not? โ˜‘ FIFO queue

    โ˜‘ High latency โ˜‘ Greedy users Are we OK?
  5. ๐Ÿง‘๐Ÿณ ๐Ÿง‘๐Ÿณ ๐Ÿง‘๐Ÿณ LMOVE queue:tenant_1 queue:sq|<worker ID>|tenant_1 RIGHT LEFT LMOVE

    queue:tenant_2 queue:sq|<worker ID>|tenant_2 RIGHT LEFT ... LMOVE queue:tenant_N queue:sq|<worker ID>|tenant_N RIGHT LEFT worker's "in-progress" queues per-tenant queues
  6. ๐Ÿง‘๐Ÿณ ๐Ÿง‘๐Ÿณ ๐Ÿง‘๐Ÿณ Solid Queue SELECT job_id FROM solid_queue_ready_executions WHERE

    queue_name = 'tenant_N' ORDER BY priority ASC, job_id ASC LIMIT ? FOR UPDATE SKIP LOCKED
  7. ๐Ÿง‘๐Ÿณ ๐Ÿง‘๐Ÿณ ๐Ÿง‘๐Ÿณ GoodJob ๐Ÿ‘ WITH rows AS MATERIALIZED (

    SELECT id, active_job_id FROM good_jobs WHERE queue_name = 'tenant_N' AND (<more filters>) ORDER BY priority DESC NULLS LAST, created_at ASC LIMIT ? ) SELECT id FROM rows WHERE pg_try_advisory_lock(<lock hash based on active_job_id>) LIMIT 1
  8. (with what we have) 1. Shuf fl e-sharding 2. Interruptible

    iteration 3. Throttling 4. Per-tenant queues Let's fix this!
  9. ๐Ÿ• ๐Ÿ• ๐Ÿ• ๐Ÿ• ๐Ÿ• ๐Ÿ• Alex ๐Ÿ• Amy ๐Ÿ•

    Joe ๐Ÿ• Sam ๐Ÿ• Sam ๐Ÿ• Amy ๐Ÿ• A to I J to R S to Z ๐Ÿง‘๐Ÿณ ๐Ÿง‘๐Ÿณ ๐Ÿง‘๐Ÿณ
  10. ๐Ÿง‘๐Ÿณ ๐Ÿท Pentagon ๐Ÿ• ๐Ÿง‘๐Ÿณ ๐Ÿง‘๐Ÿณ A to I J

    to R S to Z Amy ๐Ÿ• ๐Ÿท Pentagon ๐Ÿ• ๐Ÿ’ค Joe ๐Ÿ•
  11. ๐Ÿง‘๐Ÿณ ๐Ÿท Pentagon ๐Ÿ• ๐Ÿง‘๐Ÿณ ๐Ÿง‘๐Ÿณ A to I J

    to R S to Z Amy ๐Ÿ• ๐Ÿท Pentagon ๐Ÿ• Joe ๐Ÿ• J to R
  12. ๐Ÿง‘๐Ÿณ ๐Ÿท Pentagon ๐Ÿ• ๐Ÿง‘๐Ÿณ ๐Ÿง‘๐Ÿณ A to I J

    to R S to Z Amy ๐Ÿ• ๐Ÿท Pentagon ๐Ÿ• Joe ๐Ÿ• J to R Sam ๐Ÿ•
  13. ๐Ÿง‘๐Ÿณ ๐Ÿท Pentagon ๐Ÿ• ๐Ÿง‘๐Ÿณ ๐Ÿง‘๐Ÿณ A to I J

    to R S to Z Amy ๐Ÿ• ๐Ÿท Pentagon ๐Ÿ• Joe ๐Ÿ• J to R Sam ๐Ÿ• A to I S to Z
  14. Shuffle-sharding TL;DR: good when you have enough workload to fi

    ll all shards. No hogging? Noโ€”but it's less likely by a factor of <shard count>. Perfectly fair? Noโ€”but it affects fewer people when it's not fair. Full resource utilization? Usually, yesโ€”unless there are too many shards. Does it scale? Yesโ€”in fact, it gets more effective.
  15. Interruptible iteration No hogging? Noโ€”it could happen if one tenant

    enqueues too many batches simultaneously. Perfectly fair? Noโ€”it stops being fair when someone hogs the queue. Full resource utilization? Usually, yesโ€”except when there are less than <worker count> jobs in the queue. Does it scale? Yesโ€”using cursors to track progress is cheap. TL;DR: good when the workload comes in large batches.
  16. ๐Ÿชฃ ๐Ÿšฟ ๐Ÿ•ณ ๐Ÿ’ง ๐Ÿ’ง ๐Ÿ’ง ๐Ÿ’ง ๐Ÿ’ง ๐Ÿ’ง ๐Ÿ’ง

    ๐Ÿ’ง ๐Ÿ’ง over time, the bucket leaks allowing further orders bucket has a limited capacity each new order fi lls the bucket
  17. ๐Ÿชฃ ๐Ÿšฟ ๐Ÿ•ณ ๐Ÿ’ง ๐Ÿ’ง ๐Ÿ’ง ๐Ÿ’ง ๐Ÿ’ง ๐Ÿ’ง ๐Ÿ’ง

    ๐Ÿ’ง ๐Ÿ’ฆ ๐Ÿ’ฆ uh-oh, over fl ow! bucket has a limited capacity each new order fi lls the bucket over time, the bucket leaks allowing further orders
  18. Throttling No hogging? Yesโ€”everyone will get at least a little

    work done. Perfectly fair? Noโ€”especially if the workload is bursty by nature. Full resource utilization? Yesโ€”with weighted queues support. Does it scale? Yesโ€”especially with leaky buckets. TL;DR: good when the workload is well distributed over time.
  19. ๐Ÿ• ๐Ÿ• ๐Ÿ• ๐ŸŽฒ ๐Ÿ• ๐Ÿ• ๐Ÿ• ๐Ÿ• ๐Ÿ• ๐Ÿง‘๐Ÿณ

    ๐Ÿง‘๐Ÿณ ๐Ÿง‘๐Ÿณ ๐Ÿ“’ ๐Ÿ• ๐Ÿ• ๐Ÿ• ๐Ÿง™
  20. Per-tenant queues + custom scheduler No hogging? Yesโ€”it's basically communism.

    Perfectly fair? Yesโ€”as fair as you care to make it. Full resource utilization? Yesโ€”with zero changes to the underlying infra. Does it scale? ๐Ÿค”
  21. Per-tenant queues + custom scheduler No hogging? Yesโ€”it's basically communism.

    Perfectly fair? Yesโ€”as fair as you care to make it. Full resource utilization? Yesโ€”with zero changes to the underlying infra. Does it scale? Usually, yesโ€”unless the scheduler is slower than workers. TL;DR: good when the workload is heavy enough to forget about the bottleneck.
  22. โœจ AI work fl ows โฌ‡ Data imports ๐Ÿ“Š Report

    generation ๐Ÿ–ผ Media processing (where fairness matters) Examples! ` ... and more!
  23. โœจ AI work fl ows โฌ‡ Data imports ๐Ÿ“Š Report

    generation What limits throughput? I/O wait
  24. โœจ AI work fl ows โฌ‡ Data imports ๐Ÿ“Š Report

    generation I/O wait API rate limits Database GPU* * if self-hosted What limits throughput?
  25. So, which one? Shuf fl e- sharding Interruptible iteration Throttling

    Per-tenant queues No hogging? โŒ โŒ โœ… โœ… Perfectly fair? โŒ โŒ โŒ โœ… Full resource utilization? ๐ŸŸง ๐ŸŸง โœ… โœ… Does it scale? โœ… โœ… โœ… ๐ŸŸง

More Decks by Alexander Baygeldin

Other Decks in Technology