High availability by offloading work - background jobs, message queues, or Kafka

High availability by offloading work - background jobs, message queues, or Kafka

Talk at RubyConf Colombia 2019

5e8e44a4f6632772c47925006aff31d9?s=128

Kerstin Puschke

September 20, 2019
Tweet

Transcript

  1. 15.

    @titanoboa42 community of
 users
 can interact meaningfully
 with the system

    whenever needed High Availability community of
 users
 can interact meaningfully
 with the system whenever needed
  2. 34.

    @titanoboa42 • No breaking changes to job parameters • Changes

    need to be backwards compatible until legacy jobs have been processed Job queued and processed by different versions
  3. 37.

    @titanoboa42 • “At least” vs. “at most” once delivery •

    Idempotent jobs & at least once delivery No exactly once delivery
  4. 40.

    @titanoboa42 • Don’t queue from within a db transaction •

    Job runs before commit, or in case of rollback Non-transactional queuing
  5. 41.

    @titanoboa42 • Don’t queue from within a db transaction •

    Job runs before commit, or in case of rollback • Commit first: Job not guaranteed to be queued Non-transactional queuing
  6. 50.

    @titanoboa42 • SAGA events/choreography: jobs queue jobs • easy to

    build, hard to maintain or debug Out of order delivery
  7. 51.

    @titanoboa42 • SAGA events/choreography: jobs queue jobs • easy to

    build, hard to maintain or debug • SAGA command/orchestrator Out of order delivery
  8. 55.
  9. 57.

    @titanoboa42 • Aborted and requeued • Job may not finish

    before being aborted again Long running jobs - Sidekiq
  10. 60.

    @titanoboa42 • Split job into collection and task to be

    done • Checkpoint after iteration & requeue Large collections
  11. 64.

    @titanoboa42 • Shutdown workers anytime • Disaster prevention • Data

    integrity Interruptible job with automatic resuming
  12. 73.

    @titanoboa42 • Based on task queues • Complex overall system,

    simple concrete jobs • Great for monolith Background jobs
  13. 86.

    @titanoboa42 Topic with queues
 provides
 advanced routing App Server Business


    Partners Support
 Contracts Orders Messaging Middleware
  14. 87.

    @titanoboa42 Topic with queues
 provides
 advanced routing App Server Message

    Queue Business
 Partners Support
 Contracts Orders Message Queue Messaging Middleware
  15. 95.

    @titanoboa42 • Avoid n:m routing • Better representation of domain:


    multiple messages routed 1:n, n:1 instead Keep breaking changes manageable
  16. 98.

    @titanoboa42 • No exactly once delivery • No strong consistency

    • Out of order delivery Lack of guarantees
  17. 105.
  18. 106.

    @titanoboa42 • Based on queues and topics • Complex overall

    system • Simple message consumers Message oriented middleware
  19. 114.

    @titanoboa42 • Events persisted into append-only log • Consumers read

    shared log • Stateless broker (no queues) Event log
  20. 125.

    @titanoboa42 • Single source of truth (e.g. for event sourcing)

    • High throughput applications Event logs
  21. 132.

    @titanoboa42 • Shared log, no queues • For event sourcing

    & high throughput applications Event logs