Upgrade to Pro — share decks privately, control downloads, hide ads and more …

EuRuKo 2018: Scaling a monolith isn't scaling microservices

EuRuKo 2018: Scaling a monolith isn't scaling microservices

talk at EuRuKo 2018 about background jobs and message oriented middleware

Kerstin Puschke

August 25, 2018
Tweet

More Decks by Kerstin Puschke

Other Decks in Programming

Transcript

  1. Kerstin Puschke
    @titanoboa42
    Scaling a monolith

    isn’t scaling microservices

    View Slide

  2. View Slide

  3. @titanoboa42
    Workload distribution 

    in different architectures

    View Slide

  4. @titanoboa42
    • Background jobs
    Outline

    View Slide

  5. @titanoboa42
    • Background jobs
    • Features & Challenges
    Outline

    View Slide

  6. @titanoboa42
    • Background jobs
    • Features & Challenges
    • When to choose
    Outline

    View Slide

  7. @titanoboa42
    • Message oriented middleware
    Outline

    View Slide

  8. @titanoboa42
    • Message oriented middleware
    • Features & Challenges
    Outline

    View Slide

  9. @titanoboa42
    • Message oriented middleware
    • Features & Challenges
    • When to choose
    Outline

    View Slide

  10. @titanoboa42
    • Event log
    Outline

    View Slide

  11. @titanoboa42
    • Event log
    • Summary
    Outline

    View Slide

  12. @titanoboa42
    Background jobs

    View Slide

  13. @titanoboa42
    • Resque
    Background job backends

    View Slide

  14. @titanoboa42
    • Resque
    • Sidekiq
    Background job backends

    View Slide

  15. @titanoboa42
    • Resque
    • Sidekiq
    • …
    Background job backends

    View Slide

  16. @titanoboa42
    Background job:

    Unit of work 

    to be done later
    App
    Server
    Worker

    View Slide

  17. @titanoboa42
    Asynchronous
    communication
    App
    Server
    Message
    Queue
    Worker

    View Slide

  18. @titanoboa42
    Asynchronous
    communication
    App
    Server
    Message
    Queue
    Worker
    Task
    Queue

    View Slide

  19. @titanoboa42
    Asynchronous
    communication
    App
    Server
    Message
    Queue
    Worker Worker
    Worker
    Task
    Queue

    View Slide

  20. @titanoboa42
    Background job backend:

    task queue & broker

    View Slide

  21. @titanoboa42
    Encapsulating

    async communication

    View Slide

  22. @titanoboa42
    Features

    View Slide

  23. @titanoboa42
    Task
    Queue
    Response times
    App
    Server
    Worker

    View Slide

  24. @titanoboa42
    Task
    Queue
    Spikeability
    App
    Server
    Worker

    View Slide

  25. @titanoboa42
    Task
    Queue
    Parallelization
    App
    Server
    Worker Worker
    Worker

    View Slide

  26. @titanoboa42
    Task
    Queue
    Retries
    App
    Server
    Worker Worker
    Worker

    View Slide

  27. @titanoboa42
    Prioritization
    App
    Server
    Worker Worker
    High Prio
    Queue
    Low Prio
    Queue

    View Slide

  28. @titanoboa42
    Prioritization
    App
    Server
    Worker Worker
    High Prio
    Queue
    Low Prio
    Queue

    View Slide

  29. @titanoboa42
    Prioritization
    App
    Server
    Worker Worker
    High Prio
    Queue
    Low Prio
    Queue

    View Slide

  30. @titanoboa42
    Mastering challenges

    View Slide

  31. @titanoboa42
    No exactly once delivery

    View Slide

  32. @titanoboa42
    • “At least” vs. “at most” once delivery
    No exactly once delivery

    View Slide

  33. @titanoboa42
    • “At least” vs. “at most” once delivery
    • Idempotent jobs & at least once delivery
    No exactly once delivery

    View Slide

  34. @titanoboa42
    Out of order delivery

    View Slide

  35. @titanoboa42
    • If order matters, queue sequentially
    Out of order delivery

    View Slide

  36. @titanoboa42
    • If order matters, queue sequentially
    • First job queues follow up jobs
    Out of order delivery

    View Slide

  37. @titanoboa42
    Long running jobs - Resque

    View Slide

  38. @titanoboa42
    • Prevent worker shutdown
    Long running jobs - Resque

    View Slide

  39. @titanoboa42
    • Prevent worker shutdown
    • No deployments
    Long running jobs - Resque

    View Slide

  40. @titanoboa42
    • Prevent worker shutdown
    • No deployments
    • Not cloud-friendly
    Long running jobs - Resque

    View Slide

  41. @titanoboa42
    • Aborted and requeued
    Long running jobs - Sidekiq

    View Slide

  42. @titanoboa42
    • Aborted and requeued
    • Job may not finish before being aborted again
    Long running jobs - Sidekiq

    View Slide

  43. @titanoboa42
    Large collections

    View Slide

  44. @titanoboa42
    • Split job into collection and task to be done
    Large collections

    View Slide

  45. @titanoboa42
    • Split job into collection and task to be done
    • Checkpoint after iteration & requeue
    Large collections

    View Slide

  46. @titanoboa42
    Interruptible job with automatic resuming

    View Slide

  47. @titanoboa42
    • Shutdown workers anytime
    Interruptible job with automatic resuming

    View Slide

  48. @titanoboa42
    • Shutdown workers anytime
    • Disaster prevention
    Interruptible job with automatic resuming

    View Slide

  49. @titanoboa42
    • Shutdown workers anytime
    • Disaster prevention
    • Data integrity
    Interruptible job with automatic resuming

    View Slide

  50. @titanoboa42
    Abstracting scaling issues

    simplifies 

    concrete background jobs

    View Slide

  51. @titanoboa42
    github.com

    /Shopify/job-iteration

    View Slide

  52. @titanoboa42
    When to choose

    background jobs

    View Slide

  53. @titanoboa42
    Task
    Queue
    Background jobs are
    ruby objects
    App
    Server
    Worker

    View Slide

  54. @titanoboa42
    Task
    Queue
    Background jobs are
    ruby objects
    App
    Server
    Worker
    Broker
    Broker

    View Slide

  55. @titanoboa42
    Great fit for 

    monolithic architecture

    View Slide

  56. @titanoboa42
    Background jobs

    Summary

    View Slide

  57. @titanoboa42
    • Great features based on task queues
    Background jobs

    View Slide

  58. @titanoboa42
    • Great features based on task queues
    • Abstracting the complexity of jobs running in
    the future and / or in parallel
    Background jobs

    View Slide

  59. @titanoboa42
    • Simple concrete jobs
    Background jobs

    View Slide

  60. @titanoboa42
    • Simple concrete jobs
    • Complex overall system
    Background jobs

    View Slide

  61. @titanoboa42
    • Simple concrete jobs
    • Complex overall system
    • Great for monolithic architecture
    Background jobs

    View Slide

  62. @titanoboa42
    Message oriented middleware

    View Slide

  63. @titanoboa42
    • Implementations: RabbitMQ, ActiveMQ,…
    Message oriented middleware

    View Slide

  64. @titanoboa42
    • Implementations: RabbitMQ, ActiveMQ,…
    • Protocols: AMQP, MQTT, Stomp,…
    Message oriented middleware

    View Slide

  65. @titanoboa42
    Message
    Queue
    Messaging
    Middleware
    App
    Server
    Producer
    Data-based interface
    Worker
    Consumer

    View Slide

  66. @titanoboa42
    Message
    Queue
    Messaging
    Middleware
    App
    Server
    Producer
    Data-based interface
    Worker
    Consumer
    Broker

    View Slide

  67. @titanoboa42
    Features

    View Slide

  68. @titanoboa42
    Decoupling

    View Slide

  69. @titanoboa42
    Interoperability

    View Slide

  70. @titanoboa42
    Command messages

    View Slide

  71. @titanoboa42
    Event messages

    View Slide

  72. @titanoboa42
    Propagating updates
    Business
    Partners
    Support
    Contracts
    Orders

    View Slide

  73. @titanoboa42
    Propagating updates
    Business
    Partners
    Support
    Contracts
    Orders

    View Slide

  74. @titanoboa42
    REST/webhooks

    has disadvantages
    Business
    Partners
    Support
    Contracts
    Orders

    View Slide

  75. @titanoboa42
    REST/webhooks

    has disadvantages
    Business
    Partners
    Orders

    View Slide

  76. @titanoboa42
    REST/webhooks

    has disadvantages
    Business
    Partners
    Support
    Contracts
    Orders

    View Slide

  77. @titanoboa42
    REST/webhooks

    has disadvantages
    Business
    Partners
    Support
    Contracts
    Orders Invoices

    View Slide

  78. @titanoboa42
    Messaging

    Middleware
    Resiliency
    Business
    Partners
    Invoices

    View Slide

  79. @titanoboa42
    Messaging

    Middleware
    Resiliency
    Business
    Partners

    View Slide

  80. @titanoboa42
    Messaging

    Middleware
    Resiliency
    Business
    Partners
    Invoices

    View Slide

  81. @titanoboa42
    Topic with queues

    provides

    advanced routing
    App
    Server
    Business

    Partners
    Support

    Contracts
    Orders
    Messaging
    Middleware

    View Slide

  82. @titanoboa42
    Topic with queues

    provides

    advanced routing
    App
    Server
    Message
    Queue
    Business

    Partners
    Support

    Contracts
    Orders
    Message
    Queue
    Messaging
    Middleware

    View Slide

  83. @titanoboa42
    Messaging
    Middleware
    Anonymity for
    producer and
    consumer
    Business
    Partners
    Support
    Contracts
    Orders

    View Slide

  84. @titanoboa42
    Messaging
    Middleware
    Anonymity for
    producer and
    consumer
    Business
    Partners
    Invoices
    Support
    Contracts
    Orders

    View Slide

  85. @titanoboa42
    Messaging
    Middleware
    Anonymity for
    producer and
    consumer
    FraudScore
    Orders
    Support
    Contracts

    View Slide

  86. @titanoboa42
    Messaging
    Middleware
    Anonymity for
    producer and
    consumer
    Invoices
    FraudScore
    Orders
    Support
    Contracts

    View Slide

  87. @titanoboa42
    Mastering challenges

    View Slide

  88. @titanoboa42
    Keep breaking changes manageable

    View Slide

  89. @titanoboa42
    • Avoid n:m routing
    Keep breaking changes manageable

    View Slide

  90. @titanoboa42
    • Avoid n:m routing
    • Better representation of domain:

    multiple messages routed 1:n, n:1 instead
    Keep breaking changes manageable

    View Slide

  91. @titanoboa42
    • Messages are kept if not consumed
    Remove queues if consumer is gone

    View Slide

  92. @titanoboa42
    • Messages are kept if not consumed
    • Growing queue becomes problematic
    Remove queues if consumer is gone

    View Slide

  93. @titanoboa42
    • “At least” vs. “at most” once delivery
    No exactly once delivery

    View Slide

  94. @titanoboa42
    • “At least” vs. “at most” once delivery
    • Idempotent consumer & at least once delivery
    No exactly once delivery

    View Slide

  95. @titanoboa42
    • If order matters, queue sequentially
    Out of order delivery

    View Slide

  96. @titanoboa42
    • If order matters, queue sequentially
    • First consumer queues second message
    Out of order delivery

    View Slide

  97. @titanoboa42
    Event-carried state transfer

    requires order information

    View Slide

  98. @titanoboa42
    • Payload contains updated data
    Event-carried state transfer

    requires order information

    View Slide

  99. @titanoboa42
    • Payload contains updated data
    • Avoids requests to service that owns data
    Event-carried state transfer

    requires order information

    View Slide

  100. @titanoboa42
    • Payload contains updated data
    • Avoids requests to service that owns data
    • Provide order information (e.g. updated_at)
    Event-carried state transfer

    requires order information

    View Slide

  101. @titanoboa42
    Event notifications are commutative

    View Slide

  102. @titanoboa42
    • Payload has id only
    Event notifications are commutative

    View Slide

  103. @titanoboa42
    • Payload has id only
    • Consumer requests data separately
    Event notifications are commutative

    View Slide

  104. @titanoboa42
    • Payload has id only
    • Consumer requests data separately
    • Consumer doesn’t learn every state
    Event notifications are commutative

    View Slide

  105. @titanoboa42
    No replayability

    View Slide

  106. @titanoboa42
    • Messages removed after processing
    No replayability

    View Slide

  107. @titanoboa42
    • Messages removed after processing
    • No single source of truth for event sourcing
    No replayability

    View Slide

  108. @titanoboa42
    When to choose

    message oriented middleware

    View Slide

  109. @titanoboa42
    Great fit for 

    stateful, distributed architecture

    View Slide

  110. @titanoboa42
    Message oriented middleware

    Summary

    View Slide

  111. @titanoboa42
    • Great features based on queues and topics
    Message oriented middleware

    View Slide

  112. @titanoboa42
    • Great features based on queues and topics
    • Abstracting the complexity of pub sub
    Message oriented middleware

    View Slide

  113. @titanoboa42
    • Abstracting the complexity of messages being
    consumed in the future / in parallel
    Message oriented middleware

    View Slide

  114. @titanoboa42
    • Abstracting the complexity of messages being
    consumed in the future / in parallel
    • Simple concrete message consumers
    Message oriented middleware

    View Slide

  115. @titanoboa42
    • Complex overall system
    Message oriented middleware

    View Slide

  116. @titanoboa42
    • Complex overall system
    • Great for stateful, distributed architecture
    Message oriented middleware

    View Slide

  117. @titanoboa42
    Event log

    View Slide

  118. @titanoboa42
    • Kafka
    Event logs

    View Slide

  119. @titanoboa42
    • Kafka
    • …
    Event logs

    View Slide

  120. @titanoboa42
    • Events persisted into append-only log
    Event log

    View Slide

  121. @titanoboa42
    • Events persisted into append-only log
    • Consumers read shared log
    Event log

    View Slide

  122. @titanoboa42
    • Events persisted into append-only log
    • Consumers read shared log
    • Stateless broker (no queues)
    Event log

    View Slide

  123. @titanoboa42
    Consumers 

    easy to remove

    View Slide

  124. @titanoboa42
    High throughput 

    data streaming

    View Slide

  125. @titanoboa42
    Replayability

    View Slide

  126. @titanoboa42
    • Single source of truth for event sourcing
    Replayability

    View Slide

  127. @titanoboa42
    Great fit for 

    event sourcing

    and real time applications

    View Slide

  128. @titanoboa42
    Summary

    View Slide

  129. @titanoboa42
    • Queues
    Background jobs

    View Slide

  130. @titanoboa42
    • Queues
    • Simple concrete jobs
    Background jobs

    View Slide

  131. @titanoboa42
    • Queues
    • Simple concrete jobs
    • For monolithic architecture
    Background jobs

    View Slide

  132. @titanoboa42
    • Topics and Queues
    Message oriented middleware

    View Slide

  133. @titanoboa42
    • Topics and Queues
    • Simple, stateless message consumers
    Message oriented middleware

    View Slide

  134. @titanoboa42
    • Topics and Queues
    • Simple, stateless message consumers
    • For distributed, stateful architecture
    Message oriented middleware

    View Slide

  135. @titanoboa42
    • Shared log, no queues
    Event logs

    View Slide

  136. @titanoboa42
    • Shared log, no queues
    • Stateful consumers
    Event logs

    View Slide

  137. @titanoboa42
    • Shared log, no queues
    • Stateful consumers
    • For event sourcing & real time applications
    Event logs

    View Slide

  138. @titanoboa42
    BFCM video

    View Slide

  139. @titanoboa42
    BFCM video

    View Slide

  140. Thanks!

    Questions?

    @titanoboa42


    https://www.shopify.com/careers

    View Slide