Upgrade to Pro — share decks privately, control downloads, hide ads and more …

EuRuKo 2018: Scaling a monolith isn't scaling microservices

EuRuKo 2018: Scaling a monolith isn't scaling microservices

talk at EuRuKo 2018 about background jobs and message oriented middleware

Kerstin Puschke

August 25, 2018
Tweet

More Decks by Kerstin Puschke

Other Decks in Programming

Transcript

  1. Kerstin Puschke
    @titanoboa42
    Scaling a monolith

    isn’t scaling microservices

    View full-size slide

  2. @titanoboa42
    Workload distribution 

    in different architectures

    View full-size slide

  3. @titanoboa42
    • Background jobs
    Outline

    View full-size slide

  4. @titanoboa42
    • Background jobs
    • Features & Challenges
    Outline

    View full-size slide

  5. @titanoboa42
    • Background jobs
    • Features & Challenges
    • When to choose
    Outline

    View full-size slide

  6. @titanoboa42
    • Message oriented middleware
    Outline

    View full-size slide

  7. @titanoboa42
    • Message oriented middleware
    • Features & Challenges
    Outline

    View full-size slide

  8. @titanoboa42
    • Message oriented middleware
    • Features & Challenges
    • When to choose
    Outline

    View full-size slide

  9. @titanoboa42
    • Event log
    Outline

    View full-size slide

  10. @titanoboa42
    • Event log
    • Summary
    Outline

    View full-size slide

  11. @titanoboa42
    Background jobs

    View full-size slide

  12. @titanoboa42
    • Resque
    Background job backends

    View full-size slide

  13. @titanoboa42
    • Resque
    • Sidekiq
    Background job backends

    View full-size slide

  14. @titanoboa42
    • Resque
    • Sidekiq
    • …
    Background job backends

    View full-size slide

  15. @titanoboa42
    Background job:

    Unit of work 

    to be done later
    App
    Server
    Worker

    View full-size slide

  16. @titanoboa42
    Asynchronous
    communication
    App
    Server
    Message
    Queue
    Worker

    View full-size slide

  17. @titanoboa42
    Asynchronous
    communication
    App
    Server
    Message
    Queue
    Worker
    Task
    Queue

    View full-size slide

  18. @titanoboa42
    Asynchronous
    communication
    App
    Server
    Message
    Queue
    Worker Worker
    Worker
    Task
    Queue

    View full-size slide

  19. @titanoboa42
    Background job backend:

    task queue & broker

    View full-size slide

  20. @titanoboa42
    Encapsulating

    async communication

    View full-size slide

  21. @titanoboa42
    Features

    View full-size slide

  22. @titanoboa42
    Task
    Queue
    Response times
    App
    Server
    Worker

    View full-size slide

  23. @titanoboa42
    Task
    Queue
    Spikeability
    App
    Server
    Worker

    View full-size slide

  24. @titanoboa42
    Task
    Queue
    Parallelization
    App
    Server
    Worker Worker
    Worker

    View full-size slide

  25. @titanoboa42
    Task
    Queue
    Retries
    App
    Server
    Worker Worker
    Worker

    View full-size slide

  26. @titanoboa42
    Prioritization
    App
    Server
    Worker Worker
    High Prio
    Queue
    Low Prio
    Queue

    View full-size slide

  27. @titanoboa42
    Prioritization
    App
    Server
    Worker Worker
    High Prio
    Queue
    Low Prio
    Queue

    View full-size slide

  28. @titanoboa42
    Prioritization
    App
    Server
    Worker Worker
    High Prio
    Queue
    Low Prio
    Queue

    View full-size slide

  29. @titanoboa42
    Mastering challenges

    View full-size slide

  30. @titanoboa42
    No exactly once delivery

    View full-size slide

  31. @titanoboa42
    • “At least” vs. “at most” once delivery
    No exactly once delivery

    View full-size slide

  32. @titanoboa42
    • “At least” vs. “at most” once delivery
    • Idempotent jobs & at least once delivery
    No exactly once delivery

    View full-size slide

  33. @titanoboa42
    Out of order delivery

    View full-size slide

  34. @titanoboa42
    • If order matters, queue sequentially
    Out of order delivery

    View full-size slide

  35. @titanoboa42
    • If order matters, queue sequentially
    • First job queues follow up jobs
    Out of order delivery

    View full-size slide

  36. @titanoboa42
    Long running jobs - Resque

    View full-size slide

  37. @titanoboa42
    • Prevent worker shutdown
    Long running jobs - Resque

    View full-size slide

  38. @titanoboa42
    • Prevent worker shutdown
    • No deployments
    Long running jobs - Resque

    View full-size slide

  39. @titanoboa42
    • Prevent worker shutdown
    • No deployments
    • Not cloud-friendly
    Long running jobs - Resque

    View full-size slide

  40. @titanoboa42
    • Aborted and requeued
    Long running jobs - Sidekiq

    View full-size slide

  41. @titanoboa42
    • Aborted and requeued
    • Job may not finish before being aborted again
    Long running jobs - Sidekiq

    View full-size slide

  42. @titanoboa42
    Large collections

    View full-size slide

  43. @titanoboa42
    • Split job into collection and task to be done
    Large collections

    View full-size slide

  44. @titanoboa42
    • Split job into collection and task to be done
    • Checkpoint after iteration & requeue
    Large collections

    View full-size slide

  45. @titanoboa42
    Interruptible job with automatic resuming

    View full-size slide

  46. @titanoboa42
    • Shutdown workers anytime
    Interruptible job with automatic resuming

    View full-size slide

  47. @titanoboa42
    • Shutdown workers anytime
    • Disaster prevention
    Interruptible job with automatic resuming

    View full-size slide

  48. @titanoboa42
    • Shutdown workers anytime
    • Disaster prevention
    • Data integrity
    Interruptible job with automatic resuming

    View full-size slide

  49. @titanoboa42
    Abstracting scaling issues

    simplifies 

    concrete background jobs

    View full-size slide

  50. @titanoboa42
    github.com

    /Shopify/job-iteration

    View full-size slide

  51. @titanoboa42
    When to choose

    background jobs

    View full-size slide

  52. @titanoboa42
    Task
    Queue
    Background jobs are
    ruby objects
    App
    Server
    Worker

    View full-size slide

  53. @titanoboa42
    Task
    Queue
    Background jobs are
    ruby objects
    App
    Server
    Worker
    Broker
    Broker

    View full-size slide

  54. @titanoboa42
    Great fit for 

    monolithic architecture

    View full-size slide

  55. @titanoboa42
    Background jobs

    Summary

    View full-size slide

  56. @titanoboa42
    • Great features based on task queues
    Background jobs

    View full-size slide

  57. @titanoboa42
    • Great features based on task queues
    • Abstracting the complexity of jobs running in
    the future and / or in parallel
    Background jobs

    View full-size slide

  58. @titanoboa42
    • Simple concrete jobs
    Background jobs

    View full-size slide

  59. @titanoboa42
    • Simple concrete jobs
    • Complex overall system
    Background jobs

    View full-size slide

  60. @titanoboa42
    • Simple concrete jobs
    • Complex overall system
    • Great for monolithic architecture
    Background jobs

    View full-size slide

  61. @titanoboa42
    Message oriented middleware

    View full-size slide

  62. @titanoboa42
    • Implementations: RabbitMQ, ActiveMQ,…
    Message oriented middleware

    View full-size slide

  63. @titanoboa42
    • Implementations: RabbitMQ, ActiveMQ,…
    • Protocols: AMQP, MQTT, Stomp,…
    Message oriented middleware

    View full-size slide

  64. @titanoboa42
    Message
    Queue
    Messaging
    Middleware
    App
    Server
    Producer
    Data-based interface
    Worker
    Consumer

    View full-size slide

  65. @titanoboa42
    Message
    Queue
    Messaging
    Middleware
    App
    Server
    Producer
    Data-based interface
    Worker
    Consumer
    Broker

    View full-size slide

  66. @titanoboa42
    Features

    View full-size slide

  67. @titanoboa42
    Decoupling

    View full-size slide

  68. @titanoboa42
    Interoperability

    View full-size slide

  69. @titanoboa42
    Command messages

    View full-size slide

  70. @titanoboa42
    Event messages

    View full-size slide

  71. @titanoboa42
    Propagating updates
    Business
    Partners
    Support
    Contracts
    Orders

    View full-size slide

  72. @titanoboa42
    Propagating updates
    Business
    Partners
    Support
    Contracts
    Orders

    View full-size slide

  73. @titanoboa42
    REST/webhooks

    has disadvantages
    Business
    Partners
    Support
    Contracts
    Orders

    View full-size slide

  74. @titanoboa42
    REST/webhooks

    has disadvantages
    Business
    Partners
    Orders

    View full-size slide

  75. @titanoboa42
    REST/webhooks

    has disadvantages
    Business
    Partners
    Support
    Contracts
    Orders

    View full-size slide

  76. @titanoboa42
    REST/webhooks

    has disadvantages
    Business
    Partners
    Support
    Contracts
    Orders Invoices

    View full-size slide

  77. @titanoboa42
    Messaging

    Middleware
    Resiliency
    Business
    Partners
    Invoices

    View full-size slide

  78. @titanoboa42
    Messaging

    Middleware
    Resiliency
    Business
    Partners

    View full-size slide

  79. @titanoboa42
    Messaging

    Middleware
    Resiliency
    Business
    Partners
    Invoices

    View full-size slide

  80. @titanoboa42
    Topic with queues

    provides

    advanced routing
    App
    Server
    Business

    Partners
    Support

    Contracts
    Orders
    Messaging
    Middleware

    View full-size slide

  81. @titanoboa42
    Topic with queues

    provides

    advanced routing
    App
    Server
    Message
    Queue
    Business

    Partners
    Support

    Contracts
    Orders
    Message
    Queue
    Messaging
    Middleware

    View full-size slide

  82. @titanoboa42
    Messaging
    Middleware
    Anonymity for
    producer and
    consumer
    Business
    Partners
    Support
    Contracts
    Orders

    View full-size slide

  83. @titanoboa42
    Messaging
    Middleware
    Anonymity for
    producer and
    consumer
    Business
    Partners
    Invoices
    Support
    Contracts
    Orders

    View full-size slide

  84. @titanoboa42
    Messaging
    Middleware
    Anonymity for
    producer and
    consumer
    FraudScore
    Orders
    Support
    Contracts

    View full-size slide

  85. @titanoboa42
    Messaging
    Middleware
    Anonymity for
    producer and
    consumer
    Invoices
    FraudScore
    Orders
    Support
    Contracts

    View full-size slide

  86. @titanoboa42
    Mastering challenges

    View full-size slide

  87. @titanoboa42
    Keep breaking changes manageable

    View full-size slide

  88. @titanoboa42
    • Avoid n:m routing
    Keep breaking changes manageable

    View full-size slide

  89. @titanoboa42
    • Avoid n:m routing
    • Better representation of domain:

    multiple messages routed 1:n, n:1 instead
    Keep breaking changes manageable

    View full-size slide

  90. @titanoboa42
    • Messages are kept if not consumed
    Remove queues if consumer is gone

    View full-size slide

  91. @titanoboa42
    • Messages are kept if not consumed
    • Growing queue becomes problematic
    Remove queues if consumer is gone

    View full-size slide

  92. @titanoboa42
    • “At least” vs. “at most” once delivery
    No exactly once delivery

    View full-size slide

  93. @titanoboa42
    • “At least” vs. “at most” once delivery
    • Idempotent consumer & at least once delivery
    No exactly once delivery

    View full-size slide

  94. @titanoboa42
    • If order matters, queue sequentially
    Out of order delivery

    View full-size slide

  95. @titanoboa42
    • If order matters, queue sequentially
    • First consumer queues second message
    Out of order delivery

    View full-size slide

  96. @titanoboa42
    Event-carried state transfer

    requires order information

    View full-size slide

  97. @titanoboa42
    • Payload contains updated data
    Event-carried state transfer

    requires order information

    View full-size slide

  98. @titanoboa42
    • Payload contains updated data
    • Avoids requests to service that owns data
    Event-carried state transfer

    requires order information

    View full-size slide

  99. @titanoboa42
    • Payload contains updated data
    • Avoids requests to service that owns data
    • Provide order information (e.g. updated_at)
    Event-carried state transfer

    requires order information

    View full-size slide

  100. @titanoboa42
    Event notifications are commutative

    View full-size slide

  101. @titanoboa42
    • Payload has id only
    Event notifications are commutative

    View full-size slide

  102. @titanoboa42
    • Payload has id only
    • Consumer requests data separately
    Event notifications are commutative

    View full-size slide

  103. @titanoboa42
    • Payload has id only
    • Consumer requests data separately
    • Consumer doesn’t learn every state
    Event notifications are commutative

    View full-size slide

  104. @titanoboa42
    No replayability

    View full-size slide

  105. @titanoboa42
    • Messages removed after processing
    No replayability

    View full-size slide

  106. @titanoboa42
    • Messages removed after processing
    • No single source of truth for event sourcing
    No replayability

    View full-size slide

  107. @titanoboa42
    When to choose

    message oriented middleware

    View full-size slide

  108. @titanoboa42
    Great fit for 

    stateful, distributed architecture

    View full-size slide

  109. @titanoboa42
    Message oriented middleware

    Summary

    View full-size slide

  110. @titanoboa42
    • Great features based on queues and topics
    Message oriented middleware

    View full-size slide

  111. @titanoboa42
    • Great features based on queues and topics
    • Abstracting the complexity of pub sub
    Message oriented middleware

    View full-size slide

  112. @titanoboa42
    • Abstracting the complexity of messages being
    consumed in the future / in parallel
    Message oriented middleware

    View full-size slide

  113. @titanoboa42
    • Abstracting the complexity of messages being
    consumed in the future / in parallel
    • Simple concrete message consumers
    Message oriented middleware

    View full-size slide

  114. @titanoboa42
    • Complex overall system
    Message oriented middleware

    View full-size slide

  115. @titanoboa42
    • Complex overall system
    • Great for stateful, distributed architecture
    Message oriented middleware

    View full-size slide

  116. @titanoboa42
    Event log

    View full-size slide

  117. @titanoboa42
    • Kafka
    Event logs

    View full-size slide

  118. @titanoboa42
    • Kafka
    • …
    Event logs

    View full-size slide

  119. @titanoboa42
    • Events persisted into append-only log
    Event log

    View full-size slide

  120. @titanoboa42
    • Events persisted into append-only log
    • Consumers read shared log
    Event log

    View full-size slide

  121. @titanoboa42
    • Events persisted into append-only log
    • Consumers read shared log
    • Stateless broker (no queues)
    Event log

    View full-size slide

  122. @titanoboa42
    Consumers 

    easy to remove

    View full-size slide

  123. @titanoboa42
    High throughput 

    data streaming

    View full-size slide

  124. @titanoboa42
    Replayability

    View full-size slide

  125. @titanoboa42
    • Single source of truth for event sourcing
    Replayability

    View full-size slide

  126. @titanoboa42
    Great fit for 

    event sourcing

    and real time applications

    View full-size slide

  127. @titanoboa42
    Summary

    View full-size slide

  128. @titanoboa42
    • Queues
    Background jobs

    View full-size slide

  129. @titanoboa42
    • Queues
    • Simple concrete jobs
    Background jobs

    View full-size slide

  130. @titanoboa42
    • Queues
    • Simple concrete jobs
    • For monolithic architecture
    Background jobs

    View full-size slide

  131. @titanoboa42
    • Topics and Queues
    Message oriented middleware

    View full-size slide

  132. @titanoboa42
    • Topics and Queues
    • Simple, stateless message consumers
    Message oriented middleware

    View full-size slide

  133. @titanoboa42
    • Topics and Queues
    • Simple, stateless message consumers
    • For distributed, stateful architecture
    Message oriented middleware

    View full-size slide

  134. @titanoboa42
    • Shared log, no queues
    Event logs

    View full-size slide

  135. @titanoboa42
    • Shared log, no queues
    • Stateful consumers
    Event logs

    View full-size slide

  136. @titanoboa42
    • Shared log, no queues
    • Stateful consumers
    • For event sourcing & real time applications
    Event logs

    View full-size slide

  137. @titanoboa42
    BFCM video

    View full-size slide

  138. @titanoboa42
    BFCM video

    View full-size slide

  139. Thanks!

    Questions?

    @titanoboa42


    https://www.shopify.com/careers

    View full-size slide