EuRuKo 2018: Scaling a monolith isn't scaling microservices

EuRuKo 2018: Scaling a monolith isn't scaling microservices

talk at EuRuKo 2018 about background jobs and message oriented middleware

5e8e44a4f6632772c47925006aff31d9?s=128

Kerstin Puschke

August 25, 2018
Tweet

Transcript

  1. Kerstin Puschke @titanoboa42 Scaling a monolith
 isn’t scaling microservices

  2. None
  3. @titanoboa42 Workload distribution 
 in different architectures

  4. @titanoboa42 • Background jobs Outline

  5. @titanoboa42 • Background jobs • Features & Challenges Outline

  6. @titanoboa42 • Background jobs • Features & Challenges • When

    to choose Outline
  7. @titanoboa42 • Message oriented middleware Outline

  8. @titanoboa42 • Message oriented middleware • Features & Challenges Outline

  9. @titanoboa42 • Message oriented middleware • Features & Challenges •

    When to choose Outline
  10. @titanoboa42 • Event log Outline

  11. @titanoboa42 • Event log • Summary Outline

  12. @titanoboa42 Background jobs

  13. @titanoboa42 • Resque Background job backends

  14. @titanoboa42 • Resque • Sidekiq Background job backends

  15. @titanoboa42 • Resque • Sidekiq • … Background job backends

  16. @titanoboa42 Background job:
 Unit of work 
 to be done

    later App Server Worker
  17. @titanoboa42 Asynchronous communication App Server Message Queue Worker

  18. @titanoboa42 Asynchronous communication App Server Message Queue Worker Task Queue

  19. @titanoboa42 Asynchronous communication App Server Message Queue Worker Worker Worker

    Task Queue
  20. @titanoboa42 Background job backend:
 task queue & broker

  21. @titanoboa42 Encapsulating
 async communication

  22. @titanoboa42 Features

  23. @titanoboa42 Task Queue Response times App Server Worker

  24. @titanoboa42 Task Queue Spikeability App Server Worker

  25. @titanoboa42 Task Queue Parallelization App Server Worker Worker Worker

  26. @titanoboa42 Task Queue Retries App Server Worker Worker Worker

  27. @titanoboa42 Prioritization App Server Worker Worker High Prio Queue Low

    Prio Queue
  28. @titanoboa42 Prioritization App Server Worker Worker High Prio Queue Low

    Prio Queue
  29. @titanoboa42 Prioritization App Server Worker Worker High Prio Queue Low

    Prio Queue
  30. @titanoboa42 Mastering challenges

  31. @titanoboa42 No exactly once delivery

  32. @titanoboa42 • “At least” vs. “at most” once delivery No

    exactly once delivery
  33. @titanoboa42 • “At least” vs. “at most” once delivery •

    Idempotent jobs & at least once delivery No exactly once delivery
  34. @titanoboa42 Out of order delivery

  35. @titanoboa42 • If order matters, queue sequentially Out of order

    delivery
  36. @titanoboa42 • If order matters, queue sequentially • First job

    queues follow up jobs Out of order delivery
  37. @titanoboa42 Long running jobs - Resque

  38. @titanoboa42 • Prevent worker shutdown Long running jobs - Resque

  39. @titanoboa42 • Prevent worker shutdown • No deployments Long running

    jobs - Resque
  40. @titanoboa42 • Prevent worker shutdown • No deployments • Not

    cloud-friendly Long running jobs - Resque
  41. @titanoboa42 • Aborted and requeued Long running jobs - Sidekiq

  42. @titanoboa42 • Aborted and requeued • Job may not finish

    before being aborted again Long running jobs - Sidekiq
  43. @titanoboa42 Large collections

  44. @titanoboa42 • Split job into collection and task to be

    done Large collections
  45. @titanoboa42 • Split job into collection and task to be

    done • Checkpoint after iteration & requeue Large collections
  46. @titanoboa42 Interruptible job with automatic resuming

  47. @titanoboa42 • Shutdown workers anytime Interruptible job with automatic resuming

  48. @titanoboa42 • Shutdown workers anytime • Disaster prevention Interruptible job

    with automatic resuming
  49. @titanoboa42 • Shutdown workers anytime • Disaster prevention • Data

    integrity Interruptible job with automatic resuming
  50. @titanoboa42 Abstracting scaling issues
 simplifies 
 concrete background jobs

  51. @titanoboa42 github.com
 /Shopify/job-iteration

  52. @titanoboa42 When to choose
 background jobs

  53. @titanoboa42 Task Queue Background jobs are ruby objects App Server

    Worker
  54. @titanoboa42 Task Queue Background jobs are ruby objects App Server

    Worker Broker Broker
  55. @titanoboa42 Great fit for 
 monolithic architecture

  56. @titanoboa42 Background jobs
 Summary

  57. @titanoboa42 • Great features based on task queues Background jobs

  58. @titanoboa42 • Great features based on task queues • Abstracting

    the complexity of jobs running in the future and / or in parallel Background jobs
  59. @titanoboa42 • Simple concrete jobs Background jobs

  60. @titanoboa42 • Simple concrete jobs • Complex overall system Background

    jobs
  61. @titanoboa42 • Simple concrete jobs • Complex overall system •

    Great for monolithic architecture Background jobs
  62. @titanoboa42 Message oriented middleware

  63. @titanoboa42 • Implementations: RabbitMQ, ActiveMQ,… Message oriented middleware

  64. @titanoboa42 • Implementations: RabbitMQ, ActiveMQ,… • Protocols: AMQP, MQTT, Stomp,…

    Message oriented middleware
  65. @titanoboa42 Message Queue Messaging Middleware App Server Producer Data-based interface

    Worker Consumer
  66. @titanoboa42 Message Queue Messaging Middleware App Server Producer Data-based interface

    Worker Consumer Broker
  67. @titanoboa42 Features

  68. @titanoboa42 Decoupling

  69. @titanoboa42 Interoperability

  70. @titanoboa42 Command messages

  71. @titanoboa42 Event messages

  72. @titanoboa42 Propagating updates Business Partners Support Contracts Orders

  73. @titanoboa42 Propagating updates Business Partners Support Contracts Orders

  74. @titanoboa42 REST/webhooks
 has disadvantages Business Partners Support Contracts Orders

  75. @titanoboa42 REST/webhooks
 has disadvantages Business Partners Orders

  76. @titanoboa42 REST/webhooks
 has disadvantages Business Partners Support Contracts Orders

  77. @titanoboa42 REST/webhooks
 has disadvantages Business Partners Support Contracts Orders Invoices

  78. @titanoboa42 Messaging
 Middleware Resiliency Business Partners Invoices

  79. @titanoboa42 Messaging
 Middleware Resiliency Business Partners

  80. @titanoboa42 Messaging
 Middleware Resiliency Business Partners Invoices

  81. @titanoboa42 Topic with queues
 provides
 advanced routing App Server Business


    Partners Support
 Contracts Orders Messaging Middleware
  82. @titanoboa42 Topic with queues
 provides
 advanced routing App Server Message

    Queue Business
 Partners Support
 Contracts Orders Message Queue Messaging Middleware
  83. @titanoboa42 Messaging Middleware Anonymity for producer and consumer Business Partners

    Support Contracts Orders
  84. @titanoboa42 Messaging Middleware Anonymity for producer and consumer Business Partners

    Invoices Support Contracts Orders
  85. @titanoboa42 Messaging Middleware Anonymity for producer and consumer FraudScore Orders

    Support Contracts
  86. @titanoboa42 Messaging Middleware Anonymity for producer and consumer Invoices FraudScore

    Orders Support Contracts
  87. @titanoboa42 Mastering challenges

  88. @titanoboa42 Keep breaking changes manageable

  89. @titanoboa42 • Avoid n:m routing Keep breaking changes manageable

  90. @titanoboa42 • Avoid n:m routing • Better representation of domain:


    multiple messages routed 1:n, n:1 instead Keep breaking changes manageable
  91. @titanoboa42 • Messages are kept if not consumed Remove queues

    if consumer is gone
  92. @titanoboa42 • Messages are kept if not consumed • Growing

    queue becomes problematic Remove queues if consumer is gone
  93. @titanoboa42 • “At least” vs. “at most” once delivery No

    exactly once delivery
  94. @titanoboa42 • “At least” vs. “at most” once delivery •

    Idempotent consumer & at least once delivery No exactly once delivery
  95. @titanoboa42 • If order matters, queue sequentially Out of order

    delivery
  96. @titanoboa42 • If order matters, queue sequentially • First consumer

    queues second message Out of order delivery
  97. @titanoboa42 Event-carried state transfer
 requires order information

  98. @titanoboa42 • Payload contains updated data Event-carried state transfer
 requires

    order information
  99. @titanoboa42 • Payload contains updated data • Avoids requests to

    service that owns data Event-carried state transfer
 requires order information
  100. @titanoboa42 • Payload contains updated data • Avoids requests to

    service that owns data • Provide order information (e.g. updated_at) Event-carried state transfer
 requires order information
  101. @titanoboa42 Event notifications are commutative

  102. @titanoboa42 • Payload has id only Event notifications are commutative

  103. @titanoboa42 • Payload has id only • Consumer requests data

    separately Event notifications are commutative
  104. @titanoboa42 • Payload has id only • Consumer requests data

    separately • Consumer doesn’t learn every state Event notifications are commutative
  105. @titanoboa42 No replayability

  106. @titanoboa42 • Messages removed after processing No replayability

  107. @titanoboa42 • Messages removed after processing • No single source

    of truth for event sourcing No replayability
  108. @titanoboa42 When to choose
 message oriented middleware

  109. @titanoboa42 Great fit for 
 stateful, distributed architecture

  110. @titanoboa42 Message oriented middleware
 Summary

  111. @titanoboa42 • Great features based on queues and topics Message

    oriented middleware
  112. @titanoboa42 • Great features based on queues and topics •

    Abstracting the complexity of pub sub Message oriented middleware
  113. @titanoboa42 • Abstracting the complexity of messages being consumed in

    the future / in parallel Message oriented middleware
  114. @titanoboa42 • Abstracting the complexity of messages being consumed in

    the future / in parallel • Simple concrete message consumers Message oriented middleware
  115. @titanoboa42 • Complex overall system Message oriented middleware

  116. @titanoboa42 • Complex overall system • Great for stateful, distributed

    architecture Message oriented middleware
  117. @titanoboa42 Event log

  118. @titanoboa42 • Kafka Event logs

  119. @titanoboa42 • Kafka • … Event logs

  120. @titanoboa42 • Events persisted into append-only log Event log

  121. @titanoboa42 • Events persisted into append-only log • Consumers read

    shared log Event log
  122. @titanoboa42 • Events persisted into append-only log • Consumers read

    shared log • Stateless broker (no queues) Event log
  123. @titanoboa42 Consumers 
 easy to remove

  124. @titanoboa42 High throughput 
 data streaming

  125. @titanoboa42 Replayability

  126. @titanoboa42 • Single source of truth for event sourcing Replayability

  127. @titanoboa42 Great fit for 
 event sourcing
 and real time

    applications
  128. @titanoboa42 Summary

  129. @titanoboa42 • Queues Background jobs

  130. @titanoboa42 • Queues • Simple concrete jobs Background jobs

  131. @titanoboa42 • Queues • Simple concrete jobs • For monolithic

    architecture Background jobs
  132. @titanoboa42 • Topics and Queues Message oriented middleware

  133. @titanoboa42 • Topics and Queues • Simple, stateless message consumers

    Message oriented middleware
  134. @titanoboa42 • Topics and Queues • Simple, stateless message consumers

    • For distributed, stateful architecture Message oriented middleware
  135. @titanoboa42 • Shared log, no queues Event logs

  136. @titanoboa42 • Shared log, no queues • Stateful consumers Event

    logs
  137. @titanoboa42 • Shared log, no queues • Stateful consumers •

    For event sourcing & real time applications Event logs
  138. @titanoboa42 BFCM video

  139. @titanoboa42 BFCM video

  140. Thanks!
 Questions?
 @titanoboa42
 
 https://www.shopify.com/careers