EuRuKo 2018: Scaling a monolith isn't scaling microservices

talk at EuRuKo 2018 about background jobs and message oriented middleware

Transcript

1. Kerstin Puschke
   @titanoboa42
   Scaling a monolith

   isn’t scaling microservices
   

   View Slide

2. View Slide

3. @titanoboa42
   Workload distribution 

   in different architectures
   

   View Slide

4. @titanoboa42
   • Background jobs
   Outline
   

   View Slide

5. @titanoboa42
   • Background jobs
   • Features & Challenges
   Outline
   

   View Slide

6. @titanoboa42
   • Background jobs
   • Features & Challenges
   • When to choose
   Outline
   

   View Slide

7. @titanoboa42
   • Message oriented middleware
   Outline
   

   View Slide

8. @titanoboa42
   • Message oriented middleware
   • Features & Challenges
   Outline
   

   View Slide

9. @titanoboa42
   • Message oriented middleware
   • Features & Challenges
   • When to choose
   Outline
   

   View Slide

10. @titanoboa42
    • Event log
    Outline
    

    View Slide

11. @titanoboa42
    • Event log
    • Summary
    Outline
    

    View Slide

12. @titanoboa42
    Background jobs
    

    View Slide

13. @titanoboa42
    • Resque
    Background job backends
    

    View Slide

14. @titanoboa42
    • Resque
    • Sidekiq
    Background job backends
    

    View Slide

15. @titanoboa42
    • Resque
    • Sidekiq
    • …
    Background job backends
    

    View Slide

16. @titanoboa42
    Background job:

    Unit of work 

    to be done later
    App
    Server
    Worker
    

    View Slide

17. @titanoboa42
    Asynchronous
    communication
    App
    Server
    Message
    Queue
    Worker
    

    View Slide

18. @titanoboa42
    Asynchronous
    communication
    App
    Server
    Message
    Queue
    Worker
    Task
    Queue
    

    View Slide

19. @titanoboa42
    Asynchronous
    communication
    App
    Server
    Message
    Queue
    Worker Worker
    Worker
    Task
    Queue
    

    View Slide

20. @titanoboa42
    Background job backend:

    task queue & broker
    

    View Slide

21. @titanoboa42
    Encapsulating

    async communication
    

    View Slide

22. @titanoboa42
    Features
    

    View Slide

23. @titanoboa42
    Task
    Queue
    Response times
    App
    Server
    Worker
    

    View Slide

24. @titanoboa42
    Task
    Queue
    Spikeability
    App
    Server
    Worker
    

    View Slide

25. @titanoboa42
    Task
    Queue
    Parallelization
    App
    Server
    Worker Worker
    Worker
    

    View Slide

26. @titanoboa42
    Task
    Queue
    Retries
    App
    Server
    Worker Worker
    Worker
    

    View Slide

27. @titanoboa42
    Prioritization
    App
    Server
    Worker Worker
    High Prio
    Queue
    Low Prio
    Queue
    

    View Slide

28. @titanoboa42
    Prioritization
    App
    Server
    Worker Worker
    High Prio
    Queue
    Low Prio
    Queue
    

    View Slide

29. @titanoboa42
    Prioritization
    App
    Server
    Worker Worker
    High Prio
    Queue
    Low Prio
    Queue
    

    View Slide

30. @titanoboa42
    Mastering challenges
    

    View Slide

31. @titanoboa42
    No exactly once delivery
    

    View Slide

32. @titanoboa42
    • “At least” vs. “at most” once delivery
    No exactly once delivery
    

    View Slide

33. @titanoboa42
    • “At least” vs. “at most” once delivery
    • Idempotent jobs & at least once delivery
    No exactly once delivery
    

    View Slide

34. @titanoboa42
    Out of order delivery
    

    View Slide

35. @titanoboa42
    • If order matters, queue sequentially
    Out of order delivery
    

    View Slide

36. @titanoboa42
    • If order matters, queue sequentially
    • First job queues follow up jobs
    Out of order delivery
    

    View Slide

37. @titanoboa42
    Long running jobs - Resque
    

    View Slide

38. @titanoboa42
    • Prevent worker shutdown
    Long running jobs - Resque
    

    View Slide

39. @titanoboa42
    • Prevent worker shutdown
    • No deployments
    Long running jobs - Resque
    

    View Slide

40. @titanoboa42
    • Prevent worker shutdown
    • No deployments
    • Not cloud-friendly
    Long running jobs - Resque
    

    View Slide

41. @titanoboa42
    • Aborted and requeued
    Long running jobs - Sidekiq
    

    View Slide

42. @titanoboa42
    • Aborted and requeued
    • Job may not finish before being aborted again
    Long running jobs - Sidekiq
    

    View Slide

43. @titanoboa42
    Large collections
    

    View Slide

44. @titanoboa42
    • Split job into collection and task to be done
    Large collections
    

    View Slide

45. @titanoboa42
    • Split job into collection and task to be done
    • Checkpoint after iteration & requeue
    Large collections
    

    View Slide

46. @titanoboa42
    Interruptible job with automatic resuming
    

    View Slide

47. @titanoboa42
    • Shutdown workers anytime
    Interruptible job with automatic resuming
    

    View Slide

48. @titanoboa42
    • Shutdown workers anytime
    • Disaster prevention
    Interruptible job with automatic resuming
    

    View Slide

49. @titanoboa42
    • Shutdown workers anytime
    • Disaster prevention
    • Data integrity
    Interruptible job with automatic resuming
    

    View Slide

50. @titanoboa42
    Abstracting scaling issues

    simplifies 

    concrete background jobs
    

    View Slide

51. @titanoboa42
    github.com

    /Shopify/job-iteration
    

    View Slide

52. @titanoboa42
    When to choose

    background jobs
    

    View Slide

53. @titanoboa42
    Task
    Queue
    Background jobs are
    ruby objects
    App
    Server
    Worker
    

    View Slide

54. @titanoboa42
    Task
    Queue
    Background jobs are
    ruby objects
    App
    Server
    Worker
    Broker
    Broker
    

    View Slide

55. @titanoboa42
    Great fit for 

    monolithic architecture
    

    View Slide

56. @titanoboa42
    Background jobs

    Summary
    

    View Slide

57. @titanoboa42
    • Great features based on task queues
    Background jobs
    

    View Slide

58. @titanoboa42
    • Great features based on task queues
    • Abstracting the complexity of jobs running in
    the future and / or in parallel
    Background jobs
    

    View Slide

59. @titanoboa42
    • Simple concrete jobs
    Background jobs
    

    View Slide

60. @titanoboa42
    • Simple concrete jobs
    • Complex overall system
    Background jobs
    

    View Slide

61. @titanoboa42
    • Simple concrete jobs
    • Complex overall system
    • Great for monolithic architecture
    Background jobs
    

    View Slide

62. @titanoboa42
    Message oriented middleware
    

    View Slide

63. @titanoboa42
    • Implementations: RabbitMQ, ActiveMQ,…
    Message oriented middleware
    

    View Slide

64. @titanoboa42
    • Implementations: RabbitMQ, ActiveMQ,…
    • Protocols: AMQP, MQTT, Stomp,…
    Message oriented middleware
    

    View Slide

65. @titanoboa42
    Message
    Queue
    Messaging
    Middleware
    App
    Server
    Producer
    Data-based interface
    Worker
    Consumer
    

    View Slide

66. @titanoboa42
    Message
    Queue
    Messaging
    Middleware
    App
    Server
    Producer
    Data-based interface
    Worker
    Consumer
    Broker
    

    View Slide

67. @titanoboa42
    Features
    

    View Slide

68. @titanoboa42
    Decoupling
    

    View Slide

69. @titanoboa42
    Interoperability
    

    View Slide

70. @titanoboa42
    Command messages
    

    View Slide

71. @titanoboa42
    Event messages
    

    View Slide

72. @titanoboa42
    Propagating updates
    Business
    Partners
    Support
    Contracts
    Orders
    

    View Slide

73. @titanoboa42
    Propagating updates
    Business
    Partners
    Support
    Contracts
    Orders
    

    View Slide

74. @titanoboa42
    REST/webhooks

    has disadvantages
    Business
    Partners
    Support
    Contracts
    Orders
    

    View Slide

75. @titanoboa42
    REST/webhooks

    has disadvantages
    Business
    Partners
    Orders
    

    View Slide

76. @titanoboa42
    REST/webhooks

    has disadvantages
    Business
    Partners
    Support
    Contracts
    Orders
    

    View Slide

77. @titanoboa42
    REST/webhooks

    has disadvantages
    Business
    Partners
    Support
    Contracts
    Orders Invoices
    

    View Slide

78. @titanoboa42
    Messaging

    Middleware
    Resiliency
    Business
    Partners
    Invoices
    

    View Slide

79. @titanoboa42
    Messaging

    Middleware
    Resiliency
    Business
    Partners
    

    View Slide

80. @titanoboa42
    Messaging

    Middleware
    Resiliency
    Business
    Partners
    Invoices
    

    View Slide

81. @titanoboa42
    Topic with queues

    provides

    advanced routing
    App
    Server
    Business

    Partners
    Support

    Contracts
    Orders
    Messaging
    Middleware
    

    View Slide

82. @titanoboa42
    Topic with queues

    provides

    advanced routing
    App
    Server
    Message
    Queue
    Business

    Partners
    Support

    Contracts
    Orders
    Message
    Queue
    Messaging
    Middleware
    

    View Slide

83. @titanoboa42
    Messaging
    Middleware
    Anonymity for
    producer and
    consumer
    Business
    Partners
    Support
    Contracts
    Orders
    

    View Slide

84. @titanoboa42
    Messaging
    Middleware
    Anonymity for
    producer and
    consumer
    Business
    Partners
    Invoices
    Support
    Contracts
    Orders
    

    View Slide

85. @titanoboa42
    Messaging
    Middleware
    Anonymity for
    producer and
    consumer
    FraudScore
    Orders
    Support
    Contracts
    

    View Slide

86. @titanoboa42
    Messaging
    Middleware
    Anonymity for
    producer and
    consumer
    Invoices
    FraudScore
    Orders
    Support
    Contracts
    

    View Slide

87. @titanoboa42
    Mastering challenges
    

    View Slide

88. @titanoboa42
    Keep breaking changes manageable
    

    View Slide

89. @titanoboa42
    • Avoid n:m routing
    Keep breaking changes manageable
    

    View Slide

90. @titanoboa42
    • Avoid n:m routing
    • Better representation of domain:

    multiple messages routed 1:n, n:1 instead
    Keep breaking changes manageable
    

    View Slide

91. @titanoboa42
    • Messages are kept if not consumed
    Remove queues if consumer is gone
    

    View Slide

92. @titanoboa42
    • Messages are kept if not consumed
    • Growing queue becomes problematic
    Remove queues if consumer is gone
    

    View Slide

93. @titanoboa42
    • “At least” vs. “at most” once delivery
    No exactly once delivery
    

    View Slide

94. @titanoboa42
    • “At least” vs. “at most” once delivery
    • Idempotent consumer & at least once delivery
    No exactly once delivery
    

    View Slide

95. @titanoboa42
    • If order matters, queue sequentially
    Out of order delivery
    

    View Slide

96. @titanoboa42
    • If order matters, queue sequentially
    • First consumer queues second message
    Out of order delivery
    

    View Slide

97. @titanoboa42
    Event-carried state transfer

    requires order information
    

    View Slide

98. @titanoboa42
    • Payload contains updated data
    Event-carried state transfer

    requires order information
    

    View Slide

99. @titanoboa42
    • Payload contains updated data
    • Avoids requests to service that owns data
    Event-carried state transfer

    requires order information
    

    View Slide

100. @titanoboa42
     • Payload contains updated data
     • Avoids requests to service that owns data
     • Provide order information (e.g. updated_at)
     Event-carried state transfer

     requires order information
     

     View Slide

101. @titanoboa42
     Event notifications are commutative
     

     View Slide

102. @titanoboa42
     • Payload has id only
     Event notifications are commutative
     

     View Slide

103. @titanoboa42
     • Payload has id only
     • Consumer requests data separately
     Event notifications are commutative
     

     View Slide

104. @titanoboa42
     • Payload has id only
     • Consumer requests data separately
     • Consumer doesn’t learn every state
     Event notifications are commutative
     

     View Slide

105. @titanoboa42
     No replayability
     

     View Slide

106. @titanoboa42
     • Messages removed after processing
     No replayability
     

     View Slide

107. @titanoboa42
     • Messages removed after processing
     • No single source of truth for event sourcing
     No replayability
     

     View Slide

108. @titanoboa42
     When to choose

     message oriented middleware
     

     View Slide

109. @titanoboa42
     Great fit for 

     stateful, distributed architecture
     

     View Slide

110. @titanoboa42
     Message oriented middleware

     Summary
     

     View Slide

111. @titanoboa42
     • Great features based on queues and topics
     Message oriented middleware
     

     View Slide

112. @titanoboa42
     • Great features based on queues and topics
     • Abstracting the complexity of pub sub
     Message oriented middleware
     

     View Slide

113. @titanoboa42
     • Abstracting the complexity of messages being
     consumed in the future / in parallel
     Message oriented middleware
     

     View Slide

114. @titanoboa42
     • Abstracting the complexity of messages being
     consumed in the future / in parallel
     • Simple concrete message consumers
     Message oriented middleware
     

     View Slide

115. @titanoboa42
     • Complex overall system
     Message oriented middleware
     

     View Slide

116. @titanoboa42
     • Complex overall system
     • Great for stateful, distributed architecture
     Message oriented middleware
     

     View Slide

117. @titanoboa42
     Event log
     

     View Slide

118. @titanoboa42
     • Kafka
     Event logs
     

     View Slide

119. @titanoboa42
     • Kafka
     • …
     Event logs
     

     View Slide

120. @titanoboa42
     • Events persisted into append-only log
     Event log
     

     View Slide

121. @titanoboa42
     • Events persisted into append-only log
     • Consumers read shared log
     Event log
     

     View Slide

122. @titanoboa42
     • Events persisted into append-only log
     • Consumers read shared log
     • Stateless broker (no queues)
     Event log
     

     View Slide

123. @titanoboa42
     Consumers 

     easy to remove
     

     View Slide

124. @titanoboa42
     High throughput 

     data streaming
     

     View Slide

125. @titanoboa42
     Replayability
     

     View Slide

126. @titanoboa42
     • Single source of truth for event sourcing
     Replayability
     

     View Slide

127. @titanoboa42
     Great fit for 

     event sourcing

     and real time applications
     

     View Slide

128. @titanoboa42
     Summary
     

     View Slide

129. @titanoboa42
     • Queues
     Background jobs
     

     View Slide

130. @titanoboa42
     • Queues
     • Simple concrete jobs
     Background jobs
     

     View Slide

131. @titanoboa42
     • Queues
     • Simple concrete jobs
     • For monolithic architecture
     Background jobs
     

     View Slide

132. @titanoboa42
     • Topics and Queues
     Message oriented middleware
     

     View Slide

133. @titanoboa42
     • Topics and Queues
     • Simple, stateless message consumers
     Message oriented middleware
     

     View Slide

134. @titanoboa42
     • Topics and Queues
     • Simple, stateless message consumers
     • For distributed, stateful architecture
     Message oriented middleware
     

     View Slide

135. @titanoboa42
     • Shared log, no queues
     Event logs
     

     View Slide

136. @titanoboa42
     • Shared log, no queues
     • Stateful consumers
     Event logs
     

     View Slide

137. @titanoboa42
     • Shared log, no queues
     • Stateful consumers
     • For event sourcing & real time applications
     Event logs
     

     View Slide

138. @titanoboa42
     BFCM video
     

     View Slide

139. @titanoboa42
     BFCM video
     

     View Slide

140. Thanks!

     Questions?

     @titanoboa42

     

     https://www.shopify.com/careers
     

     View Slide