$30 off During Our Annual Pro Sale. View Details »

Distributed Scheduling com Spring Boot: os desafios e percalços de implementar um job em background

Distributed Scheduling com Spring Boot: os desafios e percalços de implementar um job em background

Cedo ou tarde um desenvolvedor(a) implementará seu primeiro job em background, o que geralmente é uma tarefa simples para maioria dos sistemas pode se tornar um pesadelo em sistemas que precisam lidar com grande volumetria de dados, alta performance e paralelismo. Cenários como esses escondem diversos problemas na qual a maioria dos desenvolvedores não está acostumado, como volume de dados, falhas na rede, reprocessamento indevido, sobrecarga no banco de dados, erros de estouro de memória até indisponibilidade de todo o sistema.

Lidar com boa parte desses problemas não requer tecnologias e serviços da moda (hype), mas sim fundamentos sólidos em sistemas distribuídos. Nessa talk, pretendo apresentar como um desenvolvedor(a) experiente implementa um job em background levando em conta os principais percalços e desenha uma solução para escala horizontal ao mesmo tempo em que tira proveito das tecnologias adotadas pelo time de desenvolvimento.

Se você acredita que um job em background é uma tarefa simples, então essa talk é para você!

(GRAVAÇÃO: https://youtu.be/I_kEO_HPfBU?list=PLHMMERsvy9EyWQPru4SrJAYHEGKfkjRgP&t=130)

Rafael Ponte

August 25, 2021
Tweet

More Decks by Rafael Ponte

Other Decks in Technology

Transcript

  1. DISTRIBUTED SCHEDULING | Spring Boot os desa fi os e

    percalços de implementar um job em background
  2. None
  3. Maaaaas implementar um job em background…

  4. None
  5. https://deniseyu.io/art/

  6. Computação paralela e distribuída

  7. Cadê meu cartão?

  8. None
  9. None
  10. None
  11. db servidor

  12. db servidor

  13. db servidor

  14. db servidor

  15. db servidor

  16. db servidor

  17. db servidor

  18. db servidor

  19. db servidor

  20. db servidor

  21. Como agendar tarefas com Spring Boot?

  22. @Component public class OneJob { @Scheduled(fixedDelay = 60_000) public void

    runQuiteOften() { // lógica do job vai aqui } }
  23. @Component public class OneJob { @Scheduled(fixedDelay = 60_000) public void

    runQuiteOften() { // lógica do job vai aqui } }
  24. @Component public class OneJob { @Scheduled(fixedDelay = 60_000) public void

    runQuiteOften() { // lógica do job vai aqui } }
  25. @Component public class OneJob { @Scheduled(fixedDelay = 60_000) public void

    runQuiteOften() { // lógica do job vai aqui } }
  26. @Component public class OneJob { public void runQuiteOften() { //

    lógica do job vai aqui } }
  27. @Component public class OneJob { @Scheduled public void runQuiteOften() {

    // lógica do job vai aqui } }
  28. @Component public class OneJob { @Scheduled(fixedDelay = 60_000) public void

    runQuiteOften() { // lógica do job vai aqui } }
  29. @Component public class OneJob { @Scheduled(fixedDelay = 60_000) public void

    runQuiteOften() { // lógica do job vai aqui } }
  30. E nossa tarefa?

  31. db servidor

  32. db servidor

  33. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  34. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  35. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  36. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  37. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  38. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  39. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  40. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  41. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  42. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  43. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  44. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  45. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  46. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  47. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  48. Solução Ingênua

  49. Rafael Ponte @rponte

  50. None
  51. Fortaleza - Terra do Sol

  52. None
  53. DISTRIBUTED SCHEDULING | Spring Boot os desa fi os e

    percalços de implementar um job em background
  54. Solução Ingênua

  55. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  56. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  57. select p.* from proposal p where p.status = 'ELIGIBLE' order

    by p.created_at asc
  58. E SE…

  59. …volume de dados crescer?

  60. None
  61. None
  62. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  63. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  64. @Repository public interface ProposalRepository extends JpaRepository<Proposal, UUID> { public List<Proposal>

    findAllByStatusOrderByCreatedAtAsc(ProposalStatus status); }
  65. @Repository public interface ProposalRepository extends JpaRepository<Proposal, UUID> { public List<Proposal>

    findAllByStatusOrderByCreatedAtAsc(ProposalStatus status); }
  66. @Repository public interface ProposalRepository extends JpaRepository<Proposal, UUID> { public List<Proposal>

    findAllByStatusOrderByCreatedAtAsc(ProposalStatus status); }
  67. select p.* from proposal p where p.status = 'ELIGIBLE' order

    by p.created_at asc
  68. List<Proposal> findAllByStatusOrderByCreatedAtAsc(...);

  69. List<Proposal> findAllByStatusOrderByCreatedAtAsc(...);

  70. List<Proposal> findTop50ByStatusOrderByCreatedAtAsc(...);

  71. select p.* from proposal p where p.status = 'ELIGIBLE' order

    by p.created_at asc
  72. select p.* from proposal p where p.status = 'ELIGIBLE' order

    by p.created_at asc limit 50
  73. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  74. Mas…

  75. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  76. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } } o loop vai encerrar muito CEDO !
  77. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } } Só vai rodar no próximo agendamento!
  78. while(true)

  79. @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals

    = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); proposals.forEach(proposal -> { // processa cada proposta }); }
  80. @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { while (true)

    { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { // processa cada proposta }); } }
  81. @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { while (true)

    { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { // processa cada proposta }); } }
  82. @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { while (true)

    { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { // processa cada proposta }); } }
  83. @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { while (true)

    { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { // processa cada proposta }); } }
  84. @Component public class AttachCardsToProposalsJob { // dependências e constantes @Transactional

    @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } } }
  85. Mas ao rodar…

  86. None
  87. Hein?

  88. @Component public class AttachCardsToProposalsJob { // dependências e constantes @Transactional

    @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } } }
  89. @Component public class AttachCardsToProposalsJob { // dependências e constantes @Transactional

    @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } } } - carregando apenas 50 objetos por vez
  90. @Component public class AttachCardsToProposalsJob { // dependências e constantes @Transactional

    @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } } } - criando nova referência - carregando apenas 50 objetos por vez
  91. @Component public class AttachCardsToProposalsJob { // dependências e constantes @Transactional

    @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } } } - criando nova referência - carregando apenas 50 objetos por vez - saindo do loop na hora certa
  92. Por que?

  93. @Component public class AttachCardsToProposalsJob { // dependências e constantes @Transactional

    @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } } }
  94. @Repository public interface ProposalRepository extends JpaRepository<Proposal, UUID> { public List<Proposal>

    findAllByStatusOrderByCreatedAtAsc(ProposalStatus status); }
  95. Revisitando JPA e Hibernate

  96. Job

  97. Repository Job

  98. Repository EM Job

  99. Repository EM Job

  100. Repository EM Job

  101. Repository EM Job

  102. Repository EM Job

  103. Repository EM Job

  104. Repository EM Job

  105. Long-running transaction

  106. @Transactional Repository EM Job Long-running transaction

  107. @Transactional Repository EM Job Long-running transaction

  108. @Transactional Repository EM Job Long-running transaction

  109. @Transactional Repository EM Job (First-Level Cache) Long-running transaction

  110. @Transactional Repository EM Job (First-Level Cache) Long-running transaction

  111. A sacada aqui é entender…

  112. @Transactional Repository EM Job (First-Level Cache) Long-running transaction

  113. @Transactional EM (First-Level Cache) Long-running transaction

  114. @Transactional EM (First-Level Cache) Long-running transaction

  115. EM (First-Level Cache) @Transactional Long-running transaction

  116. EM (First-Level Cache) @Transactional Long-running transaction

  117. Repository EM Job (First-Level Cache) @Transactional Long-running transaction

  118. Repository EM Job (First-Level Cache) Long-running transaction

  119. Ahhh!!

  120. @Component public class AttachCardsToProposalsJob { // dependências e constantes @Transactional

    @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } } }
  121. @Component public class AttachCardsToProposalsJob { // dependências e constantes @Scheduled(fixedDelay

    = 60_000) public void execute() { while (true) { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } } } inicia e termina uma transação
  122. Mas…

  123. @Component public class AttachCardsToProposalsJob { // dependências e constantes @Scheduled(fixedDelay

    = 60_000) public void execute() { while (true) { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } } }
  124. @Component public class AttachCardsToProposalsJob { // dependências e constantes @Scheduled(fixedDelay

    = 60_000) public void execute() { while (true) { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); // inicia e comita transação proposal.attachTo(newCard); proposalRepository.save(proposal); // inicia e comita transação }); } } } analogo a 
 auto_commit = true
  125. @Component public class AttachCardsToProposalsJob { // dependências e constantes @Scheduled(fixedDelay

    = 60_000) public void execute() { while (true) { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); // inicia e comita transação proposal.attachTo(newCard); proposalRepository.save(proposal); // inicia e comita transação }); } } } Abrimos mão da CONSISTÊNCIA dos dados!
  126. @Component

  127. Short-running transaction

  128. Job Transaction Job

  129. Job Transaction Job inicio do job fi m do job

  130. Job Transaction Job

  131. inicio da transação fi m da transação Job Job Transaction

  132. Job Transaction Job Long-running transaction

  133. Job Transaction Job

  134. Job Transaction Job

  135. Job Transaction Job inicio da transação fi m da transação

  136. Job Transaction Job Short-running transaction

  137. Job Transaction Job

  138. Job Transaction Job

  139. Job Transaction Job

  140. Job Transaction Job

  141. Job Transaction Job

  142. Job Transaction Job Short-running transaction

  143. Repository EM Job (First-Level Cache) Short-running transaction

  144. Repository EM Job @Transactional Short-running transaction (First-Level Cache)

  145. Repository EM Job @Transactional Short-running transaction (First-Level Cache)

  146. Repository EM Job @Transactional Short-running transaction (First-Level Cache)

  147. @Transactional Repository EM Job (First-Level Cache) Short-running transaction

  148. Repository EM Job @Transactional Short-running transaction (First-Level Cache)

  149. Controle transacional programático

  150. @Component public class AttachCardsToProposalsJob { @Scheduled(fixedDelay = 60_000) public void

    execute() { while (true) { // executa código em escopo transacional } } }
  151. @Component public class AttachCardsToProposalsJob { @Scheduled(fixedDelay = 60_000) public void

    execute() { while (true) { @Transactional.execute(transaction -> { // executa código em escopo transacional }); } } }
  152. @Component public class AttachCardsToProposalsJob { @Scheduled(fixedDelay = 60_000) public void

    execute() { while (true) { @Transactional.execute(transaction -> { // executa código em escopo transacional }); } } }
  153. @Component public class AttachCardsToProposalsJob { @Scheduled(fixedDelay = 60_000) public void

    execute() { while (true) { @Transactional.execute(transaction -> { // executa código em escopo transacional }); } } }
  154. @Component public class AttachCardsToProposalsJob { @Scheduled(fixedDelay = 60_000) public void

    execute() { while (true) { @Transactional.execute(transaction -> { // executa código em escopo transacional }); } } }
  155. @Component public class AttachCardsToProposalsJob { @Scheduled(fixedDelay = 60_000) public void

    execute() { while (true) { @Transactional.execute(transaction -> { // executa código em escopo transacional }); } } }
  156. @Component public class AttachCardsToProposalsJob { @Scheduled(fixedDelay = 60_000) public void

    execute() { while (true) { @Transactional.execute(transaction -> { // executa código em escopo transacional }); } } }
  157. @Component public class AttachCardsToProposalsJob { @Scheduled(fixedDelay = 60_000) public void

    execute() { while (true) { @Transactional.execute(transaction -> { // executa código em escopo transacional }); } } }
  158. @Component public class AttachCardsToProposalsJob { @Scheduled(fixedDelay = 60_000) public void

    execute() { while (true) { transactionManager.execute(transaction -> { // executa código em escopo transacional }); } } }
  159. @Component public class AttachCardsToProposalsJob { private TransactionTemplate transactionManager; @Scheduled(fixedDelay =

    60_000) public void execute() { while (true) { transactionManager.execute(transaction -> { // executa código em escopo transacional }); } } }
  160. @Component public class AttachCardsToProposalsJob { private TransactionTemplate transactionManager; @Scheduled(fixedDelay =

    60_000) public void execute() { while (true) { transactionManager.execute(transaction -> { // executa código em escopo transacional }); } } }
  161. @Autowired private TransactionTemplate transactionManager; @Scheduled(fixedDelay = 60_000) public void execute()

    { boolean pending = true; while (pending) { pending = transactionManager.execute(transactionStatus -> { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { return false; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); return true; }); } }
  162. @Autowired private TransactionTemplate transactionManager; @Scheduled(fixedDelay = 60_000) public void execute()

    { boolean pending = true; while (pending) { pending = transactionManager.execute(transactionStatus -> { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { return false; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); return true; }); } }
  163. None
  164. inicio do job fi m do job

  165. E SE…

  166. …rodarmos a aplicação em cluster?

  167. None
  168. None
  169. None
  170. None
  171. ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE 1004 ELIGIBLE

  172. ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE 1004 ELIGIBLE

  173. ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE 1004 ELIGIBLE

  174. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE 1003 ELIGIBLE 1004 ELIGIBLE

  175. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE 1003 ELIGIBLE 1004 ELIGIBLE

  176. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE 1004 ELIGIBLE

  177. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE 1004 ELIGIBLE

  178. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE_WITH_ATTACHED_CARD 1004 ELIGIBLE

  179. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE_WITH_ATTACHED_CARD 1004 ELIGIBLE

  180. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE_WITH_ATTACHED_CARD 1004 ELIGIBLE_WITH_ATTACHED_CARD

  181. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE_WITH_ATTACHED_CARD 1004 ELIGIBLE_WITH_ATTACHED_CARD

  182. None
  183. ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE 1004 ELIGIBLE

  184. ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE 1004 ELIGIBLE

  185. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE 1003 ELIGIBLE 1004 ELIGIBLE

  186. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE 1003 ELIGIBLE 1004 ELIGIBLE

    2x
  187. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE 1004 ELIGIBLE

    2x
  188. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE 1004 ELIGIBLE

    2x 2x
  189. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE_WITH_ATTACHED_CARD 1004 ELIGIBLE

    2x 2x
  190. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE_WITH_ATTACHED_CARD 1004 ELIGIBLE

    2x 2x 2x
  191. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE_WITH_ATTACHED_CARD 1004 ELIGIBLE_WITH_ATTACHED_CARD

    2x 2x 2x
  192. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE_WITH_ATTACHED_CARD 1004 ELIGIBLE_WITH_ATTACHED_CARD

    2x 2x 2x 2x
  193. Oops!

  194. Synchronized?

  195. @Component public class AttachCardsToProposalsJob { @Scheduled(...) public synchronized void execute()

    { // executa lógica aqui } }
  196. None
  197. None
  198. None
  199. None
  200. @Component

  201. Distributed Lock?

  202. @Scheduled(...) public void execute() { if (isLeader()) { // executa

    lógica aqui } }
  203. @Scheduled(...) public void execute() { if (isLeader()) { // executa

    lógica aqui } }
  204. None
  205. None
  206. None
  207. None
  208. None
  209. None
  210. None
  211. None
  212. None
  213. None
  214. None
  215. Não!

  216. Que tal…

  217. None
  218. None
  219. @Repository public interface ProposalRepository extends JpaRepository<Proposal, UUID> { public List<Proposal>

    findTop50ByStatusOrderByCreatedAtAsc(ProposalStatus status); }
  220. @Repository public interface ProposalRepository extends JpaRepository<Proposal, UUID> { @Lock(LockModeType.PESSIMISTIC_WRITE) public

    List<Proposal> findTop50ByStatusOrderByCreatedAtAsc(ProposalStatus status); }
  221. select p.* from proposal p where p.status = 'ELIGIBLE' order

    by p.created_at asc limit 50 

  222. select p.* from proposal p where p.status = 'ELIGIBLE' order

    by p.created_at asc limit 50 
 for update
  223. ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE ... ...

  224. ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE ... ...

  225. ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE ... ...

  226. ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE ... ...

  227. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE 1003 ELIGIBLE ... ...

  228. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE 1003 ELIGIBLE ... ...

  229. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE 1003 ELIGIBLE ... ...

  230. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE 1003 ELIGIBLE ... ...

  231. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE 1003 ELIGIBLE ... ...

  232. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE 1003 ELIGIBLE ... ...

  233. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE ... ...

  234. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE ... ...

  235. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE ... ...

  236. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE ... ...

  237. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE_WITH_ATTACHED_CARD ... ...

  238. SIM!

  239. E SE…

  240. …precisarmos processar em paralelo?

  241. None
  242. None
  243. None
  244. None
  245. Para isso…

  246. 9630 9631 QUEUE ( fi la) 9629

  247. 9630 9631 QUEUE ( fi la) producer 9629 produz: 


    1000 itens/min
  248. 9630 9631 QUEUE ( fi la) producer consumer 9629 produz:

    
 1000 itens/min consume: 
 600 itens/min
  249. 9630 9631 QUEUE ( fi la) producer consumer 9629 produz:

    
 1000 itens/min consumer consume: 
 400 itens/min consume: 
 600 itens/min
  250. 9630 9631 QUEUE ( fi la) producer consumer 9629 produz:

    
 1000 itens/min consumer consume: 
 400 itens/min consume: 
 600 itens/min
  251. None
  252. Não!

  253. Que tal…

  254. None
  255. @Repository public interface ProposalRepository extends JpaRepository<Proposal, UUID> { @QueryHints({ @QueryHint(

    name = "javax.persistence.lock.timeout", value = LockOptions.SKIP_LOCKED) // org.hibernate.LockOptions }) @Lock(LockModeType.PESSIMISTIC_WRITE) public List<Proposal> findTop50ByStatusOrderByCreatedAtAsc(ProposalStatus status); }
  256. @Repository public interface ProposalRepository extends JpaRepository<Proposal, UUID> { @QueryHints({ @QueryHint(

    name = "javax.persistence.lock.timeout", value = LockOptions.SKIP_LOCKED) // org.hibernate.LockOptions }) @Lock(LockModeType.PESSIMISTIC_WRITE) public List<Proposal> findTop50ByStatusOrderByCreatedAtAsc(ProposalStatus status); }
  257. select p.* from proposal p where p.status = 'ELIGIBLE' order

    by p.created_at asc limit 50 
 for update
  258. select p.* from proposal p where p.status = 'ELIGIBLE' order

    by p.created_at asc limit 50 
 for update skip locked
  259. ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE ... ...

  260. ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE ... ...

  261. ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE ... ...

  262. ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE ... ...

  263. ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE ... ...

  264. ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE ... ...

  265. ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE ... ...

  266. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE ... ...

  267. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE ... ...

  268. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE ... ...

  269. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE ... ...

  270. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE ... ...

  271. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE ... ...

  272. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE ... ...

  273. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE ... ...

  274. Ou seja…

  275. ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE 1004 ELIGIBLE

  276. ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE 1004 ELIGIBLE

  277. SIM!

  278. E SE…

  279. …impactar no startup da aplicação?

  280. @Scheduled( fixedDelay = 60_000 ) public void execute() { //

    executa lógica aqui }
  281. initialDelay

  282. @Scheduled( fixedDelay = 60_000, initialDelay = 120_000 ) public void

    execute() { // executa lógica aqui }
  283. E SE…

  284. …precisarmos de maior throughput?

  285. Hibernate batch size

  286. @Autowired private TransactionTemplate transactionManager; @Scheduled(fixedDelay = 60_000) public void execute()

    { boolean pending = true; while (pending) { pending = transactionManager.execute(transactionStatus -> { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { return false; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); return true; }); } }
  287. @Autowired private TransactionTemplate transactionManager; @Scheduled(fixedDelay = 60_000) public void execute()

    { boolean pending = true; while (pending) { pending = transactionManager.execute(transactionStatus -> { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { return false; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); return true; }); } }
  288. @Autowired private TransactionTemplate transactionManager; @Scheduled(fixedDelay = 60_000) public void execute()

    { boolean pending = true; while (pending) { pending = transactionManager.execute(transactionStatus -> { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { return false; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); return true; }); } }
  289. @Autowired private TransactionTemplate transactionManager; @Scheduled(fixedDelay = 60_000) public void execute()

    { boolean pending = true; while (pending) { pending = transactionManager.execute(transactionStatus -> { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { return false; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); return true; }); } }
  290. @Autowired private TransactionTemplate transactionManager; @Scheduled(fixedDelay = 60_000) public void execute()

    { boolean pending = true; while (pending) { pending = transactionManager.execute(transactionStatus -> { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { return false; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); return true; }); } } Insere novo cartão Atualiza proposta com cartão
  291. Para cada iteração do loop…

  292. 2 roundtrips INSERT INTO card(cardNumber) VALUES ('5260-9991-9040-0001'); UPDATE proposal SET

    status = 'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2021;
  293. 2 roundtrips INSERT INTO card(cardNumber) VALUES ('5260-9991-9040-0001'); UPDATE proposal SET

    status = 'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2021;
  294. 2 roundtrips INSERT INTO card(cardNumber) VALUES ('5260-9991-9040-0001'); UPDATE proposal SET

    status = 'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2021;
  295. INSERT INTO card(cardNumber) VALUES ('5260-9991-9040-0001'); UPDATE proposal SET status =

    'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2021; INSERT INTO card(cardNumber) VALUES (' 5513-8141-9261-0002'); UPDATE proposal SET status = 'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2022; 4 roundtrips
  296. INSERT INTO card(cardNumber) VALUES ('5260-9991-9040-0001'); UPDATE proposal SET status =

    'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2021; INSERT INTO card(cardNumber) VALUES (' 5513-8141-9261-0002'); UPDATE proposal SET status = 'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2022; INSERT INTO card(cardNumber) VALUES (' 5599-4154-6354-0003'); UPDATE proposal SET status = 'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2023; 6 roundtrips
  297. INSERT INTO card(cardNumber) VALUES ('5260-9991-9040-0001'); UPDATE proposal SET status =

    'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2021; INSERT INTO card(cardNumber) VALUES (' 5513-8141-9261-0002'); UPDATE proposal SET status = 'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2022; INSERT INTO card(cardNumber) VALUES (' 5599-4154-6354-0003'); UPDATE proposal SET status = 'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2023; INSERT INTO card(cardNumber) VALUES (' 5419-8753-1126-0004'); UPDATE proposal SET status = 'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2024; INSERT INTO ... UPDATE proposal SET ... 100 roundtrips
  298. Latência

  299. spring: jpa: properties: hibernate: jdbc: batch_size: 50 order_inserts: true order_updates:

    true application.yml
  300. spring: jpa: properties: hibernate: jdbc: batch_size: 50 order_inserts: true order_updates:

    true application.yml
  301. 2 roundtrips INSERT INTO card(cardNumber) VALUES (?) Params: [ 


    ('5513-8141-9261-0001'), 
 ('5513-8141-9261-0002'), 
 (‘5513-8141-9261-0003'), (‘5513-8141-9261-0004'), ... 
 ] UPDATE proposal SET status = ? WHERE id = ?; Params: [ 
 (‘ELIGIBLE_WITH_ATTACHED_CARD’, 2021), 
 ('ELIGIBLE_WITH_ATTACHED_CARD', 2022), 
 ('ELIGIBLE_WITH_ATTACHED_CARD', 2023), ('ELIGIBLE_WITH_ATTACHED_CARD', 2024), ... 
 ]
  302. 2 roundtrips INSERT INTO card(cardNumber) VALUES (?) Params: [ 


    ('5513-8141-9261-0001'), 
 ('5513-8141-9261-0002'), 
 (‘5513-8141-9261-0003'), (‘5513-8141-9261-0004'), ... 
 ] UPDATE proposal SET status = ? WHERE id = ?; Params: [ 
 (‘ELIGIBLE_WITH_ATTACHED_CARD’, 2021), 
 ('ELIGIBLE_WITH_ATTACHED_CARD', 2022), 
 ('ELIGIBLE_WITH_ATTACHED_CARD', 2023), ('ELIGIBLE_WITH_ATTACHED_CARD', 2024), ... 
 ]
  303. INSERT INTO card(cardNumber) VALUES (?) Params: [ 
 ('5513-8141-9261-0001'), 


    ('5513-8141-9261-0002'), 
 (‘5513-8141-9261-0003'), (‘5513-8141-9261-0004'), ... 
 ] 2 roundtrips UPDATE proposal SET status = ? WHERE id = ?; Params: [ 
 (‘ELIGIBLE_WITH_ATTACHED_CARD’, 2021), 
 ('ELIGIBLE_WITH_ATTACHED_CARD', 2022), 
 ('ELIGIBLE_WITH_ATTACHED_CARD', 2023), ('ELIGIBLE_WITH_ATTACHED_CARD', 2024), ... 
 ]
  304. E SE…

  305. …ocorrer erro na integração?

  306. @Scheduled(...) public void execute() { boolean pending = true; while

    (pending) { pending = transactionManager.execute(txStatus -> { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { return false; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); return true; }); } }
  307. @Scheduled(...) public void execute() { boolean pending = true; while

    (pending) { pending = transactionManager.execute(txStatus -> { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { return false; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); return true; }); } }
  308. @Scheduled(...) public void execute() { boolean pending = true; while

    (pending) { pending = transactionManager.execute(txStatus -> { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { return false; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); return true; }); } }
  309. https://deniseyu.io/art/

  310. CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId());

  311. CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); espera in fi nita? timeout

  312. CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); espera in fi nita? timeout erros

    intermitentes? retry
  313. CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); espera in fi nita? timeout erros

    intermitentes? retry posso reenviar a requisição? idempotência
  314. E SE… E SE… E SE… E SE… E SE…

    E SE… E SE… E SE… E SE… E SE… E SE… E SE E SE… SE… E SE… E SE… E SE… E SE… E SE… E SE… SE… E SE…
  315. CONCLUINDO

  316. @rponte Rafael Ponte