Tweet
Tweet

Slide 1

Slide 1 text

DISTRIBUTED SCHEDULING | Spring Boot os desa fi os e percalços de implementar um job em background

Slide 2

Slide 2 text

No content

Slide 3

Slide 3 text

Maaaaas implementar um job em background…

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

https://deniseyu.io/art/

Slide 6

Slide 6 text

Computação paralela e distribuída

Slide 7

Slide 7 text

Cadê meu cartão?

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

No content

Slide 10

Slide 10 text

No content

Slide 11

Slide 11 text

db servidor

Slide 12

Slide 12 text

db servidor

Slide 13

Slide 13 text

db servidor

Slide 14

Slide 14 text

db servidor

Slide 15

Slide 15 text

db servidor

Slide 16

Slide 16 text

db servidor

Slide 17

Slide 17 text

db servidor

Slide 18

Slide 18 text

db servidor

Slide 19

Slide 19 text

db servidor

Slide 20

Slide 20 text

db servidor

Slide 21

Slide 21 text

Como agendar tarefas com Spring Boot?

Slide 22

Slide 22 text

@Component public class OneJob { @Scheduled(fixedDelay = 60_000) public void runQuiteOften() { // lógica do job vai aqui } }

Slide 23

Slide 23 text

@Component public class OneJob { @Scheduled(fixedDelay = 60_000) public void runQuiteOften() { // lógica do job vai aqui } }

Slide 24

Slide 24 text

@Component public class OneJob { @Scheduled(fixedDelay = 60_000) public void runQuiteOften() { // lógica do job vai aqui } }

Slide 25

Slide 25 text

@Component public class OneJob { @Scheduled(fixedDelay = 60_000) public void runQuiteOften() { // lógica do job vai aqui } }

Slide 26

Slide 26 text

@Component public class OneJob { public void runQuiteOften() { // lógica do job vai aqui } }

Slide 27

Slide 27 text

@Component public class OneJob { @Scheduled public void runQuiteOften() { // lógica do job vai aqui } }

Slide 28

Slide 28 text

@Component public class OneJob { @Scheduled(fixedDelay = 60_000) public void runQuiteOften() { // lógica do job vai aqui } }

Slide 29

Slide 29 text

@Component public class OneJob { @Scheduled(fixedDelay = 60_000) public void runQuiteOften() { // lógica do job vai aqui } }

Slide 30

Slide 30 text

E nossa tarefa?

Slide 31

Slide 31 text

db servidor

Slide 32

Slide 32 text

db servidor

Slide 33

Slide 33 text

@Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }

Slide 34

Slide 34 text

@Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }

Slide 35

Slide 35 text

@Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }

Slide 36

Slide 36 text

@Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }

Slide 37

Slide 37 text

@Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }

Slide 38

Slide 38 text

@Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }

Slide 39

Slide 39 text

@Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }

Slide 40

Slide 40 text

@Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }

Slide 41

Slide 41 text

@Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }

Slide 42

Slide 42 text

@Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }

Slide 43

Slide 43 text

@Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }

Slide 44

Slide 44 text

@Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }

Slide 45

Slide 45 text

@Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }

Slide 46

Slide 46 text

@Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }

Slide 47

Slide 47 text

@Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }

Slide 48

Slide 48 text

Solução Ingênua

Slide 49

Slide 49 text

Rafael Ponte @rponte

Slide 50

Slide 50 text

No content

Slide 51

Slide 51 text

Fortaleza - Terra do Sol

Slide 52

Slide 52 text

No content

Slide 53

Slide 53 text

DISTRIBUTED SCHEDULING | Spring Boot os desa fi os e percalços de implementar um job em background

Slide 54

Slide 54 text

Solução Ingênua

Slide 55

Slide 55 text

@Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }

Slide 56

Slide 56 text

@Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }

Slide 57

Slide 57 text

select p.* from proposal p where p.status = 'ELIGIBLE' order by p.created_at asc

Slide 58

Slide 58 text

E SE…

Slide 59

Slide 59 text

…volume de dados crescer?

Slide 60

Slide 60 text

No content

Slide 61

Slide 61 text

No content

Slide 62

Slide 62 text

@Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }

Slide 63

Slide 63 text

@Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }

Slide 64

Slide 64 text

@Repository public interface ProposalRepository extends JpaRepository { public List findAllByStatusOrderByCreatedAtAsc(ProposalStatus status); }

Slide 65

Slide 65 text

@Repository public interface ProposalRepository extends JpaRepository { public List findAllByStatusOrderByCreatedAtAsc(ProposalStatus status); }

Slide 66

Slide 66 text

@Repository public interface ProposalRepository extends JpaRepository { public List findAllByStatusOrderByCreatedAtAsc(ProposalStatus status); }

Slide 67

Slide 67 text

select p.* from proposal p where p.status = 'ELIGIBLE' order by p.created_at asc

Slide 68

Slide 68 text

List findAllByStatusOrderByCreatedAtAsc(...);

Slide 69

Slide 69 text

List findAllByStatusOrderByCreatedAtAsc(...);

Slide 70

Slide 70 text

List findTop50ByStatusOrderByCreatedAtAsc(...);

Slide 71

Slide 71 text

select p.* from proposal p where p.status = 'ELIGIBLE' order by p.created_at asc

Slide 72

Slide 72 text

select p.* from proposal p where p.status = 'ELIGIBLE' order by p.created_at asc limit 50

Slide 73

Slide 73 text

@Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }

Slide 74

Slide 74 text

Mas…

Slide 75

Slide 75 text

@Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }

Slide 76

Slide 76 text

@Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } } o loop vai encerrar muito CEDO !

Slide 77

Slide 77 text

@Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } } Só vai rodar no próximo agendamento!

Slide 78

Slide 78 text

while(true)

Slide 79

Slide 79 text

@Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); proposals.forEach(proposal -> { // processa cada proposta }); }

Slide 80

Slide 80 text

@Transactional @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { List proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { // processa cada proposta }); } }

Slide 81

Slide 81 text

@Transactional @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { List proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { // processa cada proposta }); } }

Slide 82

Slide 82 text

@Transactional @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { List proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { // processa cada proposta }); } }

Slide 83

Slide 83 text

@Transactional @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { List proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { // processa cada proposta }); } }

Slide 84

Slide 84 text

@Component public class AttachCardsToProposalsJob { // dependências e constantes @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { List proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } } }

Slide 85

Slide 85 text

Mas ao rodar…

Slide 86

Slide 86 text

No content

Slide 87

Slide 87 text

Hein?

Slide 88

Slide 88 text

@Component public class AttachCardsToProposalsJob { // dependências e constantes @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { List proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } } }

Slide 89

Slide 89 text

@Component public class AttachCardsToProposalsJob { // dependências e constantes @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { List proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } } } - carregando apenas 50 objetos por vez

Slide 90

Slide 90 text

@Component public class AttachCardsToProposalsJob { // dependências e constantes @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { List proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } } } - criando nova referência - carregando apenas 50 objetos por vez

Slide 91

Slide 91 text

@Component public class AttachCardsToProposalsJob { // dependências e constantes @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { List proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } } } - criando nova referência - carregando apenas 50 objetos por vez - saindo do loop na hora certa

Slide 92

Slide 92 text

Por que?

Slide 93

Slide 93 text

@Component public class AttachCardsToProposalsJob { // dependências e constantes @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { List proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } } }

Slide 94

Slide 94 text

@Repository public interface ProposalRepository extends JpaRepository { public List findAllByStatusOrderByCreatedAtAsc(ProposalStatus status); }

Slide 95

Slide 95 text

Revisitando JPA e Hibernate

Slide 96

Slide 96 text

Job

Slide 97

Slide 97 text

Repository Job

Slide 98

Slide 98 text

Repository EM Job

Slide 99

Slide 99 text

Repository EM Job

Slide 100

Slide 100 text

Repository EM Job

Slide 101

Slide 101 text

Repository EM Job

Slide 102

Slide 102 text

Repository EM Job

Slide 103

Slide 103 text

Repository EM Job

Slide 104

Slide 104 text

Repository EM Job

Slide 105

Slide 105 text

Long-running transaction

Slide 106

Slide 106 text

@Transactional Repository EM Job Long-running transaction

Slide 107

Slide 107 text

@Transactional Repository EM Job Long-running transaction

Slide 108

Slide 108 text

@Transactional Repository EM Job Long-running transaction

Slide 109

Slide 109 text

@Transactional Repository EM Job (First-Level Cache) Long-running transaction

Slide 110

Slide 110 text

@Transactional Repository EM Job (First-Level Cache) Long-running transaction

Slide 111

Slide 111 text

A sacada aqui é entender…

Slide 112

Slide 112 text

@Transactional Repository EM Job (First-Level Cache) Long-running transaction

Slide 113

Slide 113 text

@Transactional EM (First-Level Cache) Long-running transaction

Slide 114

Slide 114 text

@Transactional EM (First-Level Cache) Long-running transaction

Slide 115

Slide 115 text

EM (First-Level Cache) @Transactional Long-running transaction

Slide 116

Slide 116 text

EM (First-Level Cache) @Transactional Long-running transaction

Slide 117

Slide 117 text

Repository EM Job (First-Level Cache) @Transactional Long-running transaction

Slide 118

Slide 118 text

Repository EM Job (First-Level Cache) Long-running transaction

Slide 119

Slide 119 text

Ahhh!!

Slide 120

Slide 120 text

@Component public class AttachCardsToProposalsJob { // dependências e constantes @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { List proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } } }

Slide 121

Slide 121 text

@Component public class AttachCardsToProposalsJob { // dependências e constantes @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { List proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } } } inicia e termina uma transação

Slide 122

Slide 122 text

Mas…

Slide 123

Slide 123 text

@Component public class AttachCardsToProposalsJob { // dependências e constantes @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { List proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } } }

Slide 124

Slide 124 text

@Component public class AttachCardsToProposalsJob { // dependências e constantes @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { List proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); // inicia e comita transação proposal.attachTo(newCard); proposalRepository.save(proposal); // inicia e comita transação }); } } } analogo a 
 auto_commit = true

Slide 125

Slide 125 text

@Component public class AttachCardsToProposalsJob { // dependências e constantes @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { List proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); // inicia e comita transação proposal.attachTo(newCard); proposalRepository.save(proposal); // inicia e comita transação }); } } } Abrimos mão da CONSISTÊNCIA dos dados!

Slide 126

Slide 126 text

@Component

Slide 127

Slide 127 text

Short-running transaction

Slide 128

Slide 128 text

Job Transaction Job

Slide 129

Slide 129 text

Job Transaction Job inicio do job fi m do job

Slide 130

Slide 130 text

Job Transaction Job

Slide 131

Slide 131 text

inicio da transação fi m da transação Job Job Transaction

Slide 132

Slide 132 text

Job Transaction Job Long-running transaction

Slide 133

Slide 133 text

Job Transaction Job

Slide 134

Slide 134 text

Job Transaction Job

Slide 135

Slide 135 text

Job Transaction Job inicio da transação fi m da transação

Slide 136

Slide 136 text

Job Transaction Job Short-running transaction

Slide 137

Slide 137 text

Job Transaction Job

Slide 138

Slide 138 text

Job Transaction Job

Slide 139

Slide 139 text

Job Transaction Job

Slide 140

Slide 140 text

Job Transaction Job

Slide 141

Slide 141 text

Job Transaction Job

Slide 142

Slide 142 text

Job Transaction Job Short-running transaction

Slide 143

Slide 143 text

Repository EM Job (First-Level Cache) Short-running transaction

Slide 144

Slide 144 text

Repository EM Job @Transactional Short-running transaction (First-Level Cache)

Slide 145

Slide 145 text

Repository EM Job @Transactional Short-running transaction (First-Level Cache)

Slide 146

Slide 146 text

Repository EM Job @Transactional Short-running transaction (First-Level Cache)

Slide 147

Slide 147 text

@Transactional Repository EM Job (First-Level Cache) Short-running transaction

Slide 148

Slide 148 text

Repository EM Job @Transactional Short-running transaction (First-Level Cache)

Slide 149

Slide 149 text

Controle transacional programático

Slide 150

Slide 150 text

@Component public class AttachCardsToProposalsJob { @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { // executa código em escopo transacional } } }

Slide 151

Slide 151 text

@Component public class AttachCardsToProposalsJob { @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { @Transactional.execute(transaction -> { // executa código em escopo transacional }); } } }

Slide 152

Slide 152 text

@Component public class AttachCardsToProposalsJob { @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { @Transactional.execute(transaction -> { // executa código em escopo transacional }); } } }

Slide 153

Slide 153 text

@Component public class AttachCardsToProposalsJob { @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { @Transactional.execute(transaction -> { // executa código em escopo transacional }); } } }

Slide 154

Slide 154 text

@Component public class AttachCardsToProposalsJob { @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { @Transactional.execute(transaction -> { // executa código em escopo transacional }); } } }

Slide 155

Slide 155 text

@Component public class AttachCardsToProposalsJob { @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { @Transactional.execute(transaction -> { // executa código em escopo transacional }); } } }

Slide 156

Slide 156 text

@Component public class AttachCardsToProposalsJob { @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { @Transactional.execute(transaction -> { // executa código em escopo transacional }); } } }

Slide 157

Slide 157 text

@Component public class AttachCardsToProposalsJob { @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { @Transactional.execute(transaction -> { // executa código em escopo transacional }); } } }

Slide 158

Slide 158 text

@Component public class AttachCardsToProposalsJob { @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { transactionManager.execute(transaction -> { // executa código em escopo transacional }); } } }

Slide 159

Slide 159 text

@Component public class AttachCardsToProposalsJob { private TransactionTemplate transactionManager; @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { transactionManager.execute(transaction -> { // executa código em escopo transacional }); } } }

Slide 160

Slide 160 text

@Component public class AttachCardsToProposalsJob { private TransactionTemplate transactionManager; @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { transactionManager.execute(transaction -> { // executa código em escopo transacional }); } } }

Slide 161

Slide 161 text

@Autowired private TransactionTemplate transactionManager; @Scheduled(fixedDelay = 60_000) public void execute() { boolean pending = true; while (pending) { pending = transactionManager.execute(transactionStatus -> { List proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { return false; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); return true; }); } }

Slide 162

Slide 162 text

@Autowired private TransactionTemplate transactionManager; @Scheduled(fixedDelay = 60_000) public void execute() { boolean pending = true; while (pending) { pending = transactionManager.execute(transactionStatus -> { List proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { return false; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); return true; }); } }

Slide 163

Slide 163 text

No content

Slide 164

Slide 164 text

inicio do job fi m do job

Slide 165

Slide 165 text

E SE…

Slide 166

Slide 166 text

…rodarmos a aplicação em cluster?

Slide 167

Slide 167 text

No content

Slide 168

Slide 168 text

No content

Slide 169

Slide 169 text

No content

Slide 170

Slide 170 text

No content

Slide 171

Slide 171 text

ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE 1004 ELIGIBLE

Slide 172

Slide 172 text

ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE 1004 ELIGIBLE

Slide 173

Slide 173 text

ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE 1004 ELIGIBLE

Slide 174

Slide 174 text

ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE 1003 ELIGIBLE 1004 ELIGIBLE

Slide 175

Slide 175 text

ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE 1003 ELIGIBLE 1004 ELIGIBLE

Slide 176

Slide 176 text

ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE 1004 ELIGIBLE

Slide 177

Slide 177 text

ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE 1004 ELIGIBLE

Slide 178

Slide 178 text

ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE_WITH_ATTACHED_CARD 1004 ELIGIBLE

Slide 179

Slide 179 text

ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE_WITH_ATTACHED_CARD 1004 ELIGIBLE

Slide 180

Slide 180 text

ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE_WITH_ATTACHED_CARD 1004 ELIGIBLE_WITH_ATTACHED_CARD

Slide 181

Slide 181 text

ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE_WITH_ATTACHED_CARD 1004 ELIGIBLE_WITH_ATTACHED_CARD

Slide 182

Slide 182 text

No content

Slide 183

Slide 183 text

ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE 1004 ELIGIBLE

Slide 184

Slide 184 text

ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE 1004 ELIGIBLE

Slide 185

Slide 185 text

ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE 1003 ELIGIBLE 1004 ELIGIBLE

Slide 186

Slide 186 text

ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE 1003 ELIGIBLE 1004 ELIGIBLE 2x

Slide 187

Slide 187 text

ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE 1004 ELIGIBLE 2x

Slide 188

Slide 188 text

ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE 1004 ELIGIBLE 2x 2x

Slide 189

Slide 189 text

ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE_WITH_ATTACHED_CARD 1004 ELIGIBLE 2x 2x

Slide 190

Slide 190 text

ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE_WITH_ATTACHED_CARD 1004 ELIGIBLE 2x 2x 2x

Slide 191

Slide 191 text

ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE_WITH_ATTACHED_CARD 1004 ELIGIBLE_WITH_ATTACHED_CARD 2x 2x 2x

Slide 192

Slide 192 text

ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE_WITH_ATTACHED_CARD 1004 ELIGIBLE_WITH_ATTACHED_CARD 2x 2x 2x 2x

Slide 193

Slide 193 text

Oops!

Slide 194

Slide 194 text

Synchronized?

Slide 195

Slide 195 text

@Component public class AttachCardsToProposalsJob { @Scheduled(...) public synchronized void execute() { // executa lógica aqui } }

Slide 196

Slide 196 text

No content

Slide 197

Slide 197 text

No content

Slide 198

Slide 198 text

No content

Slide 199

Slide 199 text

No content

Slide 200

Slide 200 text

@Component

Slide 201

Slide 201 text

Distributed Lock?

Slide 202

Slide 202 text

@Scheduled(...) public void execute() { if (isLeader()) { // executa lógica aqui } }

Slide 203

Slide 203 text

@Scheduled(...) public void execute() { if (isLeader()) { // executa lógica aqui } }

Slide 204

Slide 204 text

No content

Slide 205

Slide 205 text

No content

Slide 206

Slide 206 text

No content

Slide 207

Slide 207 text

No content

Slide 208

Slide 208 text

No content

Slide 209

Slide 209 text

No content

Slide 210

Slide 210 text

No content

Slide 211

Slide 211 text

No content

Slide 212

Slide 212 text

No content

Slide 213

Slide 213 text

No content

Slide 214

Slide 214 text

No content

Slide 215

Slide 215 text

Não!

Slide 216

Slide 216 text

Que tal…

Slide 217

Slide 217 text

No content

Slide 218

Slide 218 text

No content

Slide 219

Slide 219 text

@Repository public interface ProposalRepository extends JpaRepository { public List findTop50ByStatusOrderByCreatedAtAsc(ProposalStatus status); }

Slide 220

Slide 220 text

@Repository public interface ProposalRepository extends JpaRepository { @Lock(LockModeType.PESSIMISTIC_WRITE) public List findTop50ByStatusOrderByCreatedAtAsc(ProposalStatus status); }

Slide 221

Slide 221 text

select p.* from proposal p where p.status = 'ELIGIBLE' order by p.created_at asc limit 50 


Slide 222

Slide 222 text

select p.* from proposal p where p.status = 'ELIGIBLE' order by p.created_at asc limit 50 
 for update

Slide 223

Slide 223 text

ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE ... ...

Slide 224

Slide 224 text

ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE ... ...

Slide 225

Slide 225 text

ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE ... ...

Slide 226

Slide 226 text

ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE ... ...

Slide 227

Slide 227 text

ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE 1003 ELIGIBLE ... ...

Slide 228

Slide 228 text

ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE 1003 ELIGIBLE ... ...

Slide 229

Slide 229 text

ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE 1003 ELIGIBLE ... ...

Slide 230

Slide 230 text

ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE 1003 ELIGIBLE ... ...

Slide 231

Slide 231 text

ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE 1003 ELIGIBLE ... ...

Slide 232

Slide 232 text

ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE 1003 ELIGIBLE ... ...

Slide 233

Slide 233 text

ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE ... ...

Slide 234

Slide 234 text

ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE ... ...

Slide 235

Slide 235 text

ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE ... ...

Slide 236

Slide 236 text

ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE ... ...

Slide 237

Slide 237 text

ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE_WITH_ATTACHED_CARD ... ...

Slide 238

Slide 238 text

SIM!

Slide 239

Slide 239 text

E SE…

Slide 240

Slide 240 text

…precisarmos processar em paralelo?

Slide 241

Slide 241 text

No content

Slide 242

Slide 242 text

No content

Slide 243

Slide 243 text

No content

Slide 244

Slide 244 text

No content

Slide 245

Slide 245 text

Para isso…

Slide 246

Slide 246 text

9630 9631 QUEUE ( fi la) 9629

Slide 247

Slide 247 text

9630 9631 QUEUE ( fi la) producer 9629 produz: 
 1000 itens/min

Slide 248

Slide 248 text

9630 9631 QUEUE ( fi la) producer consumer 9629 produz: 
 1000 itens/min consume: 
 600 itens/min

Slide 249

Slide 249 text

9630 9631 QUEUE ( fi la) producer consumer 9629 produz: 
 1000 itens/min consumer consume: 
 400 itens/min consume: 
 600 itens/min

Slide 250

Slide 250 text

9630 9631 QUEUE ( fi la) producer consumer 9629 produz: 
 1000 itens/min consumer consume: 
 400 itens/min consume: 
 600 itens/min

Slide 251

Slide 251 text

No content

Slide 252

Slide 252 text

Não!

Slide 253

Slide 253 text

Que tal…

Slide 254

Slide 254 text

No content

Slide 255

Slide 255 text

@Repository public interface ProposalRepository extends JpaRepository { @QueryHints({ @QueryHint( name = "javax.persistence.lock.timeout", value = LockOptions.SKIP_LOCKED) // org.hibernate.LockOptions }) @Lock(LockModeType.PESSIMISTIC_WRITE) public List findTop50ByStatusOrderByCreatedAtAsc(ProposalStatus status); }

Slide 256

Slide 256 text

@Repository public interface ProposalRepository extends JpaRepository { @QueryHints({ @QueryHint( name = "javax.persistence.lock.timeout", value = LockOptions.SKIP_LOCKED) // org.hibernate.LockOptions }) @Lock(LockModeType.PESSIMISTIC_WRITE) public List findTop50ByStatusOrderByCreatedAtAsc(ProposalStatus status); }

Slide 257

Slide 257 text

select p.* from proposal p where p.status = 'ELIGIBLE' order by p.created_at asc limit 50 
 for update

Slide 258

Slide 258 text

select p.* from proposal p where p.status = 'ELIGIBLE' order by p.created_at asc limit 50 
 for update skip locked

Slide 259

Slide 259 text

ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE ... ...

Slide 260

Slide 260 text

ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE ... ...

Slide 261

Slide 261 text

ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE ... ...

Slide 262

Slide 262 text

ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE ... ...

Slide 263

Slide 263 text

ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE ... ...

Slide 264

Slide 264 text

ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE ... ...

Slide 265

Slide 265 text

ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE ... ...

Slide 266

Slide 266 text

ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE ... ...

Slide 267

Slide 267 text

ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE ... ...

Slide 268

Slide 268 text

ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE ... ...

Slide 269

Slide 269 text

ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE ... ...

Slide 270

Slide 270 text

ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE ... ...

Slide 271

Slide 271 text

ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE ... ...

Slide 272

Slide 272 text

ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE ... ...

Slide 273

Slide 273 text

ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE ... ...

Slide 274

Slide 274 text

Ou seja…

Slide 275

Slide 275 text

ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE 1004 ELIGIBLE

Slide 276

Slide 276 text

ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE 1004 ELIGIBLE

Slide 277

Slide 277 text

SIM!

Slide 278

Slide 278 text

E SE…

Slide 279

Slide 279 text

…impactar no startup da aplicação?

Slide 280

Slide 280 text

@Scheduled( fixedDelay = 60_000 ) public void execute() { // executa lógica aqui }

Slide 281

Slide 281 text

initialDelay

Slide 282

Slide 282 text

@Scheduled( fixedDelay = 60_000, initialDelay = 120_000 ) public void execute() { // executa lógica aqui }

Slide 283

Slide 283 text

E SE…

Slide 284

Slide 284 text

…precisarmos de maior throughput?

Slide 285

Slide 285 text

Hibernate batch size

Slide 286

Slide 286 text

@Autowired private TransactionTemplate transactionManager; @Scheduled(fixedDelay = 60_000) public void execute() { boolean pending = true; while (pending) { pending = transactionManager.execute(transactionStatus -> { List proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { return false; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); return true; }); } }

Slide 287

Slide 287 text

@Autowired private TransactionTemplate transactionManager; @Scheduled(fixedDelay = 60_000) public void execute() { boolean pending = true; while (pending) { pending = transactionManager.execute(transactionStatus -> { List proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { return false; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); return true; }); } }

Slide 288

Slide 288 text

@Autowired private TransactionTemplate transactionManager; @Scheduled(fixedDelay = 60_000) public void execute() { boolean pending = true; while (pending) { pending = transactionManager.execute(transactionStatus -> { List proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { return false; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); return true; }); } }

Slide 289

Slide 289 text

@Autowired private TransactionTemplate transactionManager; @Scheduled(fixedDelay = 60_000) public void execute() { boolean pending = true; while (pending) { pending = transactionManager.execute(transactionStatus -> { List proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { return false; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); return true; }); } }

Slide 290

Slide 290 text

@Autowired private TransactionTemplate transactionManager; @Scheduled(fixedDelay = 60_000) public void execute() { boolean pending = true; while (pending) { pending = transactionManager.execute(transactionStatus -> { List proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { return false; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); return true; }); } } Insere novo cartão Atualiza proposta com cartão

Slide 291

Slide 291 text

Para cada iteração do loop…

Slide 292

Slide 292 text

2 roundtrips INSERT INTO card(cardNumber) VALUES ('5260-9991-9040-0001'); UPDATE proposal SET status = 'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2021;

Slide 293

Slide 293 text

2 roundtrips INSERT INTO card(cardNumber) VALUES ('5260-9991-9040-0001'); UPDATE proposal SET status = 'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2021;

Slide 294

Slide 294 text

2 roundtrips INSERT INTO card(cardNumber) VALUES ('5260-9991-9040-0001'); UPDATE proposal SET status = 'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2021;

Slide 295

Slide 295 text

INSERT INTO card(cardNumber) VALUES ('5260-9991-9040-0001'); UPDATE proposal SET status = 'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2021; INSERT INTO card(cardNumber) VALUES (' 5513-8141-9261-0002'); UPDATE proposal SET status = 'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2022; 4 roundtrips

Slide 296

Slide 296 text

INSERT INTO card(cardNumber) VALUES ('5260-9991-9040-0001'); UPDATE proposal SET status = 'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2021; INSERT INTO card(cardNumber) VALUES (' 5513-8141-9261-0002'); UPDATE proposal SET status = 'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2022; INSERT INTO card(cardNumber) VALUES (' 5599-4154-6354-0003'); UPDATE proposal SET status = 'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2023; 6 roundtrips

Slide 297

Slide 297 text

INSERT INTO card(cardNumber) VALUES ('5260-9991-9040-0001'); UPDATE proposal SET status = 'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2021; INSERT INTO card(cardNumber) VALUES (' 5513-8141-9261-0002'); UPDATE proposal SET status = 'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2022; INSERT INTO card(cardNumber) VALUES (' 5599-4154-6354-0003'); UPDATE proposal SET status = 'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2023; INSERT INTO card(cardNumber) VALUES (' 5419-8753-1126-0004'); UPDATE proposal SET status = 'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2024; INSERT INTO ... UPDATE proposal SET ... 100 roundtrips

Slide 298

Slide 298 text

Latência

Slide 299

Slide 299 text

spring: jpa: properties: hibernate: jdbc: batch_size: 50 order_inserts: true order_updates: true application.yml

Slide 300

Slide 300 text

spring: jpa: properties: hibernate: jdbc: batch_size: 50 order_inserts: true order_updates: true application.yml

Slide 301

Slide 301 text

2 roundtrips INSERT INTO card(cardNumber) VALUES (?) Params: [ 
 ('5513-8141-9261-0001'), 
 ('5513-8141-9261-0002'), 
 (‘5513-8141-9261-0003'), (‘5513-8141-9261-0004'), ... 
 ] UPDATE proposal SET status = ? WHERE id = ?; Params: [ 
 (‘ELIGIBLE_WITH_ATTACHED_CARD’, 2021), 
 ('ELIGIBLE_WITH_ATTACHED_CARD', 2022), 
 ('ELIGIBLE_WITH_ATTACHED_CARD', 2023), ('ELIGIBLE_WITH_ATTACHED_CARD', 2024), ... 
 ]

Slide 302

Slide 302 text

2 roundtrips INSERT INTO card(cardNumber) VALUES (?) Params: [ 
 ('5513-8141-9261-0001'), 
 ('5513-8141-9261-0002'), 
 (‘5513-8141-9261-0003'), (‘5513-8141-9261-0004'), ... 
 ] UPDATE proposal SET status = ? WHERE id = ?; Params: [ 
 (‘ELIGIBLE_WITH_ATTACHED_CARD’, 2021), 
 ('ELIGIBLE_WITH_ATTACHED_CARD', 2022), 
 ('ELIGIBLE_WITH_ATTACHED_CARD', 2023), ('ELIGIBLE_WITH_ATTACHED_CARD', 2024), ... 
 ]

Slide 303

Slide 303 text

INSERT INTO card(cardNumber) VALUES (?) Params: [ 
 ('5513-8141-9261-0001'), 
 ('5513-8141-9261-0002'), 
 (‘5513-8141-9261-0003'), (‘5513-8141-9261-0004'), ... 
 ] 2 roundtrips UPDATE proposal SET status = ? WHERE id = ?; Params: [ 
 (‘ELIGIBLE_WITH_ATTACHED_CARD’, 2021), 
 ('ELIGIBLE_WITH_ATTACHED_CARD', 2022), 
 ('ELIGIBLE_WITH_ATTACHED_CARD', 2023), ('ELIGIBLE_WITH_ATTACHED_CARD', 2024), ... 
 ]

Slide 304

Slide 304 text

E SE…

Slide 305

Slide 305 text

…ocorrer erro na integração?

Slide 306

Slide 306 text

@Scheduled(...) public void execute() { boolean pending = true; while (pending) { pending = transactionManager.execute(txStatus -> { List proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { return false; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); return true; }); } }

Slide 307

Slide 307 text

@Scheduled(...) public void execute() { boolean pending = true; while (pending) { pending = transactionManager.execute(txStatus -> { List proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { return false; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); return true; }); } }

Slide 308

Slide 308 text

@Scheduled(...) public void execute() { boolean pending = true; while (pending) { pending = transactionManager.execute(txStatus -> { List proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { return false; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); return true; }); } }

Slide 309

Slide 309 text

https://deniseyu.io/art/

Slide 310

Slide 310 text

CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId());

Slide 311

Slide 311 text

CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); espera in fi nita? timeout

Slide 312

Slide 312 text

CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); espera in fi nita? timeout erros intermitentes? retry

Slide 313

Slide 313 text

CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); espera in fi nita? timeout erros intermitentes? retry posso reenviar a requisição? idempotência

Slide 314

Slide 314 text

E SE… E SE… E SE… E SE… E SE… E SE… E SE… E SE… E SE… E SE… E SE… E SE E SE… SE… E SE… E SE… E SE… E SE… E SE… E SE… SE… E SE…

Slide 315

Slide 315 text

CONCLUINDO

Slide 316

Slide 316 text

@rponte Rafael Ponte