Tweet
Tweet

Slide 1

Slide 1 text

DDD sous pression Retour d’expérience sur la construction d’un exchange Arnaud LEMAIRE @lilobase Jean Baptiste Dusseaut @bodysplash

Slide 2

Slide 2 text

« Un métier simple mais pas facile »

Slide 3

Slide 3 text

C’est quoi un trade ? 1 BUY 2@200 2 SELL 3@200 1 2 TRADE 2@200

Slide 4

Slide 4 text

C’est quoi un trade ? 1 BUY 2@200 2 SELL 3@200 1 2 TRADE 2@200 Un ordre

Slide 5

Slide 5 text

C’est quoi un trade ? 1 BUY 2@200 2 SELL 3@200 1 2 TRADE 2@200 Un ordre Un trade

Slide 6

Slide 6 text

C’est quoi un trade ? 1 BUY 2@200 2 SELL 3@200 1 2 TRADE 2@200 3 BUY 1@200 3 2 TRADE 1@200

Slide 7

Slide 7 text

L’exécution BUY 2@202 TRADE 2@202

Slide 8

Slide 8 text

L’exécution BUY 2@202 Le carnet d’ordre TRADE 2@202

Slide 9

Slide 9 text

Le carnet d’ordre 7 BUY 1@199 2 SELL 3@202 4 BUY 2@200 3 SELL 4@203 9 BUY 3@199 1 SELL 2@204 6 SELL 3@204 UY 1@199 SELL 2@2 5 BUY 5@198 8 BUY 2@198 10 BUY 1@198

Slide 10

Slide 10 text

Le carnet d’ordre 7 BUY 1@199 2 SELL 3@202 4 BUY 2@200 3 SELL 4@203 9 BUY 3@199 1 SELL 2@204 6 SELL 3@204 UY 1@199 SELL 2@2 5 BUY 5@198 8 BUY 2@198 BUY 2@202 10 BUY 1@198

Slide 11

Slide 11 text

Le carnet d’ordre 7 BUY 1@199 2 SELL 3@202 4 BUY 2@200 3 SELL 4@203 9 BUY 3@199 1 SELL 2@204 6 SELL 3@204 UY 1@199 SELL 2@2 5 BUY 5@198 8 BUY 2@198 BUY 2@202 10 BUY 1@198 TRADE 2@202

Slide 12

Slide 12 text

Le carnet d’ordre 7 BUY 1@199 2 SELL 3@202 4 BUY 2@200 3 SELL 4@203 9 BUY 3@199 1 SELL 2@204 6 SELL 3@204 UY 1@199 SELL 2@2 5 BUY 5@198 8 BUY 2@198 BUY 2@202 10 BUY 1@198 contre-ordre TRADE 2@202

Slide 13

Slide 13 text

Le carnet d’ordre 7 BUY 1@199 2 SELL 3@202 4 BUY 2@200 3 SELL 4@203 9 BUY 3@199 1 SELL 2@204 6 SELL 3@204 UY 1@199 SELL 2@2 5 BUY 5@198 8 BUY 2@198 BUY 2@202 10 BUY 1@198 contre-ordre TRADE 2@202 Niveau de prix

Slide 14

Slide 14 text

Le carnet d’ordre 2 SELL 1@202 3 SELL 4@203 9 BUY 3@199 1 SELL 2@204 6 SELL 3@204 UY 1@199 SELL 2@2 5 BUY 5@198 8 BUY 2@198 10 BUY 1@198 7 BUY 1@199 4 BUY 2@200

Slide 15

Slide 15 text

Le carnet d’ordre BUY 2@200 2 SELL 1@202 3 SELL 4@203 9 BUY 3@199 1 SELL 2@204 6 SELL 3@204 UY 1@199 SELL 2@2 5 BUY 5@198 8 BUY 2@198 10 BUY 1@198 7 BUY 1@199 4 BUY 2@200

Slide 16

Slide 16 text

Le carnet d’ordre BUY 2@200 2 SELL 1@202 3 SELL 4@203 9 BUY 3@199 1 SELL 2@204 6 SELL 3@204 UY 1@199 SELL 2@2 5 BUY 5@198 8 BUY 2@198 10 BUY 1@198 7 BUY 1@199 4 BUY 2@200

Slide 17

Slide 17 text

Le carnet d’ordre 2 BUY 2@200 BUY 2@200 2 SELL 1@202 3 SELL 4@203 9 BUY 3@199 1 SELL 2@204 6 SELL 3@204 UY 1@199 SELL 2@2 5 BUY 5@198 8 BUY 2@198 10 BUY 1@198 7 BUY 1@199 4 BUY 2@200

Slide 18

Slide 18 text

Le carnet d’ordre 2 BUY 2@200 2 SELL 1@202 3 SELL 4@203 9 BUY 3@199 1 SELL 2@204 6 SELL 3@204 UY 1@199 SELL 2@2 5 BUY 5@198 8 BUY 2@198 10 BUY 1@198 7 BUY 1@199 4 BUY 2@200

Slide 19

Slide 19 text

Le carnet d’ordre 2 BUY 2@200 L’ordre est mis dans la file 2 SELL 1@202 3 SELL 4@203 9 BUY 3@199 1 SELL 2@204 6 SELL 3@204 UY 1@199 SELL 2@2 5 BUY 5@198 8 BUY 2@198 10 BUY 1@198 7 BUY 1@199 4 BUY 2@200

Slide 20

Slide 20 text

L’allocation 2 BUY 2@200 2 SELL 1@202 3 SELL 4@203 9 BUY 3@199 1 SELL 2@204 6 SELL 3@204 UY 1@199 SELL 2@2 5 BUY 5@198 8 BUY 2@198 10 BUY 1@198 7 BUY 1@199 4 BUY 2@200

Slide 21

Slide 21 text

L’allocation 2 BUY 2@200 2 SELL 1@202 3 SELL 4@203 9 BUY 3@199 1 SELL 2@204 6 SELL 3@204 UY 1@199 SELL 2@2 5 BUY 5@198 8 BUY 2@198 10 BUY 1@198 SELL 2@200 7 BUY 1@199 4 BUY 2@200

Slide 22

Slide 22 text

1 L’allocation : FIFO 2 BUY 2@200 SELL 2@200 4 BUY 2@200

Slide 23

Slide 23 text

1 L’allocation : FIFO 2 BUY 2@200 SELL 2@200 4 BUY 2@200 4 1 TRADE 2@200

Slide 24

Slide 24 text

L’allocation : FIFO 2 BUY 2@200 4 BUY 2@200 4 1 TRADE 2@200

Slide 25

Slide 25 text

L’allocation : FIFO 2 BUY 2@200 4 1 TRADE 2@200

Slide 26

Slide 26 text

1 L’allocation : pro-rata 2 BUY 2@200 SELL 2@200 4 BUY 2@200

Slide 27

Slide 27 text

1 L’allocation : pro-rata 2 BUY 2@200 SELL 2@200 4 BUY 2@200 4 1 TRADE 1@200 2 1 TRADE 1@200

Slide 28

Slide 28 text

1 L’allocation : pro-rata 2 BUY 2@200 SELL 2@200 4 BUY 2@200 4 1 TRADE 1@200 2 1 TRADE 1@200

Slide 29

Slide 29 text

1 L’allocation : pro-rata 2 BUY 2@200 SELL 2@200 4 BUY 2@200 4 1 TRADE 1@200 2 1 TRADE 1@200

Slide 30

Slide 30 text

1 L’allocation : pro-rata 2 BUY 2@200 SELL 2@200 4 BUY 2@200 4 1 TRADE 1@200 2 1 TRADE 1@200 2 BUY 1@200 4 BUY 1@200

Slide 31

Slide 31 text

Orders •Fill Or Kill •Immediate Or Cancel •Market •Stop orders •…

Slide 32

Slide 32 text

Nous ne sommes pas seuls

Slide 33

Slide 33 text

Arbitrage Arbitrage $5200 $5198 $5197

Slide 34

Slide 34 text

Front-running

Slide 35

Slide 35 text

Front-running

Slide 36

Slide 36 text

Front-running

Slide 37

Slide 37 text

30ms Front-running

Slide 38

Slide 38 text

30ms 90ms Front-running

Slide 39

Slide 39 text

30ms 90ms 120ms Front-running

Slide 40

Slide 40 text

30ms 90ms 120ms 40ms Front-running

Slide 41

Slide 41 text

30ms 90ms 120ms 40ms 60ms Front-running

Slide 42

Slide 42 text

30ms 90ms 120ms 40ms 60ms Front-running

Slide 43

Slide 43 text

30ms 90ms 120ms 40ms 60ms Front-running

Slide 44

Slide 44 text

30ms 90ms 120ms Front-running

Slide 45

Slide 45 text

30ms 90ms 120ms Front-running +30ms

Slide 46

Slide 46 text

30ms 90ms 120ms Front-running +30ms +90ms

Slide 47

Slide 47 text

30ms 90ms 120ms Front-running +30ms +90ms

Slide 48

Slide 48 text

30ms 90ms 120ms Front-running +30ms +90ms

Slide 49

Slide 49 text

30ms 90ms 120ms Front-running +30ms +90ms

Slide 50

Slide 50 text

Front running interne Délit d’initié 2 BUY 2@200 # BUY 2@200

Slide 51

Slide 51 text

Front running interne Délit d’initié 2 BUY 2@200 SELL 2@200 # BUY 2@200

Slide 52

Slide 52 text

Front running interne Délit d’initié 2 BUY 2@200 SELL 2@200 # BUY 2@200 Exchange

Slide 53

Slide 53 text

Une concurrence (quasi) impossible

Slide 54

Slide 54 text

Chaque exécution est fonction de l’exécution précédente Execution

Slide 55

Slide 55 text

Chaque exécution est fonction de l’exécution précédente Execution

Slide 56

Slide 56 text

Chaque exécution est fonction de l’exécution précédente Execution Execution

Slide 57

Slide 57 text

Chaque exécution est fonction de l’exécution précédente Execution Execution

Slide 58

Slide 58 text

Avec une charge, pas très prévisible

Slide 59

Slide 59 text

Avec une charge, pas très prévisible Vous avez interet a etre pret

Slide 60

Slide 60 text

Que vous avez intérêt à absorber rapidement

Slide 61

Slide 61 text

Un chemin critique qui ne s’arrête pas à l’exécution BUY 2@200

Slide 62

Slide 62 text

Un chemin critique qui ne s’arrête pas à l’exécution BUY 2@200 Et parfois notre utilisateur est vraiment une fonction !

Slide 63

Slide 63 text

Si on résume 1 million d’ordres par seconde sur un seul thread, 24h sur 24 et 7j sur 7 Le tout sans pouvoir paralléliser…

Slide 64

Slide 64 text

Un état de l’art « Discutable »

Slide 65

Slide 65 text

No content

Slide 66

Slide 66 text

No content

Slide 67

Slide 67 text

No content

Slide 68

Slide 68 text

No content

Slide 69

Slide 69 text

BlockchainTransparency

Slide 70

Slide 70 text

No content

Slide 71

Slide 71 text

Anti front-running

Slide 72

Slide 72 text

non-custody

Slide 73

Slide 73 text

Execution engine Première approche

Slide 74

Slide 74 text

Une exécution sous contrainte Single Thread Pas de concurrence

Slide 75

Slide 75 text

Une exécution sous contrainte Single thread in memory Non bloquant (donc sans I/O)

Slide 76

Slide 76 text

Une exécution sous contrainte Single thread in memory Messages entrant séquencés

Slide 77

Slide 77 text

Le ring buffer Single thread in memory

Slide 78

Slide 78 text

Le ring buffer Thread #1 Thread #2 Business Thread

Slide 79

Slide 79 text

Le pipelining Temps Step #1 Step #2 Step #3 Step #4 2 taches composée de 4 étapes

Slide 80

Slide 80 text

Le pipelining Temps Step #1 Step #2 Step #3 Step #4 1 2 taches composée de 4 étapes

Slide 81

Slide 81 text

Le pipelining Temps Step #1 Step #2 Step #3 Step #4 1 2 2 taches composée de 4 étapes

Slide 82

Slide 82 text

Le pipelining Temps Step #1 Step #2 Step #3 Step #4 1 2 3 2 taches composée de 4 étapes

Slide 83

Slide 83 text

Le pipelining Temps Step #1 Step #2 Step #3 Step #4 1 2 3 4 2 taches composée de 4 étapes

Slide 84

Slide 84 text

Le pipelining Temps Step #1 Step #2 Step #3 Step #4 1 2 3 4 5 2 taches composée de 4 étapes

Slide 85

Slide 85 text

Le pipelining Temps Step #1 Step #2 Step #3 Step #4 1 2 3 4 5 6 2 taches composée de 4 étapes

Slide 86

Slide 86 text

Le pipelining Temps Step #1 Step #2 Step #3 Step #4 1 2 3 4 5 6 7 2 taches composée de 4 étapes

Slide 87

Slide 87 text

Le pipelining Temps Step #1 Step #2 Step #3 Step #4 1 2 3 4 5 6 7 8 2 taches composée de 4 étapes

Slide 88

Slide 88 text

Le pipelining Temps Step #1 Step #2 Step #3 Step #4 1 2 3 4 5 6 7 8 2 taches composée de 4 étapes Sans pipelining : 8 ticks

Slide 89

Slide 89 text

Le pipelining Temps Step #1 Step #2 Step #3 Step #4 2 taches composée de 4 étapes

Slide 90

Slide 90 text

Le pipelining Temps Step #1 Step #2 Step #3 Step #4 1 2 taches composée de 4 étapes

Slide 91

Slide 91 text

Le pipelining Temps Step #1 Step #2 Step #3 Step #4 1 2 2 taches composée de 4 étapes

Slide 92

Slide 92 text

Le pipelining Temps Step #1 Step #2 Step #3 Step #4 1 2 3 2 taches composée de 4 étapes

Slide 93

Slide 93 text

Le pipelining Temps Step #1 Step #2 Step #3 Step #4 1 2 3 4 2 taches composée de 4 étapes

Slide 94

Slide 94 text

Le pipelining Temps Step #1 Step #2 Step #3 Step #4 1 2 3 4 5 2 taches composée de 4 étapes

Slide 95

Slide 95 text

Le pipelining Temps Step #1 Step #2 Step #3 Step #4 1 2 3 4 5 2 taches composée de 4 étapes Sans pipelining : 5 ticks

Slide 96

Slide 96 text

Le pipelining Temps Step #1 Step #2 Step #3 Step #4 1 2 3 4 5 2 taches composée de 4 étapes Sans pipelining : 5 ticks Parallélisme sans concurrence

Slide 97

Slide 97 text

Le pipelining Thread #1 Thread #2 Business Thread

Slide 98

Slide 98 text

Le pipelining Thread #1 Thread #2 Business Thread

Slide 99

Slide 99 text

Le pipelining Thread #1 Thread #2 Business Thread Lock-free : memory barrier

Slide 100

Slide 100 text

Le pipelining Logger Unmarshalling Business Thread Producer

Slide 101

Slide 101 text

Le pipelining private Disruptor createDisruptor() { Disruptor disruptor = new Disruptor <>(
 PlatformEvent ::new, bufferSize(), new ThreadFactoryBuilder()
 .setNameFormat("disruptor").build(), ProducerType.MULTI, new BlockingWaitStrategy()); disruptor.handleEventsWith(logger).and(marshaller) .then(business_handler) .then(output); return disruptor; }

Slide 102

Slide 102 text

Le pipelining private Disruptor createDisruptor() { Disruptor disruptor = new Disruptor <>(
 PlatformEvent ::new, bufferSize(), new ThreadFactoryBuilder()
 .setNameFormat("disruptor").build(), ProducerType.MULTI, new BlockingWaitStrategy()); disruptor.handleEventsWith(logger).and(marshaller) .then(business_handler) .then(output); return disruptor; } Architecture LMAX ?

Slide 103

Slide 103 text

Le parking

Slide 104

Slide 104 text

Mais… Le disruptor n’est que le commencement

Slide 105

Slide 105 text

Haute disponibilité Et aussi survivre au reboot, tout simplement Execution engine 1 Execution engine 2 Execution engine 3 ? ? ? ?

Slide 106

Slide 106 text

Execution engine 1 Event Sourcing ? Execution engine 2 ? ?

Slide 107

Slide 107 text

Execution engine 1 Command sourcing ! Execution engine 2 Execution engine 3

Slide 108

Slide 108 text

Execution engine 1 Command sourcing ! Execution engine 2 Execution engine 3 Le metier doit etre deterministe

Slide 109

Slide 109 text

Raft

Slide 110

Slide 110 text

Raft

Slide 111

Slide 111 text

Mais… Merde²

Slide 112

Slide 112 text

No content

Slide 113

Slide 113 text

UDP avec acquittement négatif Broker less

Slide 114

Slide 114 text

Json, un load tester pour votre GC MTU & taille de vos messages Encoding Binaire

Slide 115

Slide 115 text

Aeron…

Slide 116

Slide 116 text

Media Driver Sender Conductor Receiver Media Driver Sender Conductor Receiver Client Publisher Conductor Subscriber Client Subscriber Conductor Publisher Aeron: Messaging

Slide 117

Slide 117 text

Subscription Recording Replayed subscription Segments Aeron archive : replay

Slide 118

Slide 118 text

Aeron Cluster : Raft Media Driver Sender Conductor Receiver Archiving Media Driver Consensus module Clustered Service Business logic Snapshot

Slide 119

Slide 119 text

Le monde extérieur

Slide 120

Slide 120 text

Gateways Execution engine

Slide 121

Slide 121 text

Gateways Execution engine Order placed

Slide 122

Slide 122 text

Gateways Execution engine Order placed Order Canceled

Slide 123

Slide 123 text

Gateways Execution engine Order placed Order Canceled Match

Slide 124

Slide 124 text

Gateways Execution engine Order placed Order Canceled Match Gateway

Slide 125

Slide 125 text

Gateways Execution engine Order placed Order Canceled Match Gateway

Slide 126

Slide 126 text

Gateways Execution engine Order placed Order Canceled Match Gateway

Slide 127

Slide 127 text

Projections « lentes » Execution engine 1 Execution engine 2 Execution engine 3 Consommateur CRC 32 check

Slide 128

Slide 128 text

Web gateway Web socket gateway AFR Execution Engine Exchange view Accounting Internal gateway Accounting import Settlement CA OTS Blockchain / Custody

Slide 129

Slide 129 text

Web gateway Web socket gateway AFR Execution Engine Exchange view Accounting Internal gateway Accounting import Settlement CA OTS Blockchain / Custody

Slide 130

Slide 130 text

Web gateway Web socket gateway AFR Execution Engine Exchange view Accounting Internal gateway Accounting import Settlement CA OTS Blockchain / Custody

Slide 131

Slide 131 text

Web gateway Web socket gateway AFR Execution Engine Exchange view Accounting Internal gateway Accounting import Settlement CA OTS Blockchain / Custody

Slide 132

Slide 132 text

Web gateway Web socket gateway AFR Execution Engine Exchange view Accounting Internal gateway Accounting import Settlement CA OTS Blockchain / Custody

Slide 133

Slide 133 text

Web gateway Web socket gateway AFR Execution Engine Exchange view Accounting Internal gateway Accounting import Settlement CA OTS Blockchain / Custody

Slide 134

Slide 134 text

Web gateway Web socket gateway AFR Execution Engine Exchange view Accounting Internal gateway Accounting import Settlement CA OTS Blockchain / Custody

Slide 135

Slide 135 text

Web gateway Web socket gateway AFR Execution Engine Exchange view Accounting Internal gateway Accounting import Settlement CA OTS Blockchain / Custody

Slide 136

Slide 136 text

Web gateway Web socket gateway AFR Execution Engine Exchange view Accounting Internal gateway Accounting import Settlement CA OTS Blockchain / Custody

Slide 137

Slide 137 text

Web gateway Web socket gateway AFR Execution Engine Exchange view Accounting Internal gateway Accounting import Settlement CA OTS Blockchain / Custody

Slide 138

Slide 138 text

Web gateway Web socket gateway AFR Execution Engine Exchange view Accounting Internal gateway Accounting import Settlement CA OTS Blockchain / Custody

Slide 139

Slide 139 text

Web gateway Web socket gateway AFR Execution Engine Exchange view Accounting Internal gateway Accounting import Settlement CA OTS Blockchain / Custody

Slide 140

Slide 140 text

Web gateway Web socket gateway AFR Execution Engine Exchange view Accounting Internal gateway Accounting import Settlement CA OTS Blockchain / Custody

Slide 141

Slide 141 text

Web gateway Web socket gateway AFR Execution Engine Exchange view Accounting Internal gateway Accounting import Settlement CA OTS Blockchain / Custody

Slide 142

Slide 142 text

Web gateway Web socket gateway AFR Execution Engine Exchange view Accounting Internal gateway Accounting import Settlement CA OTS Blockchain / Custody

Slide 143

Slide 143 text

Web gateway Web socket gateway AFR Execution Engine Exchange view Accounting Internal gateway Accounting import Settlement CA OTS Blockchain / Custody

Slide 144

Slide 144 text

« Une implémentation complexe mais pas compliquée »

Slide 145

Slide 145 text

Mais finalement on parle de DDD ? Et le métier, il est où ?

Slide 146

Slide 146 text

En réalité il est peu impacté, la performance se construit grâce à l’orchestration de celui-ci Mais finalement on parle de DDD ? Et le métier, il est où ?

Slide 147

Slide 147 text

Just…

Slide 148

Slide 148 text

Just… •Faire attention aux allocations

Slide 149

Slide 149 text

Just… •Faire attention aux allocations •Data-structures adaptées

Slide 150

Slide 150 text

Just… •Faire attention aux allocations •Data-structures adaptées •Escape Analysis, utilisation de la pile

Slide 151

Slide 151 text

By The Way

Slide 152

Slide 152 text

By The Way •Property Based Testing

Slide 153

Slide 153 text

By The Way •Property Based Testing •Check sur les Invariants du Domain

Slide 154

Slide 154 text

Merci ! lgo.group Arnaud LEMAIRE @lilobase Jean Baptiste Dusseaut @bodysplash