Tweet
Tweet

Slide 1

Slide 1 text

Maintaining consistency in distributed systems with an effect machine

Slide 2

Slide 2 text

Carolina Pascale Software Engineer @ Nubank @CarolinaPascale github.com/carolpc instagram.com/carolpc

Slide 3

Slide 3 text

Marcelo Piva Software Engineer @ Nubank @mpivaa github.com/mpivaa instagram.com/m.pivaa

Slide 4

Slide 4 text

In the beginning there was nothing.

Slide 5

Slide 5 text

So the programmers said: “let there be a monolith”

Slide 6

Slide 6 text

And a monolith was created

Slide 7

Slide 7 text

But it was coupled and not scalable, and the programmers saw that it was not good.

Slide 8

Slide 8 text

But it was coupled and not scalable, and the programmers saw that it was not good.

Slide 9

Slide 9 text

Emergency-driven context switching

Slide 10

Slide 10 text

but it was coupled and not scalable, and the programmers saw that it was not good.

Slide 11

Slide 11 text

Hard to implement new architectural designs

Slide 12

Slide 12 text

Long onboarding for the newcomer

Slide 13

Slide 13 text

Biggest source of shared state: database as a global variable

Slide 14

Slide 14 text

Conway's law

Slide 15

Slide 15 text

But it was coupled and not scalable, and the programmers saw that it was not good.

Slide 16

Slide 16 text

So they said: “let there be microservices”.

Slide 17

Slide 17 text

and microservices were created.

Slide 18

Slide 18 text

At first it was good, but with microservices came chaos and problems

Slide 19

Slide 19 text

No content

Slide 20

Slide 20 text

At first it was good, but with microservices came chaos and problems

Slide 21

Slide 21 text

No content

Slide 22

Slide 22 text

No content

Slide 23

Slide 23 text

consistency and testing issues

Slide 24

Slide 24 text

Idempotency

Slide 25

Slide 25 text

At least once

Slide 26

Slide 26 text

Replayable

Slide 27

Slide 27 text

No content

Slide 28

Slide 28 text

Avoid effects duplication

Slide 29

Slide 29 text

Pure functions

Slide 30

Slide 30 text

Order of effects matter

Slide 31

Slide 31 text

A B C

Slide 32

Slide 32 text

A B C

Slide 33

Slide 33 text

A B C ❌

Slide 34

Slide 34 text

A B C ❌

Slide 35

Slide 35 text

A B C ❌

Slide 36

Slide 36 text

❌

Slide 37

Slide 37 text

consistency and testing issues eventual

Slide 38

Slide 38 text

A B C

Slide 39

Slide 39 text

❌ A B C

Slide 40

Slide 40 text

Coordinating

Slide 41

Slide 41 text

db-time db-time db-time A B C sync

Slide 42

Slide 42 text

db-time db-time db-time A B C

Slide 43

Slide 43 text

consistency and testing issues

Slide 44

Slide 44 text

Services need to communicate

Slide 45

Slide 45 text

No content

Slide 46

Slide 46 text

IO is hard to test

Slide 47

Slide 47 text

Unit Integration E2E

Slide 48

Slide 48 text

Isolate side-effects

Slide 49

Slide 49 text

Pure functions 2

Slide 50

Slide 50 text

What can we do?

Slide 51

Slide 51 text

First attempt: hexagonal architecture

Slide 52

Slide 52 text

Inner layer: domain model

Slide 53

Slide 53 text

Second layer: application

Slide 54

Slide 54 text

Third layer: IO

Slide 55

Slide 55 text

Imperative code and side effects on the second layer

Slide 56

Slide 56 text

Why not imperative? Harder to reason about Harder to compose Harder to test Slower feedback

Slide 57

Slide 57 text

Why not imperative? Harder to reason about Harder to compose Harder to test Slower feedback

Slide 58

Slide 58 text

Why not imperative? (block! card datomic producer) ; =>

Slide 59

Slide 59 text

Why not imperative? (defn block! [card datomic producer] (when (logic.card/can-block? card) (let [blocked-card (logic.card/block card)] (datomic.card/update! blocked-card datomic) (producer.card/status-changed! blocked-card producer))))

Slide 60

Slide 60 text

Why not imperative? Harder to reason about Harder to compose Harder to test Slower feedback

Slide 61

Slide 61 text

Why not imperative? (defn block-all! [customer cards datomic producer] (doseq [card cards] (block! card datomic producer)) (cards-blocked-notify! customer producer))

Slide 62

Slide 62 text

Why not imperative? (defn block-all! [customer cards datomic producer] (doseq [card cards] (block! card datomic producer)) (cards-blocked-notify! customer producer))

Slide 63

Slide 63 text

Why not imperative? (defn block-all! [customer cards datomic producer] (doseq [card cards] (block! card datomic producer)) (cards-blocked-notify! customer producer))

Slide 64

Slide 64 text

Why not imperative? Harder to reason about Harder to compose Harder to test Slower feedback

Slide 65

Slide 65 text

Why not imperative? Mocks & Integration Tests

Slide 66

Slide 66 text

No content

Slide 67

Slide 67 text

Why not imperative? Harder to reason about Harder to compose Harder to test Slower feedback

Slide 68

Slide 68 text

Tests ❌ Consistency ❌

Slide 69

Slide 69 text

Second attempt: Free monads

Slide 70

Slide 70 text

CODE

Slide 71

Slide 71 text

CODE AST

Slide 72

Slide 72 text

CODE AST RUNNER IMPLEMENTATION

Slide 73

Slide 73 text

No content

Slide 74

Slide 74 text

Tests ✅ Consistency ❌

Slide 75

Slide 75 text

Not simple

Slide 76

Slide 76 text

Not easy

Slide 77

Slide 77 text

No content

Slide 78

Slide 78 text

Third attempt

Slide 79

Slide 79 text

isolate all side effects leveraging functional programming

Slide 80

Slide 80 text

No content

Slide 81

Slide 81 text

(defn run-effects! [message] (let [[datoms messages] (handler message) [error db-t] (persist! datoms)] (case error :db-error (abort! error) :idempotency-error (send-messages! messages db-t) nil (send-messages! messages db-t))))

Slide 82

Slide 82 text

(let [[datoms messages] (handler message) [error db-t] (persist! datoms)] (case error :db-error (abort! error) :idempotency-error (send-messages! messages db-t) nil (send-messages! messages db-t)))

Slide 83

Slide 83 text

(let [[datoms messages] (handler message) [error db-t] (persist! datoms)] (case error :db-error (abort! error) :idempotency-error (send-messages! messages db-t) nil (send-messages! messages db-t)))

Slide 84

Slide 84 text

A deterministic ID to each message

Slide 85

Slide 85 text

(let [[datoms messages] (handler message) [error db-t] (persist! datoms)] (case error :db-error (abort! error) :idempotency-error (send-messages! messages db-t) nil (send-messages! messages db-t)))

Slide 86

Slide 86 text

(let [[datoms messages] (handler message) [error db-t] (persist! datoms)] (case error :db-error (abort! error) :idempotency-error (send-messages! messages db-t) nil (send-messages! messages db-t)))

Slide 87

Slide 87 text

(let [[datoms messages] (handler message) [error db-t] (persist! datoms)] (case error :db-error (abort! error) :idempotency-error (send-messages! messages db-t) nil (send-messages! messages db-t)))

Slide 88

Slide 88 text

(let [[datoms messages] (handler message) [error db-t] (persist! datoms)] (case error :db-error (abort! error) :idempotency-error (send-messages! messages db-t) nil (send-messages! messages db-t)))

Slide 89

Slide 89 text

(let [[datoms messages] (handler message) [error db-t] (persist! datoms)] (case error :db-error (abort! error) :idempotency-error (send-messages! messages db-t) nil (send-messages! messages db-t)))

Slide 90

Slide 90 text

so they said: “let there be an effect machine”

Slide 91

Slide 91 text

And an effect machine was created

Slide 92

Slide 92 text

Decouple decisions from effects

Slide 93

Slide 93 text

Decouple intent from execution

Slide 94

Slide 94 text

No content

Slide 95

Slide 95 text

Data as first class citizens

Slide 96

Slide 96 text

{:effects/datoms [{:db/id [:card/id "some-uuid"] :card/status :status/blocked}] :effects/requests [{:url :some-url :method :post}] :effects/messages [{:topic :card-status-changed :message {:card/status :status/blocked}}]}

Slide 97

Slide 97 text

{:effects/datoms [{:db/id [:card/id "some-uuid"] :card/status :status/blocked}] :effects/requests [{:url :some-url :method :post}] :effects/messages [{:topic :card-status-changed :message {:card/status :status/blocked}}]}

Slide 98

Slide 98 text

Declarative

Slide 99

Slide 99 text

(block card) ; => [{:effects/datoms [...] ; :effects/messages [...]}] Easier to reason about

Slide 100

Slide 100 text

(s/defschema BlockCard {:messages [(s/one StatusChanged)] :datoms [(s/one BlockedCard)]}) (defn block :- (s/either t.effects/BlockCard t.effects/Empty) [card] (if (logic.card/can-block? card) (let [blocked-card (logic.card/block card)] (-> e/empty-effects (effect.card/update blocked-card) (effect.card/status-changed blocked-card)))) e/empty-effects))

Slide 101

Slide 101 text

(defn block-all [customer cards] (-> (apply e/merge (map block cards)) (effect.card/card-blocked-notify customer))) Easier to compose

Slide 102

Slide 102 text

Atomic operations

Slide 103

Slide 103 text

(deftest block-card-test (is (= {:effects/datoms [{:db/id [:card/id "id"] :card/status :card/blocked}]} :effects/messages [{:topic :card-status-changed :message {:card/status :status/blocked}}] (block card)))) Easier to test

Slide 104

Slide 104 text

(defspec block-card-always-return-valid-effects-prop (props/for-all [card gen-card] (s/validate t.effects/BlockCard (block card)))) Easier to test

Slide 105

Slide 105 text

Global dependencies

Slide 106

Slide 106 text

Global dependencies Dependency injection

Slide 107

Slide 107 text

Dependency rejection Global dependencies Dependency injection

Slide 108

Slide 108 text

(defn block [card] (if (logic.card/can-block? card) (let [blocked-card (logic.card/block card)] (-> e/empty-effects (effect.card/update blocked-card) (effect.card/status-changed blocked-card)))) e/empty-effects))

Slide 109

Slide 109 text

(defn block [card] (if (logic.card/can-block? card) (let [blocked-card (logic.card/block card)] (-> e/empty-effects (effect.card/update blocked-card) (effect.card/status-changed blocked-card)))) e/empty-effects))

Slide 110

Slide 110 text

(s/defn run-effects! [effects :- Effects+Id components] (-> effects add-idempotency-check (run-datoms! components) (run-requests! components) after-run-requests (run-messages! components)))

Slide 111

Slide 111 text

Tests ✅ Consistency ✅

Slide 112

Slide 112 text

No content

Slide 113

Slide 113 text

https://t.me/clojurebrasil https://www.meetup.com/pt-BR/ClojureSP/

Slide 114

Slide 114 text

Thanks! We're hiring! @mpivaa github.com/mpivaa instagram.com/m.pivaa https://boards.greenhouse.io/nubank @CarolinaPascale github.com/carolpc instagram.com/carolpc