Distributed Elixir - Speaker Deck

Slide 1

Slide 1 text

It’s scary out there

Slide 2

Slide 2 text

Organisational Matters

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

We’re 1 year old!

Slide 5

Slide 5 text

Summer break (probably)

Slide 6

Slide 6 text

We’re looking for speakers!

Slide 7

Slide 7 text

It’s scary out there Distributed Systems in Elixir Poznań Elixir Meetup #8

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

Pid 1 Pid 2

Slide 10

Slide 10 text

Pid 1 Pid 2 Node A Node B

Slide 11

Slide 11 text

The basics

Slide 12

Slide 12 text

iex --name [email protected] --cookie cookie -S mix

Slide 13

Slide 13 text

Node.connect(:’[email protected]')

Slide 14

Slide 14 text

(DEMO)

Slide 15

Slide 15 text

#PID<0.94.0>

Slide 16

Slide 16 text

#PID<0.94.0> node identiﬁer (relative to current node)

Slide 17

Slide 17 text

#PID<0.94.0> node identiﬁer (relative to current node) 0 =a local process

Slide 18

Slide 18 text

#PID<0.94.0> Process id node identiﬁer (relative to current node)

Slide 19

Slide 19 text

How does it work?

Slide 20

Slide 20 text

Pid 1 Node A Pid 2 Node B

Slide 21

Slide 21 text

Pid 1 Node A Pid 2 Node B TCP Connection

Slide 22

Slide 22 text

send(pid2, msg) Pid 1 Node A Pid 2 Node B TCP Connection

Slide 23

Slide 23 text

send(pid2, msg) Pid 1 Node A Pid 2 Node B destination_node = node(pid) TCP Connection

Slide 24

Slide 24 text

send(pid2, msg) Pid 1 Node A Pid 2 Node B destination_node = node(pid) :erlang.term_to_binary(msg) TCP Connection

Slide 25

Slide 25 text

send(pid2, msg) Pid 1 Node A Pid 2 Node B destination_node = node(pid) :erlang.term_to_binary(msg) TCP Connection

Slide 26

Slide 26 text

send(pid2, msg) Pid 1 Node A Pid 2 Node B destination_node = node(pid) :erlang.term_to_binary(msg) TCP Connection :erlang.binary_to_term(encode)

Slide 27

Slide 27 text

send(pid2, msg) Pid 1 Node A receive msg Pid 2 Node B destination_node = node(pid) :erlang.term_to_binary(msg) TCP Connection :erlang.binary_to_term(encode)

Slide 28

Slide 28 text

Distributed Systems?

Slide 29

Slide 29 text

Distributed Systems? Solved!

Slide 30

Slide 30 text

Well, not exactly…

Slide 31

Slide 31 text

Difficulties

Slide 32

Slide 32 text

Node A Node B

Slide 33

Slide 33 text

Node A Node B Node C

Slide 34

Slide 34 text

Node A Node B Node C Node D

Slide 35

Slide 35 text

No content

Slide 36

Slide 36 text

A lot of messages

Slide 37

Slide 37 text

us-east-1 us-west-2

Slide 38

Slide 38 text

8 fallacies of distributed computing

Slide 39

Slide 39 text

fallacies of distributed computing 1. The network is reliable 2. Latency is zero 3. Bandwidth is inﬁnite 4. The network is secure 5. Topology doesn’t change 6. The is one administrator 7. Transport cost is zero 8. The network is homogenous

Slide 40

Slide 40 text

CAP THEOREM

Slide 41

Slide 41 text

CAP THEOREM us-west-2 us-east-1

Slide 42

Slide 42 text

CAP THEOREM us-west-2 us-east-1 Set X=5

Slide 43

Slide 43 text

CAP THEOREM us-west-2 us-east-1 Set X=5 Read X

Slide 44

Slide 44 text

CAP THEOREM us-west-2 us-east-1 Set X=5 Set X = 7

Slide 45

Slide 45 text

Consistency or Availability (under network partition)

Slide 46

Slide 46 text

Consistency or Speed In practice

Slide 47

Slide 47 text

Guarantees

Slide 48

Slide 48 text

Pid 1 Pid 2 Pid3 Guarantees m1, m2, m3 m4, m5, m6 send(pid2, m1) send(pid2, m2) send(pid2, m3) send(pid2, m4) send(pid2, m5) send(pid2, m6)

Slide 49

Slide 49 text

Pid 1 Pid 2 Pid3 Guarantees m1, m2, m3 m4, m5, m6 send(pid2, m1) send(pid2, m2) send(pid2, m3) send(pid2, m4) send(pid2, m5) send(pid2, m6) Ordering between two processes is preserved

Slide 50

Slide 50 text

Pid 1 Pid 2 Pid3 Guarantees m4, m5, m6 send(pid2, m1) send(pid2, m2) send(pid2, m3) send(pid2, m4) send(pid2, m5) send(pid2, m6) m1, m2, m3 Delivery is not guaranteed

Slide 51

Slide 51 text

Pid 1 Pid 2 Pid3 Guarantees m1, m2, m3 m4, m5, m6 send(pid2, m1) send(pid2, m2) send(pid2, m3) send(pid2, m4) send(pid2, m5) send(pid2, m6) Ordering between different processes is not guaranteed

Slide 52

Slide 52 text

[m1, m2, m3, m4, m5, m6]

Slide 53

Slide 53 text

[m1, m2, m3, m4, m5, m6] [m4, m5, m6, m1, m2, m3]

Slide 54

Slide 54 text

[m1, m2, m3, m4, m5, m6] [m4, m5, m6, m1, m2, m3] [m1, m4, m2, m5, m3, m6]

Slide 55

Slide 55 text

[m1, m2, m3, m4, m5, m6] [m4, m5, m6, m1, m2, m3] [m1, m4, m2, m5, m3, m6] [m1, m2, m3]

Slide 56

Slide 56 text

[m1, m2, m3, m4, m5, m6] [m4, m5, m6, m1, m2, m3] [m1, m4, m2, m5, m3, m6] [m1, m2, m3] [m1, m3, m5, m6]

Slide 57

Slide 57 text

[m1, m2, m3, m4, m5, m6] [m4, m5, m6, m1, m2, m3] [m1, m4, m2, m5, m3, m6] [m1, m2, m3] [m1, m3, m5, m6] []

Slide 58

Slide 58 text

[m1, m2, m3, m4, m5, m6] [m4, m5, m6, m1, m2, m3] [m1, m4, m2, m5, m3, m6] [m1, m2, m3] [m1, m3, m5, m6] [] [m1, m3, m2, m4, m5, m6]

Slide 59

Slide 59 text

[m1, m2, m3, m4, m5, m6] [m4, m5, m6, m1, m2, m3] [m1, m4, m2, m5, m3, m6] [m1, m2, m3] [m1, m3, m5, m6] [] [m1, m3, m2, m4, m5, m6] [M3, M3]

Slide 60

Slide 60 text

Phoenix Request A User Logged In

Slide 61

Slide 61 text

Phoenix Request A Phoenix Request B User Logged In User Logged OUT

Slide 62

Slide 62 text

Phoenix Request A Phoenix Request B User Logged In User Logged OUT This Can arrive ﬁrst

Slide 63

Slide 63 text

Unfortunately, things tend to work ﬁne locally

Slide 64

Slide 64 text

The Tools

Slide 65

Slide 65 text

:global

Slide 66

Slide 66 text

Pid 1 Node A Node B Pid 2

Slide 67

Slide 67 text

Pid 1 Node A Node B Pid 2 :global.register_name(“global”, self())

Slide 68

Slide 68 text

Pid 1 Node A Node B Pid 2 :global.register_name(“global”, self()) Register PId1 as “global”

Slide 69

Slide 69 text

Pid 1 Node A Node B Pid 2 :global.register_name(“global”, self()) Register PId1 as “global” Sure

Slide 70

Slide 70 text

Pid 1 Node A Node B Pid 2 :global.register_name(“global”, self()) Register PId1 as “global” Sure :global.whereis_name(“global”) = pid1

Slide 71

Slide 71 text

Pid 1 Node A Node B Pid 2 :global.register_name(“global”, self()) :global.register_name(“global”, self()) ?

Slide 72

Slide 72 text

(DEMO)

Slide 73

Slide 73 text

:global • single process registration (if everything works OK) • Favours availability over consistency • Information stored locally (reading is fast) • Registration is blocking (may be slow)

Slide 74

Slide 74 text

:PG2

Slide 75

Slide 75 text

Pid1 Pid3 Pid2 [] [] []

Slide 76

Slide 76 text

Pid1 Pid3 Pid2 :pg2.create(“my_group”) [] [] []

Slide 77

Slide 77 text

Pid1 Pid3 Pid2 [] [] [] join join :pg2.join(“my_group”, self()

Slide 78

Slide 78 text

Pid1 Pid3 Pid2 [] [pid1] [] Monitor Monitor :pg2.join(“my_group”, self()

Slide 79

Slide 79 text

Pid1 Pid3 Pid2 [pid1] [pid1] [pid1] Monitor Monitor :pg2.join(“my_group”, self()

Slide 80

Slide 80 text

Pid1 Pid3 Pid2 [pid1] [pid1] [pid1]

Slide 81

Slide 81 text

Pid1 Pid3 Pid2 :pg2.join(“my_group”, self() [pid1] [pid1, pid2] [pid1]

Slide 82

Slide 82 text

Pid1 Pid3 Pid2 join :pg2.join(“my_group”, self() join [pid1, pid2] [pid1, pid2] [pid1, pid2]

Slide 83

Slide 83 text

Pid1 Pid3 Pid2 [pid1] [pid2] [pid1]

Slide 84

Slide 84 text

Pid1 Pid3 Pid2 [pid1] [pid2] [pid1]

Slide 85

Slide 85 text

Pid1 Pid3 Pid2 [pid1] [pid2] [pid1]

Slide 86

Slide 86 text

Pid1 Pid3 Pid2 [pid1, pid2] [pid1, pid2] [pid1, pid2]

Slide 87

Slide 87 text

It will heal, but the state in inconsistent for some time

Slide 88

Slide 88 text

What does it matter?

Slide 89

Slide 89 text

Node A Pg2 Pg2 Pg2 Node B Node C

Slide 90

Slide 90 text

Node A Pg2 Pg2 Pg2 Node B Node C Phoenix Channels

Slide 91

Slide 91 text

Node A Pg2 Pg2 Pg2 Node B Node C Phoenix Presence

Slide 92

Slide 92 text

Node A Pg2 Pg2 Pg2 Node B Node C Phoenix Channels

Slide 93

Slide 93 text

:pg2 • Process groups • Favours availability over consistency • Information stored locally (reading is fast) • Registration is blocking (may be slow)

Slide 94

Slide 94 text

Strongly consistent Solutions

Slide 95

Slide 95 text

Strongly consistent Solutions • Consensus - Raft, Paxos, ZAB • Two-phase commit/THree-phase commit (2PC/3PC) • Read/Write quorums • Single database as a source of truth

Slide 96

Slide 96 text

Summary

Slide 97

Slide 97 text

Distributed Systems

Slide 98

Slide 98 text

Well, not exactly…

Slide 99

Slide 99 text

Asynchronous messages Distributed systems are all about

Slide 100

Slide 100 text

Really, there’s no magic

Slide 101

Slide 101 text

Just asynchronous messages between nodes

Slide 102

Slide 102 text

Just asynchronous messages between nodes & node failures

Slide 103

Slide 103 text

Just asynchronous messages between nodes & node failures & Communication failures

Slide 104

Slide 104 text

Just asynchronous messages between nodes & node failures & Communication failures & Network partitions

Slide 105

Slide 105 text

Tradeoffs Distributed systems are all about

Slide 106

Slide 106 text

Where to go next

Slide 107

Slide 107 text

Worth looking at • Riak_core • RAFT • Two-Phase Commit (2PC) / Three-Phase Commit (3PC) • CRDTs • LASP and Partisan

Slide 108

Slide 108 text

Free online (click!) Elixir / Erlang

Slide 109

Slide 109 text

Free PDF (Click!) Distributed Systems

Slide 110

Slide 110 text

Theory (The hard stuff)

Slide 111

Slide 111 text

• https://raft.github.io/ (Raft Consensus) • http://learnyousomeerlang.com/distribunomicon • https://www.rgoarchitects.com/Files/fallacies.pdf (Fallacies of distributed computing) • https://dzone.com/articles/better-explaining-cap-theorem (CAP Theorem) • https://medium.com/learn-elixir/message-order-and-delivery-guarantees-in-elixir- erlang-9350a3ea7541 (Elixir message delivery guarantees) • https://lasp-lang.readme.io/ (LASP) • https://arxiv.org/pdf/1802.02652.pdf (Partisan Paper) • https://bravenewgeek.com/tag/three-phase-commit/ (3PC)