Upgrade to PRO for Only $50/Year—Limited-Time Offer! 🔥

Distributed Elixir

Distributed Elixir

Presentation about some of the tools for distributed programming in Elixir

Maciej Kaszubowski

July 07, 2018
Tweet

More Decks by Maciej Kaszubowski

Other Decks in Programming

Transcript

  1. send(pid2, msg) Pid 1 Node A Pid 2 Node B

    destination_node = node(pid) TCP Connection
  2. send(pid2, msg) Pid 1 Node A Pid 2 Node B

    destination_node = node(pid) :erlang.term_to_binary(msg) TCP Connection
  3. send(pid2, msg) Pid 1 Node A Pid 2 Node B

    destination_node = node(pid) :erlang.term_to_binary(msg) TCP Connection
  4. send(pid2, msg) Pid 1 Node A Pid 2 Node B

    destination_node = node(pid) :erlang.term_to_binary(msg) TCP Connection :erlang.binary_to_term(encode)
  5. send(pid2, msg) Pid 1 Node A receive msg Pid 2

    Node B destination_node = node(pid) :erlang.term_to_binary(msg) TCP Connection :erlang.binary_to_term(encode)
  6. fallacies of distributed computing 1. The network is reliable 2.

    Latency is zero 3. Bandwidth is infinite 4. The network is secure 5. Topology doesn’t change 6. The is one administrator 7. Transport cost is zero 8. The network is homogenous
  7. Pid 1 Pid 2 Pid3 Guarantees m1, m2, m3 m4,

    m5, m6 send(pid2, m1) send(pid2, m2) send(pid2, m3) send(pid2, m4) send(pid2, m5) send(pid2, m6)
  8. Pid 1 Pid 2 Pid3 Guarantees m1, m2, m3 m4,

    m5, m6 send(pid2, m1) send(pid2, m2) send(pid2, m3) send(pid2, m4) send(pid2, m5) send(pid2, m6) Ordering between two processes is preserved
  9. Pid 1 Pid 2 Pid3 Guarantees m4, m5, m6 send(pid2,

    m1) send(pid2, m2) send(pid2, m3) send(pid2, m4) send(pid2, m5) send(pid2, m6) m1, m2, m3 Delivery is not guaranteed
  10. Pid 1 Pid 2 Pid3 Guarantees m1, m2, m3 m4,

    m5, m6 send(pid2, m1) send(pid2, m2) send(pid2, m3) send(pid2, m4) send(pid2, m5) send(pid2, m6) Ordering between different processes is not guaranteed
  11. [m1, m2, m3, m4, m5, m6] [m4, m5, m6, m1,

    m2, m3] [m1, m4, m2, m5, m3, m6]
  12. [m1, m2, m3, m4, m5, m6] [m4, m5, m6, m1,

    m2, m3] [m1, m4, m2, m5, m3, m6] [m1, m2, m3]
  13. [m1, m2, m3, m4, m5, m6] [m4, m5, m6, m1,

    m2, m3] [m1, m4, m2, m5, m3, m6] [m1, m2, m3] [m1, m3, m5, m6]
  14. [m1, m2, m3, m4, m5, m6] [m4, m5, m6, m1,

    m2, m3] [m1, m4, m2, m5, m3, m6] [m1, m2, m3] [m1, m3, m5, m6] []
  15. [m1, m2, m3, m4, m5, m6] [m4, m5, m6, m1,

    m2, m3] [m1, m4, m2, m5, m3, m6] [m1, m2, m3] [m1, m3, m5, m6] [] [m1, m3, m2, m4, m5, m6]
  16. [m1, m2, m3, m4, m5, m6] [m4, m5, m6, m1,

    m2, m3] [m1, m4, m2, m5, m3, m6] [m1, m2, m3] [m1, m3, m5, m6] [] [m1, m3, m2, m4, m5, m6] [M3, M3]
  17. Pid 1 Node A Node B Pid 2 :global.register_name(“global”, self())

    Register PId1 as “global” Sure :global.whereis_name(“global”) = pid1
  18. Pid 1 Node A Node B Pid 2 :global.register_name(“global”, self())

    :global.register_name(“global”, self()) ?
  19. :global • single process registration (if everything works OK) •

    Favours availability over consistency • Information stored locally (reading is fast) • Registration is blocking (may be slow)
  20. :pg2 • Process groups • Favours availability over consistency •

    Information stored locally (reading is fast) • Registration is blocking (may be slow)
  21. Strongly consistent Solutions • Consensus - Raft, Paxos, ZAB •

    Two-phase commit/THree-phase commit (2PC/3PC) • Read/Write quorums • Single database as a source of truth
  22. Worth looking at • Riak_core • RAFT • Two-Phase Commit

    (2PC) / Three-Phase Commit (3PC) • CRDTs • LASP and Partisan
  23. • https://raft.github.io/ (Raft Consensus) • http://learnyousomeerlang.com/distribunomicon • https://www.rgoarchitects.com/Files/fallacies.pdf (Fallacies of

    distributed computing) • https://dzone.com/articles/better-explaining-cap-theorem (CAP Theorem) • https://medium.com/learn-elixir/message-order-and-delivery-guarantees-in-elixir- erlang-9350a3ea7541 (Elixir message delivery guarantees) • https://lasp-lang.readme.io/ (LASP) • https://arxiv.org/pdf/1802.02652.pdf (Partisan Paper) • https://bravenewgeek.com/tag/three-phase-commit/ (3PC)