Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Knee-Deep Into P2P: A Tale of Fail (non-Elixir)

Knee-Deep Into P2P: A Tale of Fail (non-Elixir)

The slides for my talk at Lambda Days 2018 (www.lambdadays.org/lambdadays2018)

Fernando Mendes

March 07, 2018
Tweet

More Decks by Fernando Mendes

Other Decks in Programming

Transcript

  1. Knee-Deep Into P2P
    A Tale of Fail
    @fribmendes

    View Slide

  2. Knee-Deep Into P2P
    A Tale of Fail
    @fribmendes

    View Slide

  3. “This is SOOO boring…”
    - me, circa last year or smth

    View Slide

  4. View Slide

  5. View Slide

  6. View Slide

  7. “This is SOOO boring…”
    - me, 10 months ago

    View Slide

  8. View Slide

  9. View Slide

  10. View Slide

  11. View Slide

  12. View Slide



  13. - me, whenever I get to play
    with distributed systems

    View Slide

  14. View Slide

  15. I don’t know how to
    smart office

    View Slide

  16. I know how to
    web development

    View Slide

  17. I know how to
    web development
    … what now?

    View Slide

  18. View Slide

  19. Step 1: receive new connections

    View Slide

  20. Step 1: receive new connections
    Step 2: accept and send messages

    View Slide

  21. Step 1: receive new connections
    Step 2: accept and send messages
    Step 3: do a bunch of Steps 1 and 2

    View Slide

  22. Step 1: receive new connections

    View Slide

  23. View Slide

  24. defmodule Gossip.Server do
    def listen(pid, port) do
    {:ok, server_socket} = :gen_tcp.listen(port, @socket_opts)
    accept_loop(pid, server_socket)
    end
    defp accept_loop(pid, server_socket) do
    {:ok, client} = :gen_tcp.accept(server_socket)
    :inet.setopts(client, [active: true])
    :gen_tcp.controlling_process(client, pid)
    Gossip.accept(pid, client)
    accept_loop(pid, server_socket)
    end
    end

    View Slide

  25. defmodule Gossip.Server do
    def listen(pid, port) do
    {:ok, server_socket} = :gen_tcp.listen(port, @socket_opts)
    accept_loop(pid, server_socket)
    end
    defp accept_loop(pid, server_socket) do
    {:ok, client} = :gen_tcp.accept(server_socket)
    :inet.setopts(client, [active: true])
    :gen_tcp.controlling_process(client, pid)
    Gossip.accept(pid, client)
    accept_loop(pid, server_socket)
    end
    end

    View Slide

  26. Step 1: receive new connections
    Step 2: accept and send messages

    View Slide

  27. View Slide

  28. defmodule Gossip.Worker do
    def recv_loop(pid, socket) do
    continue = receive do
    {:tcp, _port, msg} ->
    Gossip.recv(pid, msg)
    true
    {:tcp_closed, port} ->
    :gen_tcp.close(port)
    Gossip.disconnect(pid, self())
    false
    {:send, msg} ->
    :gen_tcp.send(socket, msg)
    true
    end
    continue and recv_loop(pid, socket)
    end
    end

    View Slide

  29. defmodule Gossip.Worker do
    def recv_loop(pid, socket) do
    continue = receive do
    {:tcp, _port, msg} ->
    Gossip.recv(pid, msg)
    true
    {:tcp_closed, port} ->
    :gen_tcp.close(port)
    Gossip.disconnect(pid, self())
    false
    {:send, msg} ->
    :gen_tcp.send(socket, msg)
    true
    end
    continue and recv_loop(pid, socket)
    end
    end

    View Slide

  30. Step 1: receive new connections
    Step 2: accept and send messages
    Step 3: do a bunch of Steps 1 and 2

    View Slide

  31. Raspberry Pi #1 Raspberry Pi #2

    View Slide

  32. View Slide

  33. View Slide

  34. “Does it scale?”

    View Slide

  35. View Slide

  36. Time to copy off the internet
    find a creative solution

    View Slide

  37. View Slide

  38. g

    View Slide

  39. Gnutella

    View Slide

  40. Gnutella

    View Slide

  41. Gnutella

    View Slide

  42. Gnutella

    View Slide

  43. Gnutella

    View Slide

  44. g
    (gnutella2)

    View Slide

  45. Gnutella

    View Slide

  46. G2/Gnutella2

    View Slide

  47. G2/Gnutella2

    View Slide

  48. G2/Gnutella2

    View Slide

  49. G2/Gnutella2

    View Slide

  50. View Slide

  51. View Slide

  52. View Slide

  53. View Slide

  54. View Slide

  55. HyParView

    View Slide

  56. View Slide

  57. View Slide

  58. View Slide

  59. View Slide

  60. View Slide

  61. View Slide

  62. Plumtrees

    View Slide

  63. Optimal number of messages

    View Slide

  64. But you can’t afford to lose nodes

    View Slide

  65. View Slide

  66. View Slide

  67. View Slide

  68. View Slide

  69. View Slide

  70. View Slide

  71. View Slide

  72. View Slide

  73. View Slide

  74. View Slide

  75. “Aha! It works on my computer!”

    View Slide

  76. “Great but we need
    something to show”

    View Slide

  77. “Great but we need
    something to show”
    (aka Raspberry Pi time)

    View Slide

  78. “Sure.
    Seems legit…”
    — @iampfac

    View Slide

  79. “Wait.
    He works here!?”
    — @naps62

    View Slide

  80. “Hey, I can borrow™ someone else’s code”

    View Slide

  81. View Slide

  82. View Slide

  83. View Slide

  84. you shall not pass!

    View Slide

  85. Solution: stick everything on Raspberry Pi’s

    View Slide

  86. Things running on one Raspberry Pi

    View Slide

  87. Things running on one Raspberry Pi
    ✓BEAM

    View Slide

  88. Things running on one Raspberry Pi
    ✓BEAM
    ✓thebox (sensors)

    View Slide

  89. Things running on one Raspberry Pi
    ✓BEAM
    ✓thebox (sensors)
    ✓Phoenix app

    View Slide

  90. Things running on one Raspberry Pi
    ✓BEAM (x2)
    ✓thebox (sensors)
    ✓Phoenix app

    View Slide

  91. Things running on one Raspberry Pi
    ✓BEAM (x2)
    ✓thebox (sensors)
    ✓Phoenix app
    ✓Postgres

    View Slide

  92. Things running on one Raspberry Pi
    ✓BEAM (x2)
    ✓thebox (sensors)
    ✓Phoenix app
    ✓Postgres
    ✓Cassandra

    View Slide

  93. Things running on one Raspberry Pi
    ✓BEAM (x2)
    ✓thebox (sensors)
    ✓Phoenix app
    ✓Postgres
    ✓Cassandra
    it works!

    View Slide

  94. View Slide

  95. View Slide

  96. @antenna = Satellite::Antenna.new(host, port)
    @antenna.on(“data") do |data|
    if data.avg_temp > 25
    slack.send_msg(“people, it's too hot")
    end
    end
    @antenna.on("lights") do |payload, data|
    payload == "on" ? turn_lights_on : turn_lights_off
    end
    @antenna.watch

    View Slide

  97. “Looking good! Everything’s working!”

    View Slide

  98. View Slide

  99. State of each node:

    View Slide

  100. State of each node:
    • Last sensor readings

    View Slide

  101. State of each node:
    • Last sensor readings
    • Network map (MAC-IP)

    View Slide

  102. State of each node:
    • Last sensor readings
    • Network map (MAC-IP)
    • Target values

    View Slide

  103. State of each node:
    • Last sensor readings
    • Network map (MAC-IP)
    • Target values

    View Slide

  104. View Slide

  105. How do we handle concurrency?

    View Slide

  106. View Slide

  107. View Slide

  108. No database locks.
    No transactions.
    You’re on your own, kiddo.

    View Slide

  109. Vector Clocks

    View Slide

  110. View Slide

  111. View Slide

  112. View Slide

  113. View Slide

  114. View Slide

  115. View Slide

  116. View Slide

  117. Shared state
    and why it sucks

    View Slide

  118. Vector = (1, 0)
    Vector = (0, 1)

    View Slide

  119. CAP Theorem

    View Slide

  120. CAP Theorem
    “you’re a programmer.
    you can’t have nice things.”

    View Slide

  121. consistency
    availability partitioning

    View Slide

  122. consistency
    availability partitioning

    View Slide

  123. Eventual Consistency

    View Slide

  124. CRDTs

    View Slide

  125. Operation-Based CRDT

    View Slide

  126. Operation-Based CRDT
    commutative but not idempotent
    update exactly once

    View Slide

  127. no CRDTs

    View Slide

  128. no CRDTs

    View Slide

  129. no CRDTs

    View Slide

  130. no CRDTs

    View Slide

  131. Op-based CRDTs

    View Slide

  132. Op-based CRDTs

    View Slide

  133. Op-based CRDTs

    View Slide

  134. Op-based CRDTs

    View Slide

  135. State-Based CRDT

    View Slide

  136. State-Based CRDT
    commutative and idempotent
    heavier on the network

    View Slide

  137. State-based CRDTs

    View Slide

  138. State-based CRDTs

    View Slide

  139. State-based CRDTs

    View Slide

  140. State-based CRDTs

    View Slide

  141. View Slide

  142. View Slide

  143. View Slide

  144. View Slide

  145. View Slide

  146. View Slide

  147. View Slide

  148. Wrapping up

    View Slide

  149. System resources matter

    View Slide

  150. System resources matter
    your algorithms should
    account for them

    View Slide

  151. There are models.
    Use them.

    View Slide

  152. Distributed System Checklist

    View Slide

  153. Distributed System Checklist
    •Is the number of processes known or finite?

    View Slide

  154. Distributed System Checklist
    •Is the number of processes known or finite?
    •Is there a global notion of time?

    View Slide

  155. Distributed System Checklist
    •Is the number of processes known or finite?
    •Is there a global notion of time?
    •Is the network reliable?

    View Slide

  156. Distributed System Checklist
    •Is the number of processes known or finite?
    •Is there a global notion of time?
    •Is the network reliable?
    •Is there full connectivity?

    View Slide

  157. Distributed System Checklist
    •Is the number of processes known or finite?
    •Is there a global notion of time?
    •Is the network reliable?
    •Is there full connectivity?
    •What happens when a process crashes?

    View Slide

  158. It really doesn’t change
    that much

    View Slide

  159. CRDTs aren’t a golden hammer

    View Slide

  160. Reinventing the wheel is stupid

    View Slide

  161. View Slide

  162. Knee-Deep Into P2P
    A Tale of Fail
    @fribmendes

    View Slide