Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Knee-Deep Into P2P: A Tale of Fail (PWL Porto)

Knee-Deep Into P2P: A Tale of Fail (PWL Porto)

6497e10d8345ce6fee06048127196d6b?s=128

Fernando Mendes

May 23, 2018
Tweet

Transcript

  1. Knee-Deep Into P2P A Tale of Fail @fribmendes

  2. Knee-Deep Into P2P A Tale of Fail @fribmendes

  3. None
  4. None
  5. None
  6. None
  7. None
  8. I don’t know how to smart office

  9. I don’t know how to smart office … what now?

  10. @fribmendes me failing at photoshop

  11. None
  12. I don’t know how to smart office … what now?

  13. None
  14. None
  15. None
  16. None
  17. Step 1: receive new connections

  18. Step 1: receive new connections Step 2: accept and send

    messages
  19. Step 1: receive new connections Step 2: accept and send

    messages Step 3: do a bunch of Steps 1 and 2
  20. Step 1: receive new connections

  21. None
  22. defp accept_loop(pid, server_socket) do {:ok, client} = :gen_tcp.accept(server_socket) :inet.setopts(client, [active:

    true]) :gen_tcp.controlling_process(client, pid) Gossip.accept(pid, client) accept_loop(pid, server_socket) end
  23. defp accept_loop(pid, server_socket) do {:ok, client} = :gen_tcp.accept(server_socket) :inet.setopts(client, [active:

    true]) :gen_tcp.controlling_process(client, pid) Gossip.accept(pid, client) accept_loop(pid, server_socket) end
  24. Step 1: receive new connections Step 2: accept and send

    messages
  25. None
  26. def recv_loop(pid, socket) do receive do {:tcp, _port, msg} ->

    # process an incoming message {:tcp_closed, port} -> # close the sockets {:send, msg} -> # send an outgoing message end end end
  27. Step 1: receive new connections Step 2: accept and send

    messages Step 3: do a bunch of Steps 1 and 2
  28. Raspberry Pi #1 Raspberry Pi #2

  29. None
  30. None
  31. None
  32. “Does it scale?”

  33. None
  34. None
  35. g

  36. Gnutella

  37. Gnutella

  38. Gnutella

  39. Gnutella

  40. Gnutella

  41. g

  42. g (gnutella2)

  43. Gnutella

  44. G2/Gnutella2

  45. G2/Gnutella2

  46. G2/Gnutella2

  47. G2/Gnutella2

  48. None
  49. None
  50. None
  51. None
  52. None
  53. HyParView

  54. None
  55. None
  56. None
  57. None
  58. None
  59. None
  60. None
  61. None
  62. Plumtrees

  63. Optimal number of messages

  64. But you can’t afford to lose nodes

  65. None
  66. None
  67. None
  68. None
  69. None
  70. None
  71. None
  72. None
  73. None
  74. None
  75. None
  76. “Aha! It works on my computer!”

  77. “Aha! It works on my computer!”

  78. “Great but we need something to show”

  79. “Great but we need something to show” (aka Raspberry Pi

    time)
  80. “Guys… Is this a bomb? Are we going to die?”

    — @naps62
  81. “Hey, I can borrow™ someone else’s code”

  82. None
  83. None
  84. None
  85. you shall not pass!

  86. Stick everything on Raspberry Pi’s

  87. Things running on one Raspberry Pi

  88. Things running on one Raspberry Pi ✓BEAM

  89. Things running on one Raspberry Pi ✓BEAM ✓thebox (sensors)

  90. Things running on one Raspberry Pi ✓BEAM ✓thebox (sensors) ✓Phoenix

    app
  91. Things running on one Raspberry Pi ✓BEAM (x2) ✓thebox (sensors)

    ✓Phoenix app
  92. Things running on one Raspberry Pi ✓BEAM (x2) ✓thebox (sensors)

    ✓Phoenix app ✓Postgres
  93. Things running on one Raspberry Pi ✓BEAM (x2) ✓thebox (sensors)

    ✓Phoenix app ✓Postgres ✓Cassandra
  94. Things running on one Raspberry Pi ✓BEAM (x2) ✓thebox (sensors)

    ✓Phoenix app ✓Postgres ✓Cassandra it works!
  95. None
  96. None
  97. None
  98. “Looking good! Everything’s working!”

  99. lol, nope

  100. State of each node:

  101. State of each node: • Last sensor readings

  102. State of each node: • Last sensor readings • Network

    map (MAC-IP)
  103. State of each node: • Last sensor readings • Network

    map (MAC-IP) • Target values
  104. State of each node: • Last sensor readings • Network

    map (MAC-IP) • Target values
  105. None
  106. How do we handle concurrency?

  107. None
  108. None
  109. No database locks. No transactions. You’re on your own, kiddo.

  110. Vector Clocks

  111. None
  112. None
  113. None
  114. None
  115. None
  116. None
  117. None
  118. None
  119. Vector = (1, 0) Vector = (0, 1)

  120. CAP Theorem

  121. CAP Theorem “you’re a programmer. you can’t have nice things.”

  122. consistency availability partitioning

  123. consistency availability partitioning

  124. None
  125. Eventual Consistency

  126. CRDTs

  127. None
  128. Operation-Based CRDT

  129. Operation-Based CRDT commutative but not idempotent update exactly once

  130. no CRDTs

  131. no CRDTs

  132. no CRDTs

  133. no CRDTs

  134. Op-based CRDTs

  135. Op-based CRDTs

  136. Op-based CRDTs

  137. Op-based CRDTs

  138. State-Based CRDT

  139. State-Based CRDT commutative and idempotent heavier on the network

  140. State-based CRDTs

  141. State-based CRDTs

  142. State-based CRDTs

  143. State-based CRDTs

  144. None
  145. None
  146. None
  147. None
  148. None
  149. None
  150. None
  151. None
  152. Wrapping up

  153. System resources matter

  154. System resources matter your algorithms should account for them

  155. There are models. Use them.

  156. Distributed System Checklist

  157. Distributed System Checklist •Is the number of processes known or

    finite?
  158. Distributed System Checklist •Is the number of processes known or

    finite? •Is there a global notion of time?
  159. Distributed System Checklist •Is the number of processes known or

    finite? •Is there a global notion of time? •Is the network reliable?
  160. Distributed System Checklist •Is the number of processes known or

    finite? •Is there a global notion of time? •Is the network reliable? •Is there full connectivity?
  161. Distributed System Checklist •Is the number of processes known or

    finite? •Is there a global notion of time? •Is the network reliable? •Is there full connectivity? •What happens when a process crashes?
  162. It really doesn’t change that much

  163. CRDTs aren’t a golden hammer

  164. Reinventing the wheel is stupid

  165. None
  166. Knee-Deep Into P2P A Tale of Fail @fribmendes