Knee-Deep Into P2P: A Tale of Fail (PWL Porto)

Transcript

1. Knee-Deep Into P2P A Tale of Fail @fribmendes

2. Knee-Deep Into P2P A Tale of Fail @fribmendes

3. None
4. None
5. None
6. None
7. None

8. I don’t know how to smart office

9. I don’t know how to smart office … what now?

10. @fribmendes me failing at photoshop

11. None

12. I don’t know how to smart office … what now?

13. None
14. None
15. None
16. None

17. Step 1: receive new connections

18. Step 1: receive new connections Step 2: accept and send

    messages

19. Step 1: receive new connections Step 2: accept and send

    messages Step 3: do a bunch of Steps 1 and 2

20. Step 1: receive new connections

21. None

22. defp accept_loop(pid, server_socket) do {:ok, client} = :gen_tcp.accept(server_socket) :inet.setopts(client, [active:

    true]) :gen_tcp.controlling_process(client, pid) Gossip.accept(pid, client) accept_loop(pid, server_socket) end

23. defp accept_loop(pid, server_socket) do {:ok, client} = :gen_tcp.accept(server_socket) :inet.setopts(client, [active:

    true]) :gen_tcp.controlling_process(client, pid) Gossip.accept(pid, client) accept_loop(pid, server_socket) end

24. Step 1: receive new connections Step 2: accept and send

    messages
25. None

26. def recv_loop(pid, socket) do receive do {:tcp, _port, msg} ->

    # process an incoming message {:tcp_closed, port} -> # close the sockets {:send, msg} -> # send an outgoing message end end end

27. Step 1: receive new connections Step 2: accept and send

    messages Step 3: do a bunch of Steps 1 and 2

28. Raspberry Pi #1 Raspberry Pi #2

29. None
30. None
31. None

32. “Does it scale?”

33. None
34. None

35. g

36. Gnutella

37. Gnutella

38. Gnutella

39. Gnutella

40. Gnutella

41. g

42. g (gnutella2)

43. Gnutella

44. G2/Gnutella2

45. G2/Gnutella2

46. G2/Gnutella2

47. G2/Gnutella2

48. None
49. None
50. None
51. None
52. None

53. HyParView

54. None
55. None
56. None
57. None
58. None
59. None
60. None
61. None

62. Plumtrees

63. Optimal number of messages

64. But you can’t afford to lose nodes

65. None
66. None
67. None
68. None
69. None
70. None
71. None
72. None
73. None
74. None
75. None

76. “Aha! It works on my computer!”

77. “Aha! It works on my computer!”

78. “Great but we need something to show”

79. “Great but we need something to show” (aka Raspberry Pi

    time)

80. “Guys… Is this a bomb? Are we going to die?”

    — @naps62

81. “Hey, I can borrow™ someone else’s code”

82. None
83. None
84. None

85. you shall not pass!

86. Stick everything on Raspberry Pi’s

87. Things running on one Raspberry Pi

88. Things running on one Raspberry Pi ✓BEAM

89. Things running on one Raspberry Pi ✓BEAM ✓thebox (sensors)

90. Things running on one Raspberry Pi ✓BEAM ✓thebox (sensors) ✓Phoenix

    app

91. Things running on one Raspberry Pi ✓BEAM (x2) ✓thebox (sensors)

    ✓Phoenix app

92. Things running on one Raspberry Pi ✓BEAM (x2) ✓thebox (sensors)

    ✓Phoenix app ✓Postgres

93. Things running on one Raspberry Pi ✓BEAM (x2) ✓thebox (sensors)

    ✓Phoenix app ✓Postgres ✓Cassandra

94. Things running on one Raspberry Pi ✓BEAM (x2) ✓thebox (sensors)

    ✓Phoenix app ✓Postgres ✓Cassandra it works!
95. None
96. None
97. None

98. “Looking good! Everything’s working!”

99. lol, nope

100. State of each node:

101. State of each node: • Last sensor readings

102. State of each node: • Last sensor readings • Network

     map (MAC-IP)

103. State of each node: • Last sensor readings • Network

     map (MAC-IP) • Target values

104. State of each node: • Last sensor readings • Network

     map (MAC-IP) • Target values
105. None

106. How do we handle concurrency?

107. None
108. None

109. No database locks. No transactions. You’re on your own, kiddo.

110. Vector Clocks

111. None
112. None
113. None
114. None
115. None
116. None
117. None
118. None

119. Vector = (1, 0) Vector = (0, 1)

120. CAP Theorem

121. CAP Theorem “you’re a programmer. you can’t have nice things.”

122. consistency availability partitioning

123. consistency availability partitioning

124. None

125. Eventual Consistency

126. CRDTs

127. None

128. Operation-Based CRDT

129. Operation-Based CRDT commutative but not idempotent update exactly once

130. no CRDTs

131. no CRDTs

132. no CRDTs

133. no CRDTs

134. Op-based CRDTs

135. Op-based CRDTs

136. Op-based CRDTs

137. Op-based CRDTs

138. State-Based CRDT

139. State-Based CRDT commutative and idempotent heavier on the network

140. State-based CRDTs

141. State-based CRDTs

142. State-based CRDTs

143. State-based CRDTs

144. None
145. None
146. None
147. None
148. None
149. None
150. None
151. None

152. Wrapping up

153. System resources matter

154. System resources matter your algorithms should account for them

155. There are models. Use them.

156. Distributed System Checklist

157. Distributed System Checklist •Is the number of processes known or

     finite?

158. Distributed System Checklist •Is the number of processes known or

     finite? •Is there a global notion of time?

159. Distributed System Checklist •Is the number of processes known or

     finite? •Is there a global notion of time? •Is the network reliable?

160. Distributed System Checklist •Is the number of processes known or

     finite? •Is there a global notion of time? •Is the network reliable? •Is there full connectivity?

161. Distributed System Checklist •Is the number of processes known or

     finite? •Is there a global notion of time? •Is the network reliable? •Is there full connectivity? •What happens when a process crashes?

162. It really doesn’t change that much

163. CRDTs aren’t a golden hammer

164. Reinventing the wheel is stupid

165. None

166. Knee-Deep Into P2P A Tale of Fail @fribmendes