Knee-Deep Into P2P: A Tale of Fail (PWL Porto)

Transcript

1. Knee-Deep Into P2P A Tale of Fail @fribmendes
2. Knee-Deep Into P2P A Tale of Fail @fribmendes
8. I don’t know how to smart office
9. I don’t know how to smart office … what now?
10. @fribmendes me failing at photoshop
12. I don’t know how to smart office … what now?
17. Step 1: receive new connections
18. Step 1: receive new connections Step 2: accept and send messages
19. Step 1: receive new connections Step 2: accept and send messages Step 3: do a bunch of Steps 1 and 2
20. Step 1: receive new connections
22. defp accept_loop(pid, server_socket) do {:ok, client} = :gen_tcp.accept(server_socket) :inet.setopts(client, [active: true]) :gen_tcp.controlling_process(client, pid) Gossip.accept(pid, client) accept_loop(pid, server_socket) end
23. defp accept_loop(pid, server_socket) do {:ok, client} = :gen_tcp.accept(server_socket) :inet.setopts(client, [active: true]) :gen_tcp.controlling_process(client, pid) Gossip.accept(pid, client) accept_loop(pid, server_socket) end
24. Step 1: receive new connections Step 2: accept and send messages
26. def recv_loop(pid, socket) do receive do {:tcp, _port, msg} -> # process an incoming message {:tcp_closed, port} -> # close the sockets {:send, msg} -> # send an outgoing message end end end
27. Step 1: receive new connections Step 2: accept and send messages Step 3: do a bunch of Steps 1 and 2
28. Raspberry Pi #1 Raspberry Pi #2
32. “Does it scale?”
35. g
36. Gnutella
37. Gnutella
38. Gnutella
39. Gnutella
40. Gnutella
41. g
42. g (gnutella2)
43. Gnutella
44. G2/Gnutella2
45. G2/Gnutella2
46. G2/Gnutella2
47. G2/Gnutella2
53. HyParView
62. Plumtrees
63. Optimal number of messages
64. But you can’t afford to lose nodes
76. “Aha! It works on my computer!”
77. “Aha! It works on my computer!”
78. “Great but we need something to show”
79. “Great but we need something to show” (aka Raspberry Pi time)
80. “Guys… Is this a bomb? Are we going to die?” — @naps62
81. “Hey, I can borrow™ someone else’s code”
85. you shall not pass!
86. Stick everything on Raspberry Pi’s
87. Things running on one Raspberry Pi
88. Things running on one Raspberry Pi ✓BEAM
89. Things running on one Raspberry Pi ✓BEAM ✓thebox (sensors)
90. Things running on one Raspberry Pi ✓BEAM ✓thebox (sensors) ✓Phoenix app
91. Things running on one Raspberry Pi ✓BEAM (x2) ✓thebox (sensors) ✓Phoenix app
92. Things running on one Raspberry Pi ✓BEAM (x2) ✓thebox (sensors) ✓Phoenix app ✓Postgres
93. Things running on one Raspberry Pi ✓BEAM (x2) ✓thebox (sensors) ✓Phoenix app ✓Postgres ✓Cassandra
94. Things running on one Raspberry Pi ✓BEAM (x2) ✓thebox (sensors) ✓Phoenix app ✓Postgres ✓Cassandra it works!
98. “Looking good! Everything’s working!”
99. lol, nope
100. State of each node:
101. State of each node: • Last sensor readings
102. State of each node: • Last sensor readings • Network map (MAC-IP)
103. State of each node: • Last sensor readings • Network map (MAC-IP) • Target values
104. State of each node: • Last sensor readings • Network map (MAC-IP) • Target values
106. How do we handle concurrency?
109. No database locks. No transactions. You’re on your own, kiddo.
110. Vector Clocks
119. Vector = (1, 0) Vector = (0, 1)
120. CAP Theorem
121. CAP Theorem “you’re a programmer. you can’t have nice things.”
122. consistency availability partitioning
123. consistency availability partitioning
125. Eventual Consistency
126. CRDTs
128. Operation-Based CRDT
129. Operation-Based CRDT commutative but not idempotent update exactly once
130. no CRDTs
131. no CRDTs
132. no CRDTs
133. no CRDTs
134. Op-based CRDTs
135. Op-based CRDTs
136. Op-based CRDTs
137. Op-based CRDTs
138. State-Based CRDT
139. State-Based CRDT commutative and idempotent heavier on the network
140. State-based CRDTs
141. State-based CRDTs
142. State-based CRDTs
143. State-based CRDTs
152. Wrapping up
153. System resources matter
154. System resources matter your algorithms should account for them
155. There are models. Use them.
156. Distributed System Checklist
157. Distributed System Checklist •Is the number of processes known or finite?
158. Distributed System Checklist •Is the number of processes known or finite? •Is there a global notion of time?
159. Distributed System Checklist •Is the number of processes known or finite? •Is there a global notion of time? •Is the network reliable?
160. Distributed System Checklist •Is the number of processes known or finite? •Is there a global notion of time? •Is the network reliable? •Is there full connectivity?
161. Distributed System Checklist •Is the number of processes known or finite? •Is there a global notion of time? •Is the network reliable? •Is there full connectivity? •What happens when a process crashes?
162. It really doesn’t change that much
163. CRDTs aren’t a golden hammer
164. Reinventing the wheel is stupid
166. Knee-Deep Into P2P A Tale of Fail @fribmendes