Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Knee-Deep Into P2P: A Tale of Fail (PWL Porto)

6497e10d8345ce6fee06048127196d6b?s=47 Fernando Mendes
May 23, 2018
Programming
0
39

Knee-Deep Into P2P: A Tale of Fail (PWL Porto)

6497e10d8345ce6fee06048127196d6b?s=128

Fernando Mendes

May 23, 2018
Tweet

More Decks by Fernando Mendes

See All by Fernando Mendes
you. and the morals of technology
6497e10d8345ce6fee06048127196d6b?s=48 fribmendes
1
90
Knee-Deep Into P2P: A Tale of Fail (ElixirConf EU 2018 version)
6497e10d8345ce6fee06048127196d6b?s=48 fribmendes
0
58
Knee-Deep Into P2P: A Tale of Fail (non-Elixir)
6497e10d8345ce6fee06048127196d6b?s=48 fribmendes
0
120
A Look Into Bloom Filters
6497e10d8345ce6fee06048127196d6b?s=48 fribmendes
0
140
Bloom Filters: A Look Into Ruby
6497e10d8345ce6fee06048127196d6b?s=48 fribmendes
0
85
Programming WTF: HTML & CSS
6497e10d8345ce6fee06048127196d6b?s=48 fribmendes
4
130
Ruby: A (pointless) Workshop
6497e10d8345ce6fee06048127196d6b?s=48 fribmendes
1
130
Elixir: A Talk For College Students
6497e10d8345ce6fee06048127196d6b?s=48 fribmendes
0
110
Riding Rails
6497e10d8345ce6fee06048127196d6b?s=48 fribmendes
0
85

Other Decks in Programming

See All in Programming
OSS Forward Workshop
011714704c4a925e542d426d4cdaa4e3?s=48 giginet
2
460
A technique to implement DSL in Ruby
110cd4dcaed019b3bd9809e541afdeb6?s=48 okuramasafumi
0
830
既存画面の Jetpack Composeでの書き換え: FAANSでの事例紹介 / Case study of rewriting existing screens with Jetpack Compose
Dbab5e5a1fd54531e1119ede6b3b9e65?s=48 horie1024
0
360
マイクロインタラクション入門〜ディテイルにこだわるエンジニアリング〜
F1fc4b62755cd021f4bf9ce5543065aa?s=48 swimmyxox
0
120
よりUXに近いSLI・SLOの運用による可用性の再設計
953d2907b80dd5955c561d695ea9a494?s=48 kazumanagano
3
1k
【Qiita Night】新卒エンジニアによるSwift6与太予想
3fc2c76b81e2acd1a263f30db01a248a?s=48 eiji127
0
190
SRE NEXT 2022: Sensible Incident Management for Software Startups
C431886a52f7545f1b122b0c221e7bcd?s=48 takanabe
2
780
Let's build components, not layers
Bc7aa56e8e004e4984e954e5e4cb2bc0?s=48 thombergs
1
230
アプリのログをチーム外で活用してもらうためにやったこと
2f8b61ad31dd9c6ada4a90af746290fd?s=48 shotakashihara
0
200
Cloud-Conference-Day-Spring Cloud + Spring Webflux: como desenvolver seu primeiro microsserviço reativo em Java?
102398563fd1bf878b35dc2d5ee67cbe?s=48 kamilahsantos
1
170
The Success of Rails: Ensuring Growth for the Next 100 Years
C44e1f7e22c3f23cff7bc130871047ef?s=48 eileencodes
4
660
モバイルファーストデザインの爆速実装を考える
7303c95e41e7523fdc955b8736978c9d?s=48 tanabee8
0
170

Featured

See All Featured
Unsuck your backbone
B83d5b590577969ee79a5ad845411a7d?s=48 ammeep
659
55k
Creatively Recalculating Your Daily Design Routine
48c098b6579220a67a6ba949be6e4b5a?s=48 revolveconf
207
10k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
B3ace07412a5178ea778e7eec161fc0a?s=48 jcasabona
12
900
4 Signs Your Business is Dying
E4f5d494ebdc9e7cce1aecf3ce3e8bc1?s=48 shpigford
169
20k
How to train your dragon (web standard)
053e75a5b48b44d6dd0612795dfb326d?s=48 notwaldorf
57
3.8k
StorybookのUI Testing Handbookを読んだ
50fc0fdc2327ecbe7350645812de52e3?s=48 zakiyama
4
2k
Raft: Consensus for Rubyists
B6a8f005f39d23ffc930508ac9da68b9?s=48 vanstee
126
5.4k
How GitHub Uses GitHub to Build GitHub
78b475797a14c84799063c7cd073962f?s=48 holman
465
280k
Testing 201, or: Great Expectations
A9704266587836f7e784235e5073b93e?s=48 jmmastey
21
5.4k
The Language of Interfaces
Fde6c3a50ca518a909ef35c22c0ee0e4?s=48 destraynor
148
20k
ピンチをチャンスに：未来をつくるプロダクトロードマップ #pmconf2020
E425fe9f170efa0dfaf8ab69d7107418?s=48 aki_i
21
15k
Learning to Love Humans: Emotional Interface Design
5f50f94346fbcb23706f0707169317a4?s=48 aarron
261
37k

Transcript

  1. Knee-Deep Into P2P A Tale of Fail @fribmendes

  2. Knee-Deep Into P2P A Tale of Fail @fribmendes

  3. None
  4. None
  5. None
  6. None
  7. None

  8. I don’t know how to smart office

  9. I don’t know how to smart office … what now?

  10. @fribmendes me failing at photoshop

  11. None

  12. I don’t know how to smart office … what now?

  13. None
  14. None
  15. None
  16. None

  17. Step 1: receive new connections

  18. Step 1: receive new connections Step 2: accept and send

    messages

  19. Step 1: receive new connections Step 2: accept and send

    messages Step 3: do a bunch of Steps 1 and 2

  20. Step 1: receive new connections

  21. None

  22. defp accept_loop(pid, server_socket) do {:ok, client} = :gen_tcp.accept(server_socket) :inet.setopts(client, [active:

    true]) :gen_tcp.controlling_process(client, pid) Gossip.accept(pid, client) accept_loop(pid, server_socket) end

  23. defp accept_loop(pid, server_socket) do {:ok, client} = :gen_tcp.accept(server_socket) :inet.setopts(client, [active:

    true]) :gen_tcp.controlling_process(client, pid) Gossip.accept(pid, client) accept_loop(pid, server_socket) end

  24. Step 1: receive new connections Step 2: accept and send

    messages
  25. None

  26. def recv_loop(pid, socket) do receive do {:tcp, _port, msg} ->

    # process an incoming message {:tcp_closed, port} -> # close the sockets {:send, msg} -> # send an outgoing message end end end

  27. Step 1: receive new connections Step 2: accept and send

    messages Step 3: do a bunch of Steps 1 and 2

  28. Raspberry Pi #1 Raspberry Pi #2

  29. None
  30. None
  31. None

  32. “Does it scale?”

  33. None
  34. None

  35. g

  36. Gnutella

  37. Gnutella

  38. Gnutella

  39. Gnutella

  40. Gnutella

  41. g

  42. g (gnutella2)

  43. Gnutella

  44. G2/Gnutella2

  45. G2/Gnutella2

  46. G2/Gnutella2

  47. G2/Gnutella2

  48. None
  49. None
  50. None
  51. None
  52. None

  53. HyParView

  54. None
  55. None
  56. None
  57. None
  58. None
  59. None
  60. None
  61. None

  62. Plumtrees

  63. Optimal number of messages

  64. But you can’t afford to lose nodes

  65. None
  66. None
  67. None
  68. None
  69. None
  70. None
  71. None
  72. None
  73. None
  74. None
  75. None

  76. “Aha! It works on my computer!”

  77. “Aha! It works on my computer!”

  78. “Great but we need something to show”

  79. “Great but we need something to show” (aka Raspberry Pi

    time)

  80. “Guys… Is this a bomb? Are we going to die?”

    — @naps62

  81. “Hey, I can borrow™ someone else’s code”

  82. None
  83. None
  84. None

  85. you shall not pass!

  86. Stick everything on Raspberry Pi’s

  87. Things running on one Raspberry Pi

  88. Things running on one Raspberry Pi ✓BEAM

  89. Things running on one Raspberry Pi ✓BEAM ✓thebox (sensors)

  90. Things running on one Raspberry Pi ✓BEAM ✓thebox (sensors) ✓Phoenix

    app

  91. Things running on one Raspberry Pi ✓BEAM (x2) ✓thebox (sensors)

    ✓Phoenix app

  92. Things running on one Raspberry Pi ✓BEAM (x2) ✓thebox (sensors)

    ✓Phoenix app ✓Postgres

  93. Things running on one Raspberry Pi ✓BEAM (x2) ✓thebox (sensors)

    ✓Phoenix app ✓Postgres ✓Cassandra

  94. Things running on one Raspberry Pi ✓BEAM (x2) ✓thebox (sensors)

    ✓Phoenix app ✓Postgres ✓Cassandra it works!
  95. None
  96. None
  97. None

  98. “Looking good! Everything’s working!”

  99. lol, nope

  100. State of each node:

  101. State of each node: • Last sensor readings

  102. State of each node: • Last sensor readings • Network

    map (MAC-IP)

  103. State of each node: • Last sensor readings • Network

    map (MAC-IP) • Target values

  104. State of each node: • Last sensor readings • Network

    map (MAC-IP) • Target values
  105. None

  106. How do we handle concurrency?

  107. None
  108. None

  109. No database locks. No transactions. You’re on your own, kiddo.

  110. Vector Clocks

  111. None
  112. None
  113. None
  114. None
  115. None
  116. None
  117. None
  118. None

  119. Vector = (1, 0) Vector = (0, 1)

  120. CAP Theorem

  121. CAP Theorem “you’re a programmer. you can’t have nice things.”

  122. consistency availability partitioning

  123. consistency availability partitioning

  124. None

  125. Eventual Consistency

  126. CRDTs

  127. None

  128. Operation-Based CRDT

  129. Operation-Based CRDT commutative but not idempotent update exactly once

  130. no CRDTs

  131. no CRDTs

  132. no CRDTs

  133. no CRDTs

  134. Op-based CRDTs

  135. Op-based CRDTs

  136. Op-based CRDTs

  137. Op-based CRDTs

  138. State-Based CRDT

  139. State-Based CRDT commutative and idempotent heavier on the network

  140. State-based CRDTs

  141. State-based CRDTs

  142. State-based CRDTs

  143. State-based CRDTs

  144. None
  145. None
  146. None
  147. None
  148. None
  149. None
  150. None
  151. None

  152. Wrapping up

  153. System resources matter

  154. System resources matter your algorithms should account for them

  155. There are models. Use them.

  156. Distributed System Checklist

  157. Distributed System Checklist •Is the number of processes known or

    finite?

  158. Distributed System Checklist •Is the number of processes known or

    finite? •Is there a global notion of time?

  159. Distributed System Checklist •Is the number of processes known or

    finite? •Is there a global notion of time? •Is the network reliable?

  160. Distributed System Checklist •Is the number of processes known or

    finite? •Is there a global notion of time? •Is the network reliable? •Is there full connectivity?

  161. Distributed System Checklist •Is the number of processes known or

    finite? •Is there a global notion of time? •Is the network reliable? •Is there full connectivity? •What happens when a process crashes?

  162. It really doesn’t change that much

  163. CRDTs aren’t a golden hammer

  164. Reinventing the wheel is stupid

  165. None

  166. Knee-Deep Into P2P A Tale of Fail @fribmendes