Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Knee-Deep Into P2P: A Tale of Fail (non-Elixir)
Search
Fernando Mendes
March 07, 2018
Programming
0
170
Knee-Deep Into P2P: A Tale of Fail (non-Elixir)
The slides for my talk at Lambda Days 2018 (
www.lambdadays.org/lambdadays2018
)
Fernando Mendes
March 07, 2018
Tweet
Share
More Decks by Fernando Mendes
See All by Fernando Mendes
you. and the morals of technology
fribmendes
1
130
Knee-Deep Into P2P: A Tale of Fail (PWL Porto)
fribmendes
0
60
Knee-Deep Into P2P: A Tale of Fail (ElixirConf EU 2018 version)
fribmendes
0
160
A Look Into Bloom Filters
fribmendes
0
420
Bloom Filters: A Look Into Ruby
fribmendes
0
110
Programming WTF: HTML & CSS
fribmendes
4
160
Ruby: A (pointless) Workshop
fribmendes
1
160
Elixir: A Talk For College Students
fribmendes
0
160
Riding Rails
fribmendes
0
110
Other Decks in Programming
See All in Programming
Model Pollution
hschwentner
1
180
どの様にAIエージェントと 協業すべきだったのか?
takefumiyoshii
2
600
CSC305 Lecture 02
javiergs
PRO
1
260
Railsだからできる 例外業務に禍根を残さない 設定設計パターン
ei_ei_eiichi
0
260
Pull-Requestの内容を1クリックで動作確認可能にするワークフロー
natmark
2
450
私はどうやって技術力を上げたのか
yusukebe
43
17k
dynamic!
moro
9
6.5k
AIエージェント時代における TypeScriptスキーマ駆動開発の新たな役割
bicstone
4
1.5k
CSC509 Lecture 05
javiergs
PRO
0
290
開発生産性を上げるための生成AI活用術
starfish719
1
170
iOSアプリの信頼性を向上させる取り組み/ios-app-improve-reliability
shino8rayu9
0
150
あなたの知らない「動画広告」の世界 - iOSDC Japan 2025
ukitaka
0
380
Featured
See All Featured
Chrome DevTools: State of the Union 2024 - Debugging React & Beyond
addyosmani
7
890
Intergalactic Javascript Robots from Outer Space
tanoku
273
27k
Visualization
eitanlees
148
16k
What’s in a name? Adding method to the madness
productmarketing
PRO
23
3.7k
Imperfection Machines: The Place of Print at Facebook
scottboms
269
13k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
54
3k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
49
3.1k
XXLCSS - How to scale CSS and keep your sanity
sugarenia
248
1.3M
How to train your dragon (web standard)
notwaldorf
96
6.3k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
52
5.6k
Typedesign – Prime Four
hannesfritz
42
2.8k
For a Future-Friendly Web
brad_frost
180
9.9k
Transcript
Knee-Deep Into P2P A Tale of Fail @fribmendes
Knee-Deep Into P2P A Tale of Fail @fribmendes
“This is SOOO boring…” - me, circa last year or
smth
None
None
None
“This is SOOO boring…” - me, 10 months ago
None
None
None
None
None
“ ” - me, whenever I get to play with
distributed systems
None
I don’t know how to smart office
I know how to web development
I know how to web development … what now?
None
Step 1: receive new connections
Step 1: receive new connections Step 2: accept and send
messages
Step 1: receive new connections Step 2: accept and send
messages Step 3: do a bunch of Steps 1 and 2
Step 1: receive new connections
None
defmodule Gossip.Server do def listen(pid, port) do {:ok, server_socket} =
:gen_tcp.listen(port, @socket_opts) accept_loop(pid, server_socket) end defp accept_loop(pid, server_socket) do {:ok, client} = :gen_tcp.accept(server_socket) :inet.setopts(client, [active: true]) :gen_tcp.controlling_process(client, pid) Gossip.accept(pid, client) accept_loop(pid, server_socket) end end
defmodule Gossip.Server do def listen(pid, port) do {:ok, server_socket} =
:gen_tcp.listen(port, @socket_opts) accept_loop(pid, server_socket) end defp accept_loop(pid, server_socket) do {:ok, client} = :gen_tcp.accept(server_socket) :inet.setopts(client, [active: true]) :gen_tcp.controlling_process(client, pid) Gossip.accept(pid, client) accept_loop(pid, server_socket) end end
Step 1: receive new connections Step 2: accept and send
messages
None
defmodule Gossip.Worker do def recv_loop(pid, socket) do continue = receive
do {:tcp, _port, msg} -> Gossip.recv(pid, msg) true {:tcp_closed, port} -> :gen_tcp.close(port) Gossip.disconnect(pid, self()) false {:send, msg} -> :gen_tcp.send(socket, msg) true end continue and recv_loop(pid, socket) end end
defmodule Gossip.Worker do def recv_loop(pid, socket) do continue = receive
do {:tcp, _port, msg} -> Gossip.recv(pid, msg) true {:tcp_closed, port} -> :gen_tcp.close(port) Gossip.disconnect(pid, self()) false {:send, msg} -> :gen_tcp.send(socket, msg) true end continue and recv_loop(pid, socket) end end
Step 1: receive new connections Step 2: accept and send
messages Step 3: do a bunch of Steps 1 and 2
Raspberry Pi #1 Raspberry Pi #2
None
None
“Does it scale?”
None
Time to copy off the internet find a creative solution
None
g
Gnutella
Gnutella
Gnutella
Gnutella
Gnutella
g (gnutella2)
Gnutella
G2/Gnutella2
G2/Gnutella2
G2/Gnutella2
G2/Gnutella2
None
None
None
None
None
HyParView
None
None
None
None
None
None
Plumtrees
Optimal number of messages
But you can’t afford to lose nodes
None
None
None
None
None
None
None
None
None
None
“Aha! It works on my computer!”
“Great but we need something to show”
“Great but we need something to show” (aka Raspberry Pi
time)
“Sure. Seems legit…” — @iampfac
“Wait. He works here!?” — @naps62
“Hey, I can borrow™ someone else’s code”
None
None
None
you shall not pass!
Solution: stick everything on Raspberry Pi’s
Things running on one Raspberry Pi
Things running on one Raspberry Pi ✓BEAM
Things running on one Raspberry Pi ✓BEAM ✓thebox (sensors)
Things running on one Raspberry Pi ✓BEAM ✓thebox (sensors) ✓Phoenix
app
Things running on one Raspberry Pi ✓BEAM (x2) ✓thebox (sensors)
✓Phoenix app
Things running on one Raspberry Pi ✓BEAM (x2) ✓thebox (sensors)
✓Phoenix app ✓Postgres
Things running on one Raspberry Pi ✓BEAM (x2) ✓thebox (sensors)
✓Phoenix app ✓Postgres ✓Cassandra
Things running on one Raspberry Pi ✓BEAM (x2) ✓thebox (sensors)
✓Phoenix app ✓Postgres ✓Cassandra it works!
None
None
@antenna = Satellite::Antenna.new(host, port) @antenna.on(“data") do |data| if data.avg_temp >
25 slack.send_msg(“people, it's too hot") end end @antenna.on("lights") do |payload, data| payload == "on" ? turn_lights_on : turn_lights_off end @antenna.watch
“Looking good! Everything’s working!”
None
State of each node:
State of each node: • Last sensor readings
State of each node: • Last sensor readings • Network
map (MAC-IP)
State of each node: • Last sensor readings • Network
map (MAC-IP) • Target values
State of each node: • Last sensor readings • Network
map (MAC-IP) • Target values
None
How do we handle concurrency?
None
None
No database locks. No transactions. You’re on your own, kiddo.
Vector Clocks
None
None
None
None
None
None
None
Shared state and why it sucks
Vector = (1, 0) Vector = (0, 1)
CAP Theorem
CAP Theorem “you’re a programmer. you can’t have nice things.”
consistency availability partitioning
consistency availability partitioning
Eventual Consistency
CRDTs
Operation-Based CRDT
Operation-Based CRDT commutative but not idempotent update exactly once
no CRDTs
no CRDTs
no CRDTs
no CRDTs
Op-based CRDTs
Op-based CRDTs
Op-based CRDTs
Op-based CRDTs
State-Based CRDT
State-Based CRDT commutative and idempotent heavier on the network
State-based CRDTs
State-based CRDTs
State-based CRDTs
State-based CRDTs
None
None
None
None
None
None
None
Wrapping up
System resources matter
System resources matter your algorithms should account for them
There are models. Use them.
Distributed System Checklist
Distributed System Checklist •Is the number of processes known or
finite?
Distributed System Checklist •Is the number of processes known or
finite? •Is there a global notion of time?
Distributed System Checklist •Is the number of processes known or
finite? •Is there a global notion of time? •Is the network reliable?
Distributed System Checklist •Is the number of processes known or
finite? •Is there a global notion of time? •Is the network reliable? •Is there full connectivity?
Distributed System Checklist •Is the number of processes known or
finite? •Is there a global notion of time? •Is the network reliable? •Is there full connectivity? •What happens when a process crashes?
It really doesn’t change that much
CRDTs aren’t a golden hammer
Reinventing the wheel is stupid
None
Knee-Deep Into P2P A Tale of Fail @fribmendes