Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Knee-Deep Into P2P: A Tale of Fail (non-Elixir)
Search
Fernando Mendes
March 07, 2018
Programming
0
170
Knee-Deep Into P2P: A Tale of Fail (non-Elixir)
The slides for my talk at Lambda Days 2018 (
www.lambdadays.org/lambdadays2018
)
Fernando Mendes
March 07, 2018
Tweet
Share
More Decks by Fernando Mendes
See All by Fernando Mendes
you. and the morals of technology
fribmendes
1
130
Knee-Deep Into P2P: A Tale of Fail (PWL Porto)
fribmendes
0
60
Knee-Deep Into P2P: A Tale of Fail (ElixirConf EU 2018 version)
fribmendes
0
160
A Look Into Bloom Filters
fribmendes
0
410
Bloom Filters: A Look Into Ruby
fribmendes
0
110
Programming WTF: HTML & CSS
fribmendes
4
160
Ruby: A (pointless) Workshop
fribmendes
1
160
Elixir: A Talk For College Students
fribmendes
0
160
Riding Rails
fribmendes
0
100
Other Decks in Programming
See All in Programming
rage against annotate_predecessor
junk0612
0
160
Testing Trophyは叫ばない
toms74209200
0
560
コンテキストエンジニアリング Cursor編
kinopeee
1
750
オープンセミナー2025@広島LT技術ブログを続けるには
satoshi256kbyte
0
150
「手軽で便利」に潜む罠。 Popover API を WCAG 2.2の視点で安全に使うには
taitotnk
0
600
兎に角、コードレビュー
mitohato14
0
170
CloudflareのChat Agent Starter Kitで簡単!AIチャットボット構築
syumai
2
380
為你自己學 Python - 冷知識篇
eddie
1
340
Ruby Parser progress report 2025
yui_knk
1
290
Claude Codeで挑むOSSコントリビュート
eycjur
0
190
パッケージ設計の黒魔術/Kyoto.go#63
lufia
3
410
もうちょっといいRubyプロファイラを作りたい (2025)
osyoyu
0
240
Featured
See All Featured
Writing Fast Ruby
sferik
628
62k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
667
120k
10 Git Anti Patterns You Should be Aware of
lemiorhan
PRO
656
61k
Optimizing for Happiness
mojombo
379
70k
Visualization
eitanlees
147
16k
How To Stay Up To Date on Web Technology
chriscoyier
790
250k
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
126
53k
What's in a price? How to price your products and services
michaelherold
246
12k
Producing Creativity
orderedlist
PRO
347
40k
Six Lessons from altMBA
skipperchong
28
4k
Large-scale JavaScript Application Architecture
addyosmani
512
110k
Being A Developer After 40
akosma
90
590k
Transcript
Knee-Deep Into P2P A Tale of Fail @fribmendes
Knee-Deep Into P2P A Tale of Fail @fribmendes
“This is SOOO boring…” - me, circa last year or
smth
None
None
None
“This is SOOO boring…” - me, 10 months ago
None
None
None
None
None
“ ” - me, whenever I get to play with
distributed systems
None
I don’t know how to smart office
I know how to web development
I know how to web development … what now?
None
Step 1: receive new connections
Step 1: receive new connections Step 2: accept and send
messages
Step 1: receive new connections Step 2: accept and send
messages Step 3: do a bunch of Steps 1 and 2
Step 1: receive new connections
None
defmodule Gossip.Server do def listen(pid, port) do {:ok, server_socket} =
:gen_tcp.listen(port, @socket_opts) accept_loop(pid, server_socket) end defp accept_loop(pid, server_socket) do {:ok, client} = :gen_tcp.accept(server_socket) :inet.setopts(client, [active: true]) :gen_tcp.controlling_process(client, pid) Gossip.accept(pid, client) accept_loop(pid, server_socket) end end
defmodule Gossip.Server do def listen(pid, port) do {:ok, server_socket} =
:gen_tcp.listen(port, @socket_opts) accept_loop(pid, server_socket) end defp accept_loop(pid, server_socket) do {:ok, client} = :gen_tcp.accept(server_socket) :inet.setopts(client, [active: true]) :gen_tcp.controlling_process(client, pid) Gossip.accept(pid, client) accept_loop(pid, server_socket) end end
Step 1: receive new connections Step 2: accept and send
messages
None
defmodule Gossip.Worker do def recv_loop(pid, socket) do continue = receive
do {:tcp, _port, msg} -> Gossip.recv(pid, msg) true {:tcp_closed, port} -> :gen_tcp.close(port) Gossip.disconnect(pid, self()) false {:send, msg} -> :gen_tcp.send(socket, msg) true end continue and recv_loop(pid, socket) end end
defmodule Gossip.Worker do def recv_loop(pid, socket) do continue = receive
do {:tcp, _port, msg} -> Gossip.recv(pid, msg) true {:tcp_closed, port} -> :gen_tcp.close(port) Gossip.disconnect(pid, self()) false {:send, msg} -> :gen_tcp.send(socket, msg) true end continue and recv_loop(pid, socket) end end
Step 1: receive new connections Step 2: accept and send
messages Step 3: do a bunch of Steps 1 and 2
Raspberry Pi #1 Raspberry Pi #2
None
None
“Does it scale?”
None
Time to copy off the internet find a creative solution
None
g
Gnutella
Gnutella
Gnutella
Gnutella
Gnutella
g (gnutella2)
Gnutella
G2/Gnutella2
G2/Gnutella2
G2/Gnutella2
G2/Gnutella2
None
None
None
None
None
HyParView
None
None
None
None
None
None
Plumtrees
Optimal number of messages
But you can’t afford to lose nodes
None
None
None
None
None
None
None
None
None
None
“Aha! It works on my computer!”
“Great but we need something to show”
“Great but we need something to show” (aka Raspberry Pi
time)
“Sure. Seems legit…” — @iampfac
“Wait. He works here!?” — @naps62
“Hey, I can borrow™ someone else’s code”
None
None
None
you shall not pass!
Solution: stick everything on Raspberry Pi’s
Things running on one Raspberry Pi
Things running on one Raspberry Pi ✓BEAM
Things running on one Raspberry Pi ✓BEAM ✓thebox (sensors)
Things running on one Raspberry Pi ✓BEAM ✓thebox (sensors) ✓Phoenix
app
Things running on one Raspberry Pi ✓BEAM (x2) ✓thebox (sensors)
✓Phoenix app
Things running on one Raspberry Pi ✓BEAM (x2) ✓thebox (sensors)
✓Phoenix app ✓Postgres
Things running on one Raspberry Pi ✓BEAM (x2) ✓thebox (sensors)
✓Phoenix app ✓Postgres ✓Cassandra
Things running on one Raspberry Pi ✓BEAM (x2) ✓thebox (sensors)
✓Phoenix app ✓Postgres ✓Cassandra it works!
None
None
@antenna = Satellite::Antenna.new(host, port) @antenna.on(“data") do |data| if data.avg_temp >
25 slack.send_msg(“people, it's too hot") end end @antenna.on("lights") do |payload, data| payload == "on" ? turn_lights_on : turn_lights_off end @antenna.watch
“Looking good! Everything’s working!”
None
State of each node:
State of each node: • Last sensor readings
State of each node: • Last sensor readings • Network
map (MAC-IP)
State of each node: • Last sensor readings • Network
map (MAC-IP) • Target values
State of each node: • Last sensor readings • Network
map (MAC-IP) • Target values
None
How do we handle concurrency?
None
None
No database locks. No transactions. You’re on your own, kiddo.
Vector Clocks
None
None
None
None
None
None
None
Shared state and why it sucks
Vector = (1, 0) Vector = (0, 1)
CAP Theorem
CAP Theorem “you’re a programmer. you can’t have nice things.”
consistency availability partitioning
consistency availability partitioning
Eventual Consistency
CRDTs
Operation-Based CRDT
Operation-Based CRDT commutative but not idempotent update exactly once
no CRDTs
no CRDTs
no CRDTs
no CRDTs
Op-based CRDTs
Op-based CRDTs
Op-based CRDTs
Op-based CRDTs
State-Based CRDT
State-Based CRDT commutative and idempotent heavier on the network
State-based CRDTs
State-based CRDTs
State-based CRDTs
State-based CRDTs
None
None
None
None
None
None
None
Wrapping up
System resources matter
System resources matter your algorithms should account for them
There are models. Use them.
Distributed System Checklist
Distributed System Checklist •Is the number of processes known or
finite?
Distributed System Checklist •Is the number of processes known or
finite? •Is there a global notion of time?
Distributed System Checklist •Is the number of processes known or
finite? •Is there a global notion of time? •Is the network reliable?
Distributed System Checklist •Is the number of processes known or
finite? •Is there a global notion of time? •Is the network reliable? •Is there full connectivity?
Distributed System Checklist •Is the number of processes known or
finite? •Is there a global notion of time? •Is the network reliable? •Is there full connectivity? •What happens when a process crashes?
It really doesn’t change that much
CRDTs aren’t a golden hammer
Reinventing the wheel is stupid
None
Knee-Deep Into P2P A Tale of Fail @fribmendes