Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Knee-Deep Into P2P: A Tale of Fail (non-Elixir)
Search
Fernando Mendes
March 07, 2018
Programming
0
170
Knee-Deep Into P2P: A Tale of Fail (non-Elixir)
The slides for my talk at Lambda Days 2018 (
www.lambdadays.org/lambdadays2018
)
Fernando Mendes
March 07, 2018
Tweet
Share
More Decks by Fernando Mendes
See All by Fernando Mendes
you. and the morals of technology
fribmendes
1
130
Knee-Deep Into P2P: A Tale of Fail (PWL Porto)
fribmendes
0
60
Knee-Deep Into P2P: A Tale of Fail (ElixirConf EU 2018 version)
fribmendes
0
160
A Look Into Bloom Filters
fribmendes
0
420
Bloom Filters: A Look Into Ruby
fribmendes
0
110
Programming WTF: HTML & CSS
fribmendes
4
160
Ruby: A (pointless) Workshop
fribmendes
1
160
Elixir: A Talk For College Students
fribmendes
0
160
Riding Rails
fribmendes
0
110
Other Decks in Programming
See All in Programming
Devvox Belgium - Agentic AI Patterns
kdubois
1
120
Cloudflare AgentsとAI SDKでAIエージェントを作ってみた
briete
0
140
そのpreloadは必要?見過ごされたpreloadが技術的負債として爆発した日
mugitti9
2
3.3k
CSC509 Lecture 03
javiergs
PRO
0
330
Foundation Modelsを実装日本語学習アプリを作ってみた!
hypebeans
0
110
Building, Deploying, and Monitoring Ruby Web Applications with Falcon (Kaigi on Rails 2025)
ioquatix
4
2.1k
バッチ処理を「状態の記録」から「事実の記録」へ
panda728
PRO
0
150
TFLintカスタムプラグインで始める Terraformコード品質管理
bells17
2
160
私はどうやって技術力を上げたのか
yusukebe
43
18k
Introducing ReActionView: A new ActionView-Compatible ERB Engine @ Kaigi on Rails 2025, Tokyo, Japan
marcoroth
3
1k
Software Architecture
hschwentner
6
2.3k
Railsだからできる 例外業務に禍根を残さない 設定設計パターン
ei_ei_eiichi
0
690
Featured
See All Featured
The Straight Up "How To Draw Better" Workshop
denniskardys
238
140k
Stop Working from a Prison Cell
hatefulcrawdad
271
21k
The Cost Of JavaScript in 2023
addyosmani
54
9k
GraphQLとの向き合い方2022年版
quramy
49
14k
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
15
1.7k
Intergalactic Javascript Robots from Outer Space
tanoku
273
27k
What’s in a name? Adding method to the madness
productmarketing
PRO
23
3.7k
Connecting the Dots Between Site Speed, User Experience & Your Business [WebExpo 2025]
tammyeverts
9
590
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
marcduiker
32
2.3k
Fireside Chat
paigeccino
40
3.7k
Automating Front-end Workflow
addyosmani
1371
200k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
359
30k
Transcript
Knee-Deep Into P2P A Tale of Fail @fribmendes
Knee-Deep Into P2P A Tale of Fail @fribmendes
“This is SOOO boring…” - me, circa last year or
smth
None
None
None
“This is SOOO boring…” - me, 10 months ago
None
None
None
None
None
“ ” - me, whenever I get to play with
distributed systems
None
I don’t know how to smart office
I know how to web development
I know how to web development … what now?
None
Step 1: receive new connections
Step 1: receive new connections Step 2: accept and send
messages
Step 1: receive new connections Step 2: accept and send
messages Step 3: do a bunch of Steps 1 and 2
Step 1: receive new connections
None
defmodule Gossip.Server do def listen(pid, port) do {:ok, server_socket} =
:gen_tcp.listen(port, @socket_opts) accept_loop(pid, server_socket) end defp accept_loop(pid, server_socket) do {:ok, client} = :gen_tcp.accept(server_socket) :inet.setopts(client, [active: true]) :gen_tcp.controlling_process(client, pid) Gossip.accept(pid, client) accept_loop(pid, server_socket) end end
defmodule Gossip.Server do def listen(pid, port) do {:ok, server_socket} =
:gen_tcp.listen(port, @socket_opts) accept_loop(pid, server_socket) end defp accept_loop(pid, server_socket) do {:ok, client} = :gen_tcp.accept(server_socket) :inet.setopts(client, [active: true]) :gen_tcp.controlling_process(client, pid) Gossip.accept(pid, client) accept_loop(pid, server_socket) end end
Step 1: receive new connections Step 2: accept and send
messages
None
defmodule Gossip.Worker do def recv_loop(pid, socket) do continue = receive
do {:tcp, _port, msg} -> Gossip.recv(pid, msg) true {:tcp_closed, port} -> :gen_tcp.close(port) Gossip.disconnect(pid, self()) false {:send, msg} -> :gen_tcp.send(socket, msg) true end continue and recv_loop(pid, socket) end end
defmodule Gossip.Worker do def recv_loop(pid, socket) do continue = receive
do {:tcp, _port, msg} -> Gossip.recv(pid, msg) true {:tcp_closed, port} -> :gen_tcp.close(port) Gossip.disconnect(pid, self()) false {:send, msg} -> :gen_tcp.send(socket, msg) true end continue and recv_loop(pid, socket) end end
Step 1: receive new connections Step 2: accept and send
messages Step 3: do a bunch of Steps 1 and 2
Raspberry Pi #1 Raspberry Pi #2
None
None
“Does it scale?”
None
Time to copy off the internet find a creative solution
None
g
Gnutella
Gnutella
Gnutella
Gnutella
Gnutella
g (gnutella2)
Gnutella
G2/Gnutella2
G2/Gnutella2
G2/Gnutella2
G2/Gnutella2
None
None
None
None
None
HyParView
None
None
None
None
None
None
Plumtrees
Optimal number of messages
But you can’t afford to lose nodes
None
None
None
None
None
None
None
None
None
None
“Aha! It works on my computer!”
“Great but we need something to show”
“Great but we need something to show” (aka Raspberry Pi
time)
“Sure. Seems legit…” — @iampfac
“Wait. He works here!?” — @naps62
“Hey, I can borrow™ someone else’s code”
None
None
None
you shall not pass!
Solution: stick everything on Raspberry Pi’s
Things running on one Raspberry Pi
Things running on one Raspberry Pi ✓BEAM
Things running on one Raspberry Pi ✓BEAM ✓thebox (sensors)
Things running on one Raspberry Pi ✓BEAM ✓thebox (sensors) ✓Phoenix
app
Things running on one Raspberry Pi ✓BEAM (x2) ✓thebox (sensors)
✓Phoenix app
Things running on one Raspberry Pi ✓BEAM (x2) ✓thebox (sensors)
✓Phoenix app ✓Postgres
Things running on one Raspberry Pi ✓BEAM (x2) ✓thebox (sensors)
✓Phoenix app ✓Postgres ✓Cassandra
Things running on one Raspberry Pi ✓BEAM (x2) ✓thebox (sensors)
✓Phoenix app ✓Postgres ✓Cassandra it works!
None
None
@antenna = Satellite::Antenna.new(host, port) @antenna.on(“data") do |data| if data.avg_temp >
25 slack.send_msg(“people, it's too hot") end end @antenna.on("lights") do |payload, data| payload == "on" ? turn_lights_on : turn_lights_off end @antenna.watch
“Looking good! Everything’s working!”
None
State of each node:
State of each node: • Last sensor readings
State of each node: • Last sensor readings • Network
map (MAC-IP)
State of each node: • Last sensor readings • Network
map (MAC-IP) • Target values
State of each node: • Last sensor readings • Network
map (MAC-IP) • Target values
None
How do we handle concurrency?
None
None
No database locks. No transactions. You’re on your own, kiddo.
Vector Clocks
None
None
None
None
None
None
None
Shared state and why it sucks
Vector = (1, 0) Vector = (0, 1)
CAP Theorem
CAP Theorem “you’re a programmer. you can’t have nice things.”
consistency availability partitioning
consistency availability partitioning
Eventual Consistency
CRDTs
Operation-Based CRDT
Operation-Based CRDT commutative but not idempotent update exactly once
no CRDTs
no CRDTs
no CRDTs
no CRDTs
Op-based CRDTs
Op-based CRDTs
Op-based CRDTs
Op-based CRDTs
State-Based CRDT
State-Based CRDT commutative and idempotent heavier on the network
State-based CRDTs
State-based CRDTs
State-based CRDTs
State-based CRDTs
None
None
None
None
None
None
None
Wrapping up
System resources matter
System resources matter your algorithms should account for them
There are models. Use them.
Distributed System Checklist
Distributed System Checklist •Is the number of processes known or
finite?
Distributed System Checklist •Is the number of processes known or
finite? •Is there a global notion of time?
Distributed System Checklist •Is the number of processes known or
finite? •Is there a global notion of time? •Is the network reliable?
Distributed System Checklist •Is the number of processes known or
finite? •Is there a global notion of time? •Is the network reliable? •Is there full connectivity?
Distributed System Checklist •Is the number of processes known or
finite? •Is there a global notion of time? •Is the network reliable? •Is there full connectivity? •What happens when a process crashes?
It really doesn’t change that much
CRDTs aren’t a golden hammer
Reinventing the wheel is stupid
None
Knee-Deep Into P2P A Tale of Fail @fribmendes