Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Knee-Deep Into P2P: A Tale of Fail (non-Elixir)
Search
Fernando Mendes
March 07, 2018
Programming
0
170
Knee-Deep Into P2P: A Tale of Fail (non-Elixir)
The slides for my talk at Lambda Days 2018 (
www.lambdadays.org/lambdadays2018
)
Fernando Mendes
March 07, 2018
Tweet
Share
More Decks by Fernando Mendes
See All by Fernando Mendes
you. and the morals of technology
fribmendes
1
130
Knee-Deep Into P2P: A Tale of Fail (PWL Porto)
fribmendes
0
58
Knee-Deep Into P2P: A Tale of Fail (ElixirConf EU 2018 version)
fribmendes
0
150
A Look Into Bloom Filters
fribmendes
0
400
Bloom Filters: A Look Into Ruby
fribmendes
0
110
Programming WTF: HTML & CSS
fribmendes
4
150
Ruby: A (pointless) Workshop
fribmendes
1
160
Elixir: A Talk For College Students
fribmendes
0
160
Riding Rails
fribmendes
0
100
Other Decks in Programming
See All in Programming
チームで開発し事業を加速するための"良い"設計の考え方 @ サポーターズCoLab 2025-07-08
agatan
1
430
Deep Dive into ~/.claude/projects
hiragram
14
2.6k
Agentic Coding: The Future of Software Development with Agents
mitsuhiko
0
110
なんとなくわかった気になるブロックテーマ入門/contents.nagoya 2025 6.28
chiilog
1
270
おやつのお供はお決まりですか?@WWDC25 Recap -Japan-\(region).swift
shingangan
0
140
20250628_非エンジニアがバイブコーディングしてみた
ponponmikankan
0
700
PipeCDのプラグイン化で目指すところ
warashi
1
280
なぜ「共通化」を考え、失敗を繰り返すのか
rinchoku
1
650
ニーリーにおけるプロダクトエンジニア
nealle
0
860
VS Code Update for GitHub Copilot
74th
2
650
ruby.wasmで多人数リアルタイム通信ゲームを作ろう
lnit
3
490
Google Agent Development Kit でLINE Botを作ってみた
ymd65536
2
250
Featured
See All Featured
Java REST API Framework Comparison - PWX 2021
mraible
31
8.7k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
46
9.6k
Optimizing for Happiness
mojombo
379
70k
The Cult of Friendly URLs
andyhume
79
6.5k
Learning to Love Humans: Emotional Interface Design
aarron
273
40k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
50
5.5k
Facilitating Awesome Meetings
lara
54
6.4k
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
29
2.7k
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
marcduiker
30
2.1k
What’s in a name? Adding method to the madness
productmarketing
PRO
23
3.5k
Testing 201, or: Great Expectations
jmmastey
43
7.6k
Building Better People: How to give real-time feedback that sticks.
wjessup
367
19k
Transcript
Knee-Deep Into P2P A Tale of Fail @fribmendes
Knee-Deep Into P2P A Tale of Fail @fribmendes
“This is SOOO boring…” - me, circa last year or
smth
None
None
None
“This is SOOO boring…” - me, 10 months ago
None
None
None
None
None
“ ” - me, whenever I get to play with
distributed systems
None
I don’t know how to smart office
I know how to web development
I know how to web development … what now?
None
Step 1: receive new connections
Step 1: receive new connections Step 2: accept and send
messages
Step 1: receive new connections Step 2: accept and send
messages Step 3: do a bunch of Steps 1 and 2
Step 1: receive new connections
None
defmodule Gossip.Server do def listen(pid, port) do {:ok, server_socket} =
:gen_tcp.listen(port, @socket_opts) accept_loop(pid, server_socket) end defp accept_loop(pid, server_socket) do {:ok, client} = :gen_tcp.accept(server_socket) :inet.setopts(client, [active: true]) :gen_tcp.controlling_process(client, pid) Gossip.accept(pid, client) accept_loop(pid, server_socket) end end
defmodule Gossip.Server do def listen(pid, port) do {:ok, server_socket} =
:gen_tcp.listen(port, @socket_opts) accept_loop(pid, server_socket) end defp accept_loop(pid, server_socket) do {:ok, client} = :gen_tcp.accept(server_socket) :inet.setopts(client, [active: true]) :gen_tcp.controlling_process(client, pid) Gossip.accept(pid, client) accept_loop(pid, server_socket) end end
Step 1: receive new connections Step 2: accept and send
messages
None
defmodule Gossip.Worker do def recv_loop(pid, socket) do continue = receive
do {:tcp, _port, msg} -> Gossip.recv(pid, msg) true {:tcp_closed, port} -> :gen_tcp.close(port) Gossip.disconnect(pid, self()) false {:send, msg} -> :gen_tcp.send(socket, msg) true end continue and recv_loop(pid, socket) end end
defmodule Gossip.Worker do def recv_loop(pid, socket) do continue = receive
do {:tcp, _port, msg} -> Gossip.recv(pid, msg) true {:tcp_closed, port} -> :gen_tcp.close(port) Gossip.disconnect(pid, self()) false {:send, msg} -> :gen_tcp.send(socket, msg) true end continue and recv_loop(pid, socket) end end
Step 1: receive new connections Step 2: accept and send
messages Step 3: do a bunch of Steps 1 and 2
Raspberry Pi #1 Raspberry Pi #2
None
None
“Does it scale?”
None
Time to copy off the internet find a creative solution
None
g
Gnutella
Gnutella
Gnutella
Gnutella
Gnutella
g (gnutella2)
Gnutella
G2/Gnutella2
G2/Gnutella2
G2/Gnutella2
G2/Gnutella2
None
None
None
None
None
HyParView
None
None
None
None
None
None
Plumtrees
Optimal number of messages
But you can’t afford to lose nodes
None
None
None
None
None
None
None
None
None
None
“Aha! It works on my computer!”
“Great but we need something to show”
“Great but we need something to show” (aka Raspberry Pi
time)
“Sure. Seems legit…” — @iampfac
“Wait. He works here!?” — @naps62
“Hey, I can borrow™ someone else’s code”
None
None
None
you shall not pass!
Solution: stick everything on Raspberry Pi’s
Things running on one Raspberry Pi
Things running on one Raspberry Pi ✓BEAM
Things running on one Raspberry Pi ✓BEAM ✓thebox (sensors)
Things running on one Raspberry Pi ✓BEAM ✓thebox (sensors) ✓Phoenix
app
Things running on one Raspberry Pi ✓BEAM (x2) ✓thebox (sensors)
✓Phoenix app
Things running on one Raspberry Pi ✓BEAM (x2) ✓thebox (sensors)
✓Phoenix app ✓Postgres
Things running on one Raspberry Pi ✓BEAM (x2) ✓thebox (sensors)
✓Phoenix app ✓Postgres ✓Cassandra
Things running on one Raspberry Pi ✓BEAM (x2) ✓thebox (sensors)
✓Phoenix app ✓Postgres ✓Cassandra it works!
None
None
@antenna = Satellite::Antenna.new(host, port) @antenna.on(“data") do |data| if data.avg_temp >
25 slack.send_msg(“people, it's too hot") end end @antenna.on("lights") do |payload, data| payload == "on" ? turn_lights_on : turn_lights_off end @antenna.watch
“Looking good! Everything’s working!”
None
State of each node:
State of each node: • Last sensor readings
State of each node: • Last sensor readings • Network
map (MAC-IP)
State of each node: • Last sensor readings • Network
map (MAC-IP) • Target values
State of each node: • Last sensor readings • Network
map (MAC-IP) • Target values
None
How do we handle concurrency?
None
None
No database locks. No transactions. You’re on your own, kiddo.
Vector Clocks
None
None
None
None
None
None
None
Shared state and why it sucks
Vector = (1, 0) Vector = (0, 1)
CAP Theorem
CAP Theorem “you’re a programmer. you can’t have nice things.”
consistency availability partitioning
consistency availability partitioning
Eventual Consistency
CRDTs
Operation-Based CRDT
Operation-Based CRDT commutative but not idempotent update exactly once
no CRDTs
no CRDTs
no CRDTs
no CRDTs
Op-based CRDTs
Op-based CRDTs
Op-based CRDTs
Op-based CRDTs
State-Based CRDT
State-Based CRDT commutative and idempotent heavier on the network
State-based CRDTs
State-based CRDTs
State-based CRDTs
State-based CRDTs
None
None
None
None
None
None
None
Wrapping up
System resources matter
System resources matter your algorithms should account for them
There are models. Use them.
Distributed System Checklist
Distributed System Checklist •Is the number of processes known or
finite?
Distributed System Checklist •Is the number of processes known or
finite? •Is there a global notion of time?
Distributed System Checklist •Is the number of processes known or
finite? •Is there a global notion of time? •Is the network reliable?
Distributed System Checklist •Is the number of processes known or
finite? •Is there a global notion of time? •Is the network reliable? •Is there full connectivity?
Distributed System Checklist •Is the number of processes known or
finite? •Is there a global notion of time? •Is the network reliable? •Is there full connectivity? •What happens when a process crashes?
It really doesn’t change that much
CRDTs aren’t a golden hammer
Reinventing the wheel is stupid
None
Knee-Deep Into P2P A Tale of Fail @fribmendes