Tweet
Tweet

Slide 1

Slide 1 text

SCALING CONNECTIONS IN P2P APPS

Slide 2

Slide 2 text

INTRODUCTION Bartosz Sypytkowski @Horusiath [email protected] bartoszsypytkowski.com

Slide 3

Slide 3 text

— P2P vs. Client/Server — Networking 101 — Membership & peer discovery — Gossiping data efficiently AGENDA

Slide 4

Slide 4 text

CLIENT / SERVER PEER TO PEER Local network Local network “The cloud” Local network Local network

Slide 5

Slide 5 text

CLIENT / SERVER PEER TO PEER • Roles: initiator or acceptor • Cluster of predictable size • DC network: solid, fast • Servers: powerful hardware, always on • Dedicated roles (DB, app server, cache) • Roles: both initiator and acceptor • Unbounded number of members • Variadic, unpredictable network • Peers: various hardware, may require power management • No preconfigured roles

Slide 6

Slide 6 text

▪ Data ownership ▪ Always available ▪ Cost reduction ▪ “Simpler” programming model LOCAL-FIRST SOFTWARE

Slide 7

Slide 7 text

1. P2P systems 2. NOT IoT 3. Hundreds/thousands users 4. Focused on message exchange OUR USE CASES

Slide 8

Slide 8 text

CLIENT / SERVER PEER TO PEER Local network Local network “The cloud” Local network Local neatwork How two devices can discover each other?

Slide 9

Slide 9 text

NETWORK ADDRESS TRANSLATION Internet Router Server 172.23.208.1 200.100.10.1 Client

Slide 10

Slide 10 text

NETWORK ADDRESS TRANSLATION Internet Router Server 172.23.208.1 200.100.10.1 Source: 172.23.208.1 Destination: 200.100.10.1 Client

Slide 11

Slide 11 text

NETWORK ADDRESS TRANSLATION Internet Router Server 172.23.208.1 200.100.10.1 Source: 172.23.208.1 Destination: 200.100.10.1 Client

Slide 12

Slide 12 text

NETWORK ADDRESS TRANSLATION Internet Router Server 172.23.208.1 200.100.10.1 Source: 84.11.65.1 Destination: 200.100.10.1 NAT table 172.23.208.1 84.11.65.1 Client

Slide 13

Slide 13 text

NETWORK ADDRESS TRANSLATION Internet Router Server 172.23.208.1 200.100.10.1 Source: 84.11.65.1 Destination: 200.100.10.1 NAT table 172.23.208.1 84.11.65.1 Possible further source IP changes Client

Slide 14

Slide 14 text

HOW DO WE KNOW THE IP ADDRESS OF ACCEPTOR?

Slide 15

Slide 15 text

NETWORK 101 DNS 172.23.208.1 Server 200.100.10.1 DNS Server DNS records example.com 200.100.10.1 Client W hat’s the IP of exam ple.com ? 200.100.10.1

Slide 16

Slide 16 text

NETWORK 101 DNS 172.23.208.1 Server 200.100.10.1 DNS Server DNS records example.com 200.100.10.1 Client

Slide 17

Slide 17 text

WHY CAN’T WE SETUP A DNS RECORD FOR EVERY PEER?

Slide 18

Slide 18 text

STUN Router STUN Server 172.23.208.1 stun.l.google.com Client DISCOVERING PUBLIC IP

Slide 19

Slide 19 text

STUN Router STUN Server 172.23.208.1 stun.l.google.com Client DISCOVERING PUBLIC IP What is my public IP? Source: 172.23.208.1 Destination: 200.100.10.1

Slide 20

Slide 20 text

STUN Router STUN Server 172.23.208.1 stun.l.google.com Client DISCOVERING PUBLIC IP What is my public IP? Source: 84.11.65.1 Destination: 200.100.10.1 NAT table 172.23.208.1 84.11.65.1

Slide 21

Slide 21 text

STUN Router STUN Server 172.23.208.1 stun.l.google.com Client DISCOVERING PUBLIC IP Your public IP is: 84.11.65.1 NAT table 172.23.208.1 84.11.65.1

Slide 22

Slide 22 text

STUN Router STUN Server 172.23.208.1 stun.l.google.com Client DISCOVERING PUBLIC IP Your public IP is: 84.11.65.1 NAT table 172.23.208.1 84.11.65.1 Periodically send ping to keep the NAT table mapping unchanged.

Slide 23

Slide 23 text

NOT ALL FIREWALLS ENABLE DIRECT DEVICE-DEVICE CONNECTION

Slide 24

Slide 24 text

TURN Firewall 84.16.55.1 Client RELAYING MESSAGES OVER NATS/FIREWALLS TURN Server 54.23.201.1 Client

Slide 25

Slide 25 text

ICE NEGOTIATE THE CONNECTION CAPABILITIES

Slide 26

Slide 26 text

ICE NEGOTIATE WHICH CONNECTION TO USE const conn = new RTCPeerConnection({ iceServers: [ { urls: "stun:stunserver.example.com:3478", }, { urls: 'turn:turnserver.com:3478', credential: 'password', username: 'username' } ] })

Slide 27

Slide 27 text

CLIENT / SERVER PEER TO PEER Local network Local network “The cloud” Local network Local neatwork What about that part?

Slide 28

Slide 28 text

mDNS svc-1.local 192.168.0.1 svc-3.local 192.168.0.3 svc-5.local 192.168.0.5 svc-2.local 192.168.0.2 svc-4.local 192.168.0.4

Slide 29

Slide 29 text

mDNS svc-1.local 192.168.0.1 svc-3.local 192.168.0.3 svc-5.local 192.168.0.5 svc-2.local 192.168.0.2 svc-4.local 192.168.0.4 Where is svc-4.local?

Slide 30

Slide 30 text

mDNS svc-1.local 192.168.0.1 svc-3.local 192.168.0.3 svc-5.local 192.168.0.5 svc-2.local 192.168.0.2 svc-4.local 192.168.0.4 Where is svc-4.local? Where is svc-4.local? Where is svc-4.local? Where is svc-4.local? multicast

Slide 31

Slide 31 text

mDNS svc-1.local 192.168.0.1 svc-3.local 192.168.0.3 svc-5.local 192.168.0.5 svc-2.local 192.168.0.2 svc-4.local 192.168.0.4 svc-4.local = 192.168.0.4

Slide 32

Slide 32 text

mDNS svc-1.local 192.168.0.1 svc-3.local 192.168.0.3 svc-5.local 192.168.0.5 svc-2.local 192.168.0.2 svc-4.local 192.168.0.4 svc-4.local = 192.168.04 svc-4.local = 192.168.04 svc-4.local = 192.168.04 svc-4.local = 192.168.04 multicast

Slide 33

Slide 33 text

mDNS svc-1.local 192.168.0.1 svc-4.local 192.168.0.4 svc-3.local 192.168.0.3 svc-4.local 192.168.0.4 svc-5.local 192.168.0.5 svc-4.local 192.168.0.4 svc-2.local 192.168.0.2 svc-4.local 192.168.0.4 svc-4.local 192.168.0.4 svc-4.local = 192.168.0.4

Slide 34

Slide 34 text

mDNS const mdns = require('mdns') // advertise service svc-1 at port 9999 via TCP const service = mdns.createAdvertisement(mdns.tcp(), 9999, { name: 'svc-1' }) service.start() // discover services const browser = mdns.createBrowser(mdns.tcp()) browser.on('ready', () => browser.discover()) browser.on('update', (data) => { console.log(data); // { // interfaceIndex: 4, // name: svc-1', // networkInterface: 'en0', // type: {name: '', protocol: 'tcp', subtypes: []}, // replyDomain: 'local.', // fullname: 'svc-1._tcp.local.', // host: 'svc-1.local.', // port: 9999, // addresses: [ '10.1.1.50', 'fe80::21f:5bff:fecd:ce64' ] // } })

Slide 35

Slide 35 text

CLUSTERING

Slide 36

Slide 36 text

HOW TO BUILD A CLUSTER THAT COULD SPAN OVER >1K NODES USING DIFFERENT NETWORKS?

Slide 37

Slide 37 text

HYPARVIEW HYBRID PARTIAL VIEW FOR CLUSTER MEMBERSHIP

Slide 38

Slide 38 text

CONNECTING EVERYONE TO EACH OTHER DOESN’T SCALE

Slide 39

Slide 39 text

CONNECTING EVERYONE TO EACH OTHER DOESN’T SCALE BUT…

Slide 40

Slide 40 text

NAÏVE CONNECTIVITY ISSUES F A E B C D Connections limit: 4

Slide 41

Slide 41 text

NAÏVE CONNECTIVITY ISSUES F A E B C D Connections limit: 4 Can I join?

Slide 42

Slide 42 text

NAÏVE CONNECTIVITY ISSUES F A E B C D Connections limit: 4 Sorry, I’m at my limit.

Slide 43

Slide 43 text

SOLUTION: INTRODUCE PRIORITY CONNECTIONS

Slide 44

Slide 44 text

PRIORITY CONNECTIONS F A E B C D Connections limit: 4

Slide 45

Slide 45 text

PRIORITY CONNECTIONS F A E B C D Connections limit: 4 SUDO: let me join

Slide 46

Slide 46 text

PRIORITY CONNECTIONS F A E B C D Connections limit: 4 drop existing connection at random to free the pool

Slide 47

Slide 47 text

PRIORITY CONNECTIONS F A E B C D Connections limit: 4 establish new connection

Slide 48

Slide 48 text

F A E B C D Connections limit: 4 FWD(2) FWD(2) FWD(2) FORWARD NEW PEER INFOR TO OTHERS

Slide 49

Slide 49 text

F A E B C D Connections limit: 4 FWD(1) FWD(1) FWD(1) FWD(1) FWD(1) FWD(1) FORWARD NEW PEER INFOR TO OTHERS

Slide 50

Slide 50 text

NON PRIORITY CONNECTIONS F A E B C D Connections limit: 4 Can I join?

Slide 51

Slide 51 text

NON PRIORITY CONNECTIONS F A E B C D Connections limit: 4 Sorry, I’m at my limit.

Slide 52

Slide 52 text

NON PRIORITY CONNECTIONS F A E B C D Connections limit: 4

Slide 53

Slide 53 text

HYPARVIEW 101 A C B D passive view active view K J I H G F E L M N Active peers Passive peers

Slide 54

Slide 54 text

HOW TO GOSSIP MESSAGES EFFICIENTLY?

Slide 55

Slide 55 text

PLUMTREE EPIDEMIC BROADCAST TREES

Slide 56

Slide 56 text

PLUMTREE 101 A C B D K J I H G F E L M N

Slide 57

Slide 57 text

BUILDING A TREE A C B D K J I H G F E L M N

Slide 58

Slide 58 text

BUILDING A TREE A C B D K J I H G F E L M N

Slide 59

Slide 59 text

BUILDING A TREE A C B D K J I H G F E L M N

Slide 60

Slide 60 text

BUILDING A TREE A C B D K J I H G F E L M N NACK NACK

Slide 61

Slide 61 text

BUILDING A TREE A C B D K J I H G F E L M N

Slide 62

Slide 62 text

BUILDING A TREE A C B D K J I H G F E L M N

Slide 63

Slide 63 text

BUILDING A TREE A C B D K J I H G F E L M N

Slide 64

Slide 64 text

BUILDING A TREE A C B D K J I H G F E L M N NACK NACK

Slide 65

Slide 65 text

BUILDING A TREE A C B D K J I H G F E L M N

Slide 66

Slide 66 text

BUILDING A TREE A C B D K J I H G F E L M N

Slide 67

Slide 67 text

BUILDING A TREE A C B D K J I H G F E L M N NACK

Slide 68

Slide 68 text

BUILDING A TREE A C B D K J I H G F E L M N Eager peers Lazy peers

Slide 69

Slide 69 text

WHAT IF CONNECTION FAILS?

Slide 70

Slide 70 text

TREE REPAIR A C B D K J I H G F E L M N connection failure

Slide 71

Slide 71 text

TREE REPAIR A C B D K J I H G F E L M N Part of the gossip tree is disconnected from the rest

Slide 72

Slide 72 text

TREE REPAIR A C B D K J I H G F E L M N Once in a while send message to lazy peers about latest gossips ids (m1 ) IHAVE(m1 )

Slide 73

Slide 73 text

TREE REPAIR A C B D K J I H G F E L M N Message receiver awaits for m1

Slide 74

Slide 74 text

TREE REPAIR A C B D K J I H G F E L M N If m1 didn’t arrive before timeout, send graft back GRAFT(m1 )

Slide 75

Slide 75 text

TREE REPAIR A C B D K J I H G F E L M N Promote lazy peer to eager one

Slide 76

Slide 76 text

TREE REPAIR A C B D K J I H G F E L M N

Slide 77

Slide 77 text

TREE VS. MESH BROADCAST

Slide 78

Slide 78 text

SUMMARY

Slide 79

Slide 79 text

— Rainbow connections: https://www.youtube.com/watch?v=8_A1CkYfzoM — Hyparview: https://bartoszsypytkowski.com/hyparview/ — Conflict-Free Replicated Data Types: https://bartoszsypytkowski.com/tag/crdt/ — Partisan: https://github.com/lasp-lang/partisan — DotNext (Hyparview): https://github.com/dotnet/dotNext/ LINKS

Slide 80

Slide 80 text

THANK YOU