Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Scaling connections in P2P apps

Scaling connections in P2P apps

This presentation discusses various protocols that can be used to establish and propagate messages in huge clusters of nodes (>1000), focused on context of Peer to Peer systems. It covers the differences between P2P and client-server model, challenges related to local and global networks operating in P2P fashion.

Bartosz Sypytkowski

November 10, 2022
Tweet

More Decks by Bartosz Sypytkowski

Other Decks in Programming

Transcript

  1. — P2P vs. Client/Server — Networking 101 — Membership &

    peer discovery — Gossiping data efficiently AGENDA
  2. CLIENT / SERVER PEER TO PEER Local network Local network

    “The cloud” Local network Local network
  3. CLIENT / SERVER PEER TO PEER • Roles: initiator or

    acceptor • Cluster of predictable size • DC network: solid, fast • Servers: powerful hardware, always on • Dedicated roles (DB, app server, cache) • Roles: both initiator and acceptor • Unbounded number of members • Variadic, unpredictable network • Peers: various hardware, may require power management • No preconfigured roles
  4. ▪ Data ownership ▪ Always available ▪ Cost reduction ▪

    “Simpler” programming model LOCAL-FIRST SOFTWARE
  5. 1. P2P systems 2. NOT IoT 3. Hundreds/thousands users 4.

    Focused on message exchange OUR USE CASES
  6. CLIENT / SERVER PEER TO PEER Local network Local network

    “The cloud” Local network Local neatwork How two devices can discover each other?
  7. NETWORK ADDRESS TRANSLATION Internet Router Server 172.23.208.1 200.100.10.1 Source: 84.11.65.1

    Destination: 200.100.10.1 NAT table 172.23.208.1 84.11.65.1 Client
  8. NETWORK ADDRESS TRANSLATION Internet Router Server 172.23.208.1 200.100.10.1 Source: 84.11.65.1

    Destination: 200.100.10.1 NAT table 172.23.208.1 84.11.65.1 Possible further source IP changes Client
  9. NETWORK 101 DNS 172.23.208.1 Server 200.100.10.1 DNS Server DNS records

    example.com 200.100.10.1 Client W hat’s the IP of exam ple.com ? 200.100.10.1
  10. STUN Router STUN Server 172.23.208.1 stun.l.google.com Client DISCOVERING PUBLIC IP

    What is my public IP? Source: 172.23.208.1 Destination: 200.100.10.1
  11. STUN Router STUN Server 172.23.208.1 stun.l.google.com Client DISCOVERING PUBLIC IP

    What is my public IP? Source: 84.11.65.1 Destination: 200.100.10.1 NAT table 172.23.208.1 84.11.65.1
  12. STUN Router STUN Server 172.23.208.1 stun.l.google.com Client DISCOVERING PUBLIC IP

    Your public IP is: 84.11.65.1 NAT table 172.23.208.1 84.11.65.1
  13. STUN Router STUN Server 172.23.208.1 stun.l.google.com Client DISCOVERING PUBLIC IP

    Your public IP is: 84.11.65.1 NAT table 172.23.208.1 84.11.65.1 Periodically send ping to keep the NAT table mapping unchanged.
  14. ICE NEGOTIATE WHICH CONNECTION TO USE const conn = new

    RTCPeerConnection({ iceServers: [ { urls: "stun:stunserver.example.com:3478", }, { urls: 'turn:turnserver.com:3478', credential: 'password', username: 'username' } ] })
  15. CLIENT / SERVER PEER TO PEER Local network Local network

    “The cloud” Local network Local neatwork What about that part?
  16. mDNS svc-1.local 192.168.0.1 svc-3.local 192.168.0.3 svc-5.local 192.168.0.5 svc-2.local 192.168.0.2 svc-4.local

    192.168.0.4 Where is svc-4.local? Where is svc-4.local? Where is svc-4.local? Where is svc-4.local? multicast
  17. mDNS svc-1.local 192.168.0.1 svc-3.local 192.168.0.3 svc-5.local 192.168.0.5 svc-2.local 192.168.0.2 svc-4.local

    192.168.0.4 svc-4.local = 192.168.04 svc-4.local = 192.168.04 svc-4.local = 192.168.04 svc-4.local = 192.168.04 multicast
  18. mDNS svc-1.local 192.168.0.1 svc-4.local 192.168.0.4 svc-3.local 192.168.0.3 svc-4.local 192.168.0.4 svc-5.local

    192.168.0.5 svc-4.local 192.168.0.4 svc-2.local 192.168.0.2 svc-4.local 192.168.0.4 svc-4.local 192.168.0.4 svc-4.local = 192.168.0.4
  19. mDNS const mdns = require('mdns') // advertise service svc-1 at

    port 9999 via TCP const service = mdns.createAdvertisement(mdns.tcp(), 9999, { name: 'svc-1' }) service.start() // discover services const browser = mdns.createBrowser(mdns.tcp()) browser.on('ready', () => browser.discover()) browser.on('update', (data) => { console.log(data); // { // interfaceIndex: 4, // name: svc-1', // networkInterface: 'en0', // type: {name: '', protocol: 'tcp', subtypes: []}, // replyDomain: 'local.', // fullname: 'svc-1._tcp.local.', // host: 'svc-1.local.', // port: 9999, // addresses: [ '10.1.1.50', 'fe80::21f:5bff:fecd:ce64' ] // } })
  20. HOW TO BUILD A CLUSTER THAT COULD SPAN OVER >1K

    NODES USING DIFFERENT NETWORKS?
  21. NAÏVE CONNECTIVITY ISSUES F A E B C D Connections

    limit: 4 Sorry, I’m at my limit.
  22. PRIORITY CONNECTIONS F A E B C D Connections limit:

    4 drop existing connection at random to free the pool
  23. F A E B C D Connections limit: 4 FWD(2)

    FWD(2) FWD(2) FORWARD NEW PEER INFOR TO OTHERS
  24. F A E B C D Connections limit: 4 FWD(1)

    FWD(1) FWD(1) FWD(1) FWD(1) FWD(1) FORWARD NEW PEER INFOR TO OTHERS
  25. NON PRIORITY CONNECTIONS F A E B C D Connections

    limit: 4 Sorry, I’m at my limit.
  26. HYPARVIEW 101 A C B D passive view active view

    K J I H G F E L M N Active peers Passive peers
  27. BUILDING A TREE A C B D K J I

    H G F E L M N NACK NACK
  28. BUILDING A TREE A C B D K J I

    H G F E L M N NACK NACK
  29. BUILDING A TREE A C B D K J I

    H G F E L M N NACK
  30. BUILDING A TREE A C B D K J I

    H G F E L M N Eager peers Lazy peers
  31. TREE REPAIR A C B D K J I H

    G F E L M N connection failure
  32. TREE REPAIR A C B D K J I H

    G F E L M N Part of the gossip tree is disconnected from the rest
  33. TREE REPAIR A C B D K J I H

    G F E L M N Once in a while send message to lazy peers about latest gossips ids (m1 ) IHAVE(m1 )
  34. TREE REPAIR A C B D K J I H

    G F E L M N Message receiver awaits for m1
  35. TREE REPAIR A C B D K J I H

    G F E L M N If m1 didn’t arrive before timeout, send graft back GRAFT(m1 )
  36. TREE REPAIR A C B D K J I H

    G F E L M N Promote lazy peer to eager one
  37. — Rainbow connections: https://www.youtube.com/watch?v=8_A1CkYfzoM — Hyparview: https://bartoszsypytkowski.com/hyparview/ — Conflict-Free Replicated

    Data Types: https://bartoszsypytkowski.com/tag/crdt/ — Partisan: https://github.com/lasp-lang/partisan — DotNext (Hyparview): https://github.com/dotnet/dotNext/ LINKS