Build Redundant Gaming Network with Wireguard and BGP

In this talk, Date Huang will discuss the difficulty of gaming network with different firewall rule and NAT policy and the reason of choosing Wireguard and BGP. Use Wireguard for encrypted connection and BGP to build redundancy route to each player. Even a player loses the connection, others will use BGP route to access each others.

Transcript

1. Build Redundant Gaming Network with Wireguard and BGP Date Huang Edgecore Networks tjjh89017@hotmail.com
2. About me: Date Huang • Engineer, Edgecore Networks ◦ 2019 OpenInfra Day Taiwan Speaker ▪ Massive Bare-Metal Operating System Provisioning Improvement ◦ 2019 Hong Kong Open Source Conference Speaker ▪ De-centralized Bare-Metal Operating System Provisioning ◦ 2018 ISC High Performance Project Poster Demo ▪ The Design and Implementation of Bare Metal Cluster Deployment Using BitTorrent ◦ 2017 Open Source Summit North America co-Speaker ▪ Building Cloud Infra using cost-effective ARM Boards ◦ 2017 OpenStack Day Taiwan Speaker ▪ Combine Continuous Integration (CI) with OpenStack ◦ 2016 OpenStack Day Taiwan Invited Speaker ▪ OpenStack on ARM64
3. Outline • Facebook Leaf-Spine Fabric • Dynamic Routing Protocol - BGP • Wireguard VPN • Multi-player P2P gaming • NAT Firewall with PlayStation4 • Concept: Wireguard + BGP build redundant P2P gaming network
4. Traditional Network Topology http://ciscorouterswitch.over-blog.com/2018/04/cisco-s-data-center-architecture.html
5. IP Fabric (Leaf Spine Fabric) https://engineering.fb.com/production-engineering/introducing-data-center-fabric-the-next-generation-facebook-data-center-network/
6. https://engineering.fb.com/production-engineering/introducing-data-center-fabric-the-next-generation-facebook-data-center-network/
7. Spanning Tree? • One Route Active, Others Standby
8. BGP • BGP: Border Gateway Protocol • Dynamic Routing Protocol • Usually used in Internet routing path select http://ciscorouterswitch.over-blog.com/article-bgp-protocol-is-essential-in-your-ip-network-115059468.html
9. https://www.mushroomnetworks.com/blog/bgp-load-balancing-good-idea-unless-att/
10. Routing on Host (L2-less) • Treat Server as a Router Node in BGP Network • Server use BGP to know other routing • Redundancy and merge bandwidth via BGP ECMP • No L2 Broadcast, Multicast • E.g：LINE Japan L2-less Network, Project Calico https://www.slideshare.net/linecorp/ss-116867631
11. https://cumulusnetworks.com/customers/switch/
12. Wireguard • GPLv2 Open Source Tunneling Protocol • Based on Linux Kernel, Support Windows and macOS • IPv4-in-IPv6 and IPv6-in-IPv4 encapsulation • Usually used in Site-to-Site Tunneling • Better Performance than IPsec, No need hardware acceleration • More Flexibility, Less Configuration, compare with GRE and IPsec • No need setup Firewall to allow additional protocol(e.g. Allow GRE, AH, ESP) • New Tech, No Mature, No Stable https://www.wireguard.com/
14. Multi-player Gaming • Central Server ◦ Connect to central server directly ◦ High Success rate ◦ Large Scale Multi-player gaming • P2P Connection ◦ Players connect to each others ◦ Low Success rate, according to players’ network quality ◦ Small Scale Multi-player gaming
15. Firewall and NAT Type using PlayStation • Type 1: System connects to Internet directly • Type 2: System connects to Internet via Router with NAT • Type 3: System connects to Internet via Router with NAT. External system cannot connect to internal system directly. https://portforward.com/nat-types/
16. Firewall and NAT Type using PlayStation • Type 1: System connects to Internet directly • Able to connect to Type 1, 2, 3. • No Limitaion https://portforward.com/nat-types/
17. Firewall and NAT Type using PlayStation • Type 2: System connects to Internet via Router with NAT • Able to connect to Type 1, 2 • Usually need DMZ, UPnP, Port Forwarding or UDP Hole Punching to allow external system to connect to internal https://portforward.com/nat-types/
18. Firewall and NAT Type using PlayStation • Type 3: System connects to Internet via Router with NAT. External system cannot connect to internal system directly. • Only able to connect with Type 1 https://portforward.com/nat-types/
19. UDP Hole Punching • UDP Hole Punching • Let two clients exchange connection info and try to connect to each other via 3rd party server • STUN: Session Traversal Utilities for NAT • STUN is common implement for UDP Hole Punching (RFC 5389) https://bford.info/pub/net/p2pnat/
20. https://bford.info/pub/net/p2pnat/
21. UDP Hole Punching FAIL! • If Hole Punching FAIL, need use other method to connect each other • Forward two clients info to each other via TURN Server or other nodes in P2P networks • For example: PS4 Party Chat. If it’s not able to connect, 1 of players will forward the network traffic or voice to others https://forum.gamer.com.tw/C.php?bsn=5786&snA=135419
22. TURN? or not • In design of TURN server, only create 1 forwarding connection. If TURN server is offline, forwarding connection will be disable, need to re-create forwarding connection to other TURN server • No Redundancy, No Fast Recovery
23. Wireguard + BGP • Use Wireguard to connect with each others • Let Wireguard connections be Full Mesh Topology via STUN as much as possible • If no Full Mesh, connect to server, server will help to forward connection • BGP will create dynamic routing, and choose the nearest routing • Create Redundancy Gaming Network via BGP https://www.talari.com/glossary_faq/sd-wan-full-mesh/
24. Tunneling + Dynamic Routing Protocol • Tunneling ◦ GRE ◦ VxLAN ◦ IPsec ◦ OpenVPN • Dynamic Routing Protocol ◦ RIP ◦ OSPF
25. Demo • R1 10.0.0.1, 192.168.0.1 • R2 10.0.0.2, 192.168.0.2 • R3 10.0.0.3, 192.168.0.3 • R4 10.0.0.4, 192.168.0.4 • While R2 reboot, R1 still can connect to R4 via R3 R1 R2 R3 R4