Build Redundant Gaming Network with Wireguard and BGP Date Huang Edgecore Networks [email protected]

About me: Date Huang ● Engineer, Edgecore Networks ○ 2019 OpenInfra Day Taiwan Speaker ■ Massive Bare-Metal Operating System Provisioning Improvement ○ 2019 Hong Kong Open Source Conference Speaker ■ De-centralized Bare-Metal Operating System Provisioning ○ 2018 ISC High Performance Project Poster Demo ■ The Design and Implementation of Bare Metal Cluster Deployment Using BitTorrent ○ 2017 Open Source Summit North America co-Speaker ■ Building Cloud Infra using cost-effective ARM Boards ○ 2017 OpenStack Day Taiwan Speaker ■ Combine Continuous Integration (CI) with OpenStack ○ 2016 OpenStack Day Taiwan Invited Speaker ■ OpenStack on ARM64

Outline ● Facebook Leaf-Spine Fabric ● Dynamic Routing Protocol - BGP ● Wireguard VPN ● Multi-player P2P gaming ● NAT Firewall with PlayStation4 ● Concept: Wireguard + BGP build redundant P2P gaming network

Traditional Network Topology http://ciscorouterswitch.over-blog.com/2018/04/cisco-s-data-center-architecture.html

IP Fabric (Leaf Spine Fabric) https://engineering.fb.com/production-engineering/introducing-data-center-fabric-the-next-generation-facebook-data-center-network/

https://engineering.fb.com/production-engineering/introducing-data-center-fabric-the-next-generation-facebook-data-center-network/

Spanning Tree? ● One Route Active, Others Standby

BGP ● BGP: Border Gateway Protocol ● Dynamic Routing Protocol ● Usually used in Internet routing path select http://ciscorouterswitch.over-blog.com/article-bgp-protocol-is-essential-in-your-ip-network-115059468.html

https://www.mushroomnetworks.com/blog/bgp-load-balancing-good-idea-unless-att/

Routing on Host (L2-less) ● Treat Server as a Router Node in BGP Network ● Server use BGP to know other routing ● Redundancy and merge bandwidth via BGP ECMP ● No L2 Broadcast, Multicast ● E.g：LINE Japan L2-less Network, Project Calico https://www.slideshare.net/linecorp/ss-116867631

https://cumulusnetworks.com/customers/switch/

Wireguard ● GPLv2 Open Source Tunneling Protocol ● Based on Linux Kernel, Support Windows and macOS ● IPv4-in-IPv6 and IPv6-in-IPv4 encapsulation ● Usually used in Site-to-Site Tunneling ● Better Performance than IPsec, No need hardware acceleration ● More Flexibility, Less Configuration, compare with GRE and IPsec ● No need setup Firewall to allow additional protocol(e.g. Allow GRE, AH, ESP) ● New Tech, No Mature, No Stable https://www.wireguard.com/

No content

Multi-player Gaming ● Central Server ○ Connect to central server directly ○ High Success rate ○ Large Scale Multi-player gaming ● P2P Connection ○ Players connect to each others ○ Low Success rate, according to players’ network quality ○ Small Scale Multi-player gaming

Firewall and NAT Type using PlayStation ● Type 1: System connects to Internet directly ● Type 2: System connects to Internet via Router with NAT ● Type 3: System connects to Internet via Router with NAT. External system cannot connect to internal system directly. https://portforward.com/nat-types/

Firewall and NAT Type using PlayStation ● Type 1: System connects to Internet directly ● Able to connect to Type 1, 2, 3. ● No Limitaion https://portforward.com/nat-types/

Firewall and NAT Type using PlayStation ● Type 2: System connects to Internet via Router with NAT ● Able to connect to Type 1, 2 ● Usually need DMZ, UPnP, Port Forwarding or UDP Hole Punching to allow external system to connect to internal https://portforward.com/nat-types/

Firewall and NAT Type using PlayStation ● Type 3: System connects to Internet via Router with NAT. External system cannot connect to internal system directly. ● Only able to connect with Type 1 https://portforward.com/nat-types/

UDP Hole Punching ● UDP Hole Punching ● Let two clients exchange connection info and try to connect to each other via 3rd party server ● STUN: Session Traversal Utilities for NAT ● STUN is common implement for UDP Hole Punching (RFC 5389) https://bford.info/pub/net/p2pnat/

https://bford.info/pub/net/p2pnat/

UDP Hole Punching FAIL! ● If Hole Punching FAIL, need use other method to connect each other ● Forward two clients info to each other via TURN Server or other nodes in P2P networks ● For example: PS4 Party Chat. If it’s not able to connect, 1 of players will forward the network traffic or voice to others https://forum.gamer.com.tw/C.php?bsn=5786&snA=135419

TURN? or not ● In design of TURN server, only create 1 forwarding connection. If TURN server is offline, forwarding connection will be disable, need to re-create forwarding connection to other TURN server ● No Redundancy, No Fast Recovery

Wireguard + BGP ● Use Wireguard to connect with each others ● Let Wireguard connections be Full Mesh Topology via STUN as much as possible ● If no Full Mesh, connect to server, server will help to forward connection ● BGP will create dynamic routing, and choose the nearest routing ● Create Redundancy Gaming Network via BGP https://www.talari.com/glossary_faq/sd-wan-full-mesh/

Tunneling + Dynamic Routing Protocol ● Tunneling ○ GRE ○ VxLAN ○ IPsec ○ OpenVPN ● Dynamic Routing Protocol ○ RIP ○ OSPF

Demo ● R1 10.0.0.1, 192.168.0.1 ● R2 10.0.0.2, 192.168.0.2 ● R3 10.0.0.3, 192.168.0.3 ● R4 10.0.0.4, 192.168.0.4 ● While R2 reboot, R1 still can connect to R4 via R3 R1 R2 R3 R4