Build Redundant Gaming Network with Wireguard and BGP

In this talk, Date Huang will discuss the difficulty of gaming network with different firewall rule and NAT policy and the reason of choosing Wireguard and BGP. Use Wireguard for encrypted connection and BGP to build redundancy route to each player. Even a player loses the connection, others will use BGP route to access each others.

Transcript

1. Build Redundant Gaming Network
   with Wireguard and BGP
   Date Huang
   Edgecore Networks
   tjjh89017@hotmail.com
   

   View Slide

2. About me: Date Huang
   ● Engineer, Edgecore Networks
   ○ 2019 OpenInfra Day Taiwan Speaker
   ■ Massive Bare-Metal Operating System Provisioning Improvement
   ○ 2019 Hong Kong Open Source Conference Speaker
   ■ De-centralized Bare-Metal Operating System Provisioning
   ○ 2018 ISC High Performance Project Poster Demo
   ■ The Design and Implementation of Bare Metal Cluster
   Deployment Using BitTorrent
   ○ 2017 Open Source Summit North America co-Speaker
   ■ Building Cloud Infra using cost-effective ARM Boards
   ○ 2017 OpenStack Day Taiwan Speaker
   ■ Combine Continuous Integration (CI) with OpenStack
   ○ 2016 OpenStack Day Taiwan Invited Speaker
   ■ OpenStack on ARM64
   

   View Slide

3. Outline
   ● Facebook Leaf-Spine Fabric
   ● Dynamic Routing Protocol - BGP
   ● Wireguard VPN
   ● Multi-player P2P gaming
   ● NAT Firewall with PlayStation4
   ● Concept: Wireguard + BGP build redundant P2P gaming network
   

   View Slide

4. Traditional Network Topology
   http://ciscorouterswitch.over-blog.com/2018/04/cisco-s-data-center-architecture.html
   

   View Slide

5. IP Fabric (Leaf Spine Fabric)
   https://engineering.fb.com/production-engineering/introducing-data-center-fabric-the-next-generation-facebook-data-center-network/
   

   View Slide

6. https://engineering.fb.com/production-engineering/introducing-data-center-fabric-the-next-generation-facebook-data-center-network/
   

   View Slide

7. Spanning Tree?
   ● One Route Active, Others Standby
   

   View Slide

8. BGP
   ● BGP: Border Gateway Protocol
   ● Dynamic Routing Protocol
   ● Usually used in Internet routing path select
   http://ciscorouterswitch.over-blog.com/article-bgp-protocol-is-essential-in-your-ip-network-115059468.html
   

   View Slide

9. https://www.mushroomnetworks.com/blog/bgp-load-balancing-good-idea-unless-att/
   

   View Slide

10. Routing on Host (L2-less)
    ● Treat Server as a Router Node in BGP
    Network
    ● Server use BGP to know other routing
    ● Redundancy and merge bandwidth via
    BGP ECMP
    ● No L2 Broadcast, Multicast
    ● E.g：LINE Japan L2-less Network,
    Project Calico
    https://www.slideshare.net/linecorp/ss-116867631
    

    View Slide

11. https://cumulusnetworks.com/customers/switch/
    

    View Slide

12. Wireguard
    ● GPLv2 Open Source Tunneling Protocol
    ● Based on Linux Kernel, Support Windows and macOS
    ● IPv4-in-IPv6 and IPv6-in-IPv4 encapsulation
    ● Usually used in Site-to-Site Tunneling
    ● Better Performance than IPsec, No need hardware acceleration
    ● More Flexibility, Less Configuration, compare with GRE and IPsec
    ● No need setup Firewall to allow additional protocol(e.g. Allow GRE, AH, ESP)
    ● New Tech, No Mature, No Stable
    https://www.wireguard.com/
    

    View Slide

13. View Slide

14. Multi-player Gaming
    ● Central Server
    ○ Connect to central server directly
    ○ High Success rate
    ○ Large Scale Multi-player gaming
    ● P2P Connection
    ○ Players connect to each others
    ○ Low Success rate, according to players’ network quality
    ○ Small Scale Multi-player gaming
    

    View Slide

15. Firewall and NAT Type using PlayStation
    ● Type 1: System connects to Internet directly
    ● Type 2: System connects to Internet via Router with NAT
    ● Type 3: System connects to Internet via Router with NAT. External system cannot
    connect to internal system directly.
    https://portforward.com/nat-types/
    

    View Slide

16. Firewall and NAT Type using PlayStation
    ● Type 1: System connects to Internet directly
    ● Able to connect to Type 1, 2, 3.
    ● No Limitaion
    https://portforward.com/nat-types/
    

    View Slide

17. Firewall and NAT Type using PlayStation
    ● Type 2: System connects to Internet via Router with NAT
    ● Able to connect to Type 1, 2
    ● Usually need DMZ, UPnP, Port Forwarding or UDP Hole Punching to allow
    external system to connect to internal
    https://portforward.com/nat-types/
    

    View Slide

18. Firewall and NAT Type using PlayStation
    ● Type 3: System connects to Internet via Router with NAT. External system cannot
    connect to internal system directly.
    ● Only able to connect with Type 1
    https://portforward.com/nat-types/
    

    View Slide

19. UDP Hole Punching
    ● UDP Hole Punching
    ● Let two clients exchange connection info and try to connect to each other via 3rd
    party server
    ● STUN: Session Traversal Utilities for NAT
    ● STUN is common implement for UDP Hole Punching (RFC 5389)
    https://bford.info/pub/net/p2pnat/
    

    View Slide

20. https://bford.info/pub/net/p2pnat/
    

    View Slide

21. UDP Hole Punching FAIL!
    ● If Hole Punching FAIL, need use other method to connect each other
    ● Forward two clients info to each other via TURN Server or other nodes in P2P
    networks
    ● For example: PS4 Party Chat. If it’s not able to connect, 1 of players will
    forward the network traffic or voice to others
    https://forum.gamer.com.tw/C.php?bsn=5786&snA=135419
    

    View Slide

22. TURN? or not
    ● In design of TURN server, only create 1 forwarding connection. If TURN server
    is offline, forwarding connection will be disable, need to re-create forwarding
    connection to other TURN server
    ● No Redundancy, No Fast Recovery
    

    View Slide

23. Wireguard + BGP
    ● Use Wireguard to connect with each others
    ● Let Wireguard connections be Full Mesh Topology via STUN as much as
    possible
    ● If no Full Mesh, connect to server, server will help to forward connection
    ● BGP will create dynamic routing, and choose the nearest routing
    ● Create Redundancy Gaming Network via BGP
    https://www.talari.com/glossary_faq/sd-wan-full-mesh/
    

    View Slide

24. Tunneling + Dynamic Routing Protocol
    ● Tunneling
    ○ GRE
    ○ VxLAN
    ○ IPsec
    ○ OpenVPN
    ● Dynamic Routing Protocol
    ○ RIP
    ○ OSPF
    

    View Slide

25. Demo
    ● R1 10.0.0.1, 192.168.0.1
    ● R2 10.0.0.2, 192.168.0.2
    ● R3 10.0.0.3, 192.168.0.3
    ● R4 10.0.0.4, 192.168.0.4
    ● While R2 reboot, R1 still can connect to R4 via R3
    R1
    R2 R3
    R4
    

    View Slide