Slide 1

Slide 1 text

Cloud Infrastructure Interconnect with Wireguard and OSPF Date Huang 黃宇強 tjjh89017 [at] hotmail.com 2021/08/01

Slide 2

Slide 2 text

About me Date Huang 黃宇強 ● 2019 OpenInfra Day Taiwan Speaker ○ Massive Bare-Metal Operating System Provisioning Improvement ● 2019 OSC Tokyo Fall Speaker ● 2019 COScon ‘19 Speaker ● 2019 Hong Kong Open Source Conference Speaker ○ De-centralized Bare-Metal Operating System Provisioning ● 2018 ISC High Performance Project Poster Demo ○ The Design and Implementation of Bare Metal Cluster Deployment Using BitTorrent ● 2017 Open Source Summit North America co-Speaker ○ Building Cloud Infra using cost-effective ARM Boards ● 2017 OpenStack Day Taiwan Speaker ○ Combine Continuous Integration (CI) with OpenStack ● 2016 OpenStack Day Taiwan Invited Speaker ○ OpenStack on ARM64 Open Source Porject ● EZIO ● STUNMESH-go 2

Slide 3

Slide 3 text

Overview ● Cloud Interconnect Solution Brief ○ DWDM Solution Brief ○ MPLS Solution Brief ○ SD-WAN IPsec Brief ● Wireguard with OSPF ○ Wireguard brief ○ OSPF brief ● Example Topology ● STUNMESH-go ○ Wireguard Helper Tool ● More than L3 Tunnel 3

Slide 4

Slide 4 text

Cloud Interconnect Solution Brief ● DWDM Solution brief ● MPLS VPN brief ● SD-WAN IPsec brief 4

Slide 5

Slide 5 text

DWDM Solution brief - (1) ● Dense Wavelength Division Multiplexing (DWDM) ● Fiber-Optic Communications ● Distance: 40, 80, 120KM or more ● Merge multiple signal into single paired fiber 5

Slide 6

Slide 6 text

DWDM Solution brief - (2) ● Rent Dark Fiber between different location ● Connect each others with DWDM Solution 6

Slide 7

Slide 7 text

DWDM Solution brief - (3) ● Pros ○ Large Bandwidth ○ Full control ● Cons ○ Very Expensive ○ Very Complex Configuration ○ Hard to Use and Maintain 7

Slide 8

Slide 8 text

MPLS VPN brief - (1) ● Multi-Protocol Label Switching (MPLS) ● Using “Label” to determine the route ● Provided by ISP 8

Slide 9

Slide 9 text

MPLS VPN brief - (2) ● Pros ○ Convenience ○ Easy to Use and Maintain ● Cons ○ Expensive ○ Less Bandwidth 9

Slide 10

Slide 10 text

SD-WAN IPsec brief - (1) ● Connect all sites with IPsec tunnel via Broadband or Mobile Network ● Rich Redundancy between multiple WAN type to Internet ○ Primary: Broadband Network ○ Backup: Mobile Network ● Need Fixed Public IP in most of time ○ e.g. AWS VPC Customer Gateway (IPsec) need Public IPs in both side 10

Slide 11

Slide 11 text

SD-WAN IPsec brief - (2) ● Pros ○ Simple ○ Cheap for larger bandwidth ○ Convenience ○ Easy to Use and Maintain ● Cons ○ Need Fixed Public IP at Centralized Site ○ Centralized Architecture 11

Slide 12

Slide 12 text

Wireguard with OSPF ● What is Wireguard ● Why Wireguard ● What is OSPF ● Why OSPF 12

Slide 13

Slide 13 text

What is Wireguard ● Simple, Fast, Modern, Secure Tunnel ● Fast without any hardware acceleration ● Support Windows, Linux, MacOS, Android, iOS ● Algo ○ Curve25519 ○ ChaCha20 ○ Poly1305 13

Slide 14

Slide 14 text

Why Wireguard ● UDP based VPN Protocol ○ NAT and Firewall Traversal Persistence ● Built-in Keepalive ● Built-in Roaming ○ Auto adjust remote peer connection info ● Performance better than OpenVPN and IPsec (AES) ○ Wireguard have good performance without hardware crypto engine ○ Suitable to Embedded system or Network Box ● Much Simpler Configuration than OpenVPN ○ OpenVPN need 5KB config file ○ Wireguard only need several bytes ● Encapsulate IPv6-in-IPv4, IPv4-in-IPv6 14

Slide 15

Slide 15 text

What is OSPF ● Dynamic Route ○ Don’t need to bother to setup static route when new peer is added ● Built-in Keepalive ○ Fast re-route to redundant route when link status or route is changed ● Fast Convergence in small scale network 15

Slide 16

Slide 16 text

Why OSPF ● Built-in Keepalive ○ Wireguard didn’t have link status ○ It will need to send a packet to remote peer and check if receving reply to check link status ○ OSPF Hello Packet could test link status and check remote OSPF routing engine status 16

Slide 17

Slide 17 text

Wireguard with OSPF ● Fast and Simple VPN tunnel ● NAT Traversal Persistence ● Dynamic Route in Full Mesh Topology ● Fast Re-route ● Auto Check Link and Route Status 17

Slide 18

Slide 18 text

Network between Your Cloud and Office 18

Slide 19

Slide 19 text

19

Slide 20

Slide 20 text

20

Slide 21

Slide 21 text

21

Slide 22

Slide 22 text

22

Slide 23

Slide 23 text

23

Slide 24

Slide 24 text

24

Slide 25

Slide 25 text

25

Slide 26

Slide 26 text

26

Slide 27

Slide 27 text

STUNMESH-go ● Mobile Network ● UDP Hole Punching ● STUN ● STUNMESH-go 27

Slide 28

Slide 28 text

Mobile Network ● Through CGNAT ○ Full Cone NAT ○ Translate into Same Port and IP mapping when Destation IP and Port are different ● NAT Session to allow ingress traffic ○ Record Src IP, Src Port, Dst IP, Dst Port ○ Allow Firewall Rule 28

Slide 29

Slide 29 text

29 No Session From Site B to Site A Deny Site A to Site B Even Site A know NAT mapping about Site B It still cannot connect to Site B

Slide 30

Slide 30 text

UDP Hole Punching ● Let two clients exchange connection info and try to connect to each other via 3rd party server ● STUN: Session Traversal Utilities for NAT ● STUN is common implement for UDP Hole Punching (RFC 5389) 30

Slide 31

Slide 31 text

https://bford.info/pub/net/p2pnat/ 31

Slide 32

Slide 32 text

STUNMESH-go ● Wireguard helper tool to get through Full-Cone NAT ○ IP and Port translation mapping will be same even different destination ● Written in Golang ● Multiple Architecture Support ○ X86_64, MIPS ● Standalone Executable ○ Don’t need to care about library dependency ● Inspired by wireguard-p2p project ● Open Source ○ https://github.com/tjjh89017/stunmesh-go ○ GPLv2 or later ● Tested with ○ UBNT ER-X v2.0.8-hotfix.1 and Wireguard v1.0.20210424 ○ VyOS 1.4-rolling-202105200417 32

Slide 33

Slide 33 text

STUNMESH-go ● Get Public IP and Port after CG-NAT translation ○ cBPF Filter to receive packet from same UDP port as Wireguard used ○ Raw Socket to construct STUN 5389 request packet to same UDP port ● Encrypt Public Info with Wireguard Curve25519 Key ● Save Ciphertext into Cloudflare TXT Record ● Query TXT Record from Cloudflare ● Decrypt Ciphertext and Update Wireguard Peer Endpoint ● Usually, Only need to run once when initiating connection in first time ○ Or disconnect in the same time 33

Slide 34

Slide 34 text

Network between Your Cloud and Office Back to here 34

Slide 35

Slide 35 text

35

Slide 36

Slide 36 text

36

Slide 37

Slide 37 text

37

Slide 38

Slide 38 text

38

Slide 39

Slide 39 text

39

Slide 40

Slide 40 text

40

Slide 41

Slide 41 text

41

Slide 42

Slide 42 text

42

Slide 43

Slide 43 text

43

Slide 44

Slide 44 text

44

Slide 45

Slide 45 text

45

Slide 46

Slide 46 text

46

Slide 47

Slide 47 text

47

Slide 48

Slide 48 text

More than L3 Tunnel 48

Slide 49

Slide 49 text

More than L3 Tunnel ● Wireguard only encapsulate L3 packet into tunnel ○ Start from IP header ○ Without L2 MAC address, VLAN ● Some Protocols or Operations will need L2 ○ VM migration, two sites need to be in same L2 if VMs connect to each other with same subnet IPs 49

Slide 50

Slide 50 text

More than L3 Tunnel - L2 Tunnel ● VXLAN ● NVGRE ● HARD to control Broadcast, Unknown Unicast, Multicast in L2 tunnel ○ Proxy ARP ○ Static MAC table ○ Static ARP table ○ BGP-EVPN 50

Slide 51

Slide 51 text

Thanks You 51

Slide 52

Slide 52 text

Reference ● https://en.wikipedia.org/wiki/Wavelength-division_multiplexing ● https://docs.vmware.com/en/VMware-Smart-Assurance/10.1.0/mpls-man ager-user-guide-101/GUID-8EB1D677-B262-475F-9C1B-8D2D9826CC0D.ht ml ● https://www.wireguard.com/ ● https://zh.wikipedia.org/zh-tw/WireGuard ● https://github.com/tjjh89017/stunmesh-go ● https://github.com/manuels/wireguard-p2p ● https://bford.info/pub/net/p2pnat/ 52