Refresh DNS Infrastructure with Modern Datacenter Network

Slide 1

Slide 1 text

Refresh DNS Infrastructure with Modern Datacenter Network KAWAKAMI KENTO, VERDA NETWORK DEVELOPMENT TEAM, LINE CORPORATION

Slide 2

Slide 2 text

Agenda • LINE Private Cloud • Previous DNS redundancy methods • IP ANYCAST with CLOS Network

Slide 3

Slide 3 text

• Network software engineer • DNS • OpenStack • Kubernetes • Kawakami Kento • LINE Corporation • Network Development Team • LINE New grad in 2019 2

Slide 4

Slide 4 text

LINE Private Cloud 3

Slide 5

Slide 5 text

Our Services B B A 4

Slide 6

Slide 6 text

DNS as a Service Verda DNSDashboard 5

Slide 7

Slide 7 text

DNS Architecture Designate l ')#846? l *& ! A5 DNS Relay l DesignateDNS CacheDNS Auth *;9:.3 DNS Auth l 0@2+$("-< DNS Cache l Full resolver =4,@%" B>@B7/13 * %" 6

Slide 8

Slide 8 text

DNS Architecture Designate l ')#846? l *& ! A5 DNS Relay l DesignateDNS CacheDNS Auth *;9:.3 DNS Auth (VRRP => IP ANYCAST) l 0@2+$("-< DNS Cache (HWLB => IP ANYCAST) l Full resolver =4,@%" B>@B7/13 * %" 7

Slide 9

Slide 9 text

Background of the replacement 8 • (#+/'0,2M@ • 2 I5JDNS$E9 • 8D!4 • Physical MachineM@ • PM$.&0;.!)F • VerdaAVM 8G • DNS Auth/CacheBH4

Slide 10

Slide 10 text

VRRP(Virtual Router Redundancy Protocol) • VIP6APR Request6 Primary #:* ,/9$* • DNS Auth /9$>? +A • DNS Auth('. Service • DoS('-5Service"& <3 • @8 • Act-Stb)2 • * • L2)2=0 • Act-Stb47! <3%;1 9

Slide 11

Slide 11 text

HWLB(Hardware Load Balancer) • HWLB1I #*!# ($E74 • DNS Cache[email protected] 6K • DNS Auth -0D32 /C;A:8LK • )J, • &)HWLB4 • H> • #?=DNS'*"* B :5 • VIP(%)HWLB+ < • EoL • EoS 10

Slide 12

Slide 12 text

VRRP HWLB Pros Cons • -*' • Act-Act&$ " • • Act-Act&$+ • L2#, • Act-Stb ! % )( • -*' 11

Slide 13

Slide 13 text

CLOS Network • LeafSpine $08 *2 *5! • !%6+ • Spine7# Leaf&3"1# • Spine/ Super-Spine7# 4':! • East-West ( • ECMP(Equal Cost Multi Path) )< 9,3" - .; 12 [1]: LINE https://www.janog.gr.jp/meeting/janog43/program/line/

Slide 14

Slide 14 text

Full L3 CLOS Network • ToR SwitchHypervisoreBGP Peering • Hypervisor FRRCLOSVM • Hypervisor VM IP /32 13

Slide 15

Slide 15 text

BGP advertisement from VM • VMFRRHypervisorFRR Peering • HypervisorFRR ToR 14

Slide 16

Slide 16 text

Advertise DNS VIP(1/3) • BGP VM VMDNSVIP 15

Slide 17

Slide 17 text

Advertise DNS VIP(2/3) • BGPVM VM DNSVIP • DNSVMHypevisor • DNSVM VIP 16

Slide 18

Slide 18 text

Advertise DNS VIP(3/3) • BGP %"VM $#VMDNS VIP" • DNS VMHypevisor ! • DNS VMVIP" • ECMP# • CLOS NetworkBest Path % 17

Slide 19

Slide 19 text

Benefit of IP ANYCAST for DNS(1/4) • • ECMP • L3 Switch! ) • CLOS Network % $( • L2 network#' • BGP& Full L3 Network " 18

Slide 20

Slide 20 text

Benefit of IP ANYCAST for DNS(2/4) • Act-Act41(0! !$'*3 • CLOS& IP ANYCST; ECMP9 ! :<+2 • % $$)8BGP9 ;'*3 • >/ NW6- ! =. ,5#7 "% $;'*3 19

Slide 21

Slide 21 text

Benefit of IP ANYCAST for DNS(3/4) • DNS Auth DNS Cache! ( • VRRP HWLB " *' • IP ANYCAST DNS Auth/Cache !$)"%&# 20

Slide 22

Slide 22 text

Benefit of IP ANYCAST for DNS(4/4) • VIP& • IP"$ !DNS Cache %# 21

Slide 23

Slide 23 text

VRRP HWLB Pros Cons • -*' • Act-Act&$ " • • Act-Act&$+ • L2#, • Act-Stb ! % )( • -*' 22

Slide 24

Slide 24 text

DNS Server VM • DNS VM • FRR • • DNS Server • Bind • NSD • Prometheus Exporters • node_exporter • bind_exporter • nsd_exporter • Health Check Daemon • Next Page=> 23

Slide 25

Slide 25 text

DNS Server VM Traffic • DNSVMInterface2 • Mgmt Interface • Prometheus scrape • ssh • • Service Interface • DNS Query • BGP 24

Slide 26

Slide 26 text

Monitoring 25 • Prometheus • Scrape exporters • DNSExporter +' • blackbox_exporter • !Exporter" #0 • DNS QueryVIP )(% DNS$* • ! -,DNS /.&

Slide 27

Slide 27 text

Health Check Daemon • Health Check DaemonDNSVIP - +)% • VMHV5. ! BGPDown(Service Out% • DaemonDNS2&% • DNS - DNS Query,* • TTL: 1 • Destination: VIP • 6 • 31 • Down#/ "7$40' 26

Slide 28

Slide 28 text

VRRP HWLB Pros Cons • -*' • Act-Act&$ " • • Act-Act&$+ • L2#, • Act-Stb ! % )( • -*' 27

Slide 29

Slide 29 text

Disadvantage of IP ANYCAST • DNS 9@/ VM><&=?B • BGP Best Path %F:; • E#76* VM350% D0 • NW(8"2VM.8 )?B • -5%F'+ @/4C • BGP%F3A1$, E%F'+! VM1100% D0 VM1 VM2 VM3 28

Slide 30

Slide 30 text

• DNSCN4IP ANYCASTS=J,/#&I< • !.*/$9B • CLOSBVMO5; -) $H76P • ,1$'1!D9B • VM8V1&06P • VIP+"*/$9B • IP ANYCASTCN4%,/#& • BGPBest Path8VFM NW:E2> (.1 1QT • IP ANYCASTK3@URLKAQT • Health Check Daemon?G Conclusion 29

Slide 31

Slide 31 text

Discussion • DCDNS • DNS • CLOS NW 30

Slide 32

Slide 32 text

Related Documents 1. LINE !#$ https://www.janog.gr.jp/meeting/janog43/program/line/ 2. " https://dnsops.jp/event/20210625/13-kosaka.pdf 31