Refresh DNS Infrastructure with Modern Datacenter Network

JANOG48 Meeting
https://www.janog.gr.jp/meeting/janog48/linedns-en/

Verda Network Development Team, LINE Corporation
Kawakami Kento

1. Refresh DNS Infrastructure with Modern Datacenter Network KAWAKAMI KENTO, VERDA NETWORK DEVELOPMENT TEAM, LINE CORPORATION
2. Agenda • LINE Private Cloud • Previous DNS redundancy methods • IP ANYCAST with CLOS Network
3. • Network software engineer • DNS • OpenStack • Kubernetes • Kawakami Kento • LINE Corporation • Network Development Team • LINE New grad in 2019 2
4. LINE Private Cloud 3
5. Our Services B B A 4
6. DNS as a Service Verda DNS
   Dashboard
    5
7. DNS Architecture Designate l ')#846? l *& ! A5  DNS Relay l DesignateDNS CacheDNS Auth *;9:.3 DNS Auth l 0@2+$("-<  DNS Cache l Full resolver  =4 
   ,@%" B>@ B7/13 * %"  6
8. DNS Architecture Designate l ')#846? l *& ! A5  DNS Relay l DesignateDNS CacheDNS Auth *;9:.3 DNS Auth (VRRP => IP ANYCAST) l 0@2+$("-<  DNS Cache (HWLB => IP ANYCAST) l Full resolver  =4 
   ,@%" B>@ B7/13 * %"  7
9. Background of the replacement 8 • (#+/'0
   ,2M@ • 
   2 I5JDNS
   $<L 12 named.conf>E9  • 8D
   !4 • Physical MachineM@ • PM
   $.&0;.
   !)F • VerdaAVM  8G • DNS Auth/CacheBH4<PK3 • *2"2<? 7O:DNR • BH4<PK39 • C2%-SQ • CLOS NetworkA8G • CLOS Network*. 6= 8D
10. VRRP(Virtual Router Redundancy Protocol) • VIP6 APR Request6 Primary #:* ,
    /9$* • DNS Auth /9$>? +A • DNS Auth('.   Service • DoS('-5Service"& <3  • @8 • Act-Stb)2 •  * • L2)2=0 • Act-Stb47! <3 %;1 9
11. HWLB(Hardware Load Balancer) • HWLB1I #*!
    #  ($E74 • DNS Cache 9@.FG 6K • DNS Auth -0D32 /C;A:8 LK • 
    )J, • &)HWLB 4 • H> • 
    #?= DNS'*"* B :5 • VIP(%)HWLB+ <  • EoL • EoS 10
12. VRRP HWLB Pros Cons •  -*' • Act-Act&$ "   •  • Act-Act&$+ • L2#, • Act-Stb ! % )( • -*'
    11
13. CLOS Network • LeafSpine  $08 *2 *5!
     • !
    %6+ • Spine7# Leaf&3"1# • Spine/ Super-Spine7# 4': !
     • East-West ( • ECMP(Equal Cost Multi Path) )< 9,3"
    - .; 12 [1]: LINE 
      https://www.janog.gr.jp/meeting/janog43/program/line/
14. Full L3 CLOS Network • ToR SwitchHypervisoreBGP Peering • Hypervisor FRRCLOSVM   • Hypervisor 
    VM IP  /32 13
15. BGP advertisement from VM • VM FRRHypervisor FRR
    Peering • Hypervisor FRR
     ToR  14
16. Advertise DNS VIP(1/3) • BGP  VM   
    VMDNSVIP  15
17. Advertise DNS VIP(2/3) • BGP VM   
    VM DNSVIP  • DNSVM Hypevisor    •   DNSVM VIP  16
18. Advertise DNS VIP(3/3) • BGP %"VM $#
    VMDNS VIP" • DNS VMHypevisor  !  •   DNS VMVIP" • ECMP#  • CLOS Network Best Path % 17
19. Benefit of IP ANYCAST for DNS(1/4) •   
    • ECMP • L3 Switch! )   • CLOS Network %  $ ( • L2 network#' • BGP&
    
    Full L3 Network " 18
20. Benefit of IP ANYCAST for DNS(2/4) • Act-Act41 (0 ! !$'*3 • CLOS& IP ANYCST; 
    ECMP9 ! :<+2 • % $$)8BGP9 ;'*3 • >/ NW6- ! =. ,5 #7  "% $;'*3 19
21. Benefit of IP ANYCAST for DNS(3/4) • DNS Auth DNS Cache !  ( • VRRP HWLB " *' 
     • IP ANYCAST DNS Auth/Cache  !$ )"%&# 20
22. Benefit of IP ANYCAST for DNS(4/4) • VIP& • IP"$  !
    DNS Cache   %# 21
23. VRRP HWLB Pros Cons •  -*' • Act-Act&$ "   •  • Act-Act&$+ • L2#, • Act-Stb ! % )( • -*'
    22
24. DNS Server VM • DNS VM  • FRR • 
    
     
      • DNS Server • Bind • NSD • Prometheus Exporters • node_exporter • bind_exporter • nsd_exporter • Health Check Daemon • Next Page=> 23
25. DNS Server VM Traffic • DNSVMInterface2 • Mgmt Interface • Prometheus scrape • ssh •
     • Service Interface •
    DNS Query  • BGP 24
26. Monitoring 25 • Prometheus • Scrape exporters • DNS Exporter +' • blackbox_exporter •  !Exporter" #0 • DNS QueryVIP )(%  DNS $*  •  !
    -, DNS /.&
27. Health Check Daemon • Health Check DaemonDNSVIP - +)%  • VMHV5. !  BGPDown( Service Out% • DaemonDNS2&% •
    DNS
     - DNS Query,* • TTL: 1 • Destination: VIP • 6
     • 31 • Down #/ "7$40' 26
28. VRRP HWLB Pros Cons •  -*' • Act-Act&$ "   •  • Act-Act&$+ • L2#, • Act-Stb ! % )( • -*'
    27
29. Disadvantage of IP ANYCAST • DNS 9@/   VM><&=?B • BGP Best Path %F:; • E#76* VM350%  D0 • NW(8"2 VM.8 )?B
     • -5%F'+ @/4C • BGP%F3A1$,
     E%F'+! 
    VM1100%  D0 VM1 VM2 VM3 28
30. • DNSCN4IP ANYCASTS=J,/#&I<  • !.*/$9B • CLOSBVMO5; -) $H76P • ,1$'1!D9B • VM8V1&06P • VIP+"*/$9B • IP ANYCASTCN4%,/#& • BGPBest Path8VFM  
    NW:E2>  (.1 1 QT • IP ANYCASTK3@URLKA QT • Health Check Daemon?G Conclusion 29
31. Discussion • DCDNS  
     • DNS  
     • CLOS NW 
     30
32. Related Documents 1. LINE 
     !#$ https://www.janog.gr.jp/meeting/janog43/program/line/ 2. 
     
    
     "  https://dnsops.jp/event/20210625/13-kosaka.pdf 31