JANOG48 Meeting https://www.janog.gr.jp/meeting/janog48/linedns-en/
Verda Network Development Team, LINE Corporation Kawakami Kento
Refresh DNS Infrastructurewith Modern Datacenter NetworkKAWAKAMI KENTO, VERDA NETWORK DEVELOPMENT TEAM, LINE CORPORATION
View Slide
Agenda • LINE Private Cloud• Previous DNS redundancy methods• IP ANYCAST with CLOS Network
• Network software engineer• DNS• OpenStack• Kubernetes• Kawakami Kento• LINE Corporation• Network Development Team• LINE New grad in 20192
LINE Private Cloud3
Our ServicesBB A4
DNS as a ServiceVerda DNSDashboard5
DNS ArchitectureDesignatel ')#846?l *& ! A5DNS Relayl DesignateDNS CacheDNS Auth*;9:.3DNS Authl 0@2+$("-<DNS Cachel Full resolver =4,@%"B>@B7/13*%"6
DNS ArchitectureDesignatel ')#846?l *& ! A5DNS Relayl DesignateDNS CacheDNS Auth*;9:.3DNS Auth (VRRP => IP ANYCAST)l 0@2+$("-<DNS Cache (HWLB => IP ANYCAST)l Full resolver =4,@%"B>@B7/13*%"7
Background of the replacement8• (#+/'0,2M@• 2 I5JDNS$named.conf>E9 • 8D!4• Physical MachineM@• PM$.&0;.!)F• VerdaAVM8G• DNS Auth/CacheBH4• *2"2 7O:DNR• BH4• C2%-SQ• CLOS NetworkA8G• CLOS Network*. 6=8D
VRRP(Virtual Router Redundancy Protocol)• VIP6APR Request6 Primary #:*,/9$*• DNS Auth /9$>? +A• DNS Auth('. Service• DoS('-5Service"&<3• @8• Act-Stb)2• *• L2)2=0• Act-Stb47!<3%;19
HWLB(Hardware Load Balancer)• HWLB1I #*!# ($E74• DNS [email protected] 6K• DNS Auth -0D32/C;A:8LK• )J,• &)HWLB4• H>• #?=DNS'*"*B :5• VIP(%)HWLB+<• EoL• EoS10
VRRP HWLBProsCons• -*'• Act-Act&$ " • • Act-Act&$+• L2#,• Act-Stb !%)(• -*'11
CLOS Network• LeafSpine $08*2*5!• !%6+• Spine7#Leaf&3"1#• Spine/ Super-Spine7#4':! • East-West (• ECMP(Equal Cost Multi Path))<9,3" -.;12[1]: LINE https://www.janog.gr.jp/meeting/janog43/program/line/
Full L3 CLOS Network• ToR SwitchHypervisoreBGPPeering• Hypervisor FRRCLOSVM• Hypervisor VM IP /3213
BGP advertisement from VM• VMFRRHypervisorFRRPeering• HypervisorFRR ToR14
Advertise DNS VIP(1/3)• BGPVM VMDNSVIP15
Advertise DNS VIP(2/3)• BGPVM VM DNSVIP• DNSVMHypevisor• DNSVM VIP16
Advertise DNS VIP(3/3)• BGP %"VM$#VMDNS VIP"• DNS VMHypevisor !• DNS VMVIP"• ECMP# • CLOS NetworkBest Path%17
Benefit of IP ANYCAST for DNS(1/4)• • ECMP• L3 Switch! )• CLOS Network % $(• L2 network#'• BGP& Full L3Network "18
Benefit of IP ANYCAST for DNS(2/4)• Act-Act41(0!!$'*3• CLOS&IP ANYCST; ECMP9 !:<+2• % $$)8BGP9;'*3• >/ NW6- ! =.,5#7"% $;'*319
Benefit of IP ANYCAST for DNS(3/4)• DNS Auth DNS Cache! (• VRRP HWLB "*'• IP ANYCAST DNS Auth/Cache!$)"%
Benefit of IP ANYCAST for DNS(4/4)• VIP&• IP"$ !DNS Cache %#21
VRRP HWLBProsCons• -*'• Act-Act&$ " • • Act-Act&$+• L2#,• Act-Stb !%)(• -*'22
DNS Server VM• DNS VM• FRR• • DNS Server• Bind• NSD• Prometheus Exporters• node_exporter• bind_exporter• nsd_exporter• Health Check Daemon• Next Page=>23
DNS Server VM Traffic• DNSVMInterface2• Mgmt Interface• Prometheus scrape• ssh• • Service Interface• DNS Query • BGP24
Monitoring25• Prometheus• Scrape exporters• DNSExporter+'• blackbox_exporter• !Exporter"#0• DNS QueryVIP)(% DNS$*• ! -,DNS/.&
Health Check Daemon• Health Check DaemonDNSVIP -+)% • VMHV5. !BGPDown(Service Out%• DaemonDNS2&%• DNS -DNS Query,*• TTL: 1• Destination: VIP• 6• 31• Down#/"7$40'26
VRRP HWLBProsCons• -*'• Act-Act&$ " • • Act-Act&$+• L2#,• Act-Stb !%)(• -*'27
Disadvantage of IP ANYCAST• DNS 9@/VM><&=?B• BGP Best Path %F:;• E#76* VM350% D0• NW(8"2VM.8)?B• -5%F'+ @/4C• BGP%F3A1$, E%F'+! VM1100% D0VM1 VM2 VM328
• DNSCN4IP ANYCASTS=J,/#&I<• !.*/$9B• CLOSBVMO5; -) $H76P• ,1$'1!D9B• VM8V1&06P• VIP+"*/$9B• IP ANYCASTCN4%,/#&• BGPBest Path8VFM NW:E2>(.1 1QT• IP ANYCASTK3@URLKAQT• Health Check Daemon?GConclusion29
Discussion• DCDNS • DNS • CLOS NW30
Related Documents1. LINE !#$https://www.janog.gr.jp/meeting/janog43/program/line/2. " https://dnsops.jp/event/20210625/13-kosaka.pdf31