Refresh DNS Infrastructure with Modern Datacenter Network

JANOG48 Meeting
https://www.janog.gr.jp/meeting/janog48/linedns-en/

Verda Network Development Team, LINE Corporation
Kawakami Kento

Transcript

1. Refresh DNS Infrastructure
   with Modern Datacenter Network
   KAWAKAMI KENTO, VERDA NETWORK DEVELOPMENT TEAM, LINE CORPORATION
   

   View Slide

2. Agenda • LINE Private Cloud
   • Previous DNS redundancy methods
   • IP ANYCAST with CLOS Network
   

   View Slide

3. • Network software engineer
   • DNS
   • OpenStack
   • Kubernetes
   • Kawakami Kento
   • LINE Corporation
   • Network Development Team
   • LINE New grad in 2019
   2
   

   View Slide

4. LINE Private Cloud
   3
   

   View Slide

5. Our Services
   B
   B A
   4
   

   View Slide

6. DNS as a Service
   Verda DNSDashboard
   5
   

   View Slide

7. DNS Architecture
   Designate
   l ')#846?
   l *& ! A5
   DNS Relay
   l DesignateDNS CacheDNS Auth
   *;9:.3
   DNS Auth
   l 0@2+$("-<
   
   DNS Cache
   l Full resolver =4,@%"
   B>@B7/13
   *
   %"
   6
   

   View Slide

8. DNS Architecture
   Designate
   l ')#846?
   l *& ! A5
   DNS Relay
   l DesignateDNS CacheDNS Auth
   *;9:.3
   DNS Auth (VRRP => IP ANYCAST)
   l 0@2+$("-<
   
   DNS Cache (HWLB => IP ANYCAST)
   l Full resolver =4,@%"
   B>@B7/13
   *
   %"
   7
   

   View Slide

9. Background of the replacement
   8
   • (#+/'0,2M@
   • 2 I5JDNS$named.conf>E9
   
   • 8D!4
   • Physical MachineM@
   • PM$.&0;.!)F
   • VerdaAVM
   8G
   • DNS Auth/CacheBH4• *2"2 7O:DNR
   • BH4• C2%-SQ
   • CLOS NetworkA8G
   • CLOS Network*. 6=8D
   

   View Slide

10. VRRP(Virtual Router Redundancy Protocol)
    • VIP6APR Request6
    
    Primary #:*
    ,/9$*
    • DNS Auth /9$>?
    +A
    • DNS Auth('.
    Service
    • DoS('-5Service"&
    <3
    
    • @8
    • Act-Stb)2
    • *
    • L2)2=0
    • Act-Stb47!
    <3%;1
    9
    

    View Slide

11. HWLB(Hardware Load Balancer)
    • HWLB1I #*!#
    ($E74
    • DNS Cache9@.FG 6K
    • DNS Auth -0D32
    /C;A:8LK
    • )J,
    • &)HWLB4
    • H>
    • #?=DNS'*"*
    B :5
    • VIP(%)HWLB+
    <
    
    • EoL
    • EoS
    10
    

    View Slide

12. VRRP HWLB
    Pros
    Cons
    • -*'
    • Act-Act&$ "
    
    •
    • Act-Act&$+
    • L2#,
    • Act-Stb !
    %
    )(
    • -*'
    11
    

    View Slide

13. CLOS Network
    • LeafSpine $08
    *2
    *5!
    • !%6+
    • Spine7#
    Leaf&3"1#
    • Spine/ Super-Spine7#
    4':!
    • East-West (
    • ECMP(Equal Cost Multi Path)
    )<
    9,3" -
    .;
    12
    [1]: LINE
    https://www.janog.gr.jp/meeting/janog43/program/line/
    

    View Slide

14. Full L3 CLOS Network
    • ToR SwitchHypervisoreBGP
    Peering
    • Hypervisor FRRCLOSVM
    
    • Hypervisor
    VM IP
    /32
    13
    

    View Slide

15. BGP advertisement from VM
    • VMFRRHypervisorFRR
    Peering
    • HypervisorFRR ToR
    
    
    14
    

    View Slide

16. Advertise DNS VIP(1/3)
    • BGP
    VM
    VMDNSVIP
    15
    

    View Slide

17. Advertise DNS VIP(2/3)
    • BGPVM
    
    VM DNSVIP
    • DNSVMHypevisor
    
    • DNSVM VIP
    16
    

    View Slide

18. Advertise DNS VIP(3/3)
    • BGP %"VM
    $#VMDNS VIP"
    • DNS VMHypevisor !
    
    • DNS VMVIP"
    • ECMP#
    
    • CLOS NetworkBest Path
    %
    17
    

    View Slide

19. Benefit of IP ANYCAST for DNS(1/4)
    •
    
    • ECMP
    • L3 Switch!
    )
    
    • CLOS Network %
    $(
    • L2 network#'
    • BGP& Full L3
    Network "
    18
    

    View Slide

20. Benefit of IP ANYCAST for DNS(2/4)
    • Act-Act41(0!
    !$'*3
    • CLOS&
    IP ANYCST;
    
    ECMP9 !
    :<+2
    • % $$)8BGP9
    ;'*3
    • >/ NW6- ! =.
    ,5#7
    "% $;'*3
    19
    

    View Slide

21. Benefit of IP ANYCAST for DNS(3/4)
    • DNS Auth DNS Cache!
    (
    • VRRP HWLB "
    *'
    
    • IP ANYCAST DNS Auth/Cache
    !$)"%&#
    20
    

    View Slide

22. Benefit of IP ANYCAST for DNS(4/4)
    • VIP&
    • IP"$
    !DNS Cache
    
    
    %#
    21
    

    View Slide

23. VRRP HWLB
    Pros
    Cons
    • -*'
    • Act-Act&$ "
    
    •
    • Act-Act&$+
    • L2#,
    • Act-Stb !
    %
    )(
    • -*'
    22
    

    View Slide

24. DNS Server VM
    • DNS VM
    
    • FRR
    •
    
    
    • DNS Server
    • Bind
    • NSD
    • Prometheus Exporters
    • node_exporter
    • bind_exporter
    • nsd_exporter
    • Health Check Daemon
    • Next Page=>
    23
    

    View Slide

25. DNS Server VM Traffic
    • DNSVMInterface2
    • Mgmt Interface
    • Prometheus scrape
    • ssh
    •
    
    • Service Interface
    • DNS Query
    • BGP
    24
    

    View Slide

26. Monitoring
    25
    • Prometheus
    • Scrape exporters
    • DNSExporter
    +'
    • blackbox_exporter
    • !Exporter"
    #0
    • DNS QueryVIP
    )(%
    DNS$*
    
    • ! -,DNS
    
    /.&
    

    View Slide

27. Health Check Daemon
    • Health Check DaemonDNSVIP -
    
    +)%
    
    • VMHV5. !
    
    BGPDown(Service Out%
    • DaemonDNS2&%
    • DNS -
    DNS Query,*
    • TTL: 1
    • Destination: VIP
    • 6
    • 31
    • Down#/
    "7$40'
    26
    

    View Slide

28. VRRP HWLB
    Pros
    Cons
    • -*'
    • Act-Act&$ "
    
    •
    • Act-Act&$+
    • L2#,
    • Act-Stb !
    %
    )(
    • -*'
    27
    

    View Slide

29. Disadvantage of IP ANYCAST
    • DNS 9@/
    
    VM><&=?B
    • BGP Best Path %F:;
    • E#76* VM350%
    D0
    • NW(8"2VM.8
    )?B
    • -5%F'+ @/4C
    • BGP%F3A1$,
    E%F'+!
    VM1100%
    D0
    VM1 VM2 VM3
    28
    

    View Slide

30. • DNSCN4IP ANYCASTS=J,/#&I<
    • !.*/$9B
    • CLOSBVMO5; -) $H76P
    • ,1$'1!D9B
    • VM8V1&06P
    • VIP+"*/$9B
    • IP ANYCASTCN4%,/#&
    • BGPBest Path8VFM
    NW:E2>
    (.1 1QT
    • IP ANYCASTK3@URLKAQT
    • Health Check Daemon?G
    Conclusion
    29
    

    View Slide

31. Discussion
    • DCDNS
    • DNS
    
    • CLOS NW
    30
    

    View Slide

32. Related Documents
    1. LINE
    !#$
    https://www.janog.gr.jp/meeting/janog43/program/line/
    2.
    "
    https://dnsops.jp/event/20210625/13-kosaka.pdf
    31
    

    View Slide