Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Refresh DNS Infrastructure with Modern Datacenter Network

Refresh DNS Infrastructure with Modern Datacenter Network

JANOG48 Meeting
https://www.janog.gr.jp/meeting/janog48/linedns-en/

Verda Network Development Team, LINE Corporation
Kawakami Kento

LINE Developers

July 16, 2021
Tweet

More Decks by LINE Developers

Other Decks in Technology

Transcript

  1. Refresh DNS Infrastructure
    with Modern Datacenter Network
    KAWAKAMI KENTO, VERDA NETWORK DEVELOPMENT TEAM, LINE CORPORATION

    View full-size slide

  2. Agenda • LINE Private Cloud
    • Previous DNS redundancy methods
    • IP ANYCAST with CLOS Network

    View full-size slide

  3. • Network software engineer
    • DNS
    • OpenStack
    • Kubernetes
    • Kawakami Kento
    • LINE Corporation
    • Network Development Team
    • LINE New grad in 2019
    2

    View full-size slide

  4. LINE Private Cloud
    3

    View full-size slide

  5. Our Services
    B
    B A
    4

    View full-size slide

  6. DNS as a Service
    Verda DNSDashboard
    5

    View full-size slide

  7. DNS Architecture
    Designate
    l ')#846?
    l *& ! A5
    DNS Relay
    l DesignateDNS CacheDNS Auth
    *;9:.3
    DNS Auth
    l 0@2+$("-<

    DNS Cache
    l Full resolver =4,@%"
    B>@B7/13
    *
    %"
    6

    View full-size slide

  8. DNS Architecture
    Designate
    l ')#846?
    l *& ! A5
    DNS Relay
    l DesignateDNS CacheDNS Auth
    *;9:.3
    DNS Auth (VRRP => IP ANYCAST)
    l 0@2+$("-<

    DNS Cache (HWLB => IP ANYCAST)
    l Full resolver =4,@%"
    B>@B7/13
    *
    %"
    7

    View full-size slide

  9. Background of the replacement
    8
    • (#+/'0,2M@
    • 2 I5JDNS$named.conf>E9

    • 8D!4
    • Physical MachineM@
    • PM$.&0;.!)F
    • VerdaAVM
    8G
    • DNS Auth/CacheBH4• *2"2 7O:DNR
    • BH4• C2%-SQ
    • CLOS NetworkA8G
    • CLOS Network*. 6=8D

    View full-size slide

  10. VRRP(Virtual Router Redundancy Protocol)
    • VIP6APR Request6

    Primary #:*
    ,/9$*
    • DNS Auth /9$>?
    +A
    • DNS Auth('.
    Service
    • DoS('-5Service"&
    <3

    • @8
    • Act-Stb)2
    • *
    • L2)2=0
    • Act-Stb47!
    <3%;1
    9

    View full-size slide

  11. HWLB(Hardware Load Balancer)
    • HWLB1I #*!#
    ($E74
    • DNS [email protected] 6K
    • DNS Auth -0D32
    /C;A:8LK
    • )J,
    • &)HWLB4
    • H>
    • #?=DNS'*"*
    B :5
    • VIP(%)HWLB+
    <

    • EoL
    • EoS
    10

    View full-size slide

  12. VRRP HWLB
    Pros
    Cons
    • -*'
    • Act-Act&$ "


    • Act-Act&$+
    • L2#,
    • Act-Stb !
    %
    )(
    • -*'
    11

    View full-size slide

  13. CLOS Network
    • LeafSpine $08
    *2
    *5!
    • !%6+
    • Spine7#
    Leaf&3"1#
    • Spine/ Super-Spine7#
    4':!
    • East-West (
    • ECMP(Equal Cost Multi Path)
    )<
    9,3" -
    .;
    12
    [1]: LINE
    https://www.janog.gr.jp/meeting/janog43/program/line/

    View full-size slide

  14. Full L3 CLOS Network
    • ToR SwitchHypervisoreBGP
    Peering
    • Hypervisor FRRCLOSVM

    • Hypervisor
    VM IP
    /32
    13

    View full-size slide

  15. BGP advertisement from VM
    • VMFRRHypervisorFRR
    Peering
    • HypervisorFRR ToR


    14

    View full-size slide

  16. Advertise DNS VIP(1/3)
    • BGP
    VM
    VMDNSVIP
    15

    View full-size slide

  17. Advertise DNS VIP(2/3)
    • BGPVM

    VM DNSVIP
    • DNSVMHypevisor

    • DNSVM VIP
    16

    View full-size slide

  18. Advertise DNS VIP(3/3)
    • BGP %"VM
    $#VMDNS VIP"
    • DNS VMHypevisor !

    • DNS VMVIP"
    • ECMP#

    • CLOS NetworkBest Path
    %
    17

    View full-size slide

  19. Benefit of IP ANYCAST for DNS(1/4)


    • ECMP
    • L3 Switch!
    )

    • CLOS Network %
    $(
    • L2 network#'
    • BGP& Full L3
    Network "
    18

    View full-size slide

  20. Benefit of IP ANYCAST for DNS(2/4)
    • Act-Act41(0!
    !$'*3
    • CLOS&
    IP ANYCST;

    ECMP9 !
    :<+2
    • % $$)8BGP9
    ;'*3
    • >/ NW6- ! =.
    ,5#7
    "% $;'*3
    19

    View full-size slide

  21. Benefit of IP ANYCAST for DNS(3/4)
    • DNS Auth DNS Cache!
    (
    • VRRP HWLB "
    *'

    • IP ANYCAST DNS Auth/Cache
    !$)"%&#
    20

    View full-size slide

  22. Benefit of IP ANYCAST for DNS(4/4)
    • VIP&
    • IP"$
    !DNS Cache


    %#
    21

    View full-size slide

  23. VRRP HWLB
    Pros
    Cons
    • -*'
    • Act-Act&$ "


    • Act-Act&$+
    • L2#,
    • Act-Stb !
    %
    )(
    • -*'
    22

    View full-size slide

  24. DNS Server VM
    • DNS VM

    • FRR



    • DNS Server
    • Bind
    • NSD
    • Prometheus Exporters
    • node_exporter
    • bind_exporter
    • nsd_exporter
    • Health Check Daemon
    • Next Page=>
    23

    View full-size slide

  25. DNS Server VM Traffic
    • DNSVMInterface2
    • Mgmt Interface
    • Prometheus scrape
    • ssh


    • Service Interface
    • DNS Query
    • BGP
    24

    View full-size slide

  26. Monitoring
    25
    • Prometheus
    • Scrape exporters
    • DNSExporter
    +'
    • blackbox_exporter
    • !Exporter"
    #0
    • DNS QueryVIP
    )(%
    DNS$*

    • ! -,DNS

    /.&

    View full-size slide

  27. Health Check Daemon
    • Health Check DaemonDNSVIP -

    +)%

    • VMHV5. !

    BGPDown(Service Out%
    • DaemonDNS2&%
    • DNS -
    DNS Query,*
    • TTL: 1
    • Destination: VIP
    • 6
    • 31
    • Down#/
    "7$40'
    26

    View full-size slide

  28. VRRP HWLB
    Pros
    Cons
    • -*'
    • Act-Act&$ "


    • Act-Act&$+
    • L2#,
    • Act-Stb !
    %
    )(
    • -*'
    27

    View full-size slide

  29. Disadvantage of IP ANYCAST
    • DNS 9@/

    VM><&=?B
    • BGP Best Path %F:;
    • E#76* VM350%
    D0
    • NW(8"2VM.8
    )?B
    • -5%F'+ @/4C
    • BGP%F3A1$,
    E%F'+!
    VM1100%
    D0
    VM1 VM2 VM3
    28

    View full-size slide

  30. • DNSCN4IP ANYCASTS=J,/#&I<
    • !.*/$9B
    • CLOSBVMO5; -) $H76P
    • ,1$'1!D9B
    • VM8V1&06P
    • VIP+"*/$9B
    • IP ANYCASTCN4%,/#&
    • BGPBest Path8VFM
    NW:E2>
    (.1 1QT
    • IP ANYCASTK3@URLKAQT
    • Health Check Daemon?G
    Conclusion
    29

    View full-size slide

  31. Discussion
    • DCDNS
    • DNS

    • CLOS NW
    30

    View full-size slide

  32. Related Documents
    1. LINE
    !#$
    https://www.janog.gr.jp/meeting/janog43/program/line/
    2.
    "
    https://dnsops.jp/event/20210625/13-kosaka.pdf
    31

    View full-size slide