Presentation about Dynamic Multipoint VPN, its operation, phases and routing design


Dmitry Figol

June 14, 2017


  1. 3.

    Out of Scope 3 • DMVPN design • 2547oDMVPN –

    MPLS over DMVPN • DMVPN + IPsec Public Key Infrastructure (PKI) • IWAN
  2. 4.

    Overview 4 • DMVPN is Hub-and-Spoke Overlay VPN with or

    without encryption which runs over any IP network (for example, Internet) • Used for scaling IPsec VPNs • Instead of a separate tunnel per each router pair, there is one tunnel interface on each router • All VPN members are in the same subnet • For adding a new device to VPN, most of configuration can be copied from another device • Uses Multipoint GRE (mGRE), Next Hop Resolution Protocol (NHRP) and IPsec
  3. 5.

    Features 5 • DMVPN cloud has two types of addresses:

    overlay (logical) and underlay (NBMA). • Dynamic spoke discovery • Multicast replication on the hub • Dynamic spoke-to-spoke tunnels • NAT-Traversal (NAT-T) support • Front Door VRF support • IPv6 over IPv4 underlay support
  4. 6.

    mGRE 6 • Multipoint GRE (mGRE) is the key building

    block of DMVPN • Removes the necessity of specifying tunnel destination • Relies on Next Hop Resolution Protocol (NHRP) with static or dynamic mappings • Tunnel key is required (must match on both sides) to identify the tunnel interface if several tunnels with the same tunnel source are used
  5. 7.

    mGRE configuration example 7 interface Tunnel0 ip address

    ! Overlay IP tunnel source e0/0.150 tunnel mode gre multipoint tunnel key 5 !
  6. 8.

    Next Hop Resolution Protocol (NHRP) 8 • NHRP does all

    DMVPN magic • Encapsulated into GRE, Protocol number 0x2001 • Client-server protocol (the hub is Next Hop Server NHS, spokes are Next Hop Clients NHC) • NHS maintains NHRP database with NHRP mappings (logical IP to NBMA) • NHC sends NHRP Registration Request to all NHS • NHS checks for uniqueness of the claimed logical IP and replies with NHRP Registration Reply
  7. 9.

    NHRP (cont.) 9 • NHRP network-id is mandatory, locally significant

    number, used to distinguish different NHRP clouds • NHRP cleartext authentication may be used to prevent configuration mistakes • NHRP hold-time controls how long NHRP mapping is valid, NHC sends Registration Request every one third of this value. Default hold-time is 7200 seconds • NHC should have a static mapping for NHS and multicast (if needed) • Static mapping can be done not only for hub, but also for other spokes, that’s why it is needed to specify NHS additionally • NHS needs to know where to replicate multicast traffic (if needed)
  8. 10.

    NHRP flags 10 • Authoritative – mapping was obtained from

    NHS • Implicit – mapping was learned from NHRP resolution or reply • Negative – requested NBMA mapping could not be obtained. Used to prevent triggering more NHRP resolution requests while the one is being resolved • Unique – NHRP mapping cannot be overwritten with the same logical IP but different NBMA address • Registered – created from receiving NHRP registration request
  9. 11.

    NHRP flags (cont.) 11 • Used – mapping is valid

    • Router – mappings are for remote router itself to access the network with this flag • Local – list of peers where Resolution Request was sent • Nat – remote node supports NHRP NAT extension for dynamic spoke- to-spoke behind a NAT router
  10. 12.

    NHRP configuration example 12 interface Tunnel 0 ! on NHC,

    first IP is logical, the second is NBMA ip nhrp map ! on NHC specify NHS ip nhrp nhs ! on NHC specify where to send multicast ip nhrp map multicast ! on NHS specify where to replicate multicast ip nhrp map multicast dynamic ip nhrp network-id 50 ip nhrp holdtime 600 ip nhrp authentication cisco !
  11. 13.

    IPsec 13 • Encrypts all control plane DMVPN traffic (NHRP)

    and data plane • NHRP triggers IPsec before installing new mappings • IPsec notifies NHRP when encryption is ready • NHRP installs mappings, and sends registration if needed. • GRE over IPsec is used • Transport mode should be used in most cases • Remember to add PSK key for spokes if spoke-to-spoke tunnels are going to be built • If there are several mGRE interfaces with the same source, the same profile must be used with the keyword shared
  12. 14.

    IPsec configuration example 14 crypto isakmp policy 10 encr aes

    hash sha512 authentication pre-share group 14 crypto isakmp key cisco address crypto ipsec transform-set TR-SET esp-aes esp-sha-hmac mode transport crypto ipsec profile DMVPN set transform-set TR-SET ! interface Tunnel 0 tunnel protection ipsec profile DMVPN !
  13. 15.

    NAT-Transparency 15 • IPsec tunnel mode may not work in

    some NAT combinations • All DMVPN spokes must have a unique IP address after they have been NAT translated • Spokes can use Port Address Translation (PAT), Hub is allowed to use only Static NAT • UDP Port 4500 (non-isakmp) should be opened across the path • In show ip nhrp claimed NBMA address is IP address before NAT, NBMA address is IP address after NAT • Enabled by default, to disable use: (config)# no crypto ipsec nat-transparency udp-encapsulation
  14. 16.

    Miscellaneous 16 • Set IP MTU and TCP Adjust MSS

    statically to the recommended value to avoid issues with fragmentation: interface Tunnel 0 ip mtu 1400 ip tcp adjust-mss 1360 !
  15. 17.

    DMVPN Phase 1 17 • Only Hub-to-Spoke tunnels, no dynamic

    Spoke-to-Spoke tunnels • All traffic between the spokes is always going through the hub regardless if the next hop is hub or spoke • Summarization on the hub is allowed • Tunnel mode on NHC is gre ip instead of gre multipoint • tunnel destination must be specified as well
  16. 18.

    DMVPN Phase 1 – RIP 18 • Disable RIP split

    horizon on hub’s tunnel interface (if no summarization of spokes’ prefixes): • (config-if)# no ip split-horizon
  17. 19.

    DMVPN Phase 1 – EIGRP 19 • Disable EIGRP split

    horizon on hub’s tunnel interface (if no summarization of spokes’ prefixes): (config-if)# no ip split-horizon eigrp 100
  18. 20.

    DMVPN Phase 1 – OSPF 20 • The following OSPF

    network types can be used for automatic neighbor discovery: • Point-to-multipoint (most common) • Broadcast • If broadcast is used, make sure to exclude spokes from participation in DR/BDR election (set DR priority to 0) • Remember that default OSPF network type on tunnel interface is point- to-point which can result in OSPF adjacencies’ flapping on the hub
  19. 21.

    DMVPN Phase 1 – BGP 21 • Both eBGP and

    iBGP can be used • eBGP needs an individual AS number per branch (you can use 4-byte AS numbers) or allowas-in should be used on the spokes (careful!) • For iBGP each Hub should be Route-Reflector (RR) • Options above are not needed if summarization on the Hub is used
  20. 22.

    DMVPN Phase 2 22 • Dynamic Spoke-to-Spoke tunnels on demand

    based on Next Hop received from routing protocols • Routing protocols should support Third Party Next Hop feature • Summarization on the hub of Spokes’ prefixes is not allowed - summarization changes next hop to the Hub • Tunnel mode must be gre multipoint on spokes
  21. 23.

    DMVPN Phase 2 (cont.) 23 • If the packet is

    supposed to go out of the tunnel interface, the Spoke looks into its NHRP cache to find NBMA address for the next hop • If not found, the packet is sent to the Hub • Also, NHRP Resolution Request is sent to the Hub which should forward it to the remote Spoke • Remote Spoke replies with NHRP Resolution Reply containing its own NBMA address directly to the source spoke. If IPsec is used, remote spoke will initiate IPsec tunnel first. • Source spoke adds a mapping to NHRP cache and subsequent packets are going via spoke-to-spoke tunnel
  22. 24.

    DMVPN Phase 2 – RIP 24 • Advertises third party

    next hop automatically • Remember to disable split horizon on the hub
  23. 25.

    DMVPN Phase 2 – EIGRP 25 • Remember to disable

    split horizon on the hub • To use third party next hop configure the following command on hub’s tunnel interface: (config-if)# no ip next-hop-self eigrp 100
  24. 26.

    DMVPN Phase 2 – OSPF 26 • Only broadcast/non-broadcast network

    types are allowed as it is the only way to advertise third party next hop in the same area – DR will keep next hop unchanged
  25. 27.

    DMVPN Phase 2 – BGP 27 • Both eBGP and

    iBGP (RR) advertise third party next hop automatically
  26. 28.

    DMVPN Phase 3 28 • Supports dynamic Spoke-to-Spoke tunnels •

    They are triggered by the Hub using NHRP, not by routing protocols • All routes on spokes now point to the Hub • Summarization is now allowed on the Hub
  27. 29.

    DMVPN Phase 3 (cont.) 29 • Route to remote spoke’s

    subnet points to the Hub • The packet is sent to the Hub • Hub forwards the packet out of the same tunnel interface towards the spoke, recognizes this and sends NHRP Redirect message to the spoke • NHRP Redirect contains the original packet • Local Spoke now sends Resolution Request querying about destination IP towards the Hub • Hub forwards it to the remote Spoke
  28. 30.

    DMVPN Phase 3 (cont.) 30 • Remote Spoke replies directly

    to the local Spoke with NBMA address for the whole subnet from the RIB • If the NHRP shortcut is enabled, the route with received subnet is installed in the routing table as NHRP (AD 250) or if the same route already exists to the hub, next hop override (nho) is installed into CEF table and is shown with % in the routing table
  29. 31.

    DMVPN Phase 3 configuration 31 • On NHS: (config-if)# ip

    nhrp redirect • On NHC: (config-if)# ip nhrp shortcut • Verification of next hop override: # show ip route next-hop-override
  30. 33.

    DMVPN Phase 3 – EIGRP 33 • Summarization can be

    used on the hub or • Split horizon should be disabled and next-hop-self should be enabled on the hub
  31. 34.

    DMVPN Phase 3 – OSPF 34 • Only point-to-multipoint network

    type is allowed because broadcast changes next hop
  32. 35.

    DMVPN Phase 3 – BGP 35 • You can use

    aggregation on the hub for both eBGP and iBGP otherwise you need to change NH to the hub • For eBGP you need to use the following configuration on the hub: (config-router)# neighbor next-hop-self • For iBGP RR you need to use the following configuration on the hub: (config-router)# neighbor next-hop-self all
  33. 36.

    DMVPN EIGRP Design 36 • EIGRP is the easiest choice

    for DMVPN: • Summarization anywhere • Next hop can be easily changed • Filtering can be performed anywhere • Traffic engineering can be achieved using filtering/offset-lists • Stub feature limits unnecessary queries to the spokes
  34. 37.

    DMVPN OSPF Design 37 • With OSPF you can have

    the following design: • One area 0 in the whole network: in DMVPN cloud, behind the spokes, behind the hub • Area 0 behind the hub; DMVPN cloud + networks behind the spokes in another area • Area 0 is for DMVPN cloud and networks behind the spokes, while the networks behind the hub are in another area • Area 0 is only in DMVPN cloud, all networks behind the spokes and the hub are in their own areas
  35. 38.

    DMVPN OSPF Design (cont.) 38 • OSPF has the following

    issues with DMVPN: • The whole DMVPN cloud should be in one area, because it is one subnet: every link flap on the spoke will be propagated through the whole area. • You can summarize only networks behind the hub if it is ABR • Routing updates (LSA) can’t be filtered within an area • It is hard to enforce traffic engineering in DMVPN + OSPF (one of the tricks is to use network type point-to-multipoint non-broadcast with individual cost per neighbor) • For Phase 2 only broadcast and non-broadcast network types can be used (which introduces risk of misconfiguration of DR priority on the spoke, which can bring the whole network down)
  36. 39.

    DMVPN BGP Design 39 • BGP is also a good

    choice: • Increased routing policy granularity • As a compromise, increased administrative burden • iBGP vs eBGP: • iBGP RR concept perfectly matches Hub and Spoke model • eBGP requires individual AS number per branch, which may be problematic • Depends on requirements • BGP dynamic neighbors feature can be used on the hub: (config-router)# bgp listen [limit max-number] network/length peer-group peer-group-name
  37. 40.

    Front Door VRF DMVPN 40 • What if the route

    towards NBMA address is in VRF RIB? • Front door VRF DMVPN feature comes to the rescue • Instead of crypto isakmp key, configure vrf-aware keyring and let tunnel interface know in which VRF to perform lookup for NBMA address: crypto keyring keyring-name vrf vrf-name pre-shared-key address <subnet> <mask> key <password> ! interface tunnel <num> tunnel vrf <vrf-name> !
  38. 41.

    Per-Tunnel QoS for DMVPN 41 • Different QoS policies are

    defined on the Hub • Spoke requests specific QoS policy using a group string in the Registration Request • Hub applies requested QoS policy on per-spoke basis • Configuration: • on the Spoke: (config-if)# ip nhrp group <group-name> • on the Hub: (config-if)# ip nhrp map group <group-name> service-policy output <qos- policy>
  39. 42.

    Per-Tunnel QoS for DMVPN Verification 42 • # show ip

    nhrp • # show dmvpn detail • # show ip nhrp group-map • # show policy-map multipoint
  40. 43.

    IPv6 over DMVPN 43 • IPv6 over IPv4 underlay (can

    be run simultaneously with IPv4) • IPv4 over IPv6 underlay • IPv6 over IPv6 underlay
  41. 44.

    IPv4 over IPv6 underlay 44 • All NBMA addresses should

    be IPv6 • IKEv2 should be used for IPsec • Tunnel mode should be gre multipoint ipv6
  42. 45.

    IPv6 over IPv4 underlay 45 • IPv6 address configured on

    the tunnel interface • Instead of ip nhrp syntax use ipv6 nhrp • New optional syntax for NHS
  43. 46.

    IPv6 over IPv4 underlay – configuration example 46 interface Tunnel0

    ipv6 address 2001::1/64 ipv6 nhrp network-id 1 ipv6 nhrp authentication cisco ! NHS mapping with 3 commands ipv6 nhrp map 2001::5/128 ipv6 nhrp map multicast ipv6 nhrp nhs 2001::5 ! or with single command ipv6 nhrp nhs 2001::5 nbma multicast ipv6 nhrp shortcut !
  44. 47.

    IPv6 over IPv6 underlay 47 • All NBMA addresses should

    be IPv6 • IKEv2 should be used for IPsec • Tunnel mode should be gre multipoint ipv6 • All other configuration is similar to IPv6 over IPv4 underlay
  45. 48.

    Additional Resources 48 • DMVPN Explained • DMVPN Phase 3

    • NHRP Feature Guide • Cisco DMVPN: Choosing The Right Routing Protocol • Dynamic Multipoint IPsec VPNs (Using Multipoint GRE/NHRP to Scale IPsec VPNs) White Paper • DMVPN Overview • DMVPN Configuration Guide
  46. 49.