MPLS has proven a very successful WAN technology, and is now moving to the metro Its success was not based on the initial drivers for MPLS, but first because of new services (VPNs), and later for convergence and fast restoration The question now is, how much further can (and should) MPLS be taken? Does it make sense to push MPLS to the access? What does this mean? What does it entail? What issues will it raise, and how can they be dealt with?
This talk will try to answer these questions Why is MPLS in the access a Good Thing? What is needed to enable MPLS to the access? What are the major issues anticipated? Are there good answers to these problems? But first, we have to take a big step back, and look at the overall MPLS architecture of the WAN, metro and access, and of services We must overhaul our picture of MPLS networks before we can tackle MPLS in the access
Terminology and Concepts? If MPLS is to grow and apply meaningfully to access networks (and other parts of the network), we absolutely need to decouple the network architecture from the service architecture Network architecture needs to take into account geographical and administrative boundaries, scaling techniques, and network resilience: connectivity Service architecture needs to figure out how, where and when to apply services, how service nodes interact with service helpers, how new services can be brought on-line, how services can scale and be resilient
MPLS Mindset Edge: today in MPLS, we talk of “Provider Edge” routers (PEs) and “Provider only” routers (Ps) … … but there isn’t a clear definition of “edge” – there is a blurring of the “network edge”, “metro/WAN edge”, “IP edge”, “MPLS edge” and “service edge” Resilience: we talk primarily of network resilience We know what to do if a network link or node fails But what if a service node fails? Scaling: we are currently talking of protocol scaling (LDP, RSVP-TE, …) We need to talk of network and service scaling
Connectivity Blueprint Decoupling Service and Network Architectures Blueprint of Service Connectivity Service Helpers Service Node/Helper Placement Factors
from the Network The service edge is independent of other “edges” Layer 2, Layer 3, IP, service, “provider edge”, media- based, metro/WAN, access/metro … these are useful distinctions in many contexts, but not for services Service node placement must primarily be driven by the demands of the service itself, and only secondarily by the exigencies of the network Service quality, optimal deployment, changing needs as a service evolves should dictate Network constraints – physical, geographical, administrative – will play a role, but hopefully minor
Connectivity (from a services point of view) Increasing level of detail AN TN SN SH End device: biz/rez end-user, voice/video/content/cloud server, … SIP SIP
Placement Factors Business and Regulatory Model: • Retail and/or Wholesale? • Content Ownership or Partnership? Value per bit versus Cost per bit: • Policy Control for applications and ASPs • Or best effort transport, as cheap as possible Operation Model: • Single or multiple OSS domains? Service Evolution: • Growth of Unicast Video • New services – gaming, security, web hosted applications • Service take up rate forecasts • Multi service devices – e.g. PS3 as an STB, TV as a phone • Future services, service velocity Specialise or Generalise? Distribute or Centralise?
Why MPLS in the access? Broadband Forum (BBF) work Pseudowires in the access Connectivity paradigms Encapsulations Signaling The primary focus today is on broadband access Last year, the focus was on MPLS for Mobile Backhaul, so we won’t cover that part again. However, some may benefit from reviewing those slides
in the Access? First and foremost, convergence Convergence has been a nice buzz-word, and driven much change in the network, usually for the better Network convergence = common infrastructure, common devices, “de-siloization”, CapEx savings Operational convergence is also important, both for cost efficiencies and as an enabling factor for … … service convergence (FMC, 3screens and beyond), which is perhaps the most important – the ability to offer the all services over all access media
in the Access? (2) Flexible service placement With a uniform MPLS infrastructure, services can be placed wherever they are most effective Especially if this is an explicit design goal If the access, metro and WAN are all based on different technologies, moving a service around would be considerably harder, and may involve shifting boundaries or re-architecting the network Consider the current constrained placement (in many networks) of the “IP edge” at the boundary of metro (Layer 2) and WAN (MPLS)
in the Access? (3) Other benefits Forwarding based on SP-controlled mechanisms rather than subscriber-controlled addresses (MAC, IP) Better scaling (20-bit label vs. 12-bit VLAN) Better resilience mechanisms, also better integrated with service, metro and WAN resilience Dynamic signaling vs. static provisioning Mobile access is quickly migrating to MPLS Overall, a more robust infrastructure Some of this (but not all) can be obtained with Layer 2 access and MPLS pseudowires upstream
the Access: Broadband Forum What is Broadband Forum? Formerly known as DSL-Forum Name changed June 2008 Broadband Forum Mission (www.broadband-forum.org) The Broadband Forum is a worldwide organization committed to rapidly creating specifications for communication service providers and vendors that – Accelerate the development and deployment of broadband networks, Foster successful interoperability, Manage and deliver advanced IP services to the customer.
TR-144: Broadband Multi-Service Architecture & Framework Requirements Deliver multi-play services Deliver them on a converged network WT-145: Multi-service Broadband Network Functional Modules and Architecture WT – “Working Text” - denotes work in progress Will describe the technical requirements needed to meet the business requirements described by TR-144 Will describe MPLS’ role in delivering on the TR-144 requirements
applications (as required by TR-144) Residential Subscriber Services Data Services Video Services VoIP Services Legacy POTS Service Business L2/L3 VPNs Legacy T1/E1 services Mobile Backhauling Wholesale and retail
within WT-145 WT-145 is still a “working text”… but some possible applications of MPLS to help WT-145 reach its goals are: LSPs could be used to separate traffic for individual subscribers (and redirect to wholesale operators) TE-LSPs could help provide bandwidth guarantees for some services PT-MPT LSP could help deliver broadcast video service Pseudo-wire LSPs could help preserve legacy services and assist with mobile backhaul IP/MPLS VPNs and VPLS could deliver VPN services
the Access Pseudowires (PWs) are a logical choice to connect an AN to a SN (and vice versa) An important task of an AN is to translate a physical customer ID (e.g., port number, local loop ID) to a logical one (ATM VP/VC, Ethernet VLAN) This task can be subsumed by an MPLS label However, there are many choices here: Types: point-to-point, point-to-multipoint, any-to-any Encapsulations: PPP, ATM, Ethernet, IP, … The next few slides will examine each of these options as to which are appropriate for AN-SN connections
Connectivity Paradigms Point-to-point PWs: ANs connect only to the SNs that serve them, and vice versa Point-to-multipoint PWs: SNs connect to a number of ANs for multicast traffic (e.g., IPTV) Any-to-any PWs (such as VPLS): ANs and SNs connect to each other in a full mesh Any-to-any connectivity is 1. not needed (outside the connectivity blueprint) 2. not desirable (AN-to-AN connectivity may pose a security hole and/or bypass legal requirements)
Encapsulations There are several options here, and much depends on the data plane capabilities of the access nodes The simpler the encapsulation, the easier the SN’s work (less encap/decap needed) The AN’s work may also be easier Also, there is less confusion in whether the customer ID is carried by the PW label or the L2 header
Encapsulation: P2P The most straightforward option (from the AN’s point of view) is straight Layer 2 over MPLS ATM or Ethernet encaps for ATM/Ethernet DSLAMs If the protocol on the wire is PPPoX, another option is to translate PPPoX to PPPoMPLS at the AN; essentially, use PPP PWs AN to SN This has many benefits, but may be hard for some ANs The simplest encapsulation is IP PWs Strip the entire L2 header from the customer packet and ship the IP payload to the SN with a PW label Only possible for some models of broadband transport
Encapsulation: P2MP The best option here is a P2MP IP PW The IP header carries the multicast group (which might represent an IPTV channel) The AN does further replication to the relevant customer ports based on the IP header Another option is an Ethernet PW Here, the Ethernet header would have a multicast destination MAC address identifying the mcast group However, there is some loss of information in mapping IP multicast groups to Ethernet MAC addresses The AN does replication as an Ethernet switch would
& Signaling for ANs: How? PWs are usually signaled by BGP or targeted LDP However, ANs may not support these protocols for some time On the other hand, provisioning every PW on every AN is a major undertaking The next few slides offer some alternatives
Setup: Option A AN TN SN AN2 Say AN2 uses label 1003 for local loop #3 AN2 sends traffic to TN with label 1003 for sub on local loop 3. TN then pushes label stack <TL, AL>: TL to reach SN, AL to identify AN2. Resultant label stack: <TL, AL, 1003> for sub. For SN, label stack is <TL’, AL’, 1003> for return traffic to same sub Option A: no config/protocol on AN, just a fixed mapping of subs on local loops to labels: LL-k maps to label 1000 + k TN signals to SN a PW per port to AN via BGP or T-LDP. Say TN uses <TL, AL> to reach SN for AN2, and SN uses <TL’, AL’> to reach AN2 AN2 has a static mapping of local loop k to label 1000+k
Setup: Option A If there are multiple services per sub/local loop, the AN may use different label blocks per service E.g., voice on local loop k maps to label 1000 + k video on local loop k maps to label 2000 + k HSI on local loop k maps to label 3000 + k If required, the upstream TN can direct traffic for different services to different SNs (“multi-edge”) based on the label block The TN must establish one PW per AN port per service and map the traffic to the right SN In Option A, the TN is somewhat service-aware
Setup: Option B Option B: AN tells SN via ANCP how many local loop ports it has; SN gives AN a label block (or a block per service) TN pushes label L1 to reach SN Much simpler configuration on upstream TN: just the SN(s) for each connected AN. This option only needs a two-label stack, unlike option A, which needs a 3-label stack TN still involved in setting up AN-SN LSPs ANCP Correlation of sub and PW is achieved simultaneously
Setup: Option C Option C: direct PW signaling (via BGP or T-LDP) from AN to SN • Unlikely that BGP or T-LDP can/will fully replace ANCP; thus, both protocols will be needed between AN and SN • May be some time before ANs are capable of PW signaling • TN here is completely service-unaware • Correlation of sub to PW can be done via BGP communities or PW “site ID” (BGP), or PW “VC ID” (T-LDP)
protected by TE-LSP TE-LSP provides MPLS connectivity between SN and AN (subscriber PWs stack on TE-LSP) MPLS Fast-Reroute ensures rapid service restoration around a failure Protection path backs up LSP in case of failure
Setup Important subject: multicast delivery is a hot topic. However, not easy: P2MP MPLS signaling is relatively new, and P2MP PW signaling even newer One idea is to build a P2MP LSP from an SN to all TNs upstream of ANs, with UHP. These TNs snoop IGMP messages, and deliver native IP or Ethernet packets to ANs MPLS Native IGMP
Setup If AN is capable of P2MP MPLS signaling, it can dynamically join/leave P2MP tree In this case, on the first IGMP join for a group, AN joins P2MP tree. AN does multicast replication from P2MP tree to local loops MPLS IGMP
On Demand Subscriber Initiated – Self Service Application Initiated SRC Service Profile Initiated • Activate on Login • ToD Activated • Volume/Time Controlled Portal Server with SRC- PE portal API • Turbo • Tiered Internet • VoD • Games • Streaming Media • Video Conferencing Network Detection Initiated DPI or IDP Platforms • P2P Controls • Threat Mitigation IMS Service Complex • VoIP • Video Telephony • Multi-media SN SH end device end device
Part of service activation is the verification that the service can reliably be delivered to the subscriber Across devices Reachability Subscriber separation Deliverability/CAC Within the SN Internal resource availability CAC
for Service Delivery Reachability Subscriber access protocols DHCPv4v6/PPP/PPPoE Liveness detection Autosensed subscribers AN-SC Cooperation AN may add context to an access protocol to more clearly identify a subscriber DHCP option 82 PPPoE ACI ANCP may provide additional information Line rate end device end device ANCP
for Service Delivery Subscriber Separation Need to ensure services are only delivered to correct subscribers Need to prevent a subscriber from impacting the service of another subscriber Service Separation At any device where multiple services may merge together toward the same subscriber, that device needs to be able to differentiate between the services Single Edge vs. Multi Edge CoS bits (e.g., 802.1p or EXP) vs separate service identifiers (e.g., VLAN per service) end devices end device ANCP
for Service Delivery Deliverability with QoS Need to ensure the requested service can be delivered with the expected quality E.g., Can’t deliver 2 5Mb video streams to a user with 8Mb access Need to prevent a subscriber from impacting the service of another subscriber E.g., If the aggregate video bandwidth from SN to AN is 10G, don’t allow an additional video flow Essentially need some understanding of the network topology and bandwidths from SN to edge device MPLS can help Resiliancy Bandwidth reservation to AN end devices end device ANCP
Requirements Subscriber Awareness Dbase of authenticated subscriber to logical access point (e.g. VLAN) Service definitions (either locally configured or provided by SH) Relationship to subscribers Required behaviors to be delivered by the SN Topology Awareness Need to know the network topology for services that require network validation before delivery Need routing state… but need not be a “router” SN may deliver L2 services as well as L3 and higher An MPLS aware SN can use MPLS-TE state to evaluate the ability of the network to deliver an additional service end devices end device ANCP
Requirements Service Delivery Capabilities Different services place different requirements on the SN Simple Stateless filters, rate limiters Complex IPTV (Multicast), DPI capabilities such as P2P control Local CAC Any SN must provide per-subscriber/per-service accounting end devices end device ANCP
the Metro: Adds resiliency to the metro network… but typically doesn’t extent to the AN or SN (requiring additional VLAN config with mappings to LSPs) Typical Metro/Access Network Evolution end devices ANCP AN TN SN SH Metro 1 WAN Subscriber separation via VLANs (stacked/ unstacked) Metro MPLS
the Metro: Extending MPLS from SN to AN provides additional resiliency, removes the VLAN configuration overhead, and retains the characteristics needed to deliver per- subscriber services Typical Metro/Access Network Evolution end devices ANCP AN TN SN SH Metro 1 WAN Subscriber separation via LSPs Metro MPLS
Networks Mobile services WAN RAN Wireless Access Mobile Core MPLS MPLS enables a transition away from TDM/ATM access networks allowing service delivery over a common infrastructure (e.g., Ethernet)
PWs deliver subscribers from AN Subscribers association with VPN by ANCP info ANCP Subscriber line id can replace VLAN as an identifier to associate a subscriber with a VPN Traditional association via DHCP/PPPoE Subscriber traffic delivered via PWs end devices ANCP L2 L3
Convergence Different service-specific networks may have a common infrastructure element… in MPLS May create opportunity to converge those networks
Scaling Domains Scaling a network to 10-100K nodes requires some form of routing hierarchy Typically, this is achieved via IGP areas (OSPF/ISIS) or via Autonomous Systems (BGP) This choice is left to each individual Service Provider There is good visibility within a domain, both for IP routing and for MPLS connectivity To scale IP routing, the loopbacks of nodes in each domain should be summarizable Within a domain, one may choose to deploy LDP or RSVP-TE, or a combination of both
Scaling Regions Scaling a network to 10-100K nodes requires some form of routing hierarchy Typically, this is achieved via IGP areas (OSPF/ISIS) or via Autonomous Systems (BGP) This choice is left to each individual Service Provider There is good visibility within a region, both for IP routing and for MPLS connectivity To scale IP routing, the loopbacks of nodes in each region should be summarizable Within a region, one may choose to deploy LDP or RSVP-TE, or a combination of both
Scaling (2) Inter-region At the edge of a region is a set of “Region Border Routers” (or Border Node (BN)) – either ABRs or ASBRs, depending on the choice of region types A BN is just a Transport Node at a region boundary BNs provide IP connectivity across regions BNs run “labeled” BGP (RFC 3107) to glue intra- region LSPs to form inter-region LSPs Basic idea: LDP and/or RSVP-TE LSPs remain strictly within a region. The only way out of a region for MPLS connectivity is via labeled BGP
Scaling (3) The addition of regions and the need for inter- region LSPs means that SNs may belong to different regions. This requires a modification to the connectivity blueprint, as follows: Intra-region: AN1 ⇔ SN1 ⇔ SN2 ⇔ AN2 One region hop: AN1 ⇔ SN1 ⇔ BN ⇔ SN2 ⇔ AN2 Two region hops: AN1 ⇔ SN1 ⇔ BN1 ⇔ BN2 ⇔ SN2 ⇔ AN2 region 1 region 2 region 3
on BGP: Infrastructure vs. Service The BGP used on BNs is considered infrastructure BGP. Contrast this with the BGP used on SNs – service BGP Infrastructure BGP is for network connectivity and other infrastructural purposes. It is not used for services, and is independent of customers The scale and churn related to infrastructure BGP is small, as are the demands on CPU and memory Service BGP is for enabling Internet, VPNs and other services, and is directly related to customer (or peering) needs Here, the scale is large and processing demands high
Sizing Network dimensioning Let’s say there are 101 regions in the network: 100 metros, and one WAN joining them Each metro has 500 nodes: 20 SNs, with 40 ANs, each dual-homed to a pair of SNs (for a total of 400 ANs), and 80 TNs The WAN has 20 SNs, 200 BNs and 30 TNs Thus, there are 200 BNs, 2,020 SNs, 8,030 TNs, and 40,000 ANs, for a total of 50,250 nodes in the network These numbers are just an example, for back-of-the- envelope computations and sanity-checking
MPLS Connectivity Number of LSPs Keep in mind the connectivity blueprint Intra-region LSPs Each node has 500 entries in its LFIB Inter-region LSPs = number of SNs = 2,020 labeled BGP routes These are maintained on SNs and BNs only Other inter-region LSPs are not needed The most state that a node (BN) has is ~4,000 entries in its LFIB, far short of 50,000 that one might expect Most nodes (ANs & TNs) in fact have only 500 entries
MPLS Connectivity Note that RSVP-TE LSPs push the limits of scalability much more than LDP LSPs The following LSPs are needed: AN-to-SN LSPs (40 ANs per SN) * 20 SNs * 2 (bidir) = 1600 per metro SN-to-BN LSPs: 20*2*2 = 80 per metro SN-to-SN (intra-metro) LSPs: 20*19 = 380 per metro BN-to-BN LSPs: 2002 = 40,000 LSPs (in WAN) Any other LSPs are not needed, even unwanted The total number of desired LSPs is far short of the 50,0002 = 2.5 billion that one might naively expect
State for a Node Number of infrastructure BGP labeled routes: Routes are needed only for SNs and BNs: 2,020 Churn occurs here only when a new SN is brought on line, moved or removed, or when BNs are added or moved (rare), or when SNs or BNs fail/recover There is tremendous room for growth! This will be needed as services become more distributed State on a SN in a metro: RSVP-TE: 61*2 = 122 LSPs LDP: 500 LSPs Infrastructure BGP on SNs: 2,020 labeled routes
Connectivity OSS operations, SNMP, even telnet, all use IP … … as do Service Helpers and service signaling May be some day these will all be MPLS-enabled, but for now, we’re assuming that these are IP-based We don’t assume a separate network for maintenance, so we need to look at IP scaling If the loopbacks in each region are nicely summarizable, then each region will have 500 loopbacks plus 100*(small number of prefixes) It is highly recommended that nodes have unnumbered interfaces, and that these are not advertised in the IGP
marschen2007
mza
baasie
nikkihalliwell
marcelosomers
jonoalderson
chriscoyier
greggifford
stephaniewalter
seathinner
mraible
colly
![Thank You! [email protected] [email protected]](https://files.speakerdeck.com/presentations/6c495e502a8c01305bdc22000a9d07cb/slide_63.jpg){kind=link}