Enterprise QoS Workshop - Interop NYC 2013

This workshop discusses QoS fundamentals, describes an enterprise QoS strategy for converged infrastructure, and walks through code examples generated on Cisco routers & switches.

Read the blog posts I wrote in preparation for this presentation here: http://ethancbanks.com/tag/enterprise-qos/

Transcript

1. Enterprise QoS Delivering Collaboration Applications End-to-End Ethan Banks, CCIE #20655 Packet Pushers Podcast Host http://packetpushers.net
2. Introduction – 1 Who am I? •  Ethan Banks, CCIE #20655. •  Real-life network architect. •  Co-host of the Packet Pushers Podcast. •  Freelance writer for Network Computing, Network World, TechTarget.
3. Introduction – 2 What is this workshop all about? Delivering collaboration applications across a converged network using QoS to improve timely packet delivery.
4. Introduction – 3 How will we cover the information? In three hours, we’ll… •  Discuss QoS fundamentals •  Design a QoS strategy •  Review code that implements QoS
5. Introduction – 4 Logistical comments. •  This is a 3 hour workshop. Coffee break is from 10:30a – 10:45a at the Conference Club. •  After this session, lunch is from 12:00p – 1:30p on the South Concourse. •  If you need to take a break, please do that. •  If you want to ask a question, do that as well.
6. Introduction – 5 Who is the target audience for this workshop? •  The network engineer who has a good handle on networking & IP fundamentals. •  Competency with Cisco switches & routers will help, but is not required.
7. What is QoS? – 1 What is Quality of Service? •  Quality of service is a group of tools that form a traffic delivery policy.
8. What is QoS? – 2 Why do engineers hate QoS? •  Confusing terms •  Confusing use-cases •  Inconsistent syntax
9. What is QoS? – 3 What problems can QoS solve? •  Poor voice call quality. •  Poor video call quality. •  Bandwidth hogs.
10. What is QoS? – 4 What problems can QoS *not* solve? •  Undersized pipes. •  Physical layer issues. •  Problems in a provider’s cloud. •  Bad network design. •  High latency.
11. Why QoS? – 1 What QoS problem does an enterprise typically have? •  All applications are on the same network, but require different network characteristics to work right. •  Most of the time vs. all of the time.
12. Why QoS? – 2 Why do voice & video need “kid glove” treatment? •  Real-time delivery is key. •  TCP vs. UDP protocol behavior.
13. Why QoS? – 3 Do non-collaboration applications need QoS? •  Maybe. It depends. •  Control plane policing protects network devices. •  How “mission critical” is a mission critical app? •  The lack of infinite bandwidth.
14. QoS = packet delivery at all costs…? – 1 Isn’t packet delivery the most important thing? •  Absolutely not. TIMELY packet delivery is the most important thing.
15. QoS = packet delivery at all costs…? – 2 So, oversized buffers are bad? •  Yes! Let’s talk about bufferbloat & TCP.
16. QoS = packet delivery at all costs…? – 3 Shouldn’t voice traffic be TCP so we’re sure it gets there? •  Not really…because if it arrives too late, what was the point of delivering it? •  Delivery “whenever” doesn’t work for real-time traffic.
17. QoS Terminology – 1 ToS •  ToS stands for “type of service.” The ToS value is stored in a byte of the IP header of a packet. The 8 bits making up the byte are broken into two fields. In modern networks, the first 6 bits represent the Differentiated Service Code Point (DSCP) value, while the last two set Explicit Congestion Notification (ECN) values.
18. QoS Terminology – 2 PHB •  PHB stands for “per hop behavior.” PHBs are RFC- defined collections of generally agreed upon ToS values. RFC2597 defines the “Assured Forwarding PHB Group”, while RFC2598 defines “An Expedited Forwarding PHB”.
19. QoS Terminology – 3 Interface •  Usually the physical port on a network device that accepts or sends traffic in the form or electrical or optical signals. It is possible to apply QoS policies to virtual or aggregated interfaces. There are QoS nuances that apply to virtual & aggregated interfaces as compared to physical interfaces.
20. QoS Terminology – 4 Transmit ring •  Contains a packet ready to be sent across the wire. If the Tx-ring already has something in it, then the physical interface is already sending something (i.e. is congested). If the Tx-ring is empty, the physical media is ready to send something (i.e. is not congested).
21. QoS Terminology – 5 Ingress vs. egress •  Terms of directionality referring to traffic, and are always in the context of an interface. “Ingress” traffic refers to traffic that is flowing into an interface. “Egress” traffic refers to traffic that is flowing out of an interface.
22. QoS Terminology – 6 Queueing •  If a queue is a waiting line, then QoS queuing is the process of handling backed-up packets (i.e. putting them in a waiting line, so to speak). Queueing only happens when the interface is too busy to send all the packets that have arrived at the device.
23. QoS Terminology – 7 Buffer •  The area in which queued packets wait. The larger the buffer, the more packets that can be held. Buffers should be sized appropriately to handle the type of application traffic they are intended to service.
24. QoS Terminology – 8 Congestion management •  A set of QoS tools that help determine what packets get sent in what order when an interface is congested (i.e. has more traffic to send than the interface bandwidth is capable of carrying). The idea here is, “Okay, we have a problem. How are we going to handle this problem?”
25. QoS Terminology – 9 Congestion avoidance •  QoS tools that try to prevent an interface from becoming congested by proactively dropping packets. The idea here is, “We’re about to have a problem. What can we do to avoid this becoming a problem?”
26. QoS Terminology – 10 Class-Based Weighted Fair Queueing •  Weighted fair queueing allows a network device to identify flows on the wire and share bandwidth equally among the flows. Class-based WFQ takes this logic one step further by allowing the network operator to identify specific traffic classes and instruct the network device via how bandwidth should be allocated to those classes.
27. QoS Terminology – 11 Low Latency Queueing •  High priority queue that preempts other traffic classes without starving them for bandwidth. •  An LLQ reserves bandwidth for a traffic class •  An LLQ also limits the amount of bandwidth that traffic class can take. •  An LLQ will dequeue packets on a regular time-slice to minimize jitter.
28. QoS Terminology – 12 Weighted Random Early Detection •  When an interface is congested, the WRED algorithm can be applied to drop traffic from certain flows. The point of WRED is to reduce interface congestion by encouraging TCP flows to slow down through drops.
29. QoS Terminology – 13 TCP Global Synchronization •  Occurs when backed up packets from all flows arriving at a congested interface are dropped at the same time. The TCP behavior of these flows then become synchronized, as all of them slow down and ramp back up at the same time.
30. QoS Terminology – 14 Shaping vs. policing •  Rate limiters. •  Shaping has a buffer that allows for bursts of traffic with fewer drops. •  Policing offers no buffer; traffic that exceeds the policed rate is dropped.
31. QoS Terminology – 15 Cisco NBAR •  Network Based Application Recognition is a way to identify traffic flows from specific applications such as HTTP, HTTP, FTP, etc. transiting a network device.
32. QoS Terminology – 16 Cisco MQC •  The Modular QoS Command Line Interface is a way to describe a QoS policy in a common was across many Cisco platforms. The MQC policy language is used to create traffic classes with class maps. Policy maps create QoS policies that inform the network device how to handle the traffic class identified by the class map. Policy maps are applied to interfaces.
33. QoS Traffic Marking – 1 What does it mean to “mark” traffic? •  Embedding a value in the L2 and/or L3 headers of traffic. •  L2 = the 3 802.1p “CoS” bits in the 802.1q header •  L3 = first 6 bits of the ToS byte in an IP header, making a DSCP value
34. QoS Traffic Marking – 2 Why do we need L2 & L3 marks? •  Because Ethernet frames and IP packets are different things.
35. QoS Traffic Marking – 3 Where should marking be done? •  As close to the source as possible (and by the source itself if possible). •  But let’s make sure we trust the traffic source.
36. QoS Traffic Marking – 4 Once marked, always marked, right? •  Probably not. It’s a matter of trust. •  Where are your trust boundaries? •  Do you know if your switches trust or strip marks by default? •  CoS values should be mapped to DSCP values. Not all media is Ethernet.
37. QoS Traffic Marking – 5 Is marking really that big a deal? I can classify in other ways. •  Marking brings consistency. •  Marking avoids transient network attributes like NAT, tunneling, and alternate media types.
38. Cisco AutoQoS - Introduction What is Cisco’s AutoQoS? •  A QoS template available on many Cisco devices to handle common enterprise needs. •  An automation tool. •  A good place to start, but perhaps not perfect. •  Getting a little long in the tooth? •  A handy introduction to queueing concepts on Catalyst switches.
39. Cisco AutoQoS - Global Commands - 1 mls qos •  Enable QoS features on this switch. They are not enabled by default.
40. Cisco AutoQoS - Global Commands - 2 mls qos map policed-dscp 0 10 18 24 46 to 8 •  In general, “mls qos map” commands translate one type of mark to another type of mark. •  In this command, DSCP values of 0, 10, 18, 24, and 46 are changed to 8 when traffic with those marks exceeds a specific rate limit defined by a policer.
41. Cisco AutoQoS - Global Commands - 3 mls qos map cos-dscp 0 8 16 24 32 46 48 56 •  This maps CoS values to DSCP values, and the numbers are positionally significant. CoS 0 = DSCP 0, CoS 1 = DSCP 8, CoS 2 = DSCP 16, etc.
42. Cisco AutoQoS – Global Commands - 4 mls qos srr-queue input bandwidth 70 30 •  Bandwidth is allocated in this command with weights. A weight of 70 is assigned to queue 1, and a weight of 30 is assigned to queue 2. Note that the values do NOT have to add up to 100. These values are *weights* and not percentages. That said, it is common to have the bandwidth queue assignments equal a percentage, as that is more human-readable.
43. Cisco AutoQoS – Global Commands - 5 mls qos srr-queue input threshold 1 80 90 •  Setting the threshold below tells the switch at what point it should start tail-dropping queued packets and from which traffic-class.
44. Cisco AutoQoS – Global Commands - 6 mls qos srr-queue input cos-map queue 1 threshold 2 3 mls qos srr-queue input cos-map queue 1 threshold 3 6 7 mls qos srr-queue input cos-map queue 2 threshold 1 4 •  The commands group maps specific traffic marks into specific thresholds for ingress queues, as outlined previously.
45. Cisco AutoQoS – Global Commands - 7 mls qos srr-queue input dscp-map queue 1 threshold 2 24 mls qos srr-queue input dscp-map queue 1 threshold 3 48 49 50 51 52 53 54 55 mls qos srr-queue input dscp-map queue 1 threshold 3 56 57 58 59 60 61 62 63 mls qos srr-queue input dscp-map queue 2 threshold 3 32 33 40 41 42 43 44 45 mls qos srr-queue input dscp-map queue 2 threshold 3 46 47 •  In this group of commands, packets with DSCP mark 24 are mapped into threshold 2 of ingress queue 1. DSCP marks 48-55 are mapped to threshold 3 of ingress queue 1. Etc.
46. Cisco AutoQoS – Global Commands - 8 mls qos srr-queue input priority-queue 2 bandwidth 30 • The “priority-queue” element here is important, as it defines very specific dequeueing behavior for the traffic that ends up in this queue.
47. Cisco AutoQoS – Global Commands - 9 mls qos srr-queue output cos-map queue 1 threshold 3 4 5 mls qos srr-queue output dscp-map queue 1 threshold 3 46 47 •  There are several more CoS & DSCP map value commands. •  Note that queue 1 is optionally a “priority queue”.
48. Cisco AutoQoS – Global Commands - 10 mls qos queue-set output 1 threshold 1 100 100 50 200 mls qos queue-set output 1 threshold 2 125 125 100 400 mls qos queue-set output 1 threshold 3 100 100 100 400 mls qos queue-set output 1 threshold 4 60 150 50 200 •  These output (egress) threshold commands establish the tail drop behavior for queues as they fill up with different classes of traffic. Note the keyword “queue- set”. You can define two different queue sets, and then apply one queue-set to one interface, and a different one to a different interface, resulting in different queueing behavior.
49. Cisco AutoQoS – Global Commands - 11 mls qos queue-set output 1 buffers 15 25 40 20 •  This command allocates what percentage of available buffer space will be allocated to which queues.
50. Cisco AutoQoS – MQC Commands - 1 ip access-list extended AUTOQOS-ACL-DEFAULT permit ip any any ! class-map match-all AUTOQOS_VOIP_DATA_CLASS match ip dscp ef class-map match-all AUTOQOS_DEFAULT_CLASS match access-group name AUTOQOS-ACL-DEFAULT class-map match-all AUTOQOS_VOIP_SIGNAL_CLASS match ip dscp cs3
51. Cisco AutoQoS – MQC Commands - 2 policy-map AUTOQOS-SRND4-CISCOPHONE-POLICY class AUTOQOS_VOIP_DATA_CLASS set dscp ef police 128000 8000 exceed-action policed-dscp-transmit class AUTOQOS_VOIP_SIGNAL_CLASS set dscp cs3 police 32000 8000 exceed-action policed-dscp-transmit class AUTOQOS_DEFAULT_CLASS set dscp default police 10000000 8000 exceed-action policed-dscp-transmit
52. Cisco AutoQoS – Interface Commands interface GigabitEthernet1/0/1 srr-queue bandwidth share 1 30 35 5 priority-queue out mls qos trust device cisco-phone mls qos trust cos auto qos voip cisco-phone service-policy input AUTOQOS-SRND4-CISCOPHONE-POLICY
53. Cisco AutoQoS – Parting Thoughts AutoQoS fixes everything, right? •  Reminder that some complain it’s getting a bit old. •  Mixed environments might have mixed AutoQoS results. •  AutoQoS makes deployment easier, but not troubleshooting.
54. QoS At The WAN Edge - 1 What does a provider do with my QoS policy? •  By default, probably nothing. •  Their marks might not be your marks. •  They’ll have their own scheme you can use, probably at a cost. •  They might not have as many classes as you.
55. QoS At The WAN Edge - 2 How do I use my provider’s QoS? •  Enable QoS functionality on the circuit. •  Mark your traffic the way the provider expects.
56. QoS At The WAN Edge - 3 How do I mark my traffic so that the provider will recognize it correctly? •  Probably IP Precedence is in use. •  DSCP to IP PREC is backwards compatible, but beware of collisions.
57. QoS At The WAN Edge - 4 What do you mean, “collisions”? BINARY | DECIMAL | PHBs 000+(000-111) | DSCP 00-07 = PREC 0 | Default, best effort 001+(000-111) | DSCP 08-15 = PREC 1 | CS1, AF11, AF12, AF13 010+(000-111) | DSCP 16-23 = PREC 2 | CS2, AF21, AF22, AF23 011+(000-111) | DSCP 24-31 = PREC 3 | CS3, AF31, AF32, AF33 100+(000-111) | DSCP 32-39 = PREC 4 | CS4, AF41, AF42, AF43 101+(000-111) | DSCP 40-47 = PREC 5 | CS5, EF 110+(000-111) | DSCP 48-55 = PREC 6 | CS6 111+(000-111) | DSCP 56-63 = PREC 7 | CS7
58. QoS At The WAN Edge – MQC Policy - 1 class-map match-any WAN-PROVIDER_P1 description REAL-TIME VOICE match precedence 5 class-map match-any WAN-PROVIDER_P2 description STREAMING VIDEO match precedence 4 6 7 class-map match-any WAN-PROVIDER_P3 description CALL SIGNALLING match precedence 2 3 class-map match-any WAN-PROVIDER_P4 description BEST EFFORT match precedence 0 1
59. QoS At The WAN Edge – MQC Policy - 2 policy-map OUTBOUND class WAN-PROVIDER_P1 priority 300 class WAN-PROVIDER_P2 bandwidth 750 class WAN-PROVIDER_P3 bandwidth 40 class class-default fair-queue
60. QoS At The WAN Edge – MQC Policy - 3 interface GigabitEthernet0/1 bandwidth 10000 ip address 10.11.12.2 255.255.255.252 service-policy output OUTBOUND
61. QoS At The WAN Edge – MQC Nested Policy - 1 How can we handle QoS for a pipe that’s throttled downstream? •  If traffic is shaped downstream, the local interface will never be congested. •  Solution? Create artificial congestion with a shaping policy. Nest the prioritization policy inside it.
62. QoS At The WAN Edge – MQC Nested Policy - 2 policy-map PRIORITIZE class WAN-PROVIDER_P1 priority 300 class WAN-PROVIDER_P2 bandwidth 750 class WAN-PROVIDER_P3 bandwidth 40 class class-default fair-queue
63. QoS At The WAN Edge – MQC Nested Policy - 3 policy-map OUTBOUND class class-default shape average 100000000 service-policy PRIORITIZE ! interface GigabitEthernet0/1 bandwidth 100000 ip address 10.11.12.2 255.255.255.252 service-policy output OUTBOUND
64. QoS Packet Walk – Introduction - 1 What does a QoS policy accomplish if an interface is not congested? •  Congestion management tools – nothing. •  Congestion avoidance tools – nothing. •  Rate limiters (policers & shapers) do their job. •  Marking policies do their job.
65. QoS Packet Walk – Introduction - 2 Packet Walk – Key Elements – Marking TRAFFIC | IP TOS BYTE | 802.1Q/802.1p Real-Time Voice | DSCP 46 (EF) | CoS 5 Call Signaling (Control) | DSCP 24 (CS3) | CoS 3 Video | DSCP 34 (AF41) | CoS 4
66. QoS Packet Walk – Introduction - 3 Packet Walk – Key Elements – Traffic Path 1.  PHONE 1 2.  ACCESS SWITCH 1 3.  CORE SWITCH 1 4.  WAN ROUTER 1 5.  PROVIDER CLOUD 6.  WAN ROUTER 2 7.  CORE SWITCH 2 8.  ACCESS SWITCH 2 9.  PHONE 2
67. QoS Packet Walk – Uncongested - 1 Step 1 - Phone1 marks traffic before it sends it into the switch. Step 2 - Marked traffic arrives at the switch on the ingress port. Marking policy is applied. Traffic is switched to the egress port and forwarded.
68. QoS Packet Walk – Uncongested - 2 Step 3 – Core Switch 1 forwards traffic to WAN Router 1. Marks are trusted and preserved. Step 4 – Traffic is forwarded into the provider cloud.
69. QoS Packet Walk – Uncongested - 3 Step 5 – The provider honors our marked traffic and transports it across the cloud appropriately. Step 6 – WAN Router 2 receives the traffic and forwards it to Core Switch 2.
70. QoS Packet Walk – Uncongested - 4 Step 7 – Core Switch 2 receives the traffic, trusts DSCP values (might map to CoS values), and forwards. Step 8 – Access Switch 2 receives the traffic, trusting marks, forwards to the egress port (the one connected to Phone 2).
71. QoS Packet Walk – Uncongested - 5 Step 9 – Phone 2 receives the traffic and makes a noise in someone’s ear.
72. QoS Packet Walk – Congested - 1 We will introduce two points of congestion. 1.  The WAN router uplink to the cloud. 2.  The uplink to the phone. Let’s skip the steps that are the same as the “uncongested” packet walk…
73. QoS Packet Walk – Congested - 2 Step 4 – WAN router uplink is congested, traffic queues. •  IP precedence 5 traffic will be serviced by LLQ. •  IP precedence 3 & 4 traffic will be services by CBWFQ. •  Everything else will be fair-queued.
74. QoS Packet Walk – Congested - 3 Step 8 – Egress interface of Access Switch 2 is congested, traffic queues. •  Traffic marked DSCP EF (46) will be mapped to output queue 1. This is a priority queue. •  Traffic marked AF41 (34) will be mapped to output queue 2. •  Traffic marked CS3 (24) will be mapped to output queue 2 as well. •  Unmarked traffic will be mapped to queue 3.
75. QoS Collaborative Q & A Q & A
76. Stay In Touch! Where you can find me… •  http://PacketPushers.net Podcast (iTunes) •  http://ethancbanks.com Blog •  http://NetworkComputing.com Blog •  @ecbanks Twitter •  ethan.banks@packetpushers.net E-mail •  Also LinkedIn & Google Plus