to Meter Band. (Change between OF1.3.3/1.3.4) • 2024/11/21 • clean up to make this public on Speaker Deck • https://speakerdeck.com/ebiken OpenFlow Specification Summary | 2024/11/21 | @ebiken 2
5 OpenFlow Tables 6 OpenFlow Channel and Control Channel 7 The OpenFlow Switch Protocol Appendix A: Header file openflow.h Appendix B: Release Notes OpenFlow Specification Document Structure (1.3.4) a. Components of OpenFlow Switch. c. Overview of OpenFlow protocol between Switch and Controller. b. Packet handling, table structure and other components. (i.e. group, meter, counter etc.) d. Protocol format and structures. (Implementation & debugging details.) OpenFlow Specification Summary | 2024/11/21 | @ebiken 4
to OpenFlow Switch. • Could be Physical or Logical. • OpenFlow-hybrid switch can have some ports assigned as OpenFlow ports, and others as normal (L2/L3) switch ports. • Standard / Reserved ports. • Standard Ports • Can be used as both ingress and output ports. • Difference of Logical port against Physical port. • Could have extra pipeline field called Tunnel-ID associated with it. • Both it’s logical and underlying physical port are reported when sent to the controller. a. Components of OpenFlow Switch OpenFlow Specification Summary | 2024/11/21 | @ebiken 7
generic forwarding actions. • Example: • Sending to controller. • Flooding. • Forwarding using non-OpenFlow methods. • Optional ports • Not required to support Optional ports. • OpenFlow-only switches do not support the NORMAL and the FLOOD port. • Flooding • Forwarding packets to the FLOOD port depends on switch implementation and configuration. • Using a group of type all enables the controller to more flexibly implement flooding. a. Components of OpenFlow Switch OpenFlow Specification Summary | 2024/11/21 | @ebiken 8
be implemented. • ALL (output) • Copy of the packet is sent to all standard ports. (Excluding the packet ingress port and ports configured OFPPC_NO_FWD) • CONTROLLER (ingress | output) • Represents control channel with the Controllers. • Output: Encapsulated in packet-in message. • Ingress: Identifies packet originating from the controller. • TABLE • Represents the start of OpenFlow pipeline. • Only valid in an output action in the list of actions of packet-out message. • IN_PORT (output) • Represents the packet ingress port. • Output: Send the packet through its ingress port. • ANY • Used in some OpenFlow requests when no port is specified. (i.e. Port is wild carded.) a. Components of OpenFlow Switch OpenFlow Specification Summary | 2024/11/21 | @ebiken 9
to be implemented. • OpenFlow-only switches do not support the NORMAL and the FLOOD port. • LOCAL (ingress | output) • Represents the switch's local networking stack and its management stack. • Enables remote entities to interact with the switch via the OpenFlow network, rather than via a separate control network. • Could be used for in-band controller connection. • NORMAL (output) • Represents forwarding using the traditional non-OpenFlow pipeline of switch. • OpenFlow switch must indicate it doesn’t support this action, if it cannot forward packets from OpenFlow to normal pipeline. • FLOOD (output) • Represents flooding using the traditional non-OpenFlow pipeline of the switch. • Send packet to all ports except for ingress nor ports in OFPPS_BLOCKED state. • The switch may also use the packet VLAN ID or other criteria to select which ports to use for flooding. a. Components of OpenFlow Switch OpenFlow Specification Summary | 2024/11/21 | @ebiken 10
• Via OpenFlow Configuration protocol. • Port Config bits in Port Structures. (7.2.1) • Useful for the controller to implement protocols such as STP or BFD. • enum ofp_port_config • OFPPC_PORT_DOWN • Port is administratively down. • OFPPC_NO_RECV • Drop all packets received by port. • OFPPC_NO_FWD • Drop packets forwarded to port. • OFPPC_OFPPC_NO_PACKET_IN • Do not send packet-in msgs for port. • Port state change by the Switch • Change based on the underlying port mechanism. (ex: link down) • Switch sends an OFPT_PORT_STATUS message to notify the controller. • Port change impact to flow tables • Never changes the content of the flow tables. • Packet forwarded to non-existent ports are just dropped. • Port change impact to group tables • Never change the content of the group tables. • Behavior of some group may change through liveness checking. • Flow entries / Group entries • The controller should clean up any flow/group entries referencing removed port if needed. • Port number could be later reused for different physical/logical port. a. Components of OpenFlow Switch OpenFlow Specification Summary | 2024/11/21 | @ebiken 11
Pipeline and components • Flow Table / Flow Entry • Match Fields, Instructions • Action Set / Actions • Group Table • Meters Table • Counters • Types of OpenFlow Switch • OpenFlow-only • OpenFlow-hybrid • OpenFlow-hybrid • Support both OpenFlow operation and normal Ethernet switching operations. • Traditional L2 Ethernet switching • VLAN isolation • L3 routing (IPv4, IPv6 etc.) • ACL and QoS processing • Classification mechanism to select normal or OpenFlow pipeline is outside the scope of this specification. • Could be classified by port, VLAN ID etc. OpenFlow Specification Summary | 2024/11/21 | @ebiken 13
contains one or more flow tables. • Flow tables are sequentially numbered, starting from 0. • Each flow table contains multiple flow entries. • Flow entry • Includes “Match Fields” matched against packets. • “Instruction set” included is executed if matched. • Goto-Table Instruction • Will direct the packet to another flow table. • Only to a flow table number which is greater than it’s own. • (i.e. pipeline processing can only go forward and not backward.) • If the matching flow entry does not direct packets to another flow table, pipeline processing stops. Note: It would not automatically move to the next table without “Goto-Table” Instruction. • Action set • Associated with each packet and executed when exiting from pipeline. • A flow entry can modify action set using Write-Action or Clear-Action instruction. b. Packet handling, table structure and other components. OpenFlow Specification Summary | 2024/11/21 | @ebiken 15
• Identified by “Match Fields” and “Priority”. • Table-miss flow entry • Wildcards all fields. (all fields omitted) • Priority = 0. • More details in later slide. • Switch with Multiple Tables. • Different flow tables may not support the same subset of Match Fields nor Instructions. • This means SW can have different ASIC suited for match / instructions on each table. • Controller can discover what each table supports with “Table Features Request”. b. Packet handling, table structure and other components. Match Fields Priority Counters Instructions Timeouts Cookie Flags Main components of a flow entry in a flow table. (5.2 Table 1) • match fields • to match against packets. These consist of the ingress port and packet headers, and optionally other pipeline fields such as metadata specified by a previous table. • priority: • matching precedence of the flow entry. • counters: • updated when packets are matched. • instructions: • to modify the action set or pipeline processing. • timeouts: • maximum amount of time or idle time before flow is expired by the switch. • cookie: • opaque data value chosen by the controller. May be used by the controller to filter flow entries affected by flow statistics, flow modification and flow deletion requests. Not used when processing packets. • flags: • flags alter the way flow entries are managed, for example the flag OFPFF_SEND_FLOW_REM triggers flow removed messages for that flow entry. OpenFlow Specification Summary | 2024/11/21 | @ebiken 17
and other components. OF1.3.4 name (*) Bytes OXM_OF_IN_PORT YES 4 OXM_OF_IN_PHY_PORT no 4 OXM_OF_METADATA no 8 OXM_OF_ETH_DST YES 6 OXM_OF_ETH_SRC YES 6 OXM_OF_ETH_TYPE YES 2 OXM_OF_VLAN_VID no 2 OXM_OF_VLAN_PCP no 1 OXM_OF_IP_DSCP no 1 OXM_OF_IP_ECN no 1 OXM_OF_IP_PROTO YES 1 OXM_OF_IPV4_SRC YES 4 OXM_OF_IPV4_DST YES 4 OXM_OF_TCP_SRC YES 2 OF1.3.4 name (*) Bytes OXM_OF_TCP_DST YES 2 OXM_OF_UDP_SRC YES 2 OXM_OF_UDP_DST YES 2 OXM_OF_SCTP_SRC no 2 OXM_OF_SCTP_DST no 2 OXM_OF_ICMPV4_TYPE no 1 OXM_OF_ICMPV4_CODE no 1 OXM_OF_ARP_OP no 2 OXM_OF_ARP_SPA no 4 OXM_OF_ARP_TPA no 4 OXM_OF_ARP_SHA no 6 OXM_OF_ARP_THA no 6 OXM_OF_IPV6_SRC YES 16 OF1.3.4 name (*) Bytes OXM_OF_IPV6_DST YES 16 OXM_OF_IPV6_FLABEL no 4 OXM_OF_ICMPV6_TYPE no 1 OXM_OF_ICMPV6_CODE no 1 OXM_OF_IPV6_ND_TARGET no 16 OXM_OF_IPV6_ND_SLL no 6 OXM_OF_IPV6_ND_TLL no 6 OXM_OF_MPLS_LABEL no 4 OXM_OF_MPLS_TC no 1 OXM_OF_MPLS_BOS no 1 OXM_OF_PBB_ISID no 3 OXM_OF_TUNNEL_ID no 8 OXM_OF_IPV6_EXTHDR no 2 (*) Required in PF1.3.4? OpenFlow Specification Summary | 2024/11/21 | @ebiken 18
other components. (*) Bytes Per Flow Table 20 Reference count (active entries) YES 4 Packet Lookups no 8 Packet Matches no 8 (*) Required in PF1.3.4? (*) Bytes Per Flow 24 Received Packets no 8 Received Bytes no 8 Duration (seconds) YES 4 Duration (nanoseconds) no 4 (*) Required in PF1.3.4? See slide “Counters: List of all counters” for all counters. OpenFlow Specification Summary | 2024/11/21 | @ebiken 19
structure and other components. (*) Required in PF1.3.4? Instruction (*) Description Meter no Direct packet to the specified meter. Apply-Actions no Applies the specific action(s) immediately, without any change to the Action Set. This instruction may be used to modify the packet between two tables or to execute multiple actions of the same type. Clear-Actions no Clears all the actions in the action set immediately. Write-Actions YES Add specified set of action(s) into the current action set. Overwrite it if an action of the given type exists. Write- Metadata no Write the masked metadata value into the metadata field. (new metadata = old metadata & ˜mask | value & mask) Goto-Table YES Indicates the next table. The table-id must be greater than the current table-id. Action Description Output <port_no> Group <group_id> Set-Queue <queue_id> Push-Tag <ethertype> MPLS,PBB,VLAN Pop-Tag <ethertype> Set-Field <field_type> <value> All the Header Fields specified in Match Fields. (OXM TLV format) Set TTL <ttl> MPLS, IP Decrement TTL Copy TTL outwards out/in: IP-to-IP out/in: MPLS-to-MPLS out: IP-to-MPLS in: MPLS-to-IP Copy TTL inwards OpenFlow Specification Summary | 2024/11/21 | @ebiken 20
timeouts defined in OF1.3.4. • idle_timeout • Idle time before discarding (seconds) • hard_timeout • Max time before discarding (seconds) • Entry is permanent if set to OFP_FLOW_PERMANENT (0 seconds) • datapath / control plane should keep track of two time objects per Flow Entry. • Time last accessed/used (datapath) • Time created (control plane or datapath) • Cookie • Opaque data value chosen by the controller. • May be used by the controller to filter flow entries affected by flow statistics, flow modification and flow deletion requests. • Not used when processing packets (Not Required in datapath) • Returned as part of each flow stats and flow expired message. • Included in packet-in message (EXT-7). • If the packet was not sent by a flow, this field is set to 0xffffffffffffffff. • Enables the controller to more efficiently classify packet-in, rather than having to match the packet against the full flow table. • Cookie Mask • Can be used to filter Flow Entry in FLOW_MOD (Modify / Delete) commands. OpenFlow Specification Summary | 2024/11/21 | @ebiken b. Packet handling, table structure and other components. 21
way flow entries are managed. • 5 Flags in OF Protocol • enum ofp_flow_mod_flags {} • Must be kept as part of Flow Table Entry • OFPFF_SEND_FLOW_REM • OFPFF_NO_PKT_COUNTS • OFPFF_NO_BYT_COUNTS • Only used when receiving OF message from controller • OFPFF_CHECK_OVERLAP • OFPFF_RESET_COUNTS • OFPFF_SEND_FLOW_REM • Send flow removed message when flow expires or is deleted. • OFPFF_NO_PKT_COUNTS • Don’t keep track of packet count. • OFPFF_NO_BYT_COUNTS • Don’t keep track of byte count. • OFPFF_CHECK_OVERLAP • Check for overlapping entries first. • OFPFF_RESET_COUNTS • Reset flow packet and byte counts. OpenFlow Specification Summary | 2024/11/21 | @ebiken b. Packet handling, table structure and other components. Cookie/Cookie Mask 22
Match Field: wildcards all match field. • Priority: 0 (zero) # lowest priority (Matching logic could be different from all other regular flow entries.) • Every flow table MUST support Table- miss flow entry. • If it does not exist, by default packets unmatched by flow entries are dropped (discarded). • Switch configuration may override this default behavior. • Capabilities below MUST be supported. • Sending packets to the controller via CONTROLLER reserved port. • Packet-in reason must be “table-miss”. • Dropping packets using the Clear-Actions instruction. (May not have the same capability as other regular flow entry.) • Other behaviors are same as regular flow entries. • Does not exist by default in a flow table. • The controller may add or remove it at any time. • May expire. b. Packet handling, table structure and other components. Match Fields (All Wildcard) Priority (0) Counters Instructions Timeouts Cookie Flags OpenFlow Specification Summary | 2024/11/21 | @ebiken 23
in two ways. • Switch flow expiry mechanism. (Timeout) • Run by switch independent from the controller. • hard_timeout, idle_timeout • Request from the controller. • hard_timeout • Time since flow entry’s arrival time. • 0 (zero) : no timeout. • idle_timeout • Time since arrival time of last packet associated with the flow. • 0 (zero) : no timeout. • Removal from the controller • Sending delete flow table modification messages from the controller. • OFPFC_DELETE | OFPFC_DELETE_STRICT • Also removed as the result of removal of a group or a meter. • Flow removed message • Must send message on removal if OFPFF_SEND_FLOW_REM flag is set for the flow entry. • Each message contains: • A complete description of the flow entry. • Reason for removal. (expiry or delete) • Duration and Statistics of the flow entry. b. Packet handling, table structure and other components. OpenFlow Specification Summary | 2024/11/21 | @ebiken 24
integer • unique on the OpenFlow switch. • Group Type • Type to determine group semantics. • Required: indirect, all • Optional: select, fast failover • Counters • Updated when packets are processed by a group. • Action Buckets • An ordered list of action buckets. • Each bucket contains a set of actions to execute. • Always applied as an action set. b. Packet handling, table structure and other components. Group Identifier Group Type Counters Action Buckets Main components of a group entry in the group table. (5.6 Table 2) OpenFlow Specification Summary | 2024/11/21 | @ebiken 25
and other components. Group Identifier Group Type Counters Action Buckets Main components of a group entry in the group table. (5.6 Table 2) Group Type (*) Description Indirect YES Execute the one defined bucket in the group. (supports only a single bucket) Allows multiple flow entries or groups to point to a common group identifier. All YES Execute all buckets in the group. Used for Multicast / Broadcast. Select no Execute one bucket in the group. Failover no Enables switch to change forwarding without requiring a round trip to the controller. Liveness monitoring is required. (*) Required in PF1.3.4? OpenFlow Specification Summary | 2024/11/21 | @ebiken 26
and other components. Group Identifier Group Type Counters Action Buckets Main components of a group entry in the group table. (5.6 Table 2) • An ordered list of action buckets. • Each bucket includes below depending on Group Type. Group Type Description weight Select Relative weight of bucket. (16bit) watch_port Failover Port whose state affects whether this bucket is live. watch_group Failover Group whose state affects whether this bucket is live. Action Set All 0 or more actions associated with the bucket OpenFlow Specification Summary | 2024/11/21 | @ebiken 27
bucket in the group. (supports only a single bucket) • Allows multiple flow entries or groups to point to a common group identifier. • Faster, more efficient convergence. • e.g. next hops for IP forwarding. b. Packet handling, table structure and other components. Packet Action Bucket Indirect (required) OpenFlow Specification Summary | 2024/11/21 | @ebiken 28
the group. • Used for Multicast / Broadcast. • Packet is cloned for each bucket. • One packet is processed for each bucket. • Forwarding to the ingress port • The group must include an extra bucket with output action to the OFPP_IN_PORT reserved port. b. Packet handling, table structure and other components. Clone Packet Action Bucket Action Bucket Action Bucket Action Bucket Action Bucket Packet All (required) OpenFlow Specification Summary | 2024/11/21 | @ebiken 29
the group. • Selection algorithm • Not defined as part of OpenFlow. • Should implement equal load sharing, optionally be based on bucket weights. • e.g. • Hash on user-configured tuple. • Simple round robin. • Selection when port down. • When a port specified in a bucket goes down, switch may restrict bucket selection instead of dropping packets. • Reduce the disruption of a downed link or switch. b. Packet handling, table structure and other components. Action Bucket Action Bucket Action Bucket Action Bucket Action Bucket Packet Selection Algorithm Select (optional) OpenFlow Specification Summary | 2024/11/21 | @ebiken 30
change forwarding without requiring a round trip to the controller. • Liveness monitoring is required. • Each action bucket is associated with a specific port and/or group that controls its liveness. • (See next slide for more details) • Evaluated in the order defined by the group. • Execute the first live bucket. • Packets are dropped if no buckets are live. b. Packet handling, table structure and other components. Action Bucket Action Bucket Action Bucket Action Bucket Action Bucket Packet Liveness Monitoring Fast failover (optional) Not Live Not Live Not Live Live Live OpenFlow Specification Summary | 2024/11/21 | @ebiken 31
either if … a. watch_port is not OFPP_ANY and the port watched is live, or b. watch_group is not OFPG_ANY and the group watched is live. • A port is considered live if … • It exists in the datapath and has the OFPPS_LIVE flag set in its port state. • The OFPPS_LIVE flag must be unset (post considered not live) if … • Liveness mechanisms of the switch (outside of the OpenFlow specification) considers port not live. • The port config bit OFPPC_PORT_DOWN indicates the port is down. • The port state bit OFPPS_LINK_DOWN indicates the link is down. • A group is considered live if … • at least one of its buckets is live. • Liveness monitoring is required for “Fast Failover group”. • Other group types may optionally implement it. b. Packet handling, table structure and other components. OpenFlow Specification Summary | 2024/11/21 | @ebiken 32
other components. Group Identifier Group Type Counters Action Buckets Main components of a group entry in the group table. (5.6 Table 2) (*) Bytes Per Group 28 Reference Count (flow entries) no 4 Packet CountPacket Count no 8 Byte CountByte Count no 8 Duration (seconds) YES 4 Duration (nanoseconds) no 4 (*) Bytes Per Group Bucket 16 Packet CountPacket Count no 8 Byte CountByte Count no 8 (*) Required in PF1.3.4? See slide “Counters: List of all counters” for all counters. (*) Required in PF1.3.4? OpenFlow Specification Summary | 2024/11/21 | @ebiken 33
rate of the aggregate of all attached flow entries. • Implement simple QoS operations • Ex: rate-limiting. • Can be combined with per-port queues to implement complex QoS frameworks such as DiffServ. • Meter entries • per-flow meters directly attached to flow entries. (<=> queues are attached to ports) • Specified in instruction set in flow entry. • Multiple meters can be used in the same table, but in an exclusive way (disjoint set of flow entries) • Multiple meters can be used on the same set of packets by using them in successive flow tables. b. Packet handling, table structure and other components. Meter Entry Flow Table #0 Instructions Instructions Instructions Flow Table #n Instructions Instructions Instructions Meter Table Meter Entry Meter Entry Meter Entry cannot set 2 meters for a flow entry OpenFlow Specification Summary | 2024/11/21 | @ebiken 34
structure and other components. Meter Identifier Meter Bands Counters Main components of a meter entry in the meter table. (5.7 Table 3) 32 bit unsigned integer uniquely identifying the meter. An unordered list of meter bands. Meter Bands described in next slide. OpenFlow Specification Summary | 2024/11/21 | @ebiken 35
way to process the packet. • Three Band Types (drop | dscp | Experimenter) • drop (1) • Drop (discard) the packet. Can be used to define a rate limiter band. • dscp remark (2) • Increase the drop precedence of the DSCP field in the IP header of the packet. Can be used to define a simple DiffServ policer. • Experimenter (0xFFFF) • Vender defined. Required to use this if updating PCP/TC fields in VLAN/MPLS. • Rate (kbps or pps) • Lowest rate at which the band can apply. • Key value to identify how packet will be processed. • Burst (kbits or packets) • burst size of the band. • Arguments (type specific) • Option argument used by band. • enum ofp_meter_flags • OFPMF_KBPS / OFPMF_PKTPS (rate/burst in kbps or pps) • OFPMF_BURST (burst exists or not) • OFPMF_STATS (collect stats or not) • Simple “Rate-Limit” type of control can be implemented using meter alone. • Combine with per-port queues to implement complex QoS frameworks, such as DiffServ. (see 5.12, 7.2.2) b. Packet handling, table structure and other components. Main components of a meter band in a meter entry. (5.7.1 Table 4) OpenFlow Specification Summary | 2024/11/21 | @ebiken Band Type Burst Counters Arguments Rate 36
other components. Meter Identifier Meter Bands Counters (*) Bytes Per Meter Band 16 In Band Packet Count no 8 In Band Byte Count no 8 (*) Bytes Per Meter 28 Flow Counter no 4 Input Packet Counter no 8 Input Byte Counter no 8 Duration (seconds) YES 4 Duration (nanoseconds) no 4 (*) Required in PF1.3.4? (*) Required in PF1.3.4? OpenFlow Specification Summary | 2024/11/21 | @ebiken Band Type Burst Counters Arguments Rate 37
the datapath on an output port. • Two or more queues can be attached to a port. • Identified by a port number and a queue ID. • Two queues on different ports can have the same queue ID. • A queue can NOT be attached to two or more ports. • ofp_packet_queue • Port Id • Queue Id • Length • ofp_queue_properties • OFPQT_MIN_RATE • OFPQT_MAX_RATE • OFPQT_MIN_EXPERIMENTER This is from “7 The OpenFlow Switch Protocol” in spec, but adding slide here since it will tightly work with Meters. OpenFlow Specification Summary | 2024/11/21 | @ebiken 38
and other components. (*) Bytes Per Flow Table 20 Reference count (active entries) YES 4 Packet Lookups no 8 Packet Matches no 8 Per Flow 24 Received Packets no 8 Received Bytes no 8 Duration (seconds) YES 4 Duration (nanoseconds) no 4 Per Queue 32 Transmit Packets YES 8 Transmit Bytes no 8 Transmit Overrun Errors no 8 Duration (seconds) YES 4 Duration (nanoseconds) no 4 Per Meter Band 16 In Band Packe Count no 8 In Band Byte Count no 8 (*) Required in PF1.3.4? (*) Bytes Per Port 104 Received Packets YES 8 Transmitted Packets YES 8 Received Bytes no 8 Transmitted Bytes no 8 Receive Drops no 8 Transmit Drops no 8 Receive Errors no 8 Transmit Errors no 8 Receive Frame Alignment Errors no 8 Receive Overrun Errors no 8 Receive CRC Errors no 8 Collisions no 8 Duration (seconds) YES 4 Duration (nanoseconds) no 4 (*) Bytes Per Group 28 Reference Count (flow entries) no 4 Packet CountPacket Count no 8 Byte CountByte Count no 8 Duration (seconds) YES 4 Duration (nanoseconds) no 4 Per Group Bucket 16 Packet CountPacket Count no 8 Byte CountByte Count no 8 Per Meter 28 Flow Counter no 4 Input Packet Counter no 8 Input Byte Counter no 8 Duration (seconds) YES 4 Duration (nanoseconds) no 4 OpenFlow Specification Summary | 2024/11/21 | @ebiken 39
ebiken
koichiotomo
asei
kamina_zzz
minorun365
tohutohu
fullkaiten
oracle4engineer
sadnessojisan
linyows
iotengineer22
coincheck_recruit
aleyda
jacobian
csswizardry
ufuk
smashingmag
bluesmoon
jrom
bkeepers
chriscoyier
michaelherold
chrisshort
