SRv6 Mobile User Plane Initial POC and Implementation ENOG55 @新潟 燕三条｜ 2019年2月22日 海老澤 健太郎 Kentaro Ebisawa Toyota InfoTechnology Center Co., Ltd. SRv6データプレーン実装とモバイルへの適用 SRv6 Mobile User Plane ~ Initial POC and Implementation | ENOG55 in Nigata Tsubame Sanjyo | 2019/02/22 Happy cat’s day ☺

• Deployment options of “SRv6 Mobile User Plane” • draft-ietf-dmm-srv6-mobile-uplane • https://datatracker.ietf.org/doc/draft-ietf-dmm-srv6-mobile-uplane/ • Proof Of Concept ... Topology and Results • Partial migration of GTP to SRv6 without Control Plane change • POC Findings • POC SRv6 Implementations • VPP / FD.io (customized by Pablo@Cisco) • Barefoot SRv6 P4-16 on Tofino Switch (customized by Apresia Systems) • POC Future plans SRv6 Mobile User Plane ~ Initial POC and Implementation | ENOG55 in Nigata Tsubame Sanjyo | 2019/02/22 Table of Contents

• POC in this document was conducted during Nov ~ Dec 2018 by the “SRv6 Consortium” ... https://seg6.net/ POC Participants and roles • Satoru Matsushima (Softbank) • Overall organization. Packet bare metal cloud setup including Cisco vRouter config. • Hitoshi Kuwata (Apresia Systems) • P4 SRv6 custom development. Wedge 100BF (Tofino Switch). • Kentaro Ebisawa (Toyota ITC) (Editor of this document) • Overall deployment and testing of VPP, P4 SRv6 and GTPing. Contributors outside of the SRv6 Consortium • Physical Lab environment (Rack/Server/Switch) was provided by Yuji Sekiya Lab @ The University of Tokyo • https://www.sekiya-lab.info/ • Physical Lab and Server & Switch setup was by Ryo Nakamura (upa) • VPP used in this POC is a privately customized version developed by Pablo@Cisco • P4 SRv6 source code (without Mobile functions) was provided from Milad@Barefoot SRv6 Mobile User Plane ~ Initial POC and Implementation | ENOG55 in Nigata Tsubame Sanjyo | 2019/02/22 About the POC and this document

SRv6 Mobile User Plane ~ Initial POC and Implementation | ENOG55 in Nigata Tsubame Sanjyo | 2019/02/22 https://seg6.net/

Deployment options of “SRv6 Mobile User Plane” draft-ietf-dmm-srv6-mobile-uplane-03 https://datatracker.ietf.org/doc/draft-ietf-dmm-srv6-mobile-uplane/ SRv6 Mobile User Plane ~ Initial POC and Implementation | ENOG55 in Nigata Tsubame Sanjyo | 2019/02/22

Deployment options of “SRv6 Mobile User Plane” SRv6 Mobile User Plane ~ Initial POC and Implementation | ENOG55 in Nigata Tsubame Sanjyo | 2019/02/22 4G / 5G Core Network (focused in Uplane) U E eNB PGW Internet GTPv1-U over IPv4 User Data User Data Wireless SGi S1-U gNB SGW UPF User Data U E GTPv1-U over IPv4 User Data N3 UPF N9 N6 Wireless 4G (LTE) 5G “SRv6 Mobile User Plane” is an alternative User Plane option to GTPv1-U

draft-ietf-dmm-srv6-mobile-uplane-03 SRv6 Mobile User Plane ~ Initial POC and Implementation | ENOG55 in Nigata Tsubame Sanjyo | 2019/02/22 5.1. Traditional mode draft-ietf-dmm-srv6-mobile-uplane-03 https://datatracker.ietf.org/doc/draft-ietf-dmm-srv6-mobile-uplane/ In existing 3GPP mobile networks, an UE session is mapped 1-for-1 with a specific GTP tunnel (TEID). This 1-for-1 mapping is replicated here to replace GTP encapsulation with the SRv6 encapsulation, while not changing anything else. Uplink Packet Flow Downlink Packet Flow

draft-ietf-dmm-srv6-mobile-uplane-03 SRv6 Mobile User Plane ~ Initial POC and Implementation | ENOG55 in Nigata Tsubame Sanjyo | 2019/02/22 5.2. Enhanced Mode ... use of multiple SIDs, instead of a single SID as done in the Traditional mode. The main difference is that the SR policy MAY include SIDs for traffic engineering and service programming in addition to the UPFs SIDs. Uplink Packet Flow Downlink Packet Flow

draft-ietf-dmm-srv6-mobile-uplane-03 SRv6 Mobile User Plane ~ Initial POC and Implementation | ENOG55 in Nigata Tsubame Sanjyo | 2019/02/22 5.3. Enhanced mode with unchanged gNB GTP behavior This section describes two mechanisms for interworking with legacy gNBs that still use GTP: one for IPv4, the other for IPv6. In the interworking scenarios as illustrated in Figure 4, gNB does not support SRv6. gNB supports GTP encapsulation over IPv4 or IPv6. To achieve interworking, a SR Gateway (SRGW-UPF1) entity is added. The SRGW maps the GTP traffic into SRv6. The SRGW is not an anchor point, and maintains very little state. For this reason, both IPv4 and IPv6 methods scale to millions of UEs.

SRv6 Mobile User Plane ~ Initial POC and Implementation | ENOG55 in Nigata Tsubame Sanjyo | 2019/02/22 5.3. Enhanced mode with unchanged gNB GTP behavior 5.3.1. / 5.3.2. | Interworking with IPv6 / IPv4 GTP End.M.GTP6.D End.DT4 or End.DT6 PSP End T.M.Tmap End.DT4 or End.DT6 PSP End T.Encaps.Red End.M.GTP4.E End.M.GTP6.E T.Encaps.Red

SRv6 Mobile User Plane ~ Initial POC and Implementation | ENOG55 in Nigata Tsubame Sanjyo | 2019/02/22 Slide 6 : https://datatracker.ietf.org/meeting/103/materials/slides-103-dmm-a-smooth-migration-of-mobile-core-user-plane-from-gtp-to-srv6-00 “A Smooth Migration of Mobile Core User Plane from GTP to SRv6” IETF103 Bangkok, November, 2018 | Arashmid Akhavain, Chenchen Liu @Huawei / Pablo Camarillo @Cisco

Proof Of Concept ... Topology and Results SRv6 Mobile User Plane ~ Initial POC and Implementation | ENOG55 in Nigata Tsubame Sanjyo | 2019/02/22

POC Topology and Results SRv6 Mobile User Plane ~ Initial POC and Implementation | ENOG55 in Nigata Tsubame Sanjyo | 2019/02/22 Partial migration of GTP to SRv6 without Control Plane change SRv6 Drop-In Replacement for GTP GTP Client SRGW (P4 Switch) SRGW (VPP) GTP Server GTP/IPv4 GTP/IPv4 <= End.M.GTP4.E <= T.M.Tmap SRv6 • GTP to SRv6 stateless translation • No SRH overhead (one segment) • No change in Control Plane => T.M.Tmap => End.M.GTP4.E

POC Topology and Results SRv6 Mobile User Plane ~ Initial POC and Implementation | ENOG55 in Nigata Tsubame Sanjyo | 2019/02/22 GTP  SRv6 Translation Logic GTP Client SRGW (P4 Switch) SRGW (VPP) GTP Server GTP/IPv4 GTP/IPv4 <= End.M.GTP4.E => T.M.Tmap => End.M.GTP4.E <= T.M.Tmap SRv6 ① ② SRv6 SID : fc34:5678:ac14:0002:ac14:0001:0000:0064 Dst: 172.20.0.2 Src: 172.20.0.1 TEID: 100 PREFIX SRv6 SID ① ① ② ① ① ② Packet Captured in private network different from POC

SRv6 Mobile User Plane ~ Initial POC and Implementation | ENOG55 in Nigata Tsubame Sanjyo | 2019/02/22 POC Topology and Results GTP Client SRGW (P4 Switch) (Tofino) SRGW (VPP) GTP Server GTP/IPv4 GTP/IPv4 <= End.M.GTP4.E => T.M.Tmap => End.M.GTP4.E <= T.M.Tmap SRv6 Dst: 2604:1380:3000:757f:a00:1ce1:0:2 TEID 48.0.117.127 10.0.28.225 48.0.117.127 10.0.28.225 2604:1380:: 2001:200:: Eyes Only Includes Public IP used in actual POC environment

SRv6 Mobile User Plane ~ Initial POC and Implementation | ENOG55 in Nigata Tsubame Sanjyo | 2019/02/22 POC Topology and Results GTP Client SRGW (P4 Switch) (Tofino) SRGW (VPP) GTP Server GTP/IPv4 GTP/IPv4 <= End.M.GTP4.E => T.M.Tmap => End.M.GTP4.E <= T.M.Tmap SRv6 Dst: 2001:200:0:1ce1:3000:757f:0:2 TEID 0.0.28.225 (*) 48.0.117.127 48.0.117.127 10.0.28.225 2604:1380:: 2001:200:: Eyes Only Includes Public IP used in actual POC environment (*) SRGW (P4 Switch) translates dst addr with mask 10.x.x.x to workaround limitation due to available public IPv6 address range.

Proof Of Concept ... Findings SRv6 Mobile User Plane ~ Initial POC and Implementation | ENOG55 in Nigata Tsubame Sanjyo | 2019/02/22

Proof Of Concept ... Findings • (T.M.Tmap) (End.M.GTP4.E) Should we (not) add SRH when single SID was specified? • “6.6. T.M.Tmap” psudo code does NOT insert SRH, but just encap in IPv6 packet. • “6.5. End.M.GTP4.E” assumes there is SRH (4. pop the SRH). • If not having SRH is acceptable, then we should update End.M.GTP4.E description (psudo code) as well. • We might want to do traffic engineering together with GTP to SRv6 translation. Thus, we should update T.M.Tmap to support both cases … no SRH with single SID, push SRH when two or more SIDs was specified including the one generated as “SRGW-IPv6-LOC-FUNC:IPv4DA:IPv4SA:TUN-ID” SRv6 Mobile User Plane ~ Initial POC and Implementation | ENOG55 in Nigata Tsubame Sanjyo | 2019/02/22 Error in the Mobile Uplane draft & Improvements

Proof Of Concept ... Findings • (T.M.Tmap) GTP header info other than TEID will be lost. • Flags ... has_seq (and others) • Message Type ... GTPMSG_ECHO(1), GTPMSG_ECHOREPLY(2), GTPMSG_TPDU(255) • (T.M.Tmap) How to set correct “Next Header” to SRv6 packet? • Protocol in GTP Payload cannot be identified from the packet itself. Assumption is GTP in/out gateway understands the protocol the user (UE) is using. • Maybe “T.M.Tmap” should have a parameter to specify “Next Header” value and let controller configure it? • What will be the number for unstructured data? 59?? SRv6 Mobile User Plane ~ Initial POC and Implementation | ENOG55 in Nigata Tsubame Sanjyo | 2019/02/22 Issues due to protocol design

Proof Of Concept ... Findings • (T.M.Tmap) UDP source port number would not be stored when “T.M.Tmap” is applied. • GTP packet could change before / after going through SRv6. (Unless it is 2152) • Use port “2152” for both source and destination packet. • This should NOT be a problem because actual GTP-U session is uni-directional. • (T.M.Tmap) (End.M.GTP4.E) PREFIX length longer than 32 bits? • Operator might only have PREFIX longer than 32bits available. • ex: Very small MVNO, Private LTE or POC testing like we did. • Possible solution is to configure PREFIX with length “32+n” and set IPv4 higher n bits as part of SRv6 function parameter. SRv6 Mobile User Plane ~ Initial POC and Implementation | ENOG55 in Nigata Tsubame Sanjyo | 2019/02/22 Minor Issues / Improvements

Proof Of Concept ... Findings • (End.M.GTP4.E) Should SRGW drop packet when header of SRv6 payload was invalid from the one specified in “Next Header” field? • Or should SRGW blindly translate SRv6 to GTP? • Should SRGW log this when it found the mismatch? • Cons are too many logs generated while operator is aware (so mismatch would not happen) or doesn’t need to be aware of what UE is sending. SRv6 Mobile User Plane ~ Initial POC and Implementation | ENOG55 in Nigata Tsubame Sanjyo | 2019/02/22 Implementation considerations

Proof Of Concept ... SRv6 Implementations SRv6 Mobile User Plane ~ Initial POC and Implementation | ENOG55 in Nigata Tsubame Sanjyo | 2019/02/22

POC SRv6 Implementations SRv6 Mobile User Plane ~ Initial POC and Implementation | ENOG55 in Nigata Tsubame Sanjyo | 2019/02/22 Mobile functions (T.M.Tmap, End.M.GTP4.E) developed by ... VPP ... by Pablo@Cisco P4 Switch ... by Apresia Systems, based on P4-16 SRv6 by Milad@Barefoot But both VPP and P4 implementations are closed source (as of today) ..... Seeking to develop Open Source implementations, so anyone can reproduce POC and expand !! ☺

POC SRv6 Implementations SRv6 Mobile User Plane ~ Initial POC and Implementation | ENOG55 in Nigata Tsubame Sanjyo | 2019/02/22 SRv6 on VPP (Vector Packet Processing) https://fd.io/technology/ • Input as Packet Vector • Process based on graph of nodes (ethernet, mpls, ip6, arp …) • New functionality could be added by customizing node or creating plugin SRv6 node: sr_localsid_node, sr_policy_*_node DPDK (User Land)

POC SRv6 Implementations SRv6 Mobile User Plane ~ Initial POC and Implementation | ENOG55 in Nigata Tsubame Sanjyo | 2019/02/22 SRv6 on VPP (by FD.io project) Supported functions as of 2019/01/16 http://www.segment-routing.net/open-software/vpp/

POC SRv6 Implementations SRv6 Mobile User Plane ~ Initial POC and Implementation | ENOG55 in Nigata Tsubame Sanjyo | 2019/02/22 Extending SRv6 functions https://docs.fd.io/vpp/19.04/srv6_plugin_doc.html $ git clone https://gerrit.fd.io/r/vpp ~/vpp/src/examples/srv6-sample-localsid$ ls node.c srv6_localsid_sample.h srv6_localsid_sample.c srv6_sample_localsid_doc.md

POC SRv6 Implementations SRv6 Mobile User Plane ~ Initial POC and Implementation | ENOG55 in Nigata Tsubame Sanjyo | 2019/02/22 IETF 104 Hackathon @Prague, Czech https://trac.ietf.org/trac/ietf/meeting/wiki/104hackathon

SRv6 Mobile User Plane ~ Initial POC and Implementation | ENOG55 in Nigata Tsubame Sanjyo | 2019/02/22 POC SRv6 Implementations Reference: P4 Workshop 2018 in Tokyo “Wedge 100 BF Switch Introduction and SRv6 Demonstration | APRESIA Systems & Edgecore” Slide 8 https://www.barefootnetworks.com/products/brief-tofino/ P4-16 SRv6 Implementation running on Wedge100BF-32X

POC SRv6 Implementations SRv6 Mobile User Plane ~ Initial POC and Implementation | ENOG55 in Nigata Tsubame Sanjyo | 2019/02/22 p4srv6 ... Open Source P4-16 SRv6 on BMv2

Proof Of Concept ... Future plans SRv6 Mobile User Plane ~ Initial POC and Implementation | ENOG55 in Nigata Tsubame Sanjyo | 2019/02/22

POC Future plans • Feedback findings to IETF DMM working group and SRv6 Mobile Uplane draft • https://datatracker.ietf.org/doc/draft-ietf-dmm-srv6-mobile-uplane/ • Develop Open Source P4 SRv6 implementations (p4srv6 and more?) • Develop and upstream SRv6 Mobile Functions on VPP (at IETF) • POC next Phase • TE enabled GTP to SRv6 translation (T.M.Tmap with SRH) • Direct SRv6 to DN translation • 5.3. Enhanced mode with unchanged gNB GTP behavior 5.3.2. Interworking with IPv4 GTP • User data  GTP  SRv6  User data (DN: upstream) | Requires control plane change (S11) • Wireless UE SRv6 Mobile User Plane ~ Initial POC and Implementation | ENOG55 in Nigata Tsubame Sanjyo | 2019/02/22 Next Steps