1 SONiC コミュニティ テスト手法あれこれ Manodipto Ghose, Product Manager ([email protected]) 野田 清志, Senior System Engineer ([email protected]) Keysight/SONiC workshop Japan

2 1. SONiC コミュニティとキーサイト - Keysight’s role in SONiC Community 2. SONiCにおけるテストエリア - Test areas in SONiC 3. RDMAテスト手法とSONiC - RDMA testing in SONiC 4. この先に向けて - Call to action Agenda Keysight/SONiC workshop Japan

3 SONiC コミュニティとキーサイト - Keysight’s role in SONiC Community Keysight/SONiC workshop Japan

4 Hyperscale Cloud Provider NEMs NEMs OEMs/NEMs/ASICs Open Network Projects – Compliance & Benchmarks Test Content Google Microsoft Amazon Meta Cisco Arista Juniper Nokia ... Criteria • Open Traffic Generator • Integration into Open Test Orchestration Frameworks Products Open Networking Eco-Systems Broadcom Marvell Intel NVIDIA ... Keysight Keysight/SONiC workshop Japan

5 SONiC testing progression (2019-2024) Keysight/SONiC workshop Japan Existing sonic-mgmt. (PTF based) Integration of Keysight TGs RDMA, BGP, Reboot Performance Tests contributed SONiC DASH DPU/SmartSwitch Contribute new tests Extend to T2 Limited Good for functional Control plane tests Congestion control needs line rate TG Scalable Line rate traffic 100G/400G/800G Advance PFC features Modern API (Snappi) Tests in collaboration with Microsoft team Accepted by community Integrated in SONiC nightly regression testing Involves stateful and stateless traffic generation Testing single DPU to Smart switch use cases as per scale Extending functionality to test multi-ASIC chassis Adding new test cases

6 SONiCにおけるテストエリア - Test areas in SONiC Keysight/SONiC workshop Japan

7 T E S T I N G F O R M AT U R I T Y What should I test to be SONiC ready Testing for Maturity Keysight/SONiC workshop Japan

8 01 Integration of Keysight traffic generator Enabled sonic-mgmt. to use Keysight 100GE and 400GE traffic generators 02 Snappi – a new API New modern API, Snappi and software application integration Keysight/SONiC workshop Japan https://github.com/sonic-net/sonic-mgmt/blob/master/docs/testbed/README.testbed.Keysight.md#sonic-testbed-with-keysight-ixnetworkixload-as-traffic- generator Test bed enhancement of SONiC

9 RDMAテスト手法とSONiC - RDMA in SONiC Keysight/SONiC workshop Japan

10 KPIs of SONiC Keysight/SONiC workshop Japan

11 Why RDMA? Keysight/SONiC workshop Japan Congestion control Lossless fabric Commodity Ethernet solution RDMA PFC, ECN ROCEv2 DCQCN SONiC switches (T0 → T1 → T2) Switch fabric for AI DC

12 Keysight/SONiC workshop Japan RDMA testing on T0/T1 switches • ECN ➢ Red Accuracy ➢ Dequeue • PFCWD ➢ Basic ➢ Runtime traffic ➢ Burst storm ➢ Many to One ➢ All to All • PFC ➢ Global pause ➢ Lossless ➢ Lossy ➢ Pause response ➢ Headroom • Testbed used • Topologies

13 Methodology Keysight/SONiC workshop Japan Tx Rx Data Traffic Background Traffic Pause Storm ➢Traffic profiles marked with lossless queues 3,4 and lossy queues ➢Uses IxNetwork and Snappi ➢Tests integrated with sonic-mgmt. ➢Create congestion scenarios

14 PFC congestion scenario Keysight/SONiC workshop Japan LC 1 LC 2 FABRIC 1 1 3 3 2 5 PFC PFC Lossless 30% Lossless 30% Lossless 25% Lossless 25% 4 ➢ Background traffic with lossy priorities 0-2, 5-6 from 2 ingress ports to 1 egress port of bandwidth 25% each. ➢ Data traffic with lossless priorities 3-4 from same 2 ingress ports to same egress port of bandwidth 30% each to create congestion at egress port. ➢ Due to congestion, PFC packets are received at ingress port and transmitting rate is adjusted due to flow control. ➢ No packet loss in both background and data traffics. T2 Chassis

15 • PFC ➢ Test Plan: https://github.com/sonic-net/sonic-mgmt/tree/master/docs/testplan/pfc ➢ Test Cases: https://github.com/sonic-net/sonic-mgmt/tree/master/tests/snappi_tests/pfc • ECN ➢ Test Plan: https://github.com/sonic-net/sonic-mgmt/tree/master/docs/testplan/ecn ➢ Test Cases: https://github.com/sonic-net/sonic-mgmt/tree/master/tests/snappi_tests/ecn • PFCWD ➢ Test Plan: https://github.com/sonic-net/sonic-mgmt/tree/master/docs/testplan/pfcwd ➢ Test Cases: https://github.com/sonic-net/sonic-mgmt/tree/master/tests/snappi_tests/pfcwd Test Plan and Test Cases of RDMA on T0/T1 switches Keysight/SONiC workshop Japan Total no. of tests Supported speed modes 107 100G & 400G

16 Testbed: https://github.com/sonic-net/sonic- mgmt/blob/master/docs/testbed/README.testbed.Keysight.md Test Cases: https://github.com/sonic-net/sonic- mgmt/tree/master/tests/snappi_tests/multidut PFC congestion test plan: https://github.com/sonic-net/sonic- mgmt/blob/master/docs/testplan/PFC_Congestion_Oversubscription _Test_Plan.md T2 Testbed and Test Cases Keysight/SONiC workshop Japan

17 Next steps • Work in progress: PRs for T2 chassis framework, RDMA, bug fixes • New T2 test plan on various performance and convergence scenarios: • Cover both inbound and outbound traffic scenarios • Convergence scenarios based on triggers • Unplanned failure scenarios like link flap, BGP flaps, line card and supervisor restarts • Planned events like TSA/TSB • End to end route performance Keysight/SONiC workshop Japan

18 この先に向けて - Call to action Keysight/SONiC workshop Japan

19 Keysight/SONiC workshop Japan • Collaborate with us to add more tests to SONiC • Review RDMA areas, explore new areas like ROCEv2, DCQCN testing • Get involved in SONiC test working group • Co-develop test plan and test cases with Keysight • Network operators • Talk to us: • Manodipto Ghose: [email protected] • Kiyoshi Noda: [email protected] Call to action