Tweet
Tweet

Slide 1

Slide 1 text

Next Hop Group on SONiC Twitter: @ebiken https://github.com/ebiken Next Hop Group on SONiC | Twitter: @ebiken | 2022/11/23

Slide 2

Slide 2 text

• Next Hop Group (NHG) ... Overview • Route Entry & Next Hop • Route Entry & Next Hop (Data Structure) • Next Hop Group on SONiC • Next Hop Group on SONiC (Multipath Route) • ASIC_DB Entry • Data Base Entry Transition when configured via CLI (CONFIG_DB, APPL_DB, ASIC_DB) • Appendix • Full ASIC_DB entry attributed and relationship (config via CLI) • SWSS / SAI log example (swss.rec , sairedis.rec) (config via CLI) • Next Hop Group on FRR • nexthop_id for direct attached network • How to check ASIC entries? • References Next Hop Group on SONiC | Twitter: @ebiken | 2022/11/23 Table of Contents Wedge100BF-32X (Tofino) SONiC-OS-master.143513-dirty-20220903.195418

Slide 3

Slide 3 text

Next Hop Group (NHG) ... Overview Next Hop Group on SONiC | Twitter: @ebiken | 2022/11/23

Slide 4

Slide 4 text

• route object will store gateway (gw) and device (dev) info for each destination (prefix) • prefix is used for Longest Prefix Match and include mask length (in this context) • For some OS (NOS), you don’t need to specify dev if dev could be looked up based on gw (address) • MAC address of gw will be resolved and stored in neightbor table (for Linux) • Data Structure (how to store above info) varies for each implementations Next Hop Group on SONiC | Twitter: @ebiken | 2022/11/23 Route Entry & Next Hop A B C D ｒoute prefix: D dev : ens0 gw : B ens0 route ens1

Slide 5

Slide 5 text

Next Hop Group on SONiC | Twitter: @ebiken | 2022/11/23 Route Entry & Next Hop (Data Structure) ｒoute prefix dev gateway ｒoute prefix nexthop nexthop dev gateway ip route add via [dev ] ip route add nexthop via [dev ] ｒoute group nexthop nexthop dev gateway nexthop dev gateway nexthop dev gateway nexthop group nexthop[N] ① route entry will include nexthop info (prior to Linux 5.3) ② split route entry and nexthop object (since Linux 5.3) Note: Linux 5.2 has partially refactored code to support nexthop object. However, fib_info only includes fib_nh and nexthop is not (yet) included. Linux command

Slide 6

Slide 6 text

Next Hop Group on SONiC Next Hop Group on SONiC | Twitter: @ebiken | 2022/11/23

Slide 7

Slide 7 text

Next Hop Group on SONiC | Twitter: @ebiken | 2022/11/23 Next Hop Group on SONiC (Multipath Route) ROUTE_TABLE Key: 10.99.0.0/24 Value: 'nexthop': '10.0.0.100,10.0.0.101' 'ifname': 'Ethernet0,Ethernet0' 'weight': '1,1' ROUTE_TABLE Key: 10.99.0.0/24 Value: 'nexthop_group': 'nhg1' NEXT_HOP_GROUP_TABLE Key: nhg1 Value: 'nexthop': '10.0.0.100,10.0.0.101' 'ifname': 'Ethernet0,Ethernet0' Two ways to configure Next Hop Group (as of 20220903) ROUTE_TABLE entry only ( No NEXT_HOP_GROUP_TABLE entry ) CLI support ? APPL_DB schema ASIC_DB schema ROUTE_TABLE + NEXT_HOP_GROUP_TABLE entry Has to directly add APPL_DB entry Can be configured via CLI config route add prefix 10.99.0.0/24 nexthop 10.0.0.100 config route add prefix 10.99.0.0/24 nexthop 10.0.0.101 (Feature added in “SONiC HLD: Routing and Next Hop Table Enhancement”) https://github.com/sonic-net/SONiC/blob/master/doc/ip/next_hop_group_hld.md ASIC_DB entry schema is same for both ways

Slide 8

Slide 8 text

Next Hop Group on SONiC | Twitter: @ebiken | 2022/11/23 ASIC_DB entry (one Next Hop) One Next Hop => ASIC_DB schema is different sudo config route add prefix 10.99.0.0/24 nexthop 10.0.0.100 NEXT_HOP Key: oid: 3 Value: { NEXT_HOP_ATTR_IP: 10.0.0.100 NEXT_HOP_ATTR_ROUTER_INTERFACE_ID: 1 } ROUTE_ENTRY Key: { dest: 10.99.0.0/24 } Value: { NEXT_HOP_ID: 3 } ROUTE_ENTRY Key: { dest: 10.99.0.0/24 } Value: { NEXT_HOP_ID: 5 } NEXT_HOP_GROUP_MEMBER Key: oid: 6 Value: { NEXT_HOP_GROUP_ID: oid: 5 NEXT_HOP_ID: oid: 3 } NEXT_HOP_GROUP Key: oid: 5 NEXT_HOP Key: oid: 3 Value: { NEXT_HOP_ATTR_IP: 10.0.0.100 NEXT_HOP_ATTR_ROUTER_INTERFACE_ID: 1 } APPL_DB ROUTE_TABLE:10.99.0.0/24{'nexthop_group': 'nhg1'} NEXTHOP_GROUP_TABLE:nhg1 {'nexthop': '10.0.0.100', 'ifname': 'Ethernet0'} APPL_DB ROUTE_TABLE:10.99.0.0/24 {'nexthop': '10.0.0.100','ifname': 'Ethernet0'} ※ NEXT_HOP_GROUP will be created even if there is only one Next Hop entry

Slide 9

Slide 9 text

Next Hop Group on SONiC | Twitter: @ebiken | 2022/11/23 ASIC_DB entry (two Next Hop) Two or more Next Hops => ASIC_DB schema is same sudo config route add prefix 10.99.0.0/24 nexthop 10.0.0.100 sudo config route add prefix 10.99.0.0/24 nexthop 10.0.0.101 NEXT_HOP Key: oid: 3 Value: { NEXT_HOP_ATTR_IP: 10.0.0.100 NEXT_HOP_ATTR_ROUTER_INTERFACE_ID: 1 } ROUTE_ENTRY Key: { dest: 10.99.0.0/24 } Value: { NEXT_HOP_ID: 5 } NEXT_HOP_GROUP_MEMBER Key: oid: 7 Value: { NEXT_HOP_GROUP_ID: oid: 5 NEXT_HOP_ID: oid: 4 } NEXT_HOP_GROUP Key: oid: 5 NEXT_HOP Key: oid: 4 Value: { NEXT_HOP_ATTR_IP: 10.0.0.100 NEXT_HOP_ATTR_ROUTER_INTERFACE_ID: 1 } APPL_DB ROUTE_TABLE:10.99.0.0/24{'nexthop_group': 'nhg1'} NEXTHOP_GROUP_TABLE:nhg1 {'nexthop': '10.0.0.100,10.0.0.101', 'ifname': 'Ethernet0,Ethernet0'} APPL_DB ROUTE_TABLE:10.99.0.0/24 {'nexthop': '10.0.0.100,10.0.0.101','ifname': 'Ethernet0,Ethernet0’, 'weight': '1,1’} NEXT_HOP_GROUP_MEMBER Key: oid: 6 Value: { NEXT_HOP_GROUP_ID: oid: 5 NEXT_HOP_ID: oid: 3 }

Slide 10

Slide 10 text

Data Base Entry Transition when configured via CLI (CONFIG_DB, APPL_DB, ASIC_DB) Entry transition will be a bit complex if configured via CLI. Thus explaining it in detail. Next Hop Group on SONiC | Twitter: @ebiken | 2022/11/23

Slide 11

Slide 11 text

Next Hop Group on SONiC | Twitter: @ebiken | 2022/11/23 CONFIG_DB, APPL_DB entry transition (config via CLI) If two or more nexthop are configured, it will be appended to the value of the same entry. ROUTE_TABLE Key: 10.99.0.0/24 Value: 'nexthop': '10.0.0.100', 'ifname': 'Ethernet0' STATIC_ROUTE Key: 10.99.0.0/24 Value: "blackhole":"false", "distance":"0", "ifname":"", "nexthop":"10.0.0.100", "nexthop-vrf":"" admin@sonic:~$ sudo config route add prefix 10.99.0.0/24 nexthop 10.0.0.100 (CONFIG＿DB) (APPL＿DB) ROUTE_TABLE Key: 10.99.0.0/24 Value: 'nexthop': '10.0.0.100,10.0.0.101', 'ifname': 'Ethernet0,Ethernet0', 'weight': '1,1' STATIC_ROUTE Key: 10.99.0.0/24 Value: "blackhole":"false,false", "distance":"0,0", "ifname":",", "nexthop":"10.0.0.100,10.0.0.101", "nexthop-vrf":"," admin@sonic:~$ sudo config route add prefix 10.99.0.0/24 nexthop 10.0.0.100 admin@sonic:~$ sudo config route add prefix 10.99.0.0/24 nexthop 10.0.0.101 (CONFIG＿DB) (APPL＿DB)

Slide 12

Slide 12 text

Next Hop Group on SONiC | Twitter: @ebiken | 2022/11/23 ASIC_DB entry transition (config via CLI) admin@sonic:~$ sudo config route add prefix 10.99.0.0/24 nexthop 10.0.0.100 NEXT_HOP Key: oid: 3 Value: { NEXT_HOP_ATTR_IP: 10.0.0.100, NEXT_HOP_ATTR_ROUTER_INTERFACE_ID: 1 } ROUTE_ENTRY Key: { dest: 10.99.0.0/24, } Value: { NEXT_HOP_ID: 3 }

Slide 13

Slide 13 text

Next Hop Group on SONiC | Twitter: @ebiken | 2022/11/23 ASIC_DB entry transition (config via CLI) admin@sonic:~$ sudo config route add prefix 10.99.0.0/24 nexthop 10.0.0.100 admin@sonic:~$ sudo config route add prefix 10.99.0.0/24 nexthop 10.0.0.101 ROUTE_ENTRY Key: { dest: 10.99.0.0/24, } Value: { NEXT_HOP_ID: 3 } NEXT_HOP_GROUP Key: oid: 5 NEXT_HOP Key: oid: 3 Value: { NEXT_HOP_ATTR_IP: 10.0.0.100, NEXT_HOP_ATTR_ROUTER_INTERFACE_ID: 1 } NEXT_HOP Key: oid: 4 Value: { NEXT_HOP_ATTR_IP: 10.0.0.101, NEXT_HOP_ATTR_ROUTER_INTERFACE_ID: 1 } 1 2

Slide 14

Slide 14 text

Next Hop Group on SONiC | Twitter: @ebiken | 2022/11/23 ASIC_DB entry transition (config via CLI) admin@sonic:~$ sudo config route add prefix 10.99.0.0/24 nexthop 10.0.0.100 admin@sonic:~$ sudo config route add prefix 10.99.0.0/24 nexthop 10.0.0.101 ROUTE_ENTRY Key: { dest: 10.99.0.0/24, } Value: { NEXT_HOP_ID: 3 } NEXT_HOP_GROUP_MEMBER Key: oid: 6 Value: { NEXT_HOP_GROUP_ID: oid: 5, NEXT_HOP_ID: oid: 3 } NEXT_HOP_GROUP Key: oid: 5 NEXT_HOP_GROUP_MEMBER Key: oid: 7 Value: { NEXT_HOP_GROUP_ID: oid: 5, NEXT_HOP_ID': oid: 4 } NEXT_HOP Key: oid: 3 Value: { NEXT_HOP_ATTR_IP: 10.0.0.100, NEXT_HOP_ATTR_ROUTER_INTERFACE_ID: 1 } NEXT_HOP Key: oid: 4 Value: { NEXT_HOP_ATTR_IP: 10.0.0.101, NEXT_HOP_ATTR_ROUTER_INTERFACE_ID: 1 } 1 3 4 2

Slide 15

Slide 15 text

Next Hop Group on SONiC | Twitter: @ebiken | 2022/11/23 ASIC_DB entry transition (config via CLI) admin@sonic:~$ sudo config route add prefix 10.99.0.0/24 nexthop 10.0.0.100 admin@sonic:~$ sudo config route add prefix 10.99.0.0/24 nexthop 10.0.0.101 ROUTE_ENTRY Key: { dest: 10.99.0.0/24, } Value: { NEXT_HOP_ID: 5 } NEXT_HOP_GROUP_MEMBER Key: oid: 6 Value: { NEXT_HOP_GROUP_ID: oid: 5, NEXT_HOP_ID: oid: 3 } NEXT_HOP_GROUP Key: oid: 5 NEXT_HOP_GROUP_MEMBER Key: oid: 7 Value: { NEXT_HOP_GROUP_ID: oid: 5, NEXT_HOP_ID': oid: 4 } NEXT_HOP Key: oid: 3 Value: { NEXT_HOP_ATTR_IP: 10.0.0.100, NEXT_HOP_ATTR_ROUTER_INTERFACE_ID: 1 } NEXT_HOP Key: oid: 4 Value: { NEXT_HOP_ATTR_IP: 10.0.0.101, NEXT_HOP_ATTR_ROUTER_INTERFACE_ID: 1 } 1 3 4 2 5 ※ NEXT_HOP and NEXT_HOP_GROUP will use same oid space => No need for extra info in ROUTE_ENTRY to identify if it’s NEXT_HOP or NEXT_HOP_GROUP

Slide 16

Slide 16 text

Next Hop Group on SONiC | Twitter: @ebiken | 2022/11/23 ASIC_DB entry transition (config via CLI) admin@sonic:~$ sudo config route add prefix 10.88.0.0/24 nexthop 10.0.0.100 admin@sonic:~$ sudo config route add prefix 10.88.0.0/24 nexthop 10.0.0.101 ROUTE_ENTRY Key: { dest: 10.99.0.0/24, } Value: { NEXT_HOP_ID: 5 } NEXT_HOP_GROUP_MEMBER Key: oid: 6 Value: { NEXT_HOP_GROUP_ID: oid: 5, NEXT_HOP_ID: oid: 3 } NEXT_HOP_GROUP Key: oid: 5 NEXT_HOP_GROUP_MEMBER Key: oid: 7 Value: { NEXT_HOP_GROUP_ID: oid: 5, NEXT_HOP_ID': oid: 4 } NEXT_HOP Key: oid: 3 Value: { NEXT_HOP_ATTR_IP: 10.0.0.100, NEXT_HOP_ATTR_ROUTER_INTERFACE_ID: 1 } NEXT_HOP Key: oid: 4 Value: { NEXT_HOP_ATTR_IP: 10.0.0.101, NEXT_HOP_ATTR_ROUTER_INTERFACE_ID: 1 } 1 3 4 2 5 ROUTE_ENTRY Key: { dest: 10.88.0.0/24, } Value: { NEXT_HOP_ID: 5 } 6 ※ ROUTE_ENTRY with same Next Hop will refer to the same NEXT_HOP_GROUP

Slide 17

Slide 17 text

Appendix Next Hop Group on SONiC | Twitter: @ebiken | 2022/11/23

Slide 18

Slide 18 text

Next Hop Group on SONiC | Twitter: @ebiken | 2022/11/23 Full ASIC_DB entry attributed and relationship (config via CLI) admin@sonic:~$ sudo config route add prefix 10.99.0.0/24 nexthop 10.0.0.100 admin@sonic:~$ sudo config route add prefix 10.99.0.0/24 nexthop 10.0.0.101 admin@sonic:~$ sudo config route add prefix 10.88.0.0/24 nexthop 10.0.0.100 admin@sonic:~$ sudo config route add prefix 10.88.0.0/24 nexthop 10.0.0.101 SAI_OBJECT_TYPE_ROUTE_ENTRY Key: { "dest":"10.99.0.0/24", "switch_id":"oid:0x21000000000000", "vr":"oid:0x300000000004a" } Value: { 'SAI_ROUTE_ENTRY_ATTR_NEXT_HOP_ID': 'oid:0x40000000003a3' } SAI_OBJECT_TYPE_NEXT_HOP_GROUP_MEMBER Key: oid:0x2d0000000003a6 Value: { 'SAI_NEXT_HOP_GROUP_MEMBER_ATTR_NEXT_HOP_GROUP_ID': 'oid:0x50000000003a5’, 'SAI_NEXT_HOP_GROUP_MEMBER_ATTR_NEXT_HOP_ID': 'oid:0x40000000003a3’, 'SAI_NEXT_HOP_GROUP_MEMBER_ATTR_WEIGHT': '1', 'SAI_NEXT_HOP_GROUP_MEMBER_ATTR_SEQUENCE_ID': '1' } SAI_OBJECT_TYPE_NEXT_HOP_GROUP Key: oid:0x50000000003a5 Value: { 'SAI_NEXT_HOP_GROUP_ATTR_TYPE': 'SAI_NEXT_HOP_GROUP_TYPE_DYNAMIC_ORDERED_ECMP' } SAI_OBJECT_TYPE_NEXT_HOP_GROUP_MEMBER Key: oid:0x2d0000000003a7 Value: { 'SAI_NEXT_HOP_GROUP_MEMBER_ATTR_NEXT_HOP_GROUP_ID': 'oid:0x50000000003a5’, 'SAI_NEXT_HOP_GROUP_MEMBER_ATTR_NEXT_HOP_ID': 'oid:0x40000000003a4’, 'SAI_NEXT_HOP_GROUP_MEMBER_ATTR_WEIGHT': '1', 'SAI_NEXT_HOP_GROUP_MEMBER_ATTR_SEQUENCE_ID': '2' } SAI_OBJECT_TYPE_NEXT_HOP Key: oid:0x40000000003a3 Value: { 'SAI_NEXT_HOP_ATTR_TYPE': 'SAI_NEXT_HOP_TYPE_IP', 'SAI_NEXT_HOP_ATTR_IP': '10.0.0.100', 'SAI_NEXT_HOP_ATTR_ROUTER_INTERFACE_ID': 'oid:0x60000000003a1' } SAI_OBJECT_TYPE_NEXT_HOP Key: oid:0x40000000003a4 Value: { 'SAI_NEXT_HOP_ATTR_TYPE': 'SAI_NEXT_HOP_TYPE_IP', 'SAI_NEXT_HOP_ATTR_IP': '10.0.0.101', 'SAI_NEXT_HOP_ATTR_ROUTER_INTERFACE_ID': 'oid:0x60000000003a1' } SAI_OBJECT_TYPE_ROUTE_ENTRY Key: { "dest":"10.99.0.0/24", "switch_id":"oid:0x21000000000000", "vr":"oid:0x300000000004a" } Value: { 'SAI_ROUTE_ENTRY_ATTR_NEXT_HOP_ID': 'oid:0x50000000003a5’ } SAI_OBJECT_TYPE_ROUTE_ENTRY Key: { "dest":"10.88.0.0/24", "switch_id":"oid:0x21000000000000", "vr":"oid:0x300000000004a" } Value: { 'SAI_ROUTE_ENTRY_ATTR_NEXT_HOP_ID': 'oid:0x50000000003a5' }

Slide 19

Slide 19 text

Next Hop Group on SONiC | Twitter: @ebiken | 2022/11/23 SWSS / SAI log example (swss.rec , sairedis.rec) (config via CLI) $ tail -f /var/log/swss/swss.log ROUTE_TABLE:10.99.0.0/24|SET| • nexthop:10.0.0.100 • ifname:Ethernet0 NEIGH_TABLE:Ethernet0:10.0.0.100|SET|neigh:0c:42:a1:46:64:a2|family:IPv4 ROUTE_TABLE:10.99.0.0/24|SET| • nexthop:10.0.0.100,10.0.0.101 • ifname:Ethernet0,Ethernet0 • weight:1,1 NEIGH_TABLE:Ethernet0:10.0.0.101|SET|neigh:0c:42:a1:46:64:a2|family:IPv4 $ tail -f /var/log/swss/sairedis.rec SAI_OBJECT_TYPE_NEIGHBOR_ENTRY:{"ip":"10.0.0.100","rif":"oid:0x60000000003a1","switch_id":"oid:0x21000000000000"} • SAI_NEIGHBOR_ENTRY_ATTR_DST_MAC_ADDRESS=0C:42:A1:46:64:A2 SAI_OBJECT_TYPE_NEXT_HOP:oid:0x40000000003a3 • SAI_NEXT_HOP_ATTR_TYPE=SAI_NEXT_HOP_TYPE_IP • SAI_NEXT_HOP_ATTR_IP=10.0.0.100 • SAI_NEXT_HOP_ATTR_ROUTER_INTERFACE_ID=oid:0x60000000003a1 SAI_OBJECT_TYPE_ROUTE_ENTRY||{"dest":"10.99.0.0/24","switch_id":"oid:0x21000000000 000","vr":"oid:0x300000000004a"} • SAI_ROUTE_ENTRY_ATTR_NEXT_HOP_ID=oid:0x40000000003a3 SAI_OBJECT_TYPE_NEIGHBOR_ENTRY:{"ip":"10.0.0.101","rif":"oid:0x60000000003a1","switch_id":"oid:0x21000000000000"} • SAI_NEIGHBOR_ENTRY_ATTR_DST_MAC_ADDRESS=0C:42:A1:46:64:A2 SAI_OBJECT_TYPE_NEXT_HOP:oid:0x40000000003a4 • SAI_NEXT_HOP_ATTR_TYPE=SAI_NEXT_HOP_TYPE_IP • SAI_NEXT_HOP_ATTR_IP=10.0.0.101 • SAI_NEXT_HOP_ATTR_ROUTER_INTERFACE_ID=oid:0x60000000003a1 SAI_OBJECT_TYPE_NEXT_HOP_GROUP:oid:0x50000000003a5 • SAI_NEXT_HOP_GROUP_ATTR_TYPE=SAI_NEXT_HOP_GROUP_TYPE_DYNAMIC_ORDERED_ECMP SAI_OBJECT_TYPE_NEXT_HOP_GROUP_MEMBER ||oid:0x2d0000000003a6 • SAI_NEXT_HOP_GROUP_MEMBER_ATTR_NEXT_HOP_GROUP_ID=oid:0x50000000003a5 • SAI_NEXT_HOP_GROUP_MEMBER_ATTR_NEXT_HOP_ID=oid:0x40000000003a3 • SAI_NEXT_HOP_GROUP_MEMBER_ATTR_WEIGHT=1 • SAI_NEXT_HOP_GROUP_MEMBER_ATTR_SEQUENCE_ID=1 ||oid:0x2d0000000003a7 • SAI_NEXT_HOP_GROUP_MEMBER_ATTR_NEXT_HOP_GROUP_ID=oid:0x50000000003a5 • SAI_NEXT_HOP_GROUP_MEMBER_ATTR_NEXT_HOP_ID=oid:0x40000000003a4 • SAI_NEXT_HOP_GROUP_MEMBER_ATTR_WEIGHT=1 • SAI_NEXT_HOP_GROUP_MEMBER_ATTR_SEQUENCE_ID=2 SAI_OBJECT_TYPE_ROUTE_ENTRY||{"dest":"10.99.0.0/24","switch_id":"oid:0x21000000000 000","vr":"oid:0x300000000004a"} • SAI_ROUTE_ENTRY_ATTR_NEXT_HOP_ID=oid:0x50000000003a5

Slide 20

Slide 20 text

Next Hop Group on SONiC | Twitter: @ebiken | 2022/11/23 Next Hop Group on FRR sonic# show ip route nexthop-group Codes: K - kernel route, C - connected, S - static, R - RIP, ...snip... S>* 10.88.0.0/24 [1/0] (26) via 10.0.0.100, Ethernet0, weight 1, 00:01:39 * via 10.0.0.101, Ethernet0, weight 1, 00:01:39 S>* 10.99.0.0/24 [1/0] (26) via 10.0.0.100, Ethernet0, weight 1, 04:56:29 * via 10.0.0.101, Ethernet0, weight 1, 04:56:29 sonic# show nexthop-group rib 26 ID: 26 (zebra) RefCnt: 2 Uptime: 01:08:01 VRF: default Valid, Installed Depends: (22) (27) via 10.0.0.100, Ethernet0 (vrf default), weight 1 via 10.0.0.101, Ethernet0 (vrf default), weight 1 sonic# show nexthop-group rib 22 ID: 22 (zebra) RefCnt: 3 Uptime: 02:27:13 VRF: default Valid, Installed Interface Index: 8 via 10.0.0.100, Ethernet0 (vrf default), weight 1 Dependents: (26) sonic# show nexthop-group rib 27 ID: 27 (zebra) RefCnt: 3 Uptime: 07:19:32 VRF: default Valid, Installed Interface Index: 8 via 10.0.0.101, Ethernet0 (vrf default), weight 1 Dependents: (26) Use “vtysh” command on SONiC host to access FRR CLI admin@sonic:~$ vtysh Hello, this is FRRouting (version 8.2.2). Copyright 1996-2005 Kunihiro Ishiguro, et al. sonic# As of today, FRR do NOT use NHA_GROUP (Next Hop Attribute Group) when communicating with fpmsyncd. Thus, it will be same as when configured via CLI. In other words, APPL_DB will only have ROUTE_TABLE entry and NEXT_HOP_GROUP_TABLE entry would not be created.

Slide 21

Slide 21 text

Next Hop Group on SONiC | Twitter: @ebiken | 2022/11/23 nexthop_id for direct attached network > nexthop_id in route entry will point to router_interface (and not nexthop object) admin@sonic:~$ sonic-db-cli ASIC_DB HGETALL 'ASIC_STATE:SAI_OBJECT_TYPE_ROUTE_ENTRY:{ "dest":"10.0.0.0/24","switch_id":"oid:0x21000000000000","vr":" oid:0x300000000004a"}' {'SAI_ROUTE_ENTRY_ATTR_NEXT_HOP_ID': 'oid:0x60000000003a1’} admin@sonic:~$ sonic-db-cli ASIC_DB HGETALL ASIC_STATE:SAI_OBJECT_TYPE_ROUTER_INTERFACE:oid:0x60000000003a1 {'SAI_ROUTER_INTERFACE_ATTR_VIRTUAL_ROUTER_ID': 'oid:0x300000000004a', 'SAI_ROUTER_INTERFACE_ATTR_SRC_MAC_ADDRESS': '00:90:FB:65:D6:FE’, 'SAI_ROUTER_INTERFACE_ATTR_TYPE': 'SAI_ROUTER_INTERFACE_TYPE_PORT’, 'SAI_ROUTER_INTERFACE_ATTR_PORT_ID': 'oid:0x1000000000050’, 'SAI_ROUTER_INTERFACE_ATTR_MTU': '9100’}

Slide 22

Slide 22 text

Next Hop Group on SONiC | Twitter: @ebiken | 2022/11/23 How to check ASIC entries ? admin@sonic:~$ docker exec -it syncd bash root@sonic:/# root@sonic:/# /opt/bfn/install/bin/bfshell ******************************************** * WARNING: Authorised Access Only * ******************************************** bfshell> Use ASIC specific debug command from syncd or sonic host Actual command and output are under NDA, thus omitted from this slide deck. admin@sonic:~$ bcmsh Press Enter to show prompt. Press Ctrl+C to exit. NOTICE: Only one bcmsh or bcmcmd can connect to the shell at same time. drivshell>

Slide 23

Slide 23 text

• Improving Route Scalability: Nexthops as Separate Objects • Linux Plumbers Conf 2019: David Ahern @Cumulus • https://lpc.events/event/4/contributions/434/attachments/251/436/nexthop-objects-talk.pdf • https://www.youtube.com/watch?v=HIqvUiwDHGk • lwn.net [net: Improve route scalability via support for nexthop objects] • https://lwn.net/Articles/763950/ • FRR: Docs: • Nexthop Groups: https://docs.frrouting.org/en/latest/nexthop_groups.html • Multiple nexthop static route: https://docs.frrouting.org/en/latest/static.html#multiple- nexthop-static-route • RFC5549 Advertising IPv4 Network Layer Reachability Information with an IPv6 Next Hop • https://datatracker.ietf.org/doc/rfc5549/ • RFC8950 Advertising IPv4 Network Layer Reachability Information (NLRI) with an IPv6 Next Hop • https://datatracker.ietf.org/doc/rfc8950/ Next Hop Group on SONiC | Twitter: @ebiken | 2022/11/23 References