RMW Zenoh Configurations

Slides presented at ROS Meetup Belgium 2025

Transcript

1. RMW Zenoh Configurations Unleashing Efficiency in ROS 2 Senior Solution Architect ZettaScale Technology Julien Enoch
2. RMW Zenoh - Getting started
3. ROS 2 has a modular architecture
4. Availability Tier-1 support
5. Installation • Install rmw-zenoh-cpp package from ROS 2 apt repo: sudo apt install ros-${ROS_DISTRO}-rmw-zenoh-cpp • Switch from other rmw to rmw_zenoh_cpp: export RMW_IMPLEMENTATION=rmw_zenoh_cpp • As a quick test: • Start the Zenoh router: ros2 run rmw_zenoh_cpp rmw_zenohd • Start the talker / listener demo: ros2 run demo_nodes_cpp talker ros2 run demo_nodes_cpp listener
6. Why a Zenoh router ? • No UDP multicast • Avoid miscon fi guration issues • Prevent uncontrolled communications • Discovery via the router • Gossip protocol for Nodes interconnections • External communication via the router • Less connections, less overheads • Batching for better throughput • Smaller surface of attack • Single point to con fi gure Access Control and Downsampling Node 1 Node 2 Node 3 Node 4 rmw_zenohd
7. Connectivity Configuration
8. Default Con fi guration fi les Node 1 Node 2 Node 3 Node 4 rmw_zenohd • In /opt/ros/$ROS_DISTRO/share/rmw_zenoh_cpp/con fi g/ • For the Zenoh router: DEFAULT_RMW_ZENOH_ROUTER_CONFIG.json5 • For all the nodes: DEFAULT_RMW_ZENOH_SESSION_CONFIG.json5 { mode: "router", connect: { endpoints: [] }, listen: { endpoints: [ "tcp/[::]:7447" ] }, scouting: { gossip: { enabled: true, autoconnect: { router: [] } } } } { mode: "peer", connect: { endpoints: [ "tcp/localhost:7447" ] }, listen: { endpoints: [ "tcp/localhost:0" ] }, scouting: { gossip: { enabled: true, autoconnect: { peer: ["router", "peer"] } } } }
9. How to change the Zenoh con fi guration ? • Copy and edit the default con fi g fi les • De fi ne environment variables with path to the con fi g fi les • ZENOH_ROUTER_CONFIG_URI for rmw_zenohd • ZENOH_SESSION_CONFIG_URI for the nodes • Override some con fi g settings via the environment variable: • ZENOH_CONFIG_OVERRIDE='key/to/field=value;...' AND / OR
10. Connect to the robot Node 1 Node 2 Node 3 Node 4 rmw_zenohd rmw_zenohd ros2cli rqt teleop rviz2 192.168.42.2 • Con fi gure rmw_zenohd to connect to robot’s rmw_zenohd export ZENOH_CONFIG_OVERRIDE='connect/endpoints=["tcp/192.168.42.2:7447"]' ros2 run rmw_zenoh_cpp rmw_zenohd
11. Connect just 1 Node to the robot Node 1 Node 2 Node 3 Node 4 rmw_zenohd rviz2 192.168.42.2 • No need of a local router • Con fi gure the Node to connect to robot’s rmw_zenohd export ZENOH_CONFIG_OVERRIDE= 'mode="client";connect/endpoints=["tcp/192.168.42.2:7447"]' ros2 run rviz2 rviz2
12. Fleet connectivity rmw_zenohd robot-1 rmw_zenohd rmw_zenohd rmw_zenohd 2 possibilities: • Either the Fleet Manager connects to all robots. In Fleet Manager • Either each robot connects to the fleet manager. In each robot’s router config: { /// Which endpoints to connect to. connect: { endpoints: [ "tcp/robot-1.local:7447", "tcp/robot-2.local:7447", "tcp/robot-3.local:7447" ] } } $ZENOH_ROUTER_CONFIG_URI { /// Which endpoints to connect to. connect: { endpoints: [ "tcp/fleet-manager.local:7447", ] } } $ZENOH_ROUTER_CONFIG_URI robot-2 robot-3 fl eet-manager
13. Routers discovery via UDP multicast • How to make the Routers to automatically discover and connect to each other ? • Via the ZENOH_ROUTER_CONFIG_URI con fi g fi le: { // ... scouting: { multicast: { enabled: true, autoconnect: { router: ["router"] }, } } // ... } • Or via the ZENOH_CONFIG_OVERRIDE : export ZENOH_CONFIG_OVERRIDE='scouting/multicast/enabled=true;scouting/multicast/ autoconnect=["router"]' ros2 run rmw_zenoh_cpp rmw_zenohd
14. AMR Multi-hosts robot Node 1 Node 2 Node 3 Node 4 rmw_zenohd ARM eth Node A Node B • Better to have 1 router only • Con fi gure all the Nodes’ Sessions to: 1. connect to the router over eth 2. listen on their host’s eth IP 192.168.42.2 192.168.2.1 192.168.2.2 { mode: "peer", connect: { endpoints: [ "tcp/192.168.2.1:7447" ] }, listen: { endpoints: [ "tcp/192.168.2.2:0" ] }, } { mode: "peer", connect: { endpoints: [ "tcp/192.168.2.1:7447" ] }, listen: { endpoints: [ "tcp/192.168.2.1:0" ] }, } ZENOH_SESSION_CONFIG_URI ZENOH_SESSION_CONFIG_URI
15. Data Flow Configuration
16. Downsampling { /// ... downsampling: [ { interfaces: [ "wlan0" ], flow: ["egress"], messages: [ /// Publication (Put and Delete) "push", ], rules: [ { key_expr: "*/scan/**", freq: 1 }, { key_expr: "*/point_cloud/**", freq: 0.5 }, ], } ], /// ... } ZENOH_ROUTER_CONFIG_URI • WiFi cannot sustain the throughput of publications • Solution: con fi gure the router to downsample topics with high throughput Node 1 Node 2 Node 3 Node 4 rmw_zenohd rviz2
17. 🤔 Why those weird "key_expr" in con fi g fi le ? • Mapping of ROS Topics and Services to Zenoh key expressions: • <domain_id>/<fully_qualified_name>/<type_name>/<type_hash> • Examples: •0/chatter/std_msgs::msg::dds_::String_/RIHS01_df668c740482bbd48fb39d76a70dfd4bd59db1288… •0/robot1/chatter/std_msgs::msg::dds_::String_/RIHS01_df668c740482bbd48fb39d76a70dfd4bd5… •2/add_two_ints/example_interfaces::srv::dds_::AddTwoInts_/RIHS01_e118de6bf5eeb66a2491b5… • Zenoh key expressions matching: • * matches exactly 1 chunk (no / ) • ** matches 0, 1 or more chunks (including / characters)
18. • I don’t want some topics and services to be exposed over WiFi • Solution: con fi gure the router to control the access from WiFi Access Control (per network interface) Node 1 Node 2 Node 3 Node 4 rmw_zenohd rviz2 { /// ... access_control: { enabled: true, default_permission: "allow", /// List of rules setting permissions per key-expression rules: [ { id: "denied", messages: [ "put", "declare_subscriber", "query", "reply", "declare_queryable", "liveliness_token", ], flows:["egress","ingress"], permission: "deny", key_exprs: [ "*/_internal/**", "*/cmd_vel/**" ], } ], /// List of combinations of subjects. subjects: [ { id: "wifi", interfaces: [ "wlan0",], } ], /// The policies list associates rules to subjects policies: [ { rules: ["denied"], subjects: ["wifi"], } ] }, /// ... } ZENOH_ROUTER_CONFIG_URI
19. Zenoh QoS - Automatic Batching Zenoh automatically batches messages to: ‣ Improve Throughput ‣ Reduce Network Overheard ‣ Reduce CPU utilisation put(“foo”, ) Network put(“bar”, ) put(“baz”,) To bypass batching: use express mode
20. Zenoh QoS - Priority Network Scheduling Zenoh has building support for priority-based network scheduling It allows to segregate di ff erent priorities on separate network session put(key, Prio N ) put(key, Prio N+1 ) Network … Prio N Prio N put(key, Prio N+2 ) Prio N+2 Prio N+1 Prio N+1 Seven (7) priority levels are supported Within a session priority inversion is bounded by time to send a fragment
21. • I want to tune the routing of publications and queries when routed over WiFi • Solution: con fi gure the router to overwrite Zenoh QoS when routing to WiFi QoS overwrite (per network interface) Node 1 Node 2 Node 3 Node 4 rmw_zenohd rviz2 { /// ... qos: { network: [ { interfaces: [ "wlan0" ], messages: [ "put", "query", "reply" ], flows: ["egress"], key_exprs: ["*/point_cloud/**"], overwrite: { /// Optional new priority value priority: "data_low", /// Optional new congestion control value congestion_control: "drop", /// Optional new express value (to bypass batching) express: true, }, } ] }, /// ... } ZENOH_ROUTER_CONFIG_URI
22. • PR in review: ros2/rmw_zenoh#363 • Port to Jazzy for testing: ros2/rmw_zenoh#656 Share Memory (not yet released) { /// ... shared_memory: { enabled: true, }, /// ... } ZENOH_ROUTER_CONFIG_URI { /// ... shared_memory: { enabled: true, }, /// ... } ZENOH_SESSION_CONFIG_URI • Each Session/Router has its own SHM segment. default: 16 MB con fi g env var: ZENOH_SHM_ALLOC_SIZE • No daemon ! Automatic handshake between Sessions. • Automatic use of SHM for large payloads. default: 512 B con fi g env var: ZENOH_SHM_MESSAGE_SIZE_THRESHOLD • If SHM allocation fails, fallback to TCP
23. Patience, persistence and perspiration make an unbeatable combination for success. Thank You