ros2-perf-multihost: Automated Coordination Framework for Objective Architecture Evaluation in Distributed Systems

ROSCon Global 2026
2026/09/24
https://roscon.ros.org/2026/

Description:
The “RMW Cambrian Explosion" following Zenoh’s integration presents developers with complex middleware choices and architectural challenges. Selecting the optimal RMW and system configuration requires empirical data from actual physical hardware. We introduce ros2-perf-multihost, an open-source framework that coordinates evaluation pipelines across multiple physical devices. It enables developers to quantify how node placement and network configurations impact overall stability. Our purpose is to provide a "scientific scale" for optimizing distributed system design across edge devices and servers with real-world networks, empowering data-driven decisions for large-scale robotic systems.

GitHub: https://github.com/hal-lab-u-tokyo/ros2-perf-multihost

Transcript

1. ROSCon Global 2026 2026/09/24 ros2-perf-multihost Automated Coordination Framework for Objective Architecture Evaluation in Distributed Systems Hideki Takase, Yuma Fujiwara, Yugo Tanaka, Takumi Kudo (The University of Tokyo)
2. Rumors about RMW,,, Which DDS is good? Let's make it distributed!! Is Zenoh the future? Deploy with Docker!! Does QoS really help? So many settings,,, Let’s discuss “scientifically”!! Connect to the cloud!! 2
3. Topology and Evaluation Framework Node Topic Host https://gitlab.com/ApexAI/ performance_test https://github.com/irobotros/ros2-performance 3
4. Topology and Evaluation Framework How can we evaluate complex distributed ROS 2 systems? 4
5. ros2-perf-multihost Objective Architecture Evaluation Framework for ROS 2 distributed systems • Objectives 1. Bridge the build–evaluation gap 2. Enable systematic measurement through multi-host coordination 3. Support design decisions through repeatable experiments 4. Build a “data-driven benchmark library ecosystem” • Evaluation scope −Communication workloads and their host resource usage −Especially RMW, but application computation is not reproduced 5
6. Step1: Define Topology Node placement host1 pub1 pub2 amazon inazuma json inter1 sub1 host2 host3 Node/Topic relationships Topic settings (msg type/size, pub rate, QoS) 6
7. Step2: Generate Execution Scripts python3 manager_scripts/generate_exec_scripts.py \ topology_example/simple.json \ --ws-dir performance_ws Files for host1 Manager json exec1.sh launch1.py compose1.yml Generate once, reuse across runs • Multiple RMWs • Native / Docker Files for host2 exec2.sh launch2.py compose2.yml Files for host3 exec3.sh launch3.py compose3.yml 7
8. Step3: Run Benchmark python3 performance_test/performance_test.py \ simple \ --rmw fastdds,cyclonedds,zenoh \ RMW selection --exec-policy native \ --fastdds-discovery-server host1 \ “native” or “docker” Placement of Discovery Server / Zenoh Router result2.csv result3.csv py yml Automated workflow 1) Deploy execution files 2) Run repeated trials 3) Collect logs & results REST result1.csv sh Measurement window & trial count sh py yml REST Manager REST --zenoh-router host1 \ --warmup-time 10 --eval-time 100 --trials 10 sh py yml 8
9. Step4: Results and Analysis Condition: 3 hosts / Native / 10 s warmup + 100 s measurement / 10 trials Latency distribution (CDF [%]) Throughput and Jitter Sierra Nevada / hamburg [host1] to osaka [host2] via parana Sierra Nevada / parana / Fast DDS / trial 1 100 Received payload [B/s] / 5 s averages 150 100 50 0 75 50 0 25 25 50 75 100 Receive interval deviation [ms] / nominal interval: 100 ms 0 0.2 0 0.3 Fast DDS 0.6 0.9 Observed one-way latency [ms] Fast DDS+DS Cyclone DDS 1.2 0.0 -0.2 Zenoh 0 25 50 Elapsed measurement time [s] 75 100 Simple topology / 3,000 deliveries per cell across 10 trials Sierra Nevada / 10-trial means / N: Native, D: Docker 1.5 mean of 10 trial 800 averages QoS configuration KEEP_LAST 10 / BEST_EFFORT KEEP_LAST 1 / BEST_EFFORT KEEP_LAST 1 / RELIABLE KEEP_ALL / RELIABLE Fast DDS 3.23 0.00 0.00 0.00 Fast DDS + DS Cyclone DDS Zenoh 0.00 3.23 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Memory [MiB] CPU and memory usage CPU [%] Unreceived messages by QoS [%] 1.0 0.5 0.0 host1 host1 host2 host2 host3 host3 N D N D N D 400 0 host1 host1 host2 host2 host3 host3 N D N D N D Caution! These results are not fully reliable due to poor clock synchronization accuracy 9
10. Summary Objective Architecture Evaluation Framework for ROS 2 distributed systems • Automates multi-host evaluation pipelines • Provides scientific metrics for system integrators, RMW/middleware developers, and researchers • Supports architecture decisions, not simply picking a “winner”! 10
11. Join us!! :D • Call for Topologies!! −Share your ROS 2 topology JSON (through a pull request) −Discuss benchmark results and experimental conditions −Help build a “data-driven benchmark library ecosystem” • TODOs,,, −Clock synchronization validation and detailed performance studies ✓We hope to share more results on Discourse very soon!! −Fine-grained DDS/Zenoh configuration and tuning −Evaluation across subnets and wide-area networks including cloud −GUI support for topology editing and result visualization 11
12. May 20-21, 2027 in Osaka!!