CTRMs: Learning to Construct Cooperative Timed Roadmaps for Multi-agent Path Planning in Continuous Spaces (AAMAS 2022) Keisuke Okumura**, Ryo Yonetani*, Mai Nishimura*, and Asako Kanezaki** * OMRON SINIC X | ** Tokyo Institute of Technology Presented at International Conference on Autonomous Agents and Multiagent Systems (AAMAS) 2022 May 9-13, 2022 Ⓒ 2022 OMRON SINIC X Corporation. All Rights Reserved.

1Tokyo Institute of Technology 2OMRON SINIC X Keisuke Okumura1, Ryo Yonetani2, Mai Nishimura2 & Asako Kanezaki1 AAMAS-22, Online, May 9th–13th 2022 CTRMs: Learning to Construct Cooperative Timed Roadmaps for Multi-agent Path Planning in Continuous Spaces Ⓒ 2022 OMRON SINIC X Corporation. All Rights Reserved.

/30 2 Background collision-free path planning for multiple robots real-time, scalable, and high-quality recent progress of multi-agent pathfinding (MAPF) for discretized environments https://automation.omron.com/en/us/industries/logistics/ i.e., roadmaps Ⓒ 2022 OMRON SINIC X Corporation. All Rights Reserved.

/30 3 Roadmap design affects solution quality on non-deliberative discretized spaces ideal paths Making roadmaps denser help situations? goal start often used in MAPF studies [Stern+ SOCS-19] Ⓒ 2022 OMRON SINIC X Corporation. All Rights Reserved.

/30 4 dense sparse large small planning effort high low solution quality big impact in multi-agent cases ideal: small search space containing high-quality solutions No, there is a trade-off *produced by PRM [Kavraki+ 1996] Ⓒ 2022 OMRON SINIC X Corporation. All Rights Reserved.

/30 5 constructing small roadmaps containing high-quality solutions multi-agent path planning in continuous spaces solving multi-agent pathfinding efficiently 1. agent-specific 2. cooperative 3. timed CTRMs: cooperative timed roadmaps Our Approach Ⓒ 2022 OMRON SINIC X Corporation. All Rights Reserved.

/30 6 How to Construct CTRMs? casting as a machine learning problem from planning demonstraitons, learning important regions of each agent and interactions between agents agent-specific cooperative using the trained model to construct timed roadmaps Ⓒ 2022 OMRON SINIC X Corporation. All Rights Reserved.

/30 7 Outline of Approach new instance 𝐹!"#$ random walk sampling module next locations starts path generation compositing solution MAPF algorithm … t=0 t=1 t=2 CTRMs 𝐹!"#$ model training instances & solutions predict next locations Online Inference Offline Training CVAE: Conditional Variational Autoencoder [Sohn+ NeurIPS-15] +importance sampling [Salzmann+ ECCV-20] +multi-agent attention [Hoshen NeurIPS-17] 𝐹!"#$ : *independent from map size and #agent, not limited to homo-agents Ⓒ 2022 OMRON SINIC X Corporation. All Rights Reserved.

/30 8 Outline of Approach new instance 𝐹!"#$ random walk sampling module next locations starts path generation compositing solution MAPF algorithm … t=0 t=1 t=2 CTRMs 𝐹!"#$ model training instances & solutions predict next locations Online Inference Offline Training CVAE: Conditional Variational Autoencoder [Sohn+ NeurIPS-15] +importance sampling [Salzmann+ ECCV-20] +multi-agent attention [Hoshen NeurIPS-17] 𝐹!"#$ : Ⓒ 2022 OMRON SINIC X Corporation. All Rights Reserved.

/30 9 𝐹!"#$ next position Offline Training & Model Arch. instance & solution generative model collected by intensive computation with conventional roadmaps CVAE [Sohn+ NeurIPS-15] Ⓒ 2022 OMRON SINIC X Corporation. All Rights Reserved.

/30 10 Offline Training & Model Arch. occupancy cost-to-go env. info Ⓒ 2022 OMRON SINIC X Corporation. All Rights Reserved.

/30 11 Offline Training & Model Arch. features ? Ⓒ 2022 OMRON SINIC X Corporation. All Rights Reserved.

/30 12 Offline Training & Model Arch. + + relative positions, size, speeds, etc goal-driven features encoded by neural network Ⓒ 2022 OMRON SINIC X Corporation. All Rights Reserved.

/30 13 Offline Training & Model Arch. + + communication features learning interactions with nearby agents multiple agents multi-agent attention [Sohn+ NeurIPS-15] relative positions, size, speeds, etc env. info encoded by neural network Ⓒ 2022 OMRON SINIC X Corporation. All Rights Reserved.

/30 14 Offline Training & Model Arch. go right [0,0,1] indicator feature Ⓒ 2022 OMRON SINIC X Corporation. All Rights Reserved.

/30 15 + + + + go right [0,0,1] 𝐹!"#$ next position goal-driven features comm. features indicator feature Offline Training & Model Arch. instance & solution occupancy cost-to-go env. info relative positions, size, speeds, etc attention Ⓒ 2022 OMRON SINIC X Corporation. All Rights Reserved.

/30 16 Outline of Approach new instance 𝐹!"#$ random walk sampling module next locations starts path generation compositing solution MAPF algorithm … t=0 t=1 t=2 CTRMs 𝐹!"#$ model training instances & solutions predict next locations Online Inference Offline Training CVAE: Conditional Variational Autoencoder [Sohn+ NeurIPS-15] +importance sampling [Salzmann+ ECCV-20] +multi-agent attention [Hoshen NeurIPS-17] 𝐹!"#$ : Ⓒ 2022 OMRON SINIC X Corporation. All Rights Reserved.

/30 17 Online Inference observations for agent-i next predicted location for agent-i trained model likely to be used by planners Ⓒ 2022 OMRON SINIC X Corporation. All Rights Reserved.

/30 18 Online Inference observations for agent-1 next predicted location for agent-1 observations for agent-N observations for agent-2 next predicted location for agent-2 next predicted location for agent-N … … timestep t timestep t+1 Ⓒ 2022 OMRON SINIC X Corporation. All Rights Reserved.

/30 19 Online Inference timestep t timestep t+1 next predicted locations for all agents observations for all agents Ⓒ 2022 OMRON SINIC X Corporation. All Rights Reserved.

/30 20 Online Inference t=0 t=1 t=2 t=T t=T-1 … initial locations timed path for agent-i each path is agent-specific and cooperative hyperparameter Ⓒ 2022 OMRON SINIC X Corporation. All Rights Reserved.

/30 21 Online Inference … t=0 t=1 t=2 t=T t=T-1 Ⓒ 2022 OMRON SINIC X Corporation. All Rights Reserved.

/30 22 Online Inference … … … … compositing t=0 t=1 t=2 t=T t=T-1 timed roadmap for agent-i each roadmap is agent-specific and cooperative hyperparameter: #(path generation) Ⓒ 2022 OMRON SINIC X Corporation. All Rights Reserved.

/30 23 Outline of Approach new instance 𝐹!"#$ random walk sampling module next locations starts path generation compositing solution MAPF algorithm … t=0 t=1 t=2 CTRMs 𝐹!"#$ model training instances & solutions predict next locations Online Inference Offline Training CVAE: Conditional Variational Autoencoder [Sohn+ NeurIPS-15] +importance sampling [Salzmann+ ECCV-20] +multi-agent attention [Hoshen NeurIPS-17] 𝐹!"#$ : *independent from map size and #agent, not limited to homo-agents Ⓒ 2022 OMRON SINIC X Corporation. All Rights Reserved.

/30 24 Evaluation Ⓒ 2022 OMRON SINIC X Corporation. All Rights Reserved.

/30 25 Roadmap Visualization SPARS [Dobson & Bekris, IJRR-14] (random) simplified PRM [Karaman & Frazzoli, IJRR-11] square as agent-specific roadmaps grid as used in MAPF studies CTRMs 20-30 homo agents corresponding to 32x32 grids CTRMs produce small but effective roadmaps specific to each agent Ⓒ 2022 OMRON SINIC X Corporation. All Rights Reserved.

/30 26 Quantitative Results 0 3000 6000 saPSles Ser (agent, tiPesteS) 0.0 0.2 0.4 0.6 0.8 1.0 success rate CTR0s randRP grid 63AR6 sTuare 20-30 homo agents corresponding to 32x32 grids 100 instances solved by prioritized planning [Silver, AIIDE-05, Van Den Berg & Overmars IROS-05, etc] CTRMs contain solutions in small search spaces sparse dense params of CTRMs: #(path generation) Ⓒ 2022 OMRON SINIC X Corporation. All Rights Reserved.

/30 27 Quantitative Results 103 104 105 exSanded nRdes / agents 0 10 20 30 40 suP-Rf-cRsts / agents average Rver 40/100 instances CT50s randRP grid S3A5S sTuare 20-30 homo agents corresponding to 32x32 grids 100 instances solved by prioritized planning [Silver, AIIDE-05, Van Den Berg & Overmars IROS-05, etc] CTRMs reduce planning effort while keeping solution qualities params of CTRMs: #(path generations) sparse dense Ⓒ 2022 OMRON SINIC X Corporation. All Rights Reserved.

/30 28 Quantitative Results CT50 s randRP grid S3A5S sTuare 0 100 200 300 400 500 runtiPe (sec) x average Rver 40/100 instances rRadPaS Slanner 20-30 homo agents corresponding to 32x32 grids 100 instances solved by prioritized planning [Silver, AIIDE-05, Van Den Berg & Overmars IROS-05, etc] CTRMs achieve efficient path-planning from the end-to-end perspective sparse dense *Roadmap construction can be much faster. Check our latest implementation: https://github.com/omron-sinicx/jaxmapp Ⓒ 2022 OMRON SINIC X Corporation. All Rights Reserved.

/30 29 Quantitative Results basic no obstacles more obstacles more agents hetero agents CTRMs consistently outperform other baselines small search spaces but containing plausible solutions (Check our paper for details) reducing planning effort by orders of magnitude Ⓒ 2022 OMRON SINIC X Corporation. All Rights Reserved.

/30 30 multi-agent path planning in continuous spaces motivation effective roadmaps for multiple agents? challenge CTRMs / data-driven roadmap construction proposal reducing planning effort (e.g., runtime) significantly result Concluding Remarks project page: https://omron-sinicx.github.io/ctrm/ anytime planning, higher-dimensional spaces future work instance roadmaps solution start multi-agent pathfinding goal Ⓒ 2022 OMRON SINIC X Corporation. All Rights Reserved.