Remove Assumptions in Multi-Agent Pathfinding

Slide 1

Slide 1 text

Scalability Assumptions Keisuke Okumura1,2 12th Feb. 2024, idealworks, Munich 1University of Cambridge 2National Institute of Advanced Industrial Science and Technology (AIST), Japan kei18 https://kei18.github.io ko393@cl.cam.ac.uk Remove Assumptions in Multi-Agent Pathfinding Image by Gordon Johnson from Pixabay

Slide 2

Slide 2 text

/92 2 AP Photo/Ross D. Franklin Ross D. Franklin / AP Photo "Swarm" automation is ubiquitous

Slide 3

Slide 3 text

/92 4 underlying common problem: Okumura & Defago. IJCAI-23 collision-free pathfinding for multiple agents

Slide 4

Slide 4 text

/92 5 given agents graph goals solution paths without collisions cost total travel time, distance, makespan, etc MAPF: Multi-Agent Path Finding

Slide 5

Slide 5 text

/92 6 CBS Sharon+ AAAI-12 -> AIJ-15 ICTS Sharon+ IJCAI-11 -> AIJ-12 two-level search HCA* Silver AIIDE-05 Prioritized Planning Erdmann+ 87 decoupled planning A*+ID+OD Standley AAAI-10 M* Wagner+ IROS-11 -> AIJ-15 search for joint state space Push & Swap Luna+ IJCAI-11 Push & Rotate De Wilde+ AAMAS-13 -> JAIR-14 rule-based SAT-based Surynek+ ECAI-16 multiflow-based Yu+ WAFR-12 -> T-RO-16 reduction-based Sorry but I cannot list everything… PBS Ma+ AAAI-19 BCP Lam+ IJCAI-19 -> COR-22 extending two-level LNS Li+ IJCAI-21, AAAI-22 LaCAM* Okumura AAAI-23, IJCAI-23 meta-algo. PIBT Okumura+ IJCAI-19 -> AIJ-22 extremely scalable EECBS Li+ AAAI-21. CBSH Felner+ ICAPS-18 ICBS Boyarski+ IJCAI-15 ECBS Barer+ SoCS-14 CBS family +various enhancements on CBS MAPF Algorithm Forest 2012 1st Workshop on MAPF 2010 2019 3rd WoFMAPF 2023 5th WoMAPF my Ph.D.

Slide 6

Slide 6 text

/92 7 Recent methods can solve MAPF for 10k agents in 5 seconds on my laptop Okumura, K., Machida, M., Défago, X., & Tamura, Y. Priority Inheritance with Backtracking for Iterative Multi-agent Path Finding. AIJ. 2022. Okumura, K. Improving LaCAM for Scalable Eventually Optimal Multi-Agent Pathfinding. IJCAI. 2023.

Slide 7

Slide 7 text

/92 8 But are MAPF algorithms deploy-ready? MAPF's development is remarkable.

Slide 8

Slide 8 text

/92 9 Real world is more complicated typical research rotation aggressive moves sometimes unavailable task interaction what the industry expects asynchrony heterogeneity continuous space imperfect execution, sometimes crash com m . delay algorithms work with X agents with Y (≪ X) real agents

Slide 9

Slide 9 text

/92 10 to keep MAPF alive increase X decrease ||X – Y|| algorithms work with X agents with Y (≪ X) real agents lab research real world strategy-1: strategy-2:

Slide 10

Slide 10 text

/92 11 to keep MAPF alive increase X decrease ||X – Y|| algorithms work with X agents with Y (≪ X) real agents lab research real world strategy-1: strategy-2:

Slide 11

Slide 11 text

/92 12 One Reason for Strategy-1: Feedback Control system (e.g., warehouse) query MAPF plan state e.g., 1~10s uncertainties

Slide 12

Slide 12 text

/92 13 ~2023 2024 random-32-32-20, 409 agents, 30s timeout Okumura, K. Engineering LaCAM∗: Towards Real-Time, Large-Scale, and Near-Optimal Multi-Agent Pathfinding. AAMAS. 2024. There are sufficiently scalable methods

Slide 13

Slide 13 text

/92 14 to keep MAPF alive increase X decrease ||X – Y|| algorithms work with X agents with Y (≪ X) real agents lab research real world strategy-1: strategy-2: Lets’ remove assumptions in MAPF!

Slide 14

Slide 14 text

/92 15 Outline Remove assumptions of: synchronized actions without crashes execution pre-defined discrete space homogeneous agents representation

Slide 15

Slide 15 text

/92 16 Outline Remove assumptions of: synchronized actions without crashes execution pre-defined discrete space homogeneous agents representation

Slide 16

Slide 16 text

/92 17 Failure Demo of Synchronous Execution

Slide 17

Slide 17 text

/92 18 Financial Times, https://www.ft.com Miscoordination triggers tragedy We need safe methods Let’s think about timing uncertainties

Slide 18

Slide 18 text

/92 19 1 2 1 2 3 4 Planning Execution Delay Imperfect Execution

Slide 19

Slide 19 text

/92 20 1 2 1 2 3 4 Planning Execution wait make agents wait for delayed ones arrival time: 3 Conservative: Synchronize Forcibly

Slide 20

Slide 20 text

/92 21 Čáp, M., Gregoire, J., & Frazzoli, E. Provably safe and deadlock-free execution of multi-robot plans under delaying disturbances. IROS. 2016. Ma, H., Kumar, T. S., & Koenig, S. Multi-agent path finding with delay probabilities. AAAI. 2017. check temporal dependencies 1 2 1 2 3 4 => preserved, go arrival time: 2 1 2 1 2 3 4 Planning Execution Progressive: Preserve Temporal Dependencies

Slide 21

Slide 21 text

/92 22 1 2 1 2 3 4 1 2 3 4 5 6 delay delay propagation Typical MAPF agents actions complicated dependencies Really Smart?

Slide 22

Slide 22 text

/92 23 Raspberry Pi x8 32 robots Bluetooth Expensive Communication Cost stable network / monitoring systems for ≥1000 agents…? NON-TRIVIAL!!

Slide 23

Slide 23 text

/92 24 online intervention at runtime to preserve timing-property abandon all timing assumptions

Slide 24

Slide 24 text

/92 25 at Dalí Paris Time-Independent Multi-Agent Path Planning

Slide 25

Slide 25 text

/92 26 Okumura, K., Bonnet, F., Tamura, Y., & Défago, X. Offline Time-Independent Multiagent Path Planning. T-RO. 2023. (extended from IJCAI-22) OTIMAPP solution 2 1 4 3 0 0 1 2 MAPF solution weak to timing uncertainties OTIMAPP: Planning with Timing Uncertainties

Slide 26

Slide 26 text

/92 27 given start goal graph solution path s.t. all agents eventually reach goals regardless of action orders Problem Def. OTIMAPP

Slide 27

Slide 27 text

/92 28 no reachable* cyclic deadlock no reachable* terminal deadlock *non-reachable deadlocks exist two conditions are necessary & sufficient Solution Analysis

Slide 28

Slide 28 text

/92 29 Computational Complexity 1. finding solutions is NP-hard 2. verification is co-NP-complete main observations OTIMAPP is computationally intractable the proofs are reductions from 3-SAT

Slide 29

Slide 29 text

/92 30 MAPF avoids collisions OTIMAPP avoids deadlocks MAPF algorithms are applicable How to solve OTIMAPP? prioritized planning deadlock-based search extending conflict-based search [Sharon+ AIJ-15] extending conventional PP [Erdmann+ Algorithmica-87] agents random-32-32-10 32x32 random-64-64-10 64x64 den520d 257x256 success rate (%) ≤ 5 min Both methods can solve large OTIMAPP instances to some extent

Slide 30

Slide 30 text

/92 31 centralized style decentralized style no synchronization only local interactions Execution Demo all robots are guaranteed to reach their goals with AFADA [Kameyama+ ICRA-21] with toio obots

Slide 31

Slide 31 text

/92 32 planning stage execution stage Conventional MAPF Solution Concept

Slide 32

Slide 32 text

/92 33 eventually reaching goal status agents reactively switch executing plan OTIMAPP Solution Concept

Slide 33

Slide 33 text

/92 34 timing uncertainties crash faults

Slide 34

Slide 34 text

/92 35 goal start graph solution Conventional Solution Concept 0

Slide 35

Slide 35 text

/92 36 with Unforeseen Crash 1 crash detected => then? crashed (forever stop) online replanning offline approach: preparing backup paths from the beginning or

Slide 36

Slide 36 text

/92 37 Fault-Tolerant Solution Concept 0 primary path backup path when is detected transition rule &

Slide 37

Slide 37 text

/92 38 1 Fault-Tolerant Solution Concept

Slide 38

Slide 38 text

/92 39 2 Fault-Tolerant Solution Concept

Slide 39

Slide 39 text

/92 40 3 Fault-Tolerant Solution Concept done! more than two agents may crash => backup path of backup path

Slide 40

Slide 40 text

/92 41 Okumura, K., Tixeuil, S. Fault-Tolerant Offline Multi-Agent Path Planning. AAAI. 2023. MAPPCF: Planning with Crash Faults fault solution multiple paths assuming crashes change executing path on-demand

Slide 41

Slide 41 text

/92 42 Problem Definition of MAPPCF centralized planning followed by decentralized execution given solution s.t. all non-crashed agents eventually reach their destination, regardless of crashes (up to f ) & transition rules & maximum number of crashes f defined with failure detector & execution model Check the paper for details.

Slide 42

Slide 42 text

/92 43 Failure Detectors c.f., Chandra+ JACM-96 oracle that tells status of neighbor vertices response: 1. no agent query 2. non-crashed agent named FD 3. crashed agent anonymous FD unable to identify who crashes

Slide 43

Slide 43 text

/92 44 Execution Models synchronous model all agents act simultaneously solutions avoid collisions MAPF: multi-agent pathfinding Stern+ SOCS-19 solution solutions avoid deadlocks each agent acts spontaneously while locally avoiding collisions offline time-independent MAPP Okumura+ T-RO-23 sequential model (async) solution possible schedule how agents are scheduled at runtime

Slide 44

Slide 44 text

/92 45 Model Power Analyses synchronous model sequential model named failure detector anonymous FD SYN SEQ NFD AFD solvable in SYN unsolvable in SEQ SYN + AFD SYN + NFD SEQ + NFD SEQ + AFD weakly stronger SYN+AFD SYN+NFD SYN+AFD SYN+NFD or strictly stronger SEQ+AFD SYN+AFD solvable instances

Slide 45

Slide 45 text

/92 46 Computational Complexity 1. finding solutions is NP-hard 2. verification is co-NP-complete regardless of FD types or execution models MAPPCF is computationally intractable the proofs are reductions from 3-SAT

Slide 46

Slide 46 text

/92 47 Solver / Empirical Results MAPPCF provides better solution concept than finding disjoint paths random-32-32-10 32x32 (|V|=922) from [Stern+ SOCS-19] 30sec timeout with named FD success rate #agents DCRF/SYN disjoint paths fixed #crashes: f =1 adapted from CBS [Sharon+ AIJ-15] v.s. finding vertex disjoint paths costs / lower bound #agents DCRF/SYN disjoint paths traveling time when no crashes We develop DCRF (decoupled crash faults resolution framework) to solve MAPPCF

Slide 47

Slide 47 text

/92 48 Summary in Execution planning execution reality gaps new solution concept: offline planning with reactive execution in practice? => hybrid of (relaxed) offline & online planning

Slide 48

Slide 48 text

/92 49 Outline Remove assumptions of: synchronized actions without crashes execution pre-defined discrete space homogeneous agents representation

Slide 49

Slide 49 text

/92 50 MAPF Definition Again given agents (starts) graph goals solution paths without collisions reality is continuous

Slide 50

Slide 50 text

/92 51 Why not move diagonally? Robots follow grid Cool coordination but not efficient! Okumura, K., & Défago, X. Solving simultaneous target assignment and path planning efficiently with time-independent execution. AIJ. 2023. (extended from ICAPS-22; best student paper)

Slide 51

Slide 51 text

/92 52 obstacle Unsolvable Instance?

Slide 52

Slide 52 text

/92 53 Multi-Agent Path Planning in Continuous Spaces given agents (starts) workspace goals solution paths without collisions Finding solutions itself is tremendously challenging Spirakis, P., & Yap, C. K. Strong NP-hardness of moving many discs. IPL. 1984. Hopcroft, J. E., Schwartz, J. T., & Sharir, M. On the Complexity of Motion Planning for Multiple Independent Objects; PSPACE-Hardness of the" Warehouseman's Problem". IJRR. 1984.

Slide 53

Slide 53 text

/92 54 artificial potential field sampling-based rule-based goal start constructing roadmap Strategies to Solve Single-Agent Path Planning in Continuous Spaces

Slide 54

Slide 54 text

/92 55 state of robot: (x, y) (x, y) should be in this region configuration space random sampling & construct roadmap pathfinding on roadmap same scheme even in high-dimension SBMP: Sampling-Based Motion Planning

Slide 55

Slide 55 text

/92 56 Naïve Strategy to Solve Multi-Agent Path Planning in Continuous Spaces 1. 2. two-phase planning Construct agent-wise roadmaps by some means, e.g., SBMP (sampling-based motion planning) Solve MAPF on those roadmaps ready for heterogeneity between agents

Slide 56

Slide 56 text

/92 57 Pitfall – There is a trade-off quality & solvability planning effort small large speed & scalability complete optimal incomplete suboptimal dense representation sparse representation produced by PRM [Kavraki+ 96] ideal: small but promising representation for planning

Slide 57

Slide 57 text

/92 58 Countermeasure biased sampling sampling from important region of each agent how to identify? agent-specific features + interactions between agents 🤔 design manually?

Slide 58

Slide 58 text

/92 59 Countermeasure This is machine learning problem! supervised learning: planning demonstration as training data biased sampling sampling from important region of each agent how to identify? agent-specific features + interactions between agents

Slide 59

Slide 59 text

/92 60 offline training: predict next locations from planning demo online inference: generate agent-specific roadmaps use ML-model as an interaction-aware vertex sampler MAPF CTRMs: Data-Driven Roadmap Construction Okumura, K., Yonetani, R., Nishimura, M., & Kanezaki, A.. Ctrms: Learning to construct cooperative timed roadmaps for multi-agent path planning in continuous spaces. AAMAS. 2022.

Slide 60

Slide 60 text

/92 61 new instance 𝐹!"#$ random walk sampling module next locations starts path generation compositing solution MAPF algorith m … t=0 t=1 t=2 CTRMs 𝐹!"#$ model training instances & solutions predict next locations Online Inference Offline Training CVAE: Conditional Variational Autoencoder [Sohn+ NeurIPS-15] +importance sampling [Salzmann+ ECCV-20] +multi-agent attention [Hoshen NeurIPS-17] 𝐹!"#$ : *independent from map size and #agent, not limited to homo-agents Pipeline

Slide 61

Slide 61 text

/92 62 new instance 𝐹!"#$ random walk sampling module next locations starts path generation compositing solution MAPF algorith m … t=0 t=1 t=2 CTRMs 𝐹!"#$ model training instances & solutions predict next locations Online Inference Offline Training CVAE: Conditional Variational Autoencoder [Sohn+ NeurIPS-15] +importance sampling [Salzmann+ ECCV-20] +multi-agent attention [Hoshen NeurIPS-17] 𝐹!"#$ : *independent from map size and #agent, not limited to homo-agents Pipeline

Slide 62

Slide 62 text

/92 63 occupancy cost-to-go env. info Offline Training & Model Arch. – 1/5 𝐹!"#$ next position generative model CVAE [Sohn+ NeurIPS-15] features ?

Slide 63

Slide 63 text

/92 64 + + relative positions, size, speeds, etc goal-driven features Offline Training & Model Arch. – 2/5

Slide 64

Slide 64 text

/92 65 + + communication features learning interactions with nearby agents m ultiple agents multi-agent attention [Sohn+ NeurIPS-15] Offline Training & Model Arch. – 3/5

Slide 65

Slide 65 text

/92 66 go right [0,0,1] indicator feature Offline Training & Model Arch. – 4/5

Slide 66

Slide 66 text

/92 67 + + + + go right [0,0,1] 𝐹!"#$ next position goal-driven features comm. features indicator feature instance & solution occupancy cost-to-go env. info relative positions, size, speeds, etc attention Offline Training & Model Arch. – 5/5

Slide 67

Slide 67 text

/92 68 new instance 𝐹!"#$ random walk sampling module next locations starts path generation compositing solution MAPF algorith m … t=0 t=1 t=2 CTRMs 𝐹!"#$ model training instances & solutions predict next locations Online Inference Offline Training CVAE: Conditional Variational Autoencoder [Sohn+ NeurIPS-15] +importance sampling [Salzmann+ ECCV-20] +multi-agent attention [Hoshen NeurIPS-17] 𝐹!"#$ : *independent from map size and #agent, not limited to homo-agents Pipeline

Slide 68

Slide 68 text

/92 69 observations for agent-i next predicted location for agent-i trained model likely to be used by planners Online Inference – 1/4

Slide 69

Slide 69 text

/92 70 timestep t timestep t+1 next predicted locations for all agents observations for all agents Online Inference – 2/4

Slide 70

Slide 70 text

/92 71 t=0 t=1 t=2 t=T t=T-1 … initial locations timed path for agent-i each path is agent-specific and cooperative hyperparameter Online Inference – 3/4

Slide 71

Slide 71 text

/92 72 … … … … compositing t=0 t=1 t=2 t=T t=T-1 timed roadmap for agent-i each roadmap is agent-specific and cooperative hyperparameter: #(path generation) Online Inference – 4/4

Slide 72

Slide 72 text

/92 73 SPARS [Dobson & Bekris, IJRR-14] (random) simplified PRM [Karaman & Frazzoli, IJRR-11] square as agent-specific roadmaps grid as used in MAPF studies CTRMs 20-30 homogeneous agents corresponding to 32x32 grids CTRMs produce small but effective roadmaps specific to each agent Roadmap Visualization

Slide 73

Slide 73 text

/92 74 S R P R R R 50 RP 3 5 T 20-30 homogeneous agents corresponding to 32x32 grids 100 instances solved by prioritized planning [Silver, AIIDE-05, Van Den Berg & Overmars IROS-05, etc] CTRMs reduce planning effort while keeping solution qualities params of CTRMs: #(path generations) denser denser Quantitative Results

Slide 74

Slide 74 text

/92 75 quality & solvability planning effort small large speed & scalability building representation from learning building representation while planning Yet Another Strategy for Representation

Slide 75

Slide 75 text

/92 76 Recap: Naïve Strategy to Solve Multi-Agent Path Planning in Continuous Spaces Construct agent-wise roadmaps by some means, e.g., SBMP (sampling-based motion planning) Solve MAPF on those roadmaps 1. 2. two-phase planning

Slide 76

Slide 76 text

/92 77 Pitfall Construct agent-wise roadmaps by some means Solve MAPF on those roadmaps 1. 2. two-phase planning Roadmaps truly used in planning process? decoupled

Slide 77

Slide 77 text

/92 78 Advanced Strategy Construct agent-wise roadmaps by some means Solve MAPF on those roadmaps 1. 2. develop agent-wise roadmaps according to MAPF search progress coupled

Slide 78

Slide 78 text

/92 79 SSSP: simultaneous sampling-and-search planning Okumura, K., & Défago, X. Quick Multi-Robot Motion Planning by Combining Sampling and Search. IJCAI. 2023. solve various types of MRMP rapidly

Slide 79

Slide 79 text

/92 80 Each robot operates in its own configuration space Each robot has initial and goal configurations solution: collision-free trajectories in continuous spaces allowing heterogeneity allowing kinematics/dynamics* constraints extremely challenging! MRMP: Multi-Robot Motion Planning *not addressed in this study

Slide 80

Slide 80 text

/92 81 How to Solve MRMP? Construct agent-wise roadmaps by SBMP Solve MAPF on those roadmaps 1. 2. coupled

Slide 81

Slide 81 text

/92 82 quick MRMP multi-robot motion planning PRM [Kavraki+ 96] EST [Hsu+ ICRA-97] RRT [LaValle 98] RRT-Connect [Kuffner+ ICRA-00] PRM*/RRT* [Karaman+ IJRR-11] SPARS [Dobson+ IJRR-14] FMT* [Jason+ IJRR-15] + SBMP: sampling-based motion planning A*+OD [Standley AAAI-10] CBS [Sharon+ AIJ-15] M* [Wagner+ AIJ-15] PBS [Ma+ AAAI-19] BCP [Lam+ COR-22] LNS [Li+ IJCAI-21; AAAI-22] LaCAM* [Okumura AAAI-23; IJCAI-23] + MAPF: multi-agent pathfinding search roadmap seamless integration Algorithm Design

Slide 82

Slide 82 text

/92 83 PRM [Kavraki+ 96] EST [Hsu+ ICRA-97] RRT [LaValle 98] RRT-Connect [Kuffner+ ICRA-00] PRM*/RRT* [Karaman+ IJRR-11] SPARS [Dobson+ IJRR-14] FMT* [Jason+ IJRR-15] + SBMP: sampling-based motion planning A*+OD [Standley AAAI-10] CBS [Sharon+ AIJ-15] M* [Wagner+ AIJ-15] PBS [Ma+ AAAI-19] BCP [Lam+ COR-22] LNS [Li+ IJCAI-21; AAAI-22] LaCAM* [Okumura AAAI-23; IJCAI-23] + MAPF: multi-agent pathfinding Algorithm Design seamless integration search roadmap example algorithm: SSSP Please check the paper for details! building representation while planning

Slide 83

Slide 83 text

/92 84 runtime (sec) solved instances (%) Point2d DOF: 2N - 100 instances with 10 random seeds - each instance is randomly generated - number of robots: 2–10 - heterogeneous robots Comparison with Standard Approaches

Slide 84

Slide 84 text

/92 85 RRT-Connect [Kuffner+ ICRA-00] RRT [LaValle 98] PRM [Kavraki+ 96] CBS [Sharon+ AIJ-15] on PRMs (with adjusted heuristics) PP [Silver AIIDE-05] on PRMs SSSP two-phase planning direct application of SBMP runtime (sec) solved instances (%) Point2d DOF: 2N - 100 instances with 10 random seeds - each instance is randomly generated - number of robots: 2–10 - heterogeneous robots Comparison with Standard Approaches

Slide 85

Slide 85 text

/92 86 runtime (sec) solved instances (%) Point2d DOF: 2N Point3d DOF: 3N Line2d DOF: 3N Capsule3d DOF: 6N Arm22 DOF: 2N Arm33 DOF: 6N Dubins2d DOF: 3N Snake2d DOF: 6N

Slide 86

Slide 86 text

/92 87 runtime (sec) solved instances (%) SSSP can quickly solve various MRMP

Slide 87

Slide 87 text

/92 88

Slide 88

Slide 88 text

/92 89 Summary in Representation quality & solvability planning effort small large speed & scalability building representation from learning building representation while planning

Slide 89

Slide 89 text

/92 90 to keep MAPF alive increase X decrease ||X – Y|| lab research real world strategy-1: strategy-2: Lets’ remove assumptions in MAPF! Specifically: synchrony, without faults, continuous spaces, heterogeneity Overall Summary algorithms work with X agents with Y (≪ X) real agents

Slide 90

Slide 90 text

/92 91 Takeaways + Personal Feelings Scalable MRMP methods have not appeared. But we have some clues from LaCAM* and SSSP. I believe the last piece is learning. Even when assumptions are removed, basic MAPF algorithms serve as the foundation. Adding robustness/resilience is expensive, but worth investigating because MAPF systems will be future infrastructure; this area is still primitive.

Slide 91

Slide 91 text

/92 92 Thank you for listening! Acknowledgements to mentors / collaborators: X. Defago, Y. Tamura, F. Bonnet, M. Machida, R. Yonetani, M. Nishimura, A. Kanezaki, S. Tixeuil A. Prorok + members of PROROK Lab in Cambridge Funding: JSPS DC & Overseas Research Fellowship, JST ACT-X, Yoshida Scholarship Foundation And, Dr. Jean-Marc Alkazzi for this opportunity / collaboration! kei18 https://kei18.github.io ko393@cl.cam.ac.uk Questions / research collaboration proposals are welcome: