A Neural Path Integration Mechanism for Adaptive Vector Navigation in Autonomous Agents

A presentation by Dennis Goldschmidt from the 2015 International Joint Conference on Neural Networks in Killarney, Ireland.

Transcript

1. A Neural Path Integration Mechanism for Adaptive Vector Navigation in Autonomous Agents Dennis Goldschmidt1, Sakyasingha Dasgupta2, Florentin Wörgötter1, Poramate Manoonpong3 1 Bernstein Center for Computational Neuroscience Göttingen 2 RIKEN Brain Science Institute 3 Center of Biorobotics, Mærsk Mc-Kinney Møller Institute July 14, 2015 International Joint Conference on Neural Networks 2015 1 / 27
2. Animal navigation crucial for survival (e.g., finding food, shelter or mates) depends on multimodal sensory input and internal representations even animals with “simpler” nervous systems exhibit remarkable navigational capabilities including learning & memory 2 / 27
3. Animal navigation crucial for survival (e.g., finding food, shelter or mates) depends on multimodal sensory input and internal representations even animals with “simpler” nervous systems exhibit remarkable navigational capabilities including learning & memory Example: Social insects 2 / 27
4. Path integration in desert ants Wehner and Wehner, 1990; Wehner, 2003 after foraging, ants return to their nests on a straight path pheromone trails and landmarks cannot be used in desert ants integrate over directions and distances traveled 3 / 27
5. Polarization orientation serves as directional cue 1. Dorsal Rim Area (DRA) contains polarization- sensitive receptors: 2. Polarization orientation pattern of skylight: Homberg et al., 2011 3. Population coding of azimuthal directions in the insect central complex: Heinze/Homberg, 2007 Sakura et al., 2008 4 / 27
6. Closed-loop control architecture sensory input from compass and proprioceptive sensors (speed signal) path integration mechanism controls steering of the agent central pattern generator for insect-like walking gaits (i.e., tripod gait) Embodied Artificial Agent AMOS II Modular Robot Control Environment (MoRoCo) FTi CTr TC R0 R1 R2 L2 L1 L0 IR Lpzrobots Toolkit IR 5 / 27
7. Path integration mechanism Input layer represents compass angles with cosine response: s B) Head Direction Layer C) Odometric Modulation D) Memory Layer E) PI Output Layer ϕ A) Sensory Input +δ -δ F) Homing signal xHD i φ(t) = cos φ(t) − φi φi = 2πi N , i ∈ [0, N − 1] -1 0 1 0 60 120 180 240 300 360 Response Angle Each unit/column encodes for a preferred direction (in [0, 2π)) 6 / 27
8. Path integration mechanism Rate-based neural network with linear rectified activation function: s B) Head Direction Layer C) Odometric Modulation D) Memory Layer E) PI Output Layer ϕ A) Sensory Input +δ -δ F) Homing signal f(x) = max(0, x) HD signals are modulated by speed signal (s ∈ [0, 1]) temporal integration by self-excitatory feedback wrec = 1 decoding to output activity with single maximum (home vector angle) using cosine kernel wij = cos(φi − φj ) 7 / 27
9. Path integration mechanism s B) Head Direction Layer C) Odometric Modulation D) Memory Layer E) PI Output Layer ϕ A) Sensory Input +δ -δ F) Homing signal L-shaped run: End Start 225° 180° 270° 8 / 27
10. Homing behavior for L-shaped run 48x 9 / 27
11. Obstacle avoidance 48x 10 / 27
12. Noise robustness in L-shaped runs Single-trial trajectories & estimated positions: Estimated home position distributions: 11 / 27
13. Noise robustness in random foraging vmax = 0.1 ms−1 ˙ φ = N (0, 0.15) L = 25 m 12 / 27
14. Noise robustness in random foraging 1. Mean angular errors with respect to sensory noise (N = 360): 2. Mean angular errors with respect to number of neurons (σ = 5%): −5 0 5 10 15 20 0 2 4 6 8 10 Mean Angular Error < ε > Sensory Noise Level in % −20 0 20 40 60 80 1 10 100 1000 10000 Mean Angular Error < ε > Number of Neurons 12 / 27
15. Reproducing systematic errors Temporal integration by positive feedback with leakage λ: 1-λ Müller and Wehner, 1988 0 10 20 30 0 30 60 90 120 150 180 Angular Error ε Second Leg Angle α λ = 0.000 λ = 0.007 λ = 0.0075 λ = 0.008 λ = 0.009 Data from Müller (1989) 13 / 27
16. Conclusions We delevoped a closed-loop, neural network-based controller for path integration and homing behavior in an embodied legged agent. path integration mechanism allows for robust homing behavior even in the presence of external sensory noise reproduced behavioral aspects of desert ant systematic errors during homing ∗ Data from Wehner/Wehner, 1986 14 / 27
17. Conclusions We delevoped a closed-loop, neural network-based controller for path integration and homing behavior in an embodied legged agent. path integration mechanism allows for robust homing behavior even in the presence of external sensory noise reproduced behavioral aspects of desert ant systematic errors during homing ∗ Data from Wehner/Wehner, 1986 14 / 27
18. Conclusions We delevoped a closed-loop, neural network-based controller for path integration and homing behavior in an embodied legged agent. path integration mechanism allows for robust homing behavior even in the presence of external sensory noise reproduced behavioral aspects of desert ant systematic errors during homing ∗ Data from Wehner/Wehner, 1986 14 / 27
19. Conclusions We delevoped a closed-loop, neural network-based controller for path integration and homing behavior in an embodied legged agent. path integration mechanism allows for robust homing behavior even in the presence of external sensory noise reproduced behavioral aspects of desert ant systematic errors during homing Model Neurons N Noise σ [%] Distance L [m] Angular error [◦] Kim/Lee, 2011 100 10 5 0.29 ± 0.12 Haferlach et al., 2007 6 3 2.5 10.74 ± 2.41 Goldschmidt et al., 2015 360 2 25 1.32 ± 0.64 5 3.75 ± 3.58 10 7.08 ± 9.25 10 5 0.20 ± 0.13 Desert ant C. fortis∗ 20.2 ± 9, 9 6.2 ± 4.0 ∗ Data from Wehner/Wehner, 1986 14 / 27
20. Conclusions We delevoped a closed-loop, neural network-based controller for path integration and homing behavior in an embodied legged agent. path integration mechanism allows for robust homing behavior even in the presence of external sensory noise reproduced behavioral aspects of desert ant systematic errors during homing Model Neurons N Noise σ [%] Distance L [m] Angular error [◦] Kim/Lee, 2011 100 10 5 0.29 ± 0.12 Haferlach et al., 2007 6 3 2.5 10.74 ± 2.41 Goldschmidt et al., 2015 360 2 25 1.32 ± 0.64 5 3.75 ± 3.58 10 7.08 ± 9.25 10 5 0.20 ± 0.13 Desert ant C. fortis∗ 20.2 ± 9, 9 6.2 ± 4.0 ∗ Data from Wehner/Wehner, 1986 14 / 27
21. Conclusions We delevoped a closed-loop, neural network-based controller for path integration and homing behavior in an embodied legged agent. path integration mechanism allows for robust homing behavior even in the presence of external sensory noise reproduced behavioral aspects of desert ant systematic errors during homing Model Neurons N Noise σ [%] Distance L [m] Angular error [◦] Kim/Lee, 2011 100 10 5 0.29 ± 0.12 Haferlach et al., 2007 6 3 2.5 10.74 ± 2.41 Goldschmidt et al., 2015 360 2 25 1.32 ± 0.64 5 3.75 ± 3.58 10 7.08 ± 9.25 10 5 0.20 ± 0.13 Desert ant C. fortis∗ 20.2 ± 9, 9 6.2 ± 4.0 ∗ Data from Wehner/Wehner, 1986 14 / 27
22. Conclusions We delevoped a closed-loop, neural network-based controller for path integration and homing behavior in an embodied legged agent. path integration mechanism allows for robust homing behavior even in the presence of external sensory noise reproduced behavioral aspects of desert ant systematic errors during homing Model Neurons N Noise σ [%] Distance L [m] Angular error [◦] Kim/Lee, 2011 100 10 5 0.29 ± 0.12 Haferlach et al., 2007 6 3 2.5 10.74 ± 2.41 Goldschmidt et al., 2015 360 2 25 1.32 ± 0.64 5 3.75 ± 3.58 10 7.08 ± 9.25 10 5 0.20 ± 0.13 Desert ant C. fortis∗ 20.2 ± 9, 9 6.2 ± 4.0 ∗ Data from Wehner/Wehner, 1986 14 / 27
23. Conclusions We delevoped a closed-loop, neural network-based controller for path integration and homing behavior in an embodied legged agent. path integration mechanism allows for robust homing behavior even in the presence of external sensory noise reproduced behavioral aspects of desert ant systematic errors during homing Model Neurons N Noise σ [%] Distance L [m] Angular error [◦] Kim/Lee, 2011 100 10 5 0.29 ± 0.12 Haferlach et al., 2007 6 3 2.5 10.74 ± 2.41 Goldschmidt et al., 2015 360 2 25 1.32 ± 0.64 5 3.75 ± 3.58 10 7.08 ± 9.25 10 5 0.20 ± 0.13 Desert ant C. fortis∗ 20.2 ± 9, 9 6.2 ± 4.0 ∗ Data from Wehner/Wehner, 1986 14 / 27
24. Conclusions We delevoped a closed-loop, neural network-based controller for path integration and homing behavior in an embodied legged agent. path integration mechanism allows for robust homing behavior even in the presence of external sensory noise reproduced behavioral aspects of desert ant systematic errors during homing Model Neurons N Noise σ [%] Distance L [m] Angular error [◦] Kim/Lee, 2011 100 10 5 0.29 ± 0.12 Haferlach et al., 2007 6 3 2.5 10.74 ± 2.41 Goldschmidt et al., 2015 360 2 25 1.32 ± 0.64 5 3.75 ± 3.58 10 7.08 ± 9.25 10 5 0.20 ± 0.13 Desert ant C. fortis∗ 20.2 ± 9, 9 6.2 ± 4.0 ∗ Data from Wehner/Wehner, 1986 14 / 27
25. Conclusions We delevoped a closed-loop, neural network-based controller for path integration and homing behavior in an embodied legged agent. path integration mechanism allows for robust homing behavior even in the presence of external sensory noise reproduced behavioral aspects of desert ant systematic errors during homing Model Neurons N Noise σ [%] Distance L [m] Angular error [◦] Kim/Lee, 2011 100 10 5 0.29 ± 0.12 Haferlach et al., 2007 6 3 2.5 10.74 ± 2.41 Goldschmidt et al., 2015 360 2 25 1.32 ± 0.64 5 3.75 ± 3.58 10 7.08 ± 9.25 10 5 0.20 ± 0.13 Desert ant C. fortis∗ 20.2 ± 9, 9 6.2 ± 4.0 ∗ Data from Wehner/Wehner, 1986 14 / 27
26. Future work 1. Reward-based associative learning rule allows for spatial learning of global and local vectors 2. Real-world applications on hexapod walking robot AMOS II and quadrocopter 3. Reservoir computing for robust sensory processing and forward models of real-world data (e.g. IMU, encoders) 15 / 27
27. Future work 1. Reward-based associative learning rule allows for spatial learning of global and local vectors 2. Real-world applications on hexapod walking robot AMOS II and quadrocopter 3. Reservoir computing for robust sensory processing and forward models of real-world data (e.g. IMU, encoders) 15 / 27
28. Future work 1. Reward-based associative learning rule allows for spatial learning of global and local vectors 2. Real-world applications on hexapod walking robot AMOS II and quadrocopter 3. Reservoir computing for robust sensory processing and forward models of real-world data (e.g. IMU, encoders) 15 / 27
29. Poramate Manoonpong Sakyasingha Dasgupta Florentin Wörgötter Thank you for your attention! Any questions? 16 / 27
30. References Haferlach, T., Wessnitzer, J., Mangan, M., and Webb, B. (2007). Evolving a neural model of insect path integration. Adaptive Behavior, 15(3):273–287. Heinze, S. and Homberg, U. (2007). Maplike representation of celestial e-vector orientations in the brain of an insect. Science, 315(5814):995–997. Homberg, U., Heinze, S., Pfeiffer, K., Kinoshita, M., and el Jundi, B. (2011). Central neural coding of sky polarization in insects. Philosophical Transactions of the Royal Society B: Biological Sciences, 366(1565):680–687. Kim, D. and Lee, J. (2011). Path integration mechanism with coarse coding of neurons. Neural Processing Letters, 34(3):277–291. Müller, M. and Wehner, R. (1988). Path integration in desert ants, cataglyphis fortis. Proceedings of the National Academy of Sciences, 85(14):5287–5290. Sakura, M., Lambrinos, D., and Labhart, T. (2008). Polarized skylight navigation in insects: Model and electrophysiology of e-vector coding by neurons in the central complex. Journal of Neurophysiology, 99(2):667–682. Wehner, R. (2003). Desert ant navigation: how miniature brains solve complex tasks. Journal of Comparative Physiology A, 189(8):579–588. Wehner, R. and Wehner, S. (1986). Path integration in desert ants. approaching a long-standing puzzle in insect navigation. Monitore Zoologico Italiano - Italian Journal of Zoology, 20(3):309–331. Wehner, R. and Wehner, S. (1990). Insect navigation: use of maps or ariadne’s thread? Ethology Ecology & Evolution, 2(1):27–48. 17 / 27
31. Odometry in desert ants Wittlinger et al., 2006 Testing three different lengths (stumps, stilts, normal). Manipulations done before homeward (bottom left) and before outward journey (bottom right). ⇒ Ant odometry relies on an integration process of strides walked during the journey 18 / 27
32. Path integration and vector navigation in honey bees von Frisch, 1957 Bees communicate vector information (hive to feeder) by ”waggle dance”: Direction ∼ angle to the feeder with respect to the sun Duration ∼ distance to the feeder (750 ms ∼ 1 km) 19 / 27
33. Drosophila angular path integration in ellipsoid body Seelig/Jayaraman, 2015 20 / 27
34. CPG-based locomotion control and legged odometry Manoonpong et al., 2013 21 / 27
35. Path integration mechanism: Odometric modulation The second layer acts as a gating mechanism, in which walking speed modulates directional signals: s B) Head Direction Layer C) Odometric Modulation D) Memory Layer E) PI Output Layer ϕ A) Sensory Input +δ -δ F) Homing signal xG i (t) = f(δijxHD j + s − 1), f(x) = max(0, x), s ∈ [0, 1] 22 / 27
36. Path integration mechanism: Odometric modulation The second layer acts as a gating mechanism, in which walking speed modulates directional signals: s B) Head Direction Layer C) Odometric Modulation D) Memory Layer E) PI Output Layer ϕ A) Sensory Input +δ -δ F) Homing signal xG i (t) = f(δijxHD j + s − 1), f(x) = max(0, x), s ∈ [0, 1] 22 / 27
37. Path integration mechanism: Leaky integration of directional signals s B) Head Direction Layer C) Odometric Modulation D) Memory Layer E) PI Output Layer ϕ A) Sensory Input +δ -δ F) Homing signal xM i (t) = f δijxG j (t) + (1 − λ)xM i (t − 1) 23 / 27
38. Path integration mechanism: Decoding the path memory s B) Head Direction Layer C) Odometric Modulation D) Memory Layer E) PI Output Layer ϕ A) Sensory Input +δ -δ F) Homing signal xPI i (t) = f wijxM j (t) , wij = cos(φi − φj ) 24 / 27
39. Scalability of the system: Path length Mean angular errors with respect to path length: 0 2 4 6 8 10 12 0 10 20 30 40 50 Mean Angular Error < ε > Outbound path length in m Average error (1000 trials) 25 / 27
40. Scalability of the system: Turning rate Mean angular errors with respect to turning rate: −10 0 10 20 30 40 0 20 40 60 80 100 Mean Angular Error < ε > Average Turning Rate dφ/dt Average error (1000 trials) 26 / 27
41. Fin. 27 / 27