The medial Reticular Formation (mRF): a neural substrate for action selection? An evaluation via evolutionary computation.

The medial Reticular Formation (mRF) is located in the brainstem: it receives many sensory inputs and it can control motor actions through its projections on the spinal cord and cranial nerves. The mRF is phylogenetically one of the oldest neural structures of the brainstem, the latter being regarded as one of the oldest centers of the central nervous system. Subsequently it seems to be a low-level system for action selection.

The first model of the mRF was proposed by Kilmer and McCulloch in 1969, who already proposed that the mRF could be a "mode selector". In 2005, Humphries et al. (2005) tested the efficiency of this model in the minimal survival task defined in Girard et al. (2003). It performed poorly, but another version of it that included artificially evolved weights performed quite honorably. As a result, Humphries proposed a second model of the mRF, based on neural network formalism and taking into account new anatomical data. Nevertheless, it showed poor performances in the minimal survival task and turns out not to be anatomically very plausible.

In this Master's Thesis, we propose a new model of the mRF:
1. constrained by anatomical information about its structure,
2. constructed based on neural networks generated by artificial evolution,
3. assessed on tasks of action selection.

The model we obtain successfully manages the tasks of selection, indicating that the mRF can be used as an action selection system. We also demonstrate an anatomical property of the mRF, which coupled with the results of the paper Humphries et al. (2006) shows that it is very likely that the mRF network has a small-world structure.

http://francky.me/publications.php#mRF2011

Transcript

1. > The medial Reticular Formation (mRF): a neural substrate for action selection? An evaluation via evolutionary computation. June 22, 2011 Project funded by the (ANR-09-EMER-005-01 - EvoNeuro) Franck Dernoncourt <Franck.Dernoncourt@gmail.com> Superviseurs : Stéphane Doncieux <stephane.doncieux@isir.upmc.fr>, Benoît Girard <benoit.girard@isir.upmc.fr>
2. 29/12/2011 The medial Reticular Formation (mRF) 2 Table of contents 1.Introduction 2.Method 3.Disembodied task 4.Embodied task 5.Conclusions
3. 29/12/2011 The medial Reticular Formation (mRF) 3 1. Introduction The mRF anatomy is similar among all animals. [Nauta & Ramon-Moliner 1966] and the mRF is phylogenetically very old. The mRF seems to be a low-level system for action selection. [Birkmayer and Pilleri, 1966]: rats with injuries to the RF demonstrate severe behavioral disorders. [Woods, 1964] : rats who had undergone a complete cut in the posterior brainstem by removing the entire brain rostral to this cross-section, had a surprisingly coherent behavior. Coherent with anatomical data: Numerous sensory inputs, Many connections to the spinal cord (= potentially motor actions). (a cat’s brain)
4. 29/12/2011 The medial Reticular Formation (mRF) 4 1. Introduction Only 2 models: Model 1: Kilmer-McCulloch 1969
5. 29/12/2011 The medial Reticular Formation (mRF) 5 1. Introduction Only 2 models: Model 2: Humphries 2006. Does not take into account all anatomical data. Unfounded hypothesis: each cluster is associated to an action. Low survival time [Humphries2006].
6. 29/12/2011 The medial Reticular Formation (mRF) 6 Table of contents 1.Introduction 2.Method 3.Disembodied task 4.Embodied task 5.Conclusions
7. 29/12/2011 The medial Reticular Formation (mRF) 7 2. Method Method’s synopsis: • Identify anatomical data of the mRF, • Use selection tasks of the literature, • Generate neural network of type mRF capable of achieving the tasks (use of a multi-objective evolutionary algorithm). Example of a mRF model Example of another mRF model
8. 29/12/2011 The medial Reticular Formation (mRF) 8 2. Method List of parameters describing a network of type mRF: 1. c : the number of clusters (between 35 and 75) ; 4 2. n : the number of neurons in one cluster (ca. 30 000) ; entre 10 et 30 lPDS 3. p : the percentage of projection neurons (ca. 80%). The percentage of interneurons is therefore 1 - p ; 4. P(c) : the probability that one projection neuron project to a given cluster (P(c) = 0.25) ; ...
9. 29/12/2011 The medial Reticular Formation (mRF) 9 2. Method Multiobjective evolutionary algorithm: Population size: 500 ; Number of Generations : 500 --> 500² evaluated models
10. 29/12/2011 The medial Reticular Formation (mRF) 10 2. Method Multiobjective evolutionary algorithm: Objective 1: the mRF must take the expected decisions, depending on the selection task. Objective 2: the mRF must make frankly these decisions (contrast objective) [Prescott1999, Girard2003] Objective 3: the mRF must respect the known anatomical constraints on the mRF (objective of anatomic plausibility).
11. 29/12/2011 The medial Reticular Formation (mRF) 11 Table of contents 1.Introduction 2.Method 3.Disembodied task 4.Embodied task 5.Conclusions
12. 29/12/2011 The medial Reticular Formation (mRF) 12 3. Disembodied task Expérience : Abstract selection task. We want the MRF to act as a WTA network (Winner-Takes-All) : [Humphries2007]
13. 29/12/2011 The medial Reticular Formation (mRF) 13 3. Disembodied task Experiment : Abstract selection task.
14. 29/12/2011 The medial Reticular Formation (mRF) 14 3. Disembodied task Results obtained with mRF-type networks:
15. 29/12/2011 The medial Reticular Formation (mRF) 15 3. Disembodied task Results obtained with unconstrained networks:
16. 29/12/2011 The medial Reticular Formation (mRF) 16 3. Disembodied task Conclusions: 1. A mRF-like network can perform a selection task. 2. The data on the known anatomical MRF represent neither an advantage (because there are other network structures equally successful) nor a disadvantage for selection. 3. Humphries obtained about 75% of good decisions with his model without considering the contrast. Our method to evolve models is thus more efficient, which tends to confirm the soundness of our approach: 1. Add more neurons per cluster, 2. Remove the hypothesis of a cluster-action mapping, 3. Consider more anatomical data, 4. Use evolutionary algorithms to evolve the network structure.
17. 29/12/2011 The medial Reticular Formation (mRF) 17 Table of contents 1.Introduction 2.Method 3.Disembodied task 4.Embodied task 5.Conclusions
18. 29/12/2011 The medial Reticular Formation (mRF) 18 4. Embodied task Experiment: Evaluation with the survival task [Girard2003, Humphries2006]
19. 29/12/2011 The medial Reticular Formation (mRF) 19 4. Embodied task Results Best average on 5 tasks: Humphries: performance between random and WTA controllers
20. 29/12/2011 The medial Reticular Formation (mRF) 20 4. Embodied task Results:
21. 29/12/2011 The medial Reticular Formation (mRF) 21 4. Embodied task Conclusions: The mRF is generally more effective than a WTA and a controller even more effective than a random controller. This means that the mRF is not only able to make action selections, but that it can deal with complex situations where a WTA would not. In addition, according to our estimates, we achieved better results than those of Humphries’ model.
22. 29/12/2011 The medial Reticular Formation (mRF) 22 Table of contents 1.Introduction 2.Method 3.Disembodied task 4.Embodied task 5.Conclusions
23. 29/12/2011 The medial Reticular Formation (mRF) 23 5. Conclusion To conclude: Disembodied task: computational capacity of the MRF to perform a task selection. Embodied task : computational capacity of the MRF to perform action selection in simulated environment. mRF-like structure : neither an advantage nor a disadvantage in these two tasks . Predictions : Compare free parameters of our models with real anatomical data (not known at this time). E.g.: p (l) = p (p) = 8%.
24. 29/12/2011 The medial Reticular Formation (mRF) 24 Questions ? Project funded by the ANR (ANR-09-EMER-005-01 - EvoNeuro)
25. 29/12/2011 The medial Reticular Formation (mRF) 25 Table of contents 1.Introduction 2.Méthode 3.Résultats 4.Conclusions 5.References
26. 29/12/2011 The medial Reticular Formation (mRF) 26 5. References • [Barraud 2003] : Barraud, Charles (2003) Contribution générale à l'étude de la formation réticulée (Formatio Reticularis). Ecole Nationale Vétérinaire de Toulouse – ENVT • [Eiben 2007] : A. E. Eiben and J. E. Smith. Introduction to Evolutionary Computing. Springer, 2003. • [Humphries 2005a] : Humphries, M., Gurney, K., Prescott, T., 2005. Is There an Integrative Center in the Vertebrate Brain-Stem ? A Robotic Evaluation of a Model of the Reticular Formation Viewed as an Action Selection Device. Adaptive Behavior 13 (2), 97–113. •[Humphries 2005b] : Humphries, M.D., Gurney, K., Prescott, T.J.: The brainstem reticular formation is a small-world, not scale-free, network. Proc. Roy. Soc. B. 273 (2006) 503–511 •[Humphries 2006] : Humphries, M. D. & Prescott, T. J. (2006), Distributed action selection by a brainstem neural substrate: An embodied evaluation, From Animals to Animats 9: Proceedings of the Ninth International Conference on Simulation of Adaptive Behaviour, pp. 199-210, Springer-Verlag: Berlin. •[Kilmer 1969] : Kilmer, W., McCulloch, W., Blum, J., 1969. A model of the vertebrate central command system. International Journal of Man Machine Studies 1, 279–309. •[Scheibel 1967] : Scheibel, M.E. and Scheibel, A.B. (1967) Anatomical Basis of Attention Mechanisms in Vertebrate Brains, Pages 577-602 in The Neurosciences: A Study Program, edited by G.C. Quarton, T. Melnechuk and F.O. Schmitt; Rockefeller University Press, New York
27. 29/12/2011 The medial Reticular Formation (mRF) 27 5. References •[Siegel 1977] : Siegel, J. M. and McGinty, D.J., Pontine reticular formation neurons and motor activity, Science, 199(1978)207-208. • [Siegel 1978] : Siegel, J. M. and McGinty, D. J. Pontine reticular formation neurons and motor activity.. Science 1978; 199: 207-208. •[Siegel 1979a] : Siegel, J. M. Behavioral functions of the reticular formation.. Brain Res. Rev. 1979; 1: 69-105. •[Sigel 1979b] : Siegel, JM Behavioral relations of medullary reticular formation cells. Experimental neurology. . 1979; 65(3): 691-8. •[Siegel 1979c] : Siegel, JM Wheeler, RL McGinty, DJ Activity of medullary reticular formation neurons in the unrestrained cat during waking and sleep.. Brain research. . 1979; 179(1): 49-60.