Could simplified stimuli change how the brain performs visual search tasks?

flash talk for Neuromatch 4.0

Transcript

1. Could simplified stimuli change how the brain performs visual search tasks? David Nicholson NMC4 December 2021
2. Introduction Visual search: a real-world behavior we engage in constantly
3. Introduction In the laboratory, visual search tasks use simplified stimuli Peelen and Kastner, 2014
4. Introduction Hallmark of behavior exhibited in laboratory visual search tasks: set size effects
5. Introduction The visual system is optimized to search natural images Peelen and Kastner, 2014
6. Introduction → simplified stimuli change visual search behavior How could we test this? Peelen and Kastner, 2014
7. Methods deep neural networks for image classification AlexNet DNN architecture "plane"
8. Methods ~ state-of-the-art models of object recognition in the visual system adapted from DiCarlo and Cox 2007 AlexNet ANN architecture: ~ primate ventral visual stream "retina space" "inferior temporal cortex space" separating hyperplane
9. Methods deep neural networks optimized for image classification (Kell McDermott 2019) step
10. Methods Transfer learning to adapt pre-trained DNNs to visual search tasks https://github.com/NickleDave/searchstims
11. Methods Transfer learning to adapt pre-trained DNNs to visual search tasks the Visual Search Difficulty dataset "How Hard Can It Be? Estimating the Difficulty of Visual Search in an Image". Ionescu, et al. 2016
12. Results DNNs exhibit set size effects
13. Results Set size effects result from optimizing DNNs to classify natural images
14. Results Optimizing DNNs with natural images --> improved, human-like behavior on search tasks with natural images
15. Results Optimizing DNNs with natural images --> improved, human-like behavior on search tasks with natural images Training method Source dataset DNN architecture Accuracy (largest object) (mean (S. D.)) transfer ImageNet VGG16 0.786 (0.007) transfer ImageNet AlexNet 0.652 (0.010) initialize Pascal VOC AlexNet 0.390 (0.010) initialize Pascal VOC VGG16 0.353 (0.060) transfer search stimuli VGG16 0.262 (0.004) transfer search stimuli AlexNet 0.208 (0.000)
16. Results Optimizing DNNs with natural images --> improved, human-like behavior on search tasks with natural images Training method Source dataset DNN architecture Accuracy (largest object) (mean (S. D.)) transfer ImageNet VGG16 0.786 (0.007) transfer ImageNet AlexNet 0.652 (0.010) initialize Pascal VOC AlexNet 0.390 (0.010) initialize Pascal VOC VGG16 0.353 (0.060) transfer search stimuli VGG16 0.262 (0.004) transfer search stimuli AlexNet 0.208 (0.000)
17. Results Optimizing DNNs with natural images --> improved, human-like behavior on search tasks with natural images Training method Source dataset DNN architecture Accuracy (largest object) (mean (S. D.)) transfer ImageNet VGG16 0.786 (0.007) transfer ImageNet AlexNet 0.652 (0.010) initialize Pascal VOC AlexNet 0.390 (0.010) initialize Pascal VOC VGG16 0.353 (0.060) transfer search stimuli VGG16 0.262 (0.004) transfer search stimuli AlexNet 0.208 (0.000)
18. Results Optimizing DNNs with natural images --> improved, human-like behavior on search tasks with natural images
19. Discussion Mismatch may be impeding our ability to understand visual search behavior
20. Discussion Future work could compare behavior of different models on a benchmark set of stimuli and tasks Guided Search 6.0, Wolfe 2021
21. NickleDave Thank you! Lifelong Learning Machines program, DARPA HR0011-18-2-0019 2017 William K. and Katherine W. Estes Fund to F. Pestilli, R. Goldstone and L. Smith, Indiana University Bloomington. nicholdav