Could simplified stimuli change how the brain performs visual search tasks?

Transcript

1. Could simplified stimuli change how the brain performs visual search

   tasks? David Nicholson NMC4 December 2021

2. Introduction Visual search: a real-world behavior we engage in constantly

3. Introduction In the laboratory, visual search tasks use simplified stimuli

   Peelen and Kastner, 2014

4. Introduction Hallmark of behavior exhibited in laboratory visual search tasks:

   set size effects

5. Introduction The visual system is optimized to search natural images

   Peelen and Kastner, 2014

6. Introduction → simplified stimuli change visual search behavior How could

   we test this? Peelen and Kastner, 2014

7. Methods deep neural networks for image classification AlexNet DNN architecture

   "plane"

8. Methods ~ state-of-the-art models of object recognition in the visual

   system adapted from DiCarlo and Cox 2007 AlexNet ANN architecture: ~ primate ventral visual stream "retina space" "inferior temporal cortex space" separating hyperplane

9. Methods deep neural networks optimized for image classification (Kell McDermott

   2019) step

10. Methods Transfer learning to adapt pre-trained DNNs to visual search

    tasks https://github.com/NickleDave/searchstims

11. Methods Transfer learning to adapt pre-trained DNNs to visual search

    tasks the Visual Search Difficulty dataset "How Hard Can It Be? Estimating the Difficulty of Visual Search in an Image". Ionescu, et al. 2016

12. Results DNNs exhibit set size effects

13. Results Set size effects result from optimizing DNNs to classify

    natural images

14. Results Optimizing DNNs with natural images --> improved, human-like behavior

    on search tasks with natural images

15. Results Optimizing DNNs with natural images --> improved, human-like behavior

    on search tasks with natural images Training method Source dataset DNN architecture Accuracy (largest object) (mean (S. D.)) transfer ImageNet VGG16 0.786 (0.007) transfer ImageNet AlexNet 0.652 (0.010) initialize Pascal VOC AlexNet 0.390 (0.010) initialize Pascal VOC VGG16 0.353 (0.060) transfer search stimuli VGG16 0.262 (0.004) transfer search stimuli AlexNet 0.208 (0.000)

16. Results Optimizing DNNs with natural images --> improved, human-like behavior

    on search tasks with natural images Training method Source dataset DNN architecture Accuracy (largest object) (mean (S. D.)) transfer ImageNet VGG16 0.786 (0.007) transfer ImageNet AlexNet 0.652 (0.010) initialize Pascal VOC AlexNet 0.390 (0.010) initialize Pascal VOC VGG16 0.353 (0.060) transfer search stimuli VGG16 0.262 (0.004) transfer search stimuli AlexNet 0.208 (0.000)

17. Results Optimizing DNNs with natural images --> improved, human-like behavior

    on search tasks with natural images Training method Source dataset DNN architecture Accuracy (largest object) (mean (S. D.)) transfer ImageNet VGG16 0.786 (0.007) transfer ImageNet AlexNet 0.652 (0.010) initialize Pascal VOC AlexNet 0.390 (0.010) initialize Pascal VOC VGG16 0.353 (0.060) transfer search stimuli VGG16 0.262 (0.004) transfer search stimuli AlexNet 0.208 (0.000)

18. Results Optimizing DNNs with natural images --> improved, human-like behavior

    on search tasks with natural images

19. Discussion Mismatch may be impeding our ability to understand visual

    search behavior

20. Discussion Future work could compare behavior of different models on

    a benchmark set of stimuli and tasks Guided Search 6.0, Wolfe 2021

21. NickleDave Thank you! Lifelong Learning Machines program, DARPA HR0011-18-2-0019 2017

    William K. and Katherine W. Estes Fund to F. Pestilli, R. Goldstone and L. Smith, Indiana University Bloomington. nicholdav