Save 37% off PRO during our Black Friday Sale! »

Could simplified stimuli change how the brain performs visual search tasks?

Could simplified stimuli change how the brain performs visual search tasks?

flash talk for Neuromatch 4.0

9ae315da9dbd0b9cec19ab9b595915b2?s=128

David Nicholson

November 23, 2021
Tweet

Transcript

  1. Could simplified stimuli change how the brain performs visual search

    tasks? David Nicholson NMC4 December 2021
  2. Introduction Visual search: a real-world behavior we engage in constantly

  3. Introduction In the laboratory, visual search tasks use simplified stimuli

    Peelen and Kastner, 2014
  4. Introduction Hallmark of behavior exhibited in laboratory visual search tasks:

    set size effects
  5. Introduction The visual system is optimized to search natural images

    Peelen and Kastner, 2014
  6. Introduction → simplified stimuli change visual search behavior How could

    we test this? Peelen and Kastner, 2014
  7. Methods deep neural networks for image classification AlexNet DNN architecture

    "plane"
  8. Methods ~ state-of-the-art models of object recognition in the visual

    system adapted from DiCarlo and Cox 2007 AlexNet ANN architecture: ~ primate ventral visual stream "retina space" "inferior temporal cortex space" separating hyperplane
  9. Methods deep neural networks optimized for image classification (Kell McDermott

    2019) step
  10. Methods Transfer learning to adapt pre-trained DNNs to visual search

    tasks https://github.com/NickleDave/searchstims
  11. Methods Transfer learning to adapt pre-trained DNNs to visual search

    tasks the Visual Search Difficulty dataset "How Hard Can It Be? Estimating the Difficulty of Visual Search in an Image". Ionescu, et al. 2016
  12. Results DNNs exhibit set size effects

  13. Results Set size effects result from optimizing DNNs to classify

    natural images
  14. Results Optimizing DNNs with natural images --> improved, human-like behavior

    on search tasks with natural images
  15. Results Optimizing DNNs with natural images --> improved, human-like behavior

    on search tasks with natural images Training method Source dataset DNN architecture Accuracy (largest object) (mean (S. D.)) transfer ImageNet VGG16 0.786 (0.007) transfer ImageNet AlexNet 0.652 (0.010) initialize Pascal VOC AlexNet 0.390 (0.010) initialize Pascal VOC VGG16 0.353 (0.060) transfer search stimuli VGG16 0.262 (0.004) transfer search stimuli AlexNet 0.208 (0.000)
  16. Results Optimizing DNNs with natural images --> improved, human-like behavior

    on search tasks with natural images Training method Source dataset DNN architecture Accuracy (largest object) (mean (S. D.)) transfer ImageNet VGG16 0.786 (0.007) transfer ImageNet AlexNet 0.652 (0.010) initialize Pascal VOC AlexNet 0.390 (0.010) initialize Pascal VOC VGG16 0.353 (0.060) transfer search stimuli VGG16 0.262 (0.004) transfer search stimuli AlexNet 0.208 (0.000)
  17. Results Optimizing DNNs with natural images --> improved, human-like behavior

    on search tasks with natural images Training method Source dataset DNN architecture Accuracy (largest object) (mean (S. D.)) transfer ImageNet VGG16 0.786 (0.007) transfer ImageNet AlexNet 0.652 (0.010) initialize Pascal VOC AlexNet 0.390 (0.010) initialize Pascal VOC VGG16 0.353 (0.060) transfer search stimuli VGG16 0.262 (0.004) transfer search stimuli AlexNet 0.208 (0.000)
  18. Results Optimizing DNNs with natural images --> improved, human-like behavior

    on search tasks with natural images
  19. Discussion Mismatch may be impeding our ability to understand visual

    search behavior
  20. Discussion Future work could compare behavior of different models on

    a benchmark set of stimuli and tasks Guided Search 6.0, Wolfe 2021
  21. NickleDave Thank you! Lifelong Learning Machines program, DARPA HR0011-18-2-0019 2017

    William K. and Katherine W. Estes Fund to F. Pestilli, R. Goldstone and L. Smith, Indiana University Bloomington. nicholdav