Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Could simplified stimuli change how the brain p...
Search
David Nicholson
November 23, 2021
Research
0
84
Could simplified stimuli change how the brain performs visual search tasks?
flash talk for Neuromatch 4.0
David Nicholson
November 23, 2021
Tweet
Share
More Decks by David Nicholson
See All by David Nicholson
VocalPy: a core Python package for acoustic communication research
nickledave
0
35
sorry-no-chatgpt-PyCon-2023-lightning-talk
nickledave
0
71
pyvanot
nickledave
0
70
vak: software for automated annotation of vocalizations with neural networks
nickledave
0
84
scipy-2019-visual-search-Tensorflow-talk
nickledave
0
91
scipy-2019-lightning-talk
nickledave
0
140
Automated Annotation of Animal Vocalizations
nickledave
0
75
Neural networks for segmentation of vocalizations
nickledave
0
390
Teaching Data Science to Scientists
nickledave
0
180
Other Decks in Research
See All in Research
When Submarine Cables Go Dark: Examining the Web Services Resilience Amid Global Internet Disruptions
irvin
0
290
Delta Airlines® Customer Care in the U.S.: How to Reach Them Now
bookingcomcustomersupportusa
0
110
一人称視点映像解析の最先端(MIRU2025 チュートリアル)
takumayagi
6
3.3k
Creation and environmental applications of 15-year daily inundation and vegetation maps for Siberia by integrating satellite and meteorological datasets
satai
3
200
Streamlit 総合解説 ~ PythonistaのためのWebアプリ開発 ~
mickey_kubo
1
1.4k
2025年度人工知能学会全国大会チュートリアル講演「深層基盤モデルの数理」
taiji_suzuki
25
18k
A multimodal data fusion model for accurate and interpretable urban land use mapping with uncertainty analysis
satai
3
270
多言語カスタマーインタビューの“壁”を越える~PMと生成AIの共創~ 株式会社ジグザグ 松野 亘
watarumatsuno
0
110
SSII2025 [TS1] 光学・物理原理に基づく深層画像生成
ssii
PRO
4
4.1k
A scalable, annual aboveground biomass product for monitoring carbon impacts of ecosystem restoration projects
satai
3
160
集合間Bregmanダイバージェンスと置換不変NNによるその学習
wasyro
0
140
最適決定木を用いた処方的価格最適化
mickey_kubo
4
1.8k
Featured
See All Featured
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
194
16k
Into the Great Unknown - MozCon
thekraken
40
2k
Art, The Web, and Tiny UX
lynnandtonic
302
21k
BBQ
matthewcrist
89
9.8k
Making Projects Easy
brettharned
117
6.3k
GraphQLの誤解/rethinking-graphql
sonatard
71
11k
How To Stay Up To Date on Web Technology
chriscoyier
790
250k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
44
2.4k
Designing for humans not robots
tammielis
253
25k
Balancing Empowerment & Direction
lara
2
590
Visualization
eitanlees
147
16k
Why Our Code Smells
bkeepers
PRO
338
57k
Transcript
Could simplified stimuli change how the brain performs visual search
tasks? David Nicholson NMC4 December 2021
Introduction Visual search: a real-world behavior we engage in constantly
Introduction In the laboratory, visual search tasks use simplified stimuli
Peelen and Kastner, 2014
Introduction Hallmark of behavior exhibited in laboratory visual search tasks:
set size effects
Introduction The visual system is optimized to search natural images
Peelen and Kastner, 2014
Introduction → simplified stimuli change visual search behavior How could
we test this? Peelen and Kastner, 2014
Methods deep neural networks for image classification AlexNet DNN architecture
"plane"
Methods ~ state-of-the-art models of object recognition in the visual
system adapted from DiCarlo and Cox 2007 AlexNet ANN architecture: ~ primate ventral visual stream "retina space" "inferior temporal cortex space" separating hyperplane
Methods deep neural networks optimized for image classification (Kell McDermott
2019) step
Methods Transfer learning to adapt pre-trained DNNs to visual search
tasks https://github.com/NickleDave/searchstims
Methods Transfer learning to adapt pre-trained DNNs to visual search
tasks the Visual Search Difficulty dataset "How Hard Can It Be? Estimating the Difficulty of Visual Search in an Image". Ionescu, et al. 2016
Results DNNs exhibit set size effects
Results Set size effects result from optimizing DNNs to classify
natural images
Results Optimizing DNNs with natural images --> improved, human-like behavior
on search tasks with natural images
Results Optimizing DNNs with natural images --> improved, human-like behavior
on search tasks with natural images Training method Source dataset DNN architecture Accuracy (largest object) (mean (S. D.)) transfer ImageNet VGG16 0.786 (0.007) transfer ImageNet AlexNet 0.652 (0.010) initialize Pascal VOC AlexNet 0.390 (0.010) initialize Pascal VOC VGG16 0.353 (0.060) transfer search stimuli VGG16 0.262 (0.004) transfer search stimuli AlexNet 0.208 (0.000)
Results Optimizing DNNs with natural images --> improved, human-like behavior
on search tasks with natural images Training method Source dataset DNN architecture Accuracy (largest object) (mean (S. D.)) transfer ImageNet VGG16 0.786 (0.007) transfer ImageNet AlexNet 0.652 (0.010) initialize Pascal VOC AlexNet 0.390 (0.010) initialize Pascal VOC VGG16 0.353 (0.060) transfer search stimuli VGG16 0.262 (0.004) transfer search stimuli AlexNet 0.208 (0.000)
Results Optimizing DNNs with natural images --> improved, human-like behavior
on search tasks with natural images Training method Source dataset DNN architecture Accuracy (largest object) (mean (S. D.)) transfer ImageNet VGG16 0.786 (0.007) transfer ImageNet AlexNet 0.652 (0.010) initialize Pascal VOC AlexNet 0.390 (0.010) initialize Pascal VOC VGG16 0.353 (0.060) transfer search stimuli VGG16 0.262 (0.004) transfer search stimuli AlexNet 0.208 (0.000)
Results Optimizing DNNs with natural images --> improved, human-like behavior
on search tasks with natural images
Discussion Mismatch may be impeding our ability to understand visual
search behavior
Discussion Future work could compare behavior of different models on
a benchmark set of stimuli and tasks Guided Search 6.0, Wolfe 2021
NickleDave Thank you! Lifelong Learning Machines program, DARPA HR0011-18-2-0019 2017
William K. and Katherine W. Estes Fund to F. Pestilli, R. Goldstone and L. Smith, Indiana University Bloomington. nicholdav