Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Could simplified stimuli change how the brain p...
Search
David Nicholson
November 23, 2021
Research
0
84
Could simplified stimuli change how the brain performs visual search tasks?
flash talk for Neuromatch 4.0
David Nicholson
November 23, 2021
Tweet
Share
More Decks by David Nicholson
See All by David Nicholson
VocalPy: a core Python package for acoustic communication research
nickledave
0
33
sorry-no-chatgpt-PyCon-2023-lightning-talk
nickledave
0
70
pyvanot
nickledave
0
69
vak: software for automated annotation of vocalizations with neural networks
nickledave
0
83
scipy-2019-visual-search-Tensorflow-talk
nickledave
0
88
scipy-2019-lightning-talk
nickledave
0
140
Automated Annotation of Animal Vocalizations
nickledave
0
75
Neural networks for segmentation of vocalizations
nickledave
0
390
Teaching Data Science to Scientists
nickledave
0
180
Other Decks in Research
See All in Research
最適化と機械学習による問題解決
mickey_kubo
0
150
Computational OT #1 - Monge and Kantorovitch
gpeyre
0
220
Mechanistic Interpretability:解釈可能性研究の新たな潮流
koshiro_aoki
1
380
ストレス計測方法の確立に向けたマルチモーダルデータの活用
yurikomium
0
1k
SatCLIP: Global, General-Purpose Location Embeddings with Satellite Imagery
satai
3
250
Agentic AIとMCPを利用したサービス作成入門
mickey_kubo
0
370
LLM-as-a-Judge: 文章をLLMで評価する@教育機関DXシンポ
k141303
3
850
Principled AI ~深層学習時代における課題解決の方法論~
taniai
3
1.2k
cvpaper.challenge 10年の軌跡 / cvpaper.challenge a decade-long journey
gatheluck
1
270
ノンパラメトリック分布表現を用いた位置尤度場周辺化によるRTK-GNSSの整数アンビギュイティ推定
aoki_nosse
0
350
最適決定木を用いた処方的価格最適化
mickey_kubo
4
1.8k
Trust No Bot? Forging Confidence in AI for Software Engineering
tomzimmermann
1
250
Featured
See All Featured
Practical Orchestrator
shlominoach
190
11k
YesSQL, Process and Tooling at Scale
rocio
173
14k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
50
5.5k
RailsConf 2023
tenderlove
30
1.2k
Documentation Writing (for coders)
carmenintech
73
5k
Balancing Empowerment & Direction
lara
1
530
How to Ace a Technical Interview
jacobian
278
23k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
251
21k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
507
140k
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
kevinliebholz
34
6k
Art, The Web, and Tiny UX
lynnandtonic
301
21k
Reflections from 52 weeks, 52 projects
jeffersonlam
351
21k
Transcript
Could simplified stimuli change how the brain performs visual search
tasks? David Nicholson NMC4 December 2021
Introduction Visual search: a real-world behavior we engage in constantly
Introduction In the laboratory, visual search tasks use simplified stimuli
Peelen and Kastner, 2014
Introduction Hallmark of behavior exhibited in laboratory visual search tasks:
set size effects
Introduction The visual system is optimized to search natural images
Peelen and Kastner, 2014
Introduction → simplified stimuli change visual search behavior How could
we test this? Peelen and Kastner, 2014
Methods deep neural networks for image classification AlexNet DNN architecture
"plane"
Methods ~ state-of-the-art models of object recognition in the visual
system adapted from DiCarlo and Cox 2007 AlexNet ANN architecture: ~ primate ventral visual stream "retina space" "inferior temporal cortex space" separating hyperplane
Methods deep neural networks optimized for image classification (Kell McDermott
2019) step
Methods Transfer learning to adapt pre-trained DNNs to visual search
tasks https://github.com/NickleDave/searchstims
Methods Transfer learning to adapt pre-trained DNNs to visual search
tasks the Visual Search Difficulty dataset "How Hard Can It Be? Estimating the Difficulty of Visual Search in an Image". Ionescu, et al. 2016
Results DNNs exhibit set size effects
Results Set size effects result from optimizing DNNs to classify
natural images
Results Optimizing DNNs with natural images --> improved, human-like behavior
on search tasks with natural images
Results Optimizing DNNs with natural images --> improved, human-like behavior
on search tasks with natural images Training method Source dataset DNN architecture Accuracy (largest object) (mean (S. D.)) transfer ImageNet VGG16 0.786 (0.007) transfer ImageNet AlexNet 0.652 (0.010) initialize Pascal VOC AlexNet 0.390 (0.010) initialize Pascal VOC VGG16 0.353 (0.060) transfer search stimuli VGG16 0.262 (0.004) transfer search stimuli AlexNet 0.208 (0.000)
Results Optimizing DNNs with natural images --> improved, human-like behavior
on search tasks with natural images Training method Source dataset DNN architecture Accuracy (largest object) (mean (S. D.)) transfer ImageNet VGG16 0.786 (0.007) transfer ImageNet AlexNet 0.652 (0.010) initialize Pascal VOC AlexNet 0.390 (0.010) initialize Pascal VOC VGG16 0.353 (0.060) transfer search stimuli VGG16 0.262 (0.004) transfer search stimuli AlexNet 0.208 (0.000)
Results Optimizing DNNs with natural images --> improved, human-like behavior
on search tasks with natural images Training method Source dataset DNN architecture Accuracy (largest object) (mean (S. D.)) transfer ImageNet VGG16 0.786 (0.007) transfer ImageNet AlexNet 0.652 (0.010) initialize Pascal VOC AlexNet 0.390 (0.010) initialize Pascal VOC VGG16 0.353 (0.060) transfer search stimuli VGG16 0.262 (0.004) transfer search stimuli AlexNet 0.208 (0.000)
Results Optimizing DNNs with natural images --> improved, human-like behavior
on search tasks with natural images
Discussion Mismatch may be impeding our ability to understand visual
search behavior
Discussion Future work could compare behavior of different models on
a benchmark set of stimuli and tasks Guided Search 6.0, Wolfe 2021
NickleDave Thank you! Lifelong Learning Machines program, DARPA HR0011-18-2-0019 2017
William K. and Katherine W. Estes Fund to F. Pestilli, R. Goldstone and L. Smith, Indiana University Bloomington. nicholdav