Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Pathologies of Neural Models Make Interpretatio...
Search
Yasufumi Taniguchi
December 09, 2018
Research
1
1.8k
Pathologies of Neural Models Make Interpretations Difficult
Yasufumi Taniguchi
December 09, 2018
Tweet
Share
More Decks by Yasufumi Taniguchi
See All by Yasufumi Taniguchi
AllenNLPを使った開発
yasufumy
0
2.2k
Making Neural QA as Simple as Possible but not Simpler
yasufumy
0
96
Other Decks in Research
See All in Research
【緊急警告】日本の未来設計図 ~沈没か、再生か。国民と断行するラストチャンス~
yuutakasan
0
130
線形判別分析のPU学習による朝日歌壇短歌の分析
masakat0
0
130
クラウドのテレメトリーシステム研究動向2025年
yuukit
3
960
Adaptive fusion of multi-modal remote sensing data for optimal sub-field crop yield prediction
satai
3
220
Self-supervised audiovisual representation learning for remote sensing data
satai
3
220
数理最適化に基づく制御
mickey_kubo
5
680
Towards a More Efficient Reasoning LLM: AIMO2 Solution Summary and Introduction to Fast-Math Models
analokmaus
2
230
CSP: Self-Supervised Contrastive Spatial Pre-Training for Geospatial-Visual Representations
satai
3
210
Type Theory as a Formal Basis of Natural Language Semantics
daikimatsuoka
1
230
最適化と機械学習による問題解決
mickey_kubo
0
140
Transparency to sustain open science infrastructure - Printemps Couperin
mlarrieu
1
180
言語モデルの内部機序:解析と解釈
eumesy
PRO
49
18k
Featured
See All Featured
Building Applications with DynamoDB
mza
95
6.5k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
229
22k
Reflections from 52 weeks, 52 projects
jeffersonlam
351
20k
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
marcduiker
30
2.1k
BBQ
matthewcrist
89
9.7k
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
29
2.7k
Product Roadmaps are Hard
iamctodd
PRO
54
11k
How STYLIGHT went responsive
nonsquared
100
5.6k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
667
120k
The Illustrated Children's Guide to Kubernetes
chrisshort
48
50k
GraphQLの誤解/rethinking-graphql
sonatard
71
11k
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
irinanazarova
8
810
Transcript
ൃදऀ ୩ޱହ࢙ ҟৗͳڍಈ
!2 Pathological behavior ࣭จ͕did͚ͩͰ Ϟσϧͷग़ྗಉ͡ ֬ߴ͍
֓ཁ w NLPʹ͓͚ΔχϡʔϥϧϞσϧͷղੳख๏ΛఏҊ w Ϟσϧ͕λεΫΛղ্͘Ͱॏཁͳ୯ޠΛநग़͢Δख๏ w நग़͞Εͨ୯ޠਓʹͱͬͯҙຯෆ໌ w ҰํͰϞσϧநग़୯ޠͰਖ਼͘͠༧ଌ(Pathology) w
ղੳ݁Ռʹجͮ͘ਖ਼ଇԽ߲ΛఏҊ w ਖ਼ଇԽ߲ʹΑͬͯϞσϧͷղऍੑ্ !3
࣍ Ϟσϧղੳͷطଘख๏ ఏҊख๏ ࣮ݧ ·ͱΊ !4
Ϟσϧղੳͷطଘख๏
Ϟσϧղੳͷطଘख๏ !6 Adversarial Example Ϟσϧʹਓͷײʹ͢ΔڍಈΛͤ͞Δαϯϓϧ NLPͷλεΫ ओʹQAλεΫ Ͱύλʔϯ ਓʹͱͬͯҙຯͷͳ͍มߋ͕ɺϞσϧͷग़ྗΛܹมͤ͞Δέʔε
ਓʹͱͬͯ໌Β͔ͳมߋͰɺϞσϧ͕ग़ྗΛม͑ͳ͍έʔε
ग़ྗ͕ܹม͢Δέʔε !7 Jia et al., 2017 ΫΥʔλʔόοΫͷྸʹ͍ͭͯͷ จॻʹΫΥʔλʔόοΫͷഎ൪߸ʹ ؔ͢ΔจΛՃ Ϟσϧޡ
ग़ྗΛม͑ͳ͍έʔε !8 Mudrakarta et al., 2018 ݐͷന͍ϨϯΨ͕ରশ͔ʁ spherical (ٿঢ়ͷ) ݐͷന͍ϨϯΨ͕ٿঢ়͔ʁ
࣭จͷҙຯมԽ Ϟσϧͷ༧ଌෆม
2. ఏҊख๏
*OQVU3FEVDUJPO • ॏཁͰͳ͍୯ޠΛೖྗ͔ΒΓɺϞσϧͷڍಈΛੳ • Ϟσϧ͕ਖ਼͍͠ग़ྗΛ͢ΔͨΊʹඞཁͳ࠷୯ޠ (ॏཁ ୯ޠ) • Adversarial ExampleϞσϧʹͱͬͯͷॏཁ୯ޠʹண
*OQVU3FEVDUJPO !11 x y Ϟσϧͷ༧ଌ f( ⋅ ) χϡʔϥϧϞσϧ ೖྗܥྻ
(จจॻ) xi ೖྗܥྻͷ͋Δཁૉ (୯ޠ) g(xi |x) = f(y|x) − f(y|x−i ) ͋Δ୯ޠ ʹର͢Δ ॏཁΛఆٛ xi g i൪ͷ୯ޠΛফͨ͠ೖྗ
*OQVU3FEVDUJPO !12 g(xcontest |x) = f(y|x) − f(y|x−contest ) What
company won free advertisement due to QuickBooks contest ? What company won free advertisement due to QuickBooks contest ? g͕େ͖͚Εɺcontest͕ॏཁͳ୯ޠͱͳΔ Ϟσϧͷग़ྗʹେ͖͘د༩͍ͯ͠ΔͨΊ
*OQVU3FEVDUJPO !13 g(xi |x) = f(y|x) − f(y|x−i ) ॏཁͷ͍୯ޠΛআ
y͕มԽ͠ͳ͍Α͏ʹɺg͕࠷খͱͳΔ୯ޠiΛআ ͍ͯ͘͠
3. ࣮ݧ
ղੳͷରλεΫ 1. SQuAD w จॻͱ࣭จ͕༩͑ΒΕΔˠ࣭จʹରͯ͠Input Reduction w จॻ͔ΒղΛநग़͢ΔλεΫ 2. SNLI
w จ͕༩͑ΒΕΔˠͭͷจʹରͯ͠Input Reduction w จͷؔΛਪఆ͢ΔλεΫ 3. VQA w ը૾ͱ࣭จ͕༩͑ΒΕΔˠ࣭จʹରͯ͠Input Reduction w ղΛੜ͢ΔλεΫ !15
࣮ݧ༰ Input Reduction w Ϟσϧ͕ਖ਼͍͠ग़ྗΛ͢ΔαϯϓϧΛରʹ࣮ݧ w Input ReductionΛద༻ͨ͠ೖྗ(Reduced)ʹର͢ΔਓखධՁ w ReducedͱϥϯμϜʹ୯ޠΛམͱͨ͠߹(Random)ͷࠩҟͷධՁ
Regularization on Reduced Inputs w Input ReductionʹΑΔϞσϧͷPathological behaviorΛܰݮ͢Δਖ਼ଇԽ߲ ޙड़ ͷಋೖ !16
Reducedʹର͢ΔਓखධՁ !17 Reducedʹରͯ͠ ਓਖ਼͍͠༧ଌΛͰ ͖ͳ͍ w Reducedʹର͢Δਓͷਖ਼ w Ϟσϧͷਖ਼͕ͷαϯϓϧΛ༻
Reducedʹର͢ΔਓखධՁ !18 w ReducedͱRandomͷͲͪΒ͕ࣗવͳจ͔ w vs. Randomfifty-fiftyͱׂ͑ͨ߹ Reducedਓʹͱͬ ͯRandomͱಉ͡
Reducedͷࣄྫ !19 ʮͲ͜Ͱ࿅शͨ͠ ͔ʯΛฉ͔Ε͍ͯ ΔͷΘ͔Δ͕ɺ ʮͲͷνʔϜʯ͔ Θ͔Βͳ͍
Reducedͷฏۉ୯ޠ ͭͷλεΫͱɺ ਖ਼͢Δͷʹඞཁͳ୯ޠฏۉd
Reducedʹର͢ΔϞσϧͷ֬ !21 • Input Reductionͷద༻લޙͰϞσϧͷ ֬ʹมԽ΄ͱΜͲͳ͍ • ϞσϧӶ͍ϐʔΫΛ࣋ͭΑ͏ͳ Λֶश͍ͯ͠Δ͜ͱ͕ݪҼ
ਖ਼ଇԽ߲ͷಋೖ !22 ∑ (x,y)∈(X,Y) log(f(y|x)) + λ∑ ¯ x∈ ¯
X H(f(y| ¯ x)) Reducedʹରͯ͠ਖ਼͍͠yΛ ग़ྗ͠ʹ͘͘͢Δ ௨ৗͷతؔ Reducedαϯϓϧ௨ৗͷతؔΛֶͬͯशͨ͠ ϞσϧΛ༻͍ͯੜ
ਖ਼ଇԽ߲ͷޮՌ !23 • Ϟσϧͷਫ਼͕ඍ૿ • ਖ਼ʹඞཁͳ୯ޠ ͕૿Ճ
ਖ਼ଇԽ߲ͷޮՌ !24 ਓखධՁͷਫ਼্ Input Reductionͨ͠ೖྗ ͷղऍੑ্͕
ਖ਼ଇԽͨ͠Ϟσϧͷࣄྫ !25 Input Reductionͨ͠ೖྗ͕ਓͰ ղऍՄೳʹͳͬͨ
·ͱΊ ఏҊख๏ w NLPͷχϡʔϥϧϞσϧղੳख๏ͱͯ͠ɺInput ReductionΛఏҊ w ༧ଌʹد༩͠ͳ͍୯ޠΛೖྗ͔ΒΓɺϞσϧΛղੳ ࣮ݧ݁Ռ w ఏҊख๏Λద༻ͨ͠ೖྗਓʹͱͬͯҙຯෆ໌
w ҰํͰχϡʔϥϧϞσϧਖ਼͍͠༧ଌΛߦ͏ w ਖ਼ଇԽ߲Λಋೖ͢ΔͱϞσϧͷڍಈվળ !26