Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
文献紹介:Attention is not not Explanation
Search
Taichi Aida
August 28, 2019
Technology
0
220
文献紹介:Attention is not not Explanation
Taichi Aida
August 28, 2019
Tweet
Share
More Decks by Taichi Aida
See All by Taichi Aida
PhD Defence: Considering Temporal and Contextual Information for Lexical Semantic Change Detection
a1da4
0
120
文献紹介:A Multidimensional Framework for Evaluating Lexical Semantic Change with Social Science Applications
a1da4
1
250
YANS2024:目指せ国際会議!「ネットワーキングの極意(国際会議編)」
a1da4
0
150
言語処理学会30周年記念事業留学支援交流会@YANS2024:「学生のための短期留学」
a1da4
1
300
新入生向けチュートリアル:文献のサーベイv2
a1da4
13
9.4k
文献紹介:Isotropic Representation Can Improve Zero-Shot Cross-Lingual Transfer on Multilingual Language Models
a1da4
0
150
文献紹介:WhitenedCSE: Whitening-based Contrastive Learning of Sentence Embeddings
a1da4
1
210
文献紹介:On the Transformation of Latent Space in Fine-Tuned NLP Models
a1da4
0
78
新入生向けチュートリアル:文献のサーベイ
a1da4
0
430
Other Decks in Technology
See All in Technology
わたしのOSS活動
kazupon
2
330
LINEギフトにおけるバックエンド開発
lycorptech_jp
PRO
0
170
実は強い 非ViTな画像認識モデル
tattaka
1
920
Cracking the Coding Interview 6th Edition
gdplabs
14
28k
OPENLOGI Company Profile for engineer
hr01
1
20k
(機械学習システムでも) SLO から始める信頼性構築 - ゆる SRE#9 2025/02/21
daigo0927
0
220
CDKのコードを書く環境を作りました with Amazon Q
nobuhitomorioka
1
130
データエンジニアリング領域におけるDuckDBのユースケース
chanyou0311
5
1.7k
デスクトップだけじゃないUbuntu
mtyshibata
0
600
2025-02-21 ゆるSRE勉強会 Enhancing SRE Using AI
yoshiiryo1
1
440
【内製開発Summit 2025】イオンスマートテクノロジーの内製化組織の作り方/In-house-development-summit-AST
aeonpeople
1
430
ビジネスモデリング道場 目的と背景
masuda220
PRO
9
680
Featured
See All Featured
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
32
2.1k
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
panda_program
100
18k
Documentation Writing (for coders)
carmenintech
67
4.6k
XXLCSS - How to scale CSS and keep your sanity
sugarenia
248
1.3M
RailsConf 2023
tenderlove
29
1k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
47
5.2k
Designing for Performance
lara
604
68k
Stop Working from a Prison Cell
hatefulcrawdad
267
20k
Writing Fast Ruby
sferik
628
61k
Done Done
chrislema
182
16k
The Pragmatic Product Professional
lauravandoore
32
6.4k
The World Runs on Bad Software
bkeepers
PRO
67
11k
Transcript
จݙհʢʣ Attention is not not Explanation Sarah Wiegreffe, Yuval Pinter
EMNLP2019 Ԭٕज़Պֶେֶ ࣗવݴޠॲཧݚڀࣨɹ ૬ాɹଠҰ
"CTUSBDU w l"UUFOUJPOJTOPU&YQMBOBUJPOzͷओு&YQMBOBUJPOͷఆٛʹґଘ w ͋ΒΏΔཁૉΛߟྀͰ͖ΔΑ͏ʹɺͭͷ࣮ݧΛఏҊ͠ɺߦͳͬͨ w ΘΓͱͳΔ͕"UUFOUJPOΑΓ͏·͘ಈ࡞͠ͳ͔ͬͨͷͰɺ "UUFOUJPOͷ༗༻ੑΛ൱ఆ͢Δ͜ͱͰ͖ͳ͍͜ͱ͕Θ͔ͬͨ !2
"UUFOUJPOJTOPU&YQMBOBUJPO w"UUFOUJPOTDPSFͱॏཁͳࢦඪʢMFBWFPOFPVU HSBEJFOUʣͱͷؒʹ૬ؔ ͕ͳ͔ͬͨ w"UUFOUJPOͷΛϥϯμϜʹม͑ͨ࣌ʹ΄΅มΘΒͳ͍݁Ռ͕ग़ͨ wࢀߟɿIUUQTTQFBLFSEFDLDPNPLBZVXFOYJBOTIBPKJF BUUFOUJPOJTOPUFYQMBOBUJPO ˠ"UUFOUJPO&YQMBOBUJPOʢઆ໌Λఏڙ͢Δͷʣ ɹͰͳ͍ʁ !3
&YQFSJNFOU ɾ+8ɿઌߦݚڀ ɾjɿྨλεΫ ɾjɿࢄΛ֬ೝ ɾjɿͷஅ ɾjɿఢରతֶश !4
&YQFSJNFOUBM4FUVQ w @ ɾl"UUFOUJPOJTOPU&YQMBOBUJPOzͱ΄΅ಉ͡ ɾϞσϧɿํ-45. "UUFOUJPO !5
%FpOJUJPOT Ҿ༻ɿIUUQTTQFBLFSEFDLDPNPLBZVXFOYJBOTIBPKJFBUUFOUJPOJTOPUFYQMBOBUJPO TMJEF ɾl"UUFOUJPOJTOPU&YQMBOBUJPOzͱಉ͡ !6
&YQFSJNFOUjྨλεΫ w "UUFOUJPOͷඞཁੑΛௐͯɺྨ λεΫͱσʔληοτͷଥੑΛ֬ೝ w "UUFOUJPO͕ඞཁͰ͋Δ߹ɺ#BTF ͱ6OJGPSNʢ"UUFOUJPOͷॏΈ͕ɹ ۉҰʣͷؒͰ݁Ռ͕ѱԽ͢Δͣ w ͭͷྨλεΫͰɺ"UUFOUJPOΛ
༻͍ͯ΄ͱΜͲվળ͠ͳ͍ w Ҏ߱ɺ͜ͷͭͷதͰ445ͷΈѻ͏ !7
&YQFSJNFOUjࢄͷ֬ೝ !8 ɾઌߦݚڀͰಘΒΕͨࢄ͕ ɹਖ਼͘͠ಘΒΕ͍ͯΔ͔֬ೝ ɾॳظԽͷϥϯμϜγʔυΛ ɹ༻ͯͭ͠ͷϞσϧͰੳ ɹʢTFFEʣ ɾఢରతͳϞσϧ ɹʢBEWFSTBSZʣ
3FTVMUjࢄͷ֬ೝ !9 ɾϥϯμϜγʔυͷมߋʹରͯ͠ ɹϩόετͰ͋Δ
&YQFSJNFOUjͷஅ w 6OJGPSNɿ.-1ग़ྗΛۉʹߟྀ w 5SBJOFE.-1ɿ.-1ʹ "UUFOUJPOΛֶशͤ͞Δ w #BTF-45.ɿγϯϓϧͳ-45. ͷ"UUFOUJPOMBZFSͰී௨ʹֶश w
"EWFSTBSZɿఢରతֶशͰಘΒΕ ͨʹجͮ͘ ͭͷॏΈͷͰ j ͱಉ༷ʹྨ λεΫΛղ͖ɺൺֱ͢Δ !10
3FTVMUjͷஅ w 6OJGPSNɿ.-1ग़ྗΛۉʹߟྀ w 5SBJOFE.-1ɿ.-1ʹ "UUFOUJPOΛֶशͤ͞Δ w #BTF-45.ɿγϯϓϧͳ-45. ͷ"UUFOUJPOMBZFSͰී௨ʹֶश w
"EWFSTBSZɿఢରతֶशͰಘΒΕ ͨʹجͮ͘ w "UUFOUJPOΛ༻͍ͨ #BTF-45.͕࠷ྑ͍ ݁Ռʹͳͬͨ !11
.FUIPEjఢରతֶश w Ҏલͷͭʢj ʣ"UUFOUJPOͷ༗ҙੑΛςετ͢Δํ๏ w "UUFOUJPOʹΑΔҎ֎Ͱɺ"UUFOUJPOͱಉͷ༧ଌ͕Ͱ͖ΔΛ ಘΔͨΊʹɺఢରతֶशΛ༻͍Δ w ֬తޯ߱Լ๏Ͱߋ৽͠ɺϞσϧ.BΛಘΔ
!12
&YQMBOBUJPO w ʮઆ໌Մೳͳ"*ʯɿಁ໌ੑɺઆ໌ՄೳੑɺղऍՄೳੑ w -JQUPO ɿಁ໌ੑΛʮϞσϧͷಛఆͷ෦͕ਓ͕ؒཧղͰ͖ΔߏʹରԠ ͢Δํ๏ʯͱఆٛ w 3VEJO
ɿઆ໌ՄೳੑʻղऍՄೳੑͱఆٛ w ʮೖྗͷಛఆͷཁૉʹߴ͍"UUFOUJPOXFJHIU͕͋ΔͱɺϞσϧ͕ͦͷ༧ଌΛ ߦ͏͜ͱʹͭͳ͕Δʁʯ w ෳͷ͔Βಉ͡༧ଌ͕ಘΒΕͨ࣌ɺಁ໌ੑͷԼͷഉଞੑʹΑΓɺ "UUFOUJPO&YQMBOBUJPOͰͳ͍ͱ͞ΕΔ !13
$PODMVTJPO w &YQMBOBUJPO͔Ͳ͏͔ఆٛʹΑΔ w ༷ʑͳֶशํ๏ͰಘͨସͱͳΔɺैདྷͷ"UUFOUJPOʹൺͯ݁ Ռ͕ѱ͔ͬͨʢjʣɻ͜ΕʹΑΓɺ"UUFOUJPO͕τʔΫϯͱ༧ଌͷؔ ʹ͍ͭͯҙٛͷ͋Δ͜ͱΛֶश͍ͯ͠Δͱ͍͏͜ͱ͕ߟ͑ΒΕΔɻ !14