Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
文献紹介:Attention is not not Explanation
Search
Taichi Aida
August 28, 2019
Technology
0
210
文献紹介:Attention is not not Explanation
Taichi Aida
August 28, 2019
Tweet
Share
More Decks by Taichi Aida
See All by Taichi Aida
文献紹介:Isotropic Representation Can Improve Zero-Shot Cross-Lingual Transfer on Multilingual Language Models
a1da4
0
44
文献紹介:WhitenedCSE: Whitening-based Contrastive Learning of Sentence Embeddings
a1da4
1
55
文献紹介:On the Transformation of Latent Space in Fine-Tuned NLP Models
a1da4
0
20
新入生向けチュートリアル:文献のサーベイ
a1da4
0
96
文献紹介:Temporal Attention for Language Models
a1da4
0
170
文献紹介:Dynamic Contextualized Word Embeddings
a1da4
2
290
文献紹介:Learning Lexical Subspaces in a Distributional Vector Space
a1da4
0
230
文献紹介:Unsupervised Word Polysemy Quantification with Multiresolution Grids of Contextual Embeddings
a1da4
0
65
新入生向けチュートリアル:tmux
a1da4
0
130
Other Decks in Technology
See All in Technology
期待しすぎずに取り組む両面 TypeScript
shozawa
4
580
ビジネスとコード品質の接合点 そしてコード品質がそこに及ぼす影響 / The Intersections of Business and Engineering, and The Impact of Code Quality There
mtx2s
11
1.1k
中央集権体制からDataOpsへの転換 / centralized-to-dataops-transformation
pei0804
7
1.6k
8週連続ウェビナー_イチから学ぶFivetran
cmsuzu
0
170
依存ライブラリはどこに?
takesection
0
120
やっていきテスト
k6s4i53rx
0
160
コードレビューを支援するAI技術の応用
akkie76
3
190
家族アルバム みてねで直面してきた技術的負債 / MIXI KAG 2024
isaoshimizu
17
7.7k
私のRSpecの書き方 / How I write RSpec
tmtms
4
840
あらゆる商品を扱う商品データベースを再設計した話 / product db re-architecture
rince
8
3.9k
XRミーティング 2024-03-20
1ftseabass
PRO
0
100
GitHub最新情報キャッチアップ 2024年3月
dzeyelid
17
3.2k
Featured
See All Featured
Art, The Web, and Tiny UX
lynnandtonic
288
19k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
226
16k
Designing Dashboards & Data Visualisations in Web Apps
destraynor
225
51k
In The Pink: A Labor of Love
frogandcode
137
21k
A Modern Web Designer's Workflow
chriscoyier
689
190k
What's in a price? How to price your products and services
michaelherold
236
11k
Thoughts on Productivity
jonyablonski
57
3.8k
A designer walks into a library…
pauljervisheath
199
23k
Producing Creativity
orderedlist
PRO
335
39k
From Idea to $5000 a Month in 5 Months
shpigford
376
45k
The Invisible Side of Design
smashingmag
293
49k
Writing Fast Ruby
sferik
619
59k
Transcript
จݙհʢʣ Attention is not not Explanation Sarah Wiegreffe, Yuval Pinter
EMNLP2019 Ԭٕज़Պֶେֶ ࣗવݴޠॲཧݚڀࣨɹ ૬ాɹଠҰ
"CTUSBDU w l"UUFOUJPOJTOPU&YQMBOBUJPOzͷओு&YQMBOBUJPOͷఆٛʹґଘ w ͋ΒΏΔཁૉΛߟྀͰ͖ΔΑ͏ʹɺͭͷ࣮ݧΛఏҊ͠ɺߦͳͬͨ w ΘΓͱͳΔ͕"UUFOUJPOΑΓ͏·͘ಈ࡞͠ͳ͔ͬͨͷͰɺ "UUFOUJPOͷ༗༻ੑΛ൱ఆ͢Δ͜ͱͰ͖ͳ͍͜ͱ͕Θ͔ͬͨ !2
"UUFOUJPOJTOPU&YQMBOBUJPO w"UUFOUJPOTDPSFͱॏཁͳࢦඪʢMFBWFPOFPVU HSBEJFOUʣͱͷؒʹ૬ؔ ͕ͳ͔ͬͨ w"UUFOUJPOͷΛϥϯμϜʹม͑ͨ࣌ʹ΄΅มΘΒͳ͍݁Ռ͕ग़ͨ wࢀߟɿIUUQTTQFBLFSEFDLDPNPLBZVXFOYJBOTIBPKJF BUUFOUJPOJTOPUFYQMBOBUJPO ˠ"UUFOUJPO&YQMBOBUJPOʢઆ໌Λఏڙ͢Δͷʣ ɹͰͳ͍ʁ !3
&YQFSJNFOU ɾ+8ɿઌߦݚڀ ɾjɿྨλεΫ ɾjɿࢄΛ֬ೝ ɾjɿͷஅ ɾjɿఢରతֶश !4
&YQFSJNFOUBM4FUVQ w @ ɾl"UUFOUJPOJTOPU&YQMBOBUJPOzͱ΄΅ಉ͡ ɾϞσϧɿํ-45. "UUFOUJPO !5
%FpOJUJPOT Ҿ༻ɿIUUQTTQFBLFSEFDLDPNPLBZVXFOYJBOTIBPKJFBUUFOUJPOJTOPUFYQMBOBUJPO TMJEF ɾl"UUFOUJPOJTOPU&YQMBOBUJPOzͱಉ͡ !6
&YQFSJNFOUjྨλεΫ w "UUFOUJPOͷඞཁੑΛௐͯɺྨ λεΫͱσʔληοτͷଥੑΛ֬ೝ w "UUFOUJPO͕ඞཁͰ͋Δ߹ɺ#BTF ͱ6OJGPSNʢ"UUFOUJPOͷॏΈ͕ɹ ۉҰʣͷؒͰ݁Ռ͕ѱԽ͢Δͣ w ͭͷྨλεΫͰɺ"UUFOUJPOΛ
༻͍ͯ΄ͱΜͲվળ͠ͳ͍ w Ҏ߱ɺ͜ͷͭͷதͰ445ͷΈѻ͏ !7
&YQFSJNFOUjࢄͷ֬ೝ !8 ɾઌߦݚڀͰಘΒΕͨࢄ͕ ɹਖ਼͘͠ಘΒΕ͍ͯΔ͔֬ೝ ɾॳظԽͷϥϯμϜγʔυΛ ɹ༻ͯͭ͠ͷϞσϧͰੳ ɹʢTFFEʣ ɾఢରతͳϞσϧ ɹʢBEWFSTBSZʣ
3FTVMUjࢄͷ֬ೝ !9 ɾϥϯμϜγʔυͷมߋʹରͯ͠ ɹϩόετͰ͋Δ
&YQFSJNFOUjͷஅ w 6OJGPSNɿ.-1ग़ྗΛۉʹߟྀ w 5SBJOFE.-1ɿ.-1ʹ "UUFOUJPOΛֶशͤ͞Δ w #BTF-45.ɿγϯϓϧͳ-45. ͷ"UUFOUJPOMBZFSͰී௨ʹֶश w
"EWFSTBSZɿఢରతֶशͰಘΒΕ ͨʹجͮ͘ ͭͷॏΈͷͰ j ͱಉ༷ʹྨ λεΫΛղ͖ɺൺֱ͢Δ !10
3FTVMUjͷஅ w 6OJGPSNɿ.-1ग़ྗΛۉʹߟྀ w 5SBJOFE.-1ɿ.-1ʹ "UUFOUJPOΛֶशͤ͞Δ w #BTF-45.ɿγϯϓϧͳ-45. ͷ"UUFOUJPOMBZFSͰී௨ʹֶश w
"EWFSTBSZɿఢରతֶशͰಘΒΕ ͨʹجͮ͘ w "UUFOUJPOΛ༻͍ͨ #BTF-45.͕࠷ྑ͍ ݁Ռʹͳͬͨ !11
.FUIPEjఢରతֶश w Ҏલͷͭʢj ʣ"UUFOUJPOͷ༗ҙੑΛςετ͢Δํ๏ w "UUFOUJPOʹΑΔҎ֎Ͱɺ"UUFOUJPOͱಉͷ༧ଌ͕Ͱ͖ΔΛ ಘΔͨΊʹɺఢରతֶशΛ༻͍Δ w ֬తޯ߱Լ๏Ͱߋ৽͠ɺϞσϧ.BΛಘΔ
!12
&YQMBOBUJPO w ʮઆ໌Մೳͳ"*ʯɿಁ໌ੑɺઆ໌ՄೳੑɺղऍՄೳੑ w -JQUPO ɿಁ໌ੑΛʮϞσϧͷಛఆͷ෦͕ਓ͕ؒཧղͰ͖ΔߏʹରԠ ͢Δํ๏ʯͱఆٛ w 3VEJO
ɿઆ໌ՄೳੑʻղऍՄೳੑͱఆٛ w ʮೖྗͷಛఆͷཁૉʹߴ͍"UUFOUJPOXFJHIU͕͋ΔͱɺϞσϧ͕ͦͷ༧ଌΛ ߦ͏͜ͱʹͭͳ͕Δʁʯ w ෳͷ͔Βಉ͡༧ଌ͕ಘΒΕͨ࣌ɺಁ໌ੑͷԼͷഉଞੑʹΑΓɺ "UUFOUJPO&YQMBOBUJPOͰͳ͍ͱ͞ΕΔ !13
$PODMVTJPO w &YQMBOBUJPO͔Ͳ͏͔ఆٛʹΑΔ w ༷ʑͳֶशํ๏ͰಘͨସͱͳΔɺैདྷͷ"UUFOUJPOʹൺͯ݁ Ռ͕ѱ͔ͬͨʢjʣɻ͜ΕʹΑΓɺ"UUFOUJPO͕τʔΫϯͱ༧ଌͷؔ ʹ͍ͭͯҙٛͷ͋Δ͜ͱΛֶश͍ͯ͠Δͱ͍͏͜ͱ͕ߟ͑ΒΕΔɻ !14