Upgrade to Pro — share decks privately, control downloads, hide ads and more …

文献紹介：Attention is not not Explanation

Sponsored · Your Podcast. Everywhere. Effortlessly. Share. Educate. Inspire. Entertain. You do you. We'll handle the rest. →
Avatar for Taichi Aida Taichi Aida
August 28, 2019
Technology
0
220

文献紹介：Attention is not not Explanation

Avatar for Taichi Aida

Taichi Aida

August 28, 2019
Tweet

More Decks by Taichi Aida

See All by Taichi Aida
意味を表すベクトル表現を用いたテキスト分析
Avatar for Taichi Aida a1da4
0
95
PhD Defence: Considering Temporal and Contextual Information for Lexical Semantic Change Detection
Avatar for Taichi Aida a1da4
1
260
文献紹介：A Multidimensional Framework for Evaluating Lexical Semantic Change with Social Science Applications
Avatar for Taichi Aida a1da4
1
370
YANS2024：目指せ国際会議！「ネットワーキングの極意（国際会議編）」
Avatar for Taichi Aida a1da4
0
290
言語処理学会30周年記念事業留学支援交流会＠YANS2024：「学生のための短期留学」
Avatar for Taichi Aida a1da4
1
410
新入生向けチュートリアル：文献のサーベイv2
Avatar for Taichi Aida a1da4
16
11k
文献紹介：Isotropic Representation Can Improve Zero-Shot Cross-Lingual Transfer on Multilingual Language Models
Avatar for Taichi Aida a1da4
0
210
文献紹介：WhitenedCSE: Whitening-based Contrastive Learning of Sentence Embeddings
Avatar for Taichi Aida a1da4
1
350
文献紹介：On the Transformation of Latent Space in Fine-Tuned NLP Models
Avatar for Taichi Aida a1da4
0
130

Other Decks in Technology

See All in Technology
OCI Database Management サービス詳細
Avatar for oracle4engineer oracle4engineer
PRO
1
7.4k
Oracle AI Database 移行・アップグレード勉強会 - RAT活用編
Avatar for oracle4engineer oracle4engineer
PRO
0
110
コミュニティが変えるキャリアの地平線：コロナ禍新卒入社のエンジニアがAWSコミュニティで見つけた成長の羅針盤
Avatar for Kento Suzuki kentosuzuki
0
130
マネージャー視点で考えるプロダクトエンジニアの評価 / Evaluating Product Engineers from a Manager's Perspective
Avatar for hiro-torii hiro_torii
0
190
Oracle Base Database Service 技術詳細
Avatar for oracle4engineer oracle4engineer
PRO
15
93k
Bedrock PolicyでAmazon Bedrock Guardrails利用を強制してみた
Avatar for Yudai Jinno yuu551
0
260
今こそ学びたいKubernetesネットワーク ～CNIが繋ぐNWとプラットフォームの「フラッと」な対話
Avatar for Takuto Nagami logica0419
5
480
登壇駆動学習のすすめ — CfPのネタの見つけ方と書くときに意識していること
Avatar for おおいし bicstone
3
130
学生・新卒・ジュニアから目指すSRE
Avatar for Hiroya Onoe hiroyaonoe
2
770
SchooでVue.js/Nuxtを技術選定している理由
Avatar for Okuto Oyama yamanoku
3
210
[CV勉強会@関東 World Model 読み会] Orbis: Overcoming Challenges of Long-Horizon Prediction in Driving World Models (Mousakhan+, NeurIPS 2025)
Avatar for abemii_ abemii
0
150
日本の85%が使う公共SaaSは、どう育ったのか
Avatar for たけだ taketakekaho
1
240

Featured

See All Featured
The Art of Delivering Value - GDevCon NA Keynote
Avatar for David Neal reverentgeek
16
1.8k
Producing Creativity
Avatar for Steve Smith orderedlist
PRO
348
40k
Build The Right Thing And Hit Your Dates
Avatar for Maggie C. maggiecrowley
39
3k
End of SEO as We Know It (SMX Advanced Version)
Avatar for Michael King ipullrank
3
3.9k
AI Search:
Where Are We & What Can We Do About It?
Avatar for Aleyda Solis aleyda
0
7k
Become a Pro
Avatar for Speaker Deck speakerdeck
PRO
31
5.8k
Game over? The fight for quality and originality in the time of robots
Avatar for Wayne Barker wayneb77
1
120
How to optimise 3,500 product descriptions for ecommerce in one day using ChatGPT
Avatar for Katarina Dahlin katarinadahlin
PRO
0
3.4k
Bash Introduction
Avatar for André Augusto Costa Santos 62gerente
615
210k
Designing for Performance
Avatar for Lara Hogan lara
610
70k
BBQ
Avatar for Matthew Crist matthewcrist
89
10k
Leveraging Curiosity to Care for An Aging Population
Avatar for Cassini Nazir cassininazir
1
170

Transcript

  1. จݙ঺հʢʣ Attention is not not Explanation Sarah Wiegreffe, Yuval Pinter

    EMNLP2019 ௕Ԭٕज़Պֶେֶ ࣗવݴޠॲཧݚڀࣨɹ ૬ాɹଠҰ

  2. "CTUSBDU w l"UUFOUJPOJTOPU&YQMBOBUJPOzͷओு͸&YQMBOBUJPOͷఆٛʹґଘ w ͋ΒΏΔཁૉΛߟྀͰ͖ΔΑ͏ʹɺͭͷ࣮ݧΛఏҊ͠ɺߦͳͬͨ w ୅ΘΓͱͳΔ෼෍͕"UUFOUJPOΑΓ΋͏·͘ಈ࡞͠ͳ͔ͬͨͷͰɺ "UUFOUJPOͷ༗༻ੑΛ൱ఆ͢Δ͜ͱ͸Ͱ͖ͳ͍͜ͱ͕Θ͔ͬͨ !2

  3. "UUFOUJPOJTOPU&YQMBOBUJPO w"UUFOUJPOTDPSFͱॏཁͳࢦඪʢMFBWFPOFPVU HSBEJFOUʣͱͷؒʹ૬ؔ ͕ͳ͔ͬͨ w"UUFOUJPOͷ෼෍ΛϥϯμϜʹม͑ͨ࣌ʹ΄΅มΘΒͳ͍݁Ռ͕ग़ͨ wࢀߟɿIUUQTTQFBLFSEFDLDPNPLBZVXFOYJBOTIBPKJF BUUFOUJPOJTOPUFYQMBOBUJPO ˠ"UUFOUJPO͸&YQMBOBUJPOʢઆ໌Λఏڙ͢Δ΋ͷʣ ɹͰ͸ͳ͍ʁ !3

  4. &YQFSJNFOU ɾ+8ɿઌߦݚڀ ɾjɿ෼ྨλεΫ ɾjɿ෼ࢄΛ֬ೝ ɾjɿ෼෍ͷ਍அ ɾjɿఢରతֶश !4

  5. &YQFSJNFOUBM4FUVQ w @ ɾl"UUFOUJPOJTOPU&YQMBOBUJPOzͱ΄΅ಉ͡ ɾϞσϧɿ૒ํ޲-45. "UUFOUJPO !5

  6. %FpOJUJPOT Ҿ༻ɿIUUQTTQFBLFSEFDLDPNPLBZVXFOYJBOTIBPKJFBUUFOUJPOJTOPUFYQMBOBUJPO TMJEF ɾl"UUFOUJPOJTOPU&YQMBOBUJPOzͱಉ͡ !6

  7. &YQFSJNFOUj෼ྨλεΫ w "UUFOUJPOͷඞཁੑΛௐ΂ͯɺ෼ྨ λεΫͱσʔληοτͷଥ౰ੑΛ֬ೝ w "UUFOUJPO͕ඞཁͰ͋Δ৔߹ɺ#BTF ͱ6OJGPSNʢ"UUFOUJPOͷॏΈ͕ɹ ۉҰʣͷؒͰ݁Ռ͕ѱԽ͢Δ͸ͣ w ͭͷ෼ྨλεΫͰ͸ɺ"UUFOUJPOΛ

    ༻͍ͯ΋΄ͱΜͲվળ͠ͳ͍ w Ҏ߱ɺ͜ͷͭͷதͰ445ͷΈѻ͏ !7

  8. &YQFSJNFOUj෼ࢄͷ֬ೝ !8 ɾઌߦݚڀͰಘΒΕͨ෼ࢄ͕ ɹਖ਼͘͠ಘΒΕ͍ͯΔ͔֬ೝ ɾॳظԽͷϥϯμϜγʔυΛ ɹ࢖༻ͯͭ͠ͷϞσϧͰ෼ੳ ɹʢTFFEʣ ɾఢରతͳϞσϧ ɹʢBEWFSTBSZʣ

  9. 3FTVMUj෼ࢄͷ֬ೝ !9 ɾϥϯμϜγʔυͷมߋʹରͯ͠ ɹϩόετͰ͋Δ

  10. &YQFSJNFOUj෼෍ͷ਍அ w 6OJGPSNɿ.-1ग़ྗΛۉ౳ʹߟྀ w 5SBJOFE.-1ɿ.-1ʹ "UUFOUJPOΛֶशͤ͞Δ w #BTF-45.ɿγϯϓϧͳ-45. ͷ"UUFOUJPOMBZFSͰී௨ʹֶश w

    "EWFSTBSZɿఢରతֶशͰಘΒΕ ͨ෼෍ʹجͮ͘ ͭͷॏΈͷ෼෍Ͱ j ͱಉ༷ʹ෼ྨ λεΫΛղ͖ɺൺֱ͢Δ !10

  11. 3FTVMUj෼෍ͷ਍அ w 6OJGPSNɿ.-1ग़ྗΛۉ౳ʹߟྀ w 5SBJOFE.-1ɿ.-1ʹ "UUFOUJPOΛֶशͤ͞Δ w #BTF-45.ɿγϯϓϧͳ-45. ͷ"UUFOUJPOMBZFSͰී௨ʹֶश w

    "EWFSTBSZɿఢରతֶशͰಘΒΕ ͨ෼෍ʹجͮ͘ w "UUFOUJPOΛ༻͍ͨ #BTF-45.͕࠷΋ྑ͍ ݁Ռʹͳͬͨ !11

  12. .FUIPEjఢରతֶश w Ҏલͷͭʢj  ʣ͸"UUFOUJPOͷ༗ҙੑΛςετ͢Δํ๏ w "UUFOUJPOʹΑΔ෼෍Ҏ֎Ͱɺ"UUFOUJPOͱಉ౳ͷ༧ଌ͕Ͱ͖Δ෼෍Λ ಘΔͨΊʹɺఢରతֶशΛ༻͍Δ w ֬཰తޯ഑߱Լ๏Ͱߋ৽͠ɺϞσϧ.BΛಘΔ

    !12

  13. &YQMBOBUJPO w ʮઆ໌Մೳͳ"*ʯɿಁ໌ੑɺઆ໌ՄೳੑɺղऍՄೳੑ w -JQUPO  ɿಁ໌ੑΛʮϞσϧͷಛఆͷ෦෼͕ਓ͕ؒཧղͰ͖Δߏ଄ʹରԠ ͢Δํ๏ʯͱఆٛ w 3VEJO

     ɿઆ໌ՄೳੑʻղऍՄೳੑͱఆٛ w ʮೖྗͷಛఆͷཁૉʹߴ͍"UUFOUJPOXFJHIU͕͋ΔͱɺϞσϧ͕ͦͷ༧ଌΛ ߦ͏͜ͱʹͭͳ͕Δʁʯ w ෳ਺ͷ෼෍͔Βಉ͡༧ଌ͕ಘΒΕͨ࣌ɺಁ໌ੑͷԼͷഉଞੑʹΑΓɺ "UUFOUJPO͸&YQMBOBUJPOͰ͸ͳ͍ͱ͞ΕΔ !13

  14. $PODMVTJPO w &YQMBOBUJPO͔Ͳ͏͔͸ఆٛʹΑΔ w ༷ʑͳֶशํ๏Ͱಘͨ୅ସͱͳΔ෼෍͸ɺैདྷͷ"UUFOUJPOʹൺ΂ͯ݁ Ռ͕ѱ͔ͬͨʢjʣɻ͜ΕʹΑΓɺ"UUFOUJPO͕τʔΫϯͱ༧ଌͷؔ ܎ʹ͍ͭͯҙٛͷ͋Δ͜ͱΛֶश͍ͯ͠Δͱ͍͏͜ͱ͕ߟ͑ΒΕΔɻ !14