Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up for free
SNLP2019
Ayana Niwa
September 25, 2019
Research
1
270
SNLP2019
第11回最先端NLP勉強会 発表資料
Ayana Niwa
September 25, 2019
Tweet
Share
More Decks by Ayana Niwa
See All by Ayana Niwa
UnNatural Language Inference
ayaniwa
0
220
Trends in Natural Language Processing at NeurIPS 2019.
ayaniwa
9
3.5k
Other Decks in Research
See All in Research
中国のオープンソースムーブメント:その現状と可能性 #中国オープンソース
takasumasakazu
2
580
第10回チャンピオンズミーティング・アクエリアス杯決勝集計 / Umamusume Aquarius 2022 Final
kitachan_black
0
690
再帰化への認知的転回/the-turn-to-recursive-system
monochromegane
0
120
SketchODE: Learning neural sketch representation in continuous time
dasayan05
0
120
AI最新論文読み会2022年4月
ailaboocu
0
330
データサイエンティストと博士の専門性
mtakano
1
140
CompilerGym CGO 2022 Tutorial Part 2
chriscummins
0
470
MioGatto による数式グラウンディング データセットの構築 / nlp2022
wtsnjp
0
120
第8回チャンピオンズミーティング・サジタリウス杯ラウンド1集計 / Umamusume Sagittarius 2021 Round1
kitachan_black
0
2.2k
機械学習ベースの動画像処理における近似計算手法の検討 (CPSY 2022/03)
hashi0203
0
170
GovTechとマーケットデザイン (渋谷区山室係長)
daimoriwaki
0
140
Task-Oriented Word Segmentation (Presentation for Doctoral Dissertation)
tathi
3
240
Featured
See All Featured
Building a Scalable Design System with Sketch
lauravandoore
447
30k
Rebuilding a faster, lazier Slack
samanthasiow
62
7.2k
The Brand Is Dead. Long Live the Brand.
mthomps
45
2.7k
Practical Orchestrator
shlominoach
178
8.6k
The Straight Up "How To Draw Better" Workshop
denniskardys
225
120k
Why You Should Never Use an ORM
jnunemaker
PRO
47
5.5k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
15
910
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
7
1k
Bash Introduction
62gerente
596
210k
Writing Fast Ruby
sferik
612
57k
The Invisible Customer
myddelton
110
11k
How GitHub Uses GitHub to Build GitHub
holman
465
280k
Transcript
Probing for Semantic Classes: Diagnosing the Meaning Content of Word
Embeddings Yadollah Yaghoobzadeh, Katharina Kann, Timothy J. Hazen, Eneko Agirre, Hinrich Schutze ACL2019 11 NLP 2019/09/28 <ayana.niwa@nlp.c.titech.ac.jp>
Outline 2 H9P=O8 O8 "!PC
$@K !.S%,'3 F5T&*'$:I P=O8$.S%,'PV ! R<72$PC H/+%)- >#"!.S O8! H9E69NR<72;BJ?Q1L >#".S(rare senses)P=O8APM0 " 4D('% UG
Background 3 # " !Word2VecGrove #"NLPIR* - ELMoBERT
$'+ ) %" full-space , % & L "$( " # &!
Background 4 # "@!Semantic class>*( S-class , "@=6:-&C?4=6
B $8 SEMCAT, HyperLex… =1;+=62%A → 0# .9/ - WIKI-PSE(WIKIpedia-based resource for Probing Semantics in word Embeddings) - "@=1;+sense embedding'7) “Lamb” Food3< Living-thing5< "@ !
Background 5 # Apple Apple Apple
+ Word embeddingsense embedding Arora et al. (2018) Arora et al. (2018) Word embedding sparse coding WIKI-PSEsense embedding Word embedding Sense embedding Sense embedding
WIKI-PSE Resource 6 Wikipedia .!2,$# .! ()
S-class 113 FIGER types%+134"10 person/authorperson/politician… à person -S-class *'.!/.! & - S-class0
WIKI-PSE Resource 7 "$% %2& +:! 343,000-6.-S-class#5'5000;-!% 75,.6,.83
+*1! /4 44,250- -S-class - S-class )0organization (9food @apple@ – food @apple@ – organization
8 Word embedding (word) @apple@ WIKI-PSE
word embedding Sense embedding @apple@ @apple@ - food @apple@ - organization @ word/S-class Uniform sum (unifΣ) !" Weighted sum (wghtΣ) !" # = % " !" #&" # Aggregated word embedding #&" Sense (word/S-class) embeddings word, unifΣ, wghtΣ
Experiments 9 Problem settings SkipGram Structured SkipGram
WIKI-PSE (LR) # (MLP k*)KNN) 1. word % word embedding 2. unifΣ sense embedding " 3. wghtΣ sense embedding ! $'& (" Word embedding
Experiments 10 Probing Task 1S-class Prediction S-class
@apple@ +food ∩ +organization ∩ -event S-class
Experiments 11 MLP > LRKNN 8&!S-class ."7;5
KNN )#, *:3, <(3-/ unifΣ > word > wghtΣ 'rare sense 0$ 97< à Rare sense42%+6 unifΣ Probing Task 1S-class Prediction F11 4
Experiments 12 Probing Task 1S-class Prediction
unifΣ $ #rare sense (13,000) F1! "
Experiments 13 Probing Task 1S-class Prediction * #4(! Recall
) S-class& sense embedding+%3 Rare S-class/"12 %"0, .$- Recall Dominance S-class' Recall
Experiments 14 Probing Task 1S-class Prediction /&)5. !"$' ,(20-
3+1(#41( Personlocation,(20* S-class !"6 20% Recall Recall #4 ,(20 Typicality Recall
Experiments 15 " Probing Task 2Ambiguity
Prediction ! L2 SSKIP$" LR / KNN / MLP ! ! unifΣ > word > wghtΣ KNN à % FREQUENCY(Baseline) # &LR
NLP Application Experiments 16 wghtΣ > word >(=) unifΣ <--
Probing task #!! -the U.S. Attorney’s Office announced Friday → location Common S-classtime Rare S-class location Friday mountain unifΣ ( !+ entity mention MC ), CRMR $)& SUBJ %*' MRPC " S-class
Summary & Conclusion 17 * $2"(3 WIKI-PSE'. 1)/& 0+$2/&
1, 1)/&% # 0+$2- a a e Rare sense e harder NLP Rare sense !# e