Distant Learning for Entity Linking with Automatic Noise Detection

ACL19 sup. for journal club.

Other EL paper slides and summaries:
https://github.com/izuna385/EntityLinking_RecentTrend

Transcript

1. 1/20 1 Distant Learning for Entity Linking with Automatic Noise Detection @izuna385
2. 2/20 Entity Linking • Link mention to specific entity in Knowledge Base 2 Beam, Andrew L., et al. "Clinical Concept Embeddings Learned from Massive Sources of Medical Data." arXiv:1804.01486 (2018). entity Knowledge Base
3. 3/20 Procedure 3 1. Prepare Mention/Context vector 2. Learn/prepare Entity(inKB) representation 3. Candidate generation 4. Linking
4. 4/20 Procedure 4 1. Prepare Mention/Context vector 2. Learn/prepare Entity(inKB) representation 3. Candidate generation 4. Linking Large amount of labeled data Wikipedia-hyperlink based alias table
5. 5/20 Available: Unlabeled(no gold) text + KB
6. 6/20
7. 7/20 : scorer for linking : loss for linker expecting high link score from bag in which possibly exists gold entity expecting low link score from negative-sampled bag
8. 8/20 Under “Supervised” settings If candidate generation fail to get gold entity, we can simply add gold entity to bag. ← Shikhar et al., ACL’18
9. 9/20 Under “Distant” settings We can’t know whether candidate bag has gold entity or not. But for training g with valid data point, we want to know/classify this.
10. 10/20 : Representation for E+ bag Again, expecting g puts high score to gold(or near) entity
11. 11/20 Noisy/Valid E+ classifier E+ bag rep. Contextualized mention pN : Classify whether bag for mention is ‘noisy’ or ‘valid’ 1 0
12. 12/20 Noisy/Valid E+ classifier E+ bag rep. Contextualized mention pN : Classify whether bag for mention is ‘noisy’ or ‘valid’ 1 0 NOTE: pN doesn’t have inputs of mention-candidate surface sim.
13. 13/20 P2. LEFT
14. 14/20 Loss for training pN (noisy/valid bag classifier) with linker valid(not noisy) prob. link loss For possibly valid(= gold entity exists) bag, sum up link loss for training linker, but… 1 0
15. 15/20 Loss for training pN (noisy/valid bag classifier) with linker valid(not noisy) prob. link loss assigning ‘noisy’ to all bags easily lead loss to 0, so we can’t train linker and bag classifier. 1 0
16. 16/20 Loss for training pN (noisy/valid bag classifier) with linker valid(not noisy) prob. link loss : Hyperparameter: beliefs about noisy data points. (e.g. 0.9) noisiness mean val. for Document 1 0
17. 17/20 Loss for training pN (noisy/valid bag classifier) with linker valid(not noisy) prob. link loss : Hyperparameter: beliefs about noisy data points. (e.g. 0.9) noisiness mean val. for Document expect training linker with gold-entity-highly-possibly-exists data ↑ by adding this loss 1 0
18. 18/20 1 0
19. 19/20 Table3: Linker error rate for dev set Blue: denoising succeeded Red: denoising failure, due to flaw of candidate generation ND: denoising bags for training linker = succeeded at catching the signal of gold entity in bag
20. 20/20 confirming pN separates valid/noisy data : bag in which gold entity doesn’t exist. Figure 3: