文献紹介：Joint Embedding of Words and Labels for Text Classification

Transcript

1. 文献紹介 Joint Embedding of Words and Labels for Text Classification 長岡技術科学大学 自然言語処理研究室 多田　太郎 ２０１８年７月１９日
2. about this thesis Authors: Guoyin Wang, Chunyuan Li , Wenlin Wang, Yizhe Zhang, Dinghan Shen, Xinyuan Zhang, Ricardo Henao, Lawrence Carin : Duke University Journal reference: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, pages 1–11 2018 Association for Computational Linguistics 2
3. Abstract • Propose to view text classification as a label-word joint embedding problem. • Introduce an attention framework that measures the compatibility of embeddings between text sequences and labels. • LEAM algorithm requires much lower computational cost, and achieves comparable performance relative to the state-of-the-art. 3
4. Model 4
5. Compatibility : G Attention score : β Model 5
6. Classification on Benchmark Datasets 6
7. Experimental Results 7
8. Correlation between text sequence and label embedding 8
9. Comparison of model size and speed 9
10. Conclusions • Propose the label-embedding attentive models. • Embeds the words and labels in the same joint space, and measures the compatibility of word-label pairs to attend the document representations. • Compared with the previous methods, the LEAM algorithm requires much lower computational cost, and achieves comparable performance relative to the state-of-the-art. 10
12. results 12
13. Applications to Clinical Text 13
14. The Label Embedding Attentive Model (LEAM) to improve text classification. (i) Label-attentive text representation is informative for the downstream classification task, as it directly learns from a shared joint space. (ii) The LEAM learning procedure only involves a series of basic algebraic operations, and hence it retains the interpretability of simple models, especially when the label description is available. (iii) Our attention mechanism has fewer parameters and less computation than related methods. (iv) Demonstrating the effectiveness of our label-embedding attentive model, providing state- of-the-art results on bench mark datasets. (v) We further apply LEAM to predict the medical codes from clinical text. 14