$30 off During Our Annual Pro Sale. View Details »

文献紹介:Joint Embedding of Words and Labels for Text Classification

T.Tada
July 19, 2018
350

文献紹介:Joint Embedding of Words and Labels for Text Classification

T.Tada

July 19, 2018
Tweet

More Decks by T.Tada

Transcript

  1. 文献紹介 Joint Embedding of Words and Labels for Text Classification

    長岡技術科学大学 自然言語処理研究室 多田 太郎 2018年7月19日
  2. about this thesis Authors: Guoyin Wang, Chunyuan Li , Wenlin

    Wang, Yizhe Zhang, Dinghan Shen, Xinyuan Zhang, Ricardo Henao, Lawrence Carin : Duke University Journal reference: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, pages 1–11 2018 Association for Computational Linguistics 2
  3. Abstract • Propose to view text classification as a label-word

    joint embedding problem. • Introduce an attention framework that measures the compatibility of embeddings between text sequences and labels. • LEAM algorithm requires much lower computational cost, and achieves comparable performance relative to the state-of-the-art. 3
  4. Model 4

  5. Compatibility : G Attention score : β Model 5

  6. Classification on Benchmark Datasets 6

  7. Experimental Results 7

  8. Correlation between text sequence and label embedding 8

  9. Comparison of model size and speed 9

  10. Conclusions • Propose the label-embedding attentive models. • Embeds the

    words and labels in the same joint space, and measures the compatibility of word-label pairs to attend the document representations. • Compared with the previous methods, the LEAM algorithm requires much lower computational cost, and achieves comparable performance relative to the state-of-the-art. 10
  11. 11

  12. results 12

  13. Applications to Clinical Text 13

  14. The Label Embedding Attentive Model (LEAM) to improve text classification.

    (i) Label-attentive text representation is informative for the downstream classification task, as it directly learns from a shared joint space. (ii) The LEAM learning procedure only involves a series of basic algebraic operations, and hence it retains the interpretability of simple models, especially when the label description is available. (iii) Our attention mechanism has fewer parameters and less computation than related methods. (iv) Demonstrating the effectiveness of our label-embedding attentive model, providing state- of-the-art results on bench mark datasets. (v) We further apply LEAM to predict the medical codes from clinical text. 14