Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Baseline Needs More Love: On Simple Word-Embedd...

Avatar for katsutan katsutan
April 08, 2019

Baseline Needs More Love: On Simple Word-Embedding-Based Models and Associated Pooling Mechanisms

文献紹介

長岡技術科学大学
勝田 哲弘

Avatar for katsutan

katsutan

April 08, 2019
Tweet

More Decks by katsutan

Other Decks in Technology

Transcript

  1. Baseline Needs More Love: On Simple Word-Embedding-Based Models and Associated

    Pooling Mechanisms Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Long Papers), pages 440–450 Melbourne, Australia, July 15 - 20, 2018. 文献紹介: 長岡技術科学大学 勝田 哲弘
  2. Abstract • Simple Word-Embedding-based Models (SWEMs)と word-embedding-based RNN/CNN modelsの比較 ◦

    SWEMsが多くの場合で同等、優れた精度を示す • Parameter freeのpoolingを活用するモデル ◦ hierarchical pooling ◦ parameter数が少なく済む 2
  3. Simple Word-Embedding Model (SWEM) パラメータを持たないモデル • Average-Pooling(一番単純なモデル) • Max Pooling(CNNでのmax-over-time

    pooling に近い) • Hierarchical Pooling ◦ ウィンドウ幅nでavg-poolingを行い、その上にmax-pooling 5
  4. Extension to other languages • Sogou news corpus(a Chinese dataset

    represented by Pinyin) ◦ SWEM-concat accuracy : 91.3% ◦ SWEM-hier (window size of 5) accuracy : 96.2% ◦ CNN (95.6%) and LSTM (95.2%) • より語順に敏感な中国語においても最高精度に匹敵する 14