Upgrade to Pro — share decks privately, control downloads, hide ads and more …


A3ea3bc5dde6ae2dd6eae71da9c418b0?s=47 MARUYAMA
April 23, 2018




April 23, 2018


  1. Sentence Simplification with Memory-Augmented Neural Networks Tu Vu, Baotian Hu,

    Tsendsuren Munkhdalai and Hong Yu NAACL HLT 2018 Nagaoka University of Technology Takumi Maruyama Literature review:
  2. Abstract Ø Sentence simplification aims to simplify the content and

    structure of complex sentences Ø This paper adapts an architecture with Neural Semantic Encoders for sentence simplification Ø Experiments demonstrate the effectiveness of proposed approach on different simplification datasets 2
  3. Neural Sequence to Sequence Models Ø Attention-based Encoder-Decoder Model 3

  4. Neural Sequence to Sequence Models Ø Neural Semantic Encoders: •

    Give us unrestricted access to the entire source sequence stored in the memory • May attend to relevant words when encoding each word 4
  5. Datasets Ø Newsela: A simplification corpus of news articles composed

    by Newsela editor Ø Wikismall: Aligned complex-simple sentence pairs from English Wikipedia (EW) and Simple English Wikipedia Ø Wikilarge: A mixture of three Wikipedia datasets 5
  6. Models and Training Details Ø Data Ø Attention-based sequence to

    sequence models • LSTMLSTM: The encoder is implemented by LSTM • NSELSTM: The encoder is implemented by NSE Ø Models were tuned on the development sets, either with BLEU (-B) or SARI (-S) Dataset Train Dev. Test Newsela 94,208 1,129 1,077 Wikismall 88,837 205 100 Wikilarge 296,402 2,000 359 6
  7. Comparing Systems Ø PBSMT-R: A PBMT model with dissimilarity-based re-ranking

    Ø HYBRID: A hybrid semantic-based model that combines a simplification model and a monolingual MT model Ø SBMT-SARI: A SBMT model with simplification-specific components Ø DRESS: A deep reinforcement learning model Ø DRESS-LS: A combination of DRESS and a lexical simplification 7
  8. Evaluations Ø Automatic evaluations • BLEU • SARI Ø Human

    Judgments (five point Likert scale) • Fluency: The extent to which the output is grammatical English • Adequacy: The extent to which the output has the same meaning as the input sentence • Simplicity: The extent to which the output is simpler than the input sentence 8
  9. Results 9

  10. Results 10

  11. Results 11

  12. Results 12

  13. Results 13

  14. Conclusions Ø This paper proposes an architecture with Neural Semantic

    Encoders for sentence simplification Ø The proposed model is capable of reducing the reading difficulty of the input, while performing well in terms of grammaticality and meaning preservation 14
  15. Example model outputs 15