Sentence-Simplification-with-Memory-Augmented-Neural-Networks.pdf

Transcript

1. Sentence Simplification with Memory-Augmented Neural Networks Tu Vu, Baotian Hu,

   Tsendsuren Munkhdalai and Hong Yu NAACL HLT 2018 Nagaoka University of Technology Takumi Maruyama Literature review:

2. Abstract Ø Sentence simplification aims to simplify the content and

   structure of complex sentences Ø This paper adapts an architecture with Neural Semantic Encoders for sentence simplification Ø Experiments demonstrate the effectiveness of proposed approach on different simplification datasets 2

3. Neural Sequence to Sequence Models Ø Attention-based Encoder-Decoder Model 3

4. Neural Sequence to Sequence Models Ø Neural Semantic Encoders: •

   Give us unrestricted access to the entire source sequence stored in the memory • May attend to relevant words when encoding each word 4

5. Datasets Ø Newsela: A simplification corpus of news articles composed

   by Newsela editor Ø Wikismall: Aligned complex-simple sentence pairs from English Wikipedia (EW) and Simple English Wikipedia Ø Wikilarge: A mixture of three Wikipedia datasets 5

6. Models and Training Details Ø Data Ø Attention-based sequence to

   sequence models • LSTMLSTM: The encoder is implemented by LSTM • NSELSTM: The encoder is implemented by NSE Ø Models were tuned on the development sets, either with BLEU (-B) or SARI (-S) Dataset Train Dev. Test Newsela 94,208 1,129 1,077 Wikismall 88,837 205 100 Wikilarge 296,402 2,000 359 6

7. Comparing Systems Ø PBSMT-R: A PBMT model with dissimilarity-based re-ranking

   Ø HYBRID: A hybrid semantic-based model that combines a simplification model and a monolingual MT model Ø SBMT-SARI: A SBMT model with simplification-specific components Ø DRESS: A deep reinforcement learning model Ø DRESS-LS: A combination of DRESS and a lexical simplification 7

8. Evaluations Ø Automatic evaluations • BLEU • SARI Ø Human

   Judgments (five point Likert scale) • Fluency: The extent to which the output is grammatical English • Adequacy: The extent to which the output has the same meaning as the input sentence • Simplicity: The extent to which the output is simpler than the input sentence 8

9. Results 9

10. Results 10

11. Results 11

12. Results 12

13. Results 13

14. Conclusions Ø This paper proposes an architecture with Neural Semantic

    Encoders for sentence simplification Ø The proposed model is capable of reducing the reading difficulty of the input, while performing well in terms of grammaticality and meaning preservation 14

15. Example model outputs 15