Sentence-Simplification-with-Memory-Augmented-Neural-Networks.pdf

Transcript

1. Sentence Simplification with Memory-Augmented Neural Networks Tu Vu, Baotian Hu, Tsendsuren Munkhdalai and Hong Yu NAACL HLT 2018 Nagaoka University of Technology Takumi Maruyama Literature review:
2. Abstract Ø Sentence simplification aims to simplify the content and structure of complex sentences Ø This paper adapts an architecture with Neural Semantic Encoders for sentence simplification Ø Experiments demonstrate the effectiveness of proposed approach on different simplification datasets 2
3. Neural Sequence to Sequence Models Ø Attention-based Encoder-Decoder Model 3
4. Neural Sequence to Sequence Models Ø Neural Semantic Encoders: • Give us unrestricted access to the entire source sequence stored in the memory • May attend to relevant words when encoding each word 4
5. Datasets Ø Newsela: A simplification corpus of news articles composed by Newsela editor Ø Wikismall: Aligned complex-simple sentence pairs from English Wikipedia (EW) and Simple English Wikipedia Ø Wikilarge: A mixture of three Wikipedia datasets 5
6. Models and Training Details Ø Data Ø Attention-based sequence to sequence models • LSTMLSTM: The encoder is implemented by LSTM • NSELSTM: The encoder is implemented by NSE Ø Models were tuned on the development sets, either with BLEU (-B) or SARI (-S) Dataset Train Dev. Test Newsela 94,208 1,129 1,077 Wikismall 88,837 205 100 Wikilarge 296,402 2,000 359 6
7. Comparing Systems Ø PBSMT-R: A PBMT model with dissimilarity-based re-ranking Ø HYBRID: A hybrid semantic-based model that combines a simplification model and a monolingual MT model Ø SBMT-SARI: A SBMT model with simplification-specific components Ø DRESS: A deep reinforcement learning model Ø DRESS-LS: A combination of DRESS and a lexical simplification 7
8. Evaluations Ø Automatic evaluations • BLEU • SARI Ø Human Judgments (five point Likert scale) • Fluency: The extent to which the output is grammatical English • Adequacy: The extent to which the output has the same meaning as the input sentence • Simplicity: The extent to which the output is simpler than the input sentence 8
9. Results 9
10. Results 10
11. Results 11
12. Results 12
13. Results 13
14. Conclusions Ø This paper proposes an architecture with Neural Semantic Encoders for sentence simplification Ø The proposed model is capable of reducing the reading difficulty of the input, while performing well in terms of grammaticality and meaning preservation 14
15. Example model outputs 15