Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfer

弊研究室で行なった NAACL読み会の発表資料です。

Transcript

1. Delete, Retrieve, Generate: 
 A Simple Approach to 
 Sentiment and Style Transfer 2018.7.25 B4 Miura Biwa
   
2. Contents • Overview • Background • Method • Result • Conclusion
   
3. Overview • Propose simple models for style transfer • Process in 3 steps : Delete, Retrieve, Generate • Evaluate by Yelp, Amazon, Caption dataset • Best method is stronger than previous system
   
4. Background • Control attributes (such as sentiment and style) is very important for neural language generation(NLG) • Goal : transfer specific attribute while preserving all attribute-independent content • Difficulty : aligned sentences with the same content but different attributes are not available
   
5. Method • Delete : delete attribute marker • Retrieve : retrieve similar sentence 
 with target attribute • Generate : generate output
    %FMFUF 3FUSJFWF (FOFSBUF Y WTSD Z
6. Method • RETRIEVEONLY • TEMPLETEBASED • DELETEONLY • DELETEANDRETRIEVE
   
7. Delete • Identify attribute markers • Delete attribute markers • (“i have had this mount for about a year
 and it works great .” , “positive”)
 → “i have had this mount for about
 ɹ a year and it .”
    %FMFUF 3FUSJFWF (FOFSBUF Y WTSD Z
8. Retrieve • Look at similar sentence with target attribute • “i have had this mount for about
 a year and it .”
 →“i have had it for a while but barely
 ɹ used it .”
    %FMFUF 3FUSJFWF (FOFSBUF Y WTSD Z
9. Generate • Generate final output • RETRIEVEONLY
 - output retrieved sentence
 →“i have had it for a while but barely used it .” • TEMPLETEBASED
 - insert target attribute marker
 “i have had it for a while but barely used it .”
 →”i have had this mount for about
 ɹ a year and it barely used.”
    %FMFUF 3FUSJFWF (FOFSBUF Y WTSD Z
10. Generate • DELETEONLY
 - concat and decode extracted sentence
 and target label
 →G(“i have had this mount for about a year
 and it .”, “negative”) • DELETEANDRETRIEVE
 - concat and decode extracted sentence
 and attribute marker in retrieved sentence
 →G(“i have had this mount for about a year
 and it .”, “beraly used”)
     %FMFUF 3FUSJFWF (FOFSBUF Y WTSD Z
11. Training • DELETEONLY • AE : (extracted sentence, label)→source sentence • DELETEANDRETRIEVE • Denoising-AE : 
 (extracted sentence, attribute marker+noise)
 →source sentence
    
12. Method
    
13. Dataset
    
14. Training setting • word vector : 128 • GRU (both Decoder and Encoder) • hidden size : 512 • layer : 1 • activation function : maxout • init parameters : uniform distribution between [-0.1,0.1] • minibatch size : 256
    
15. Training setting • for attribute marker extraction • span : 4 word • λ = 1 • γ : 15(Yelp) , 5.5(Amazon) , 5(Caption)
    
16. Training setting • Distance metric for retrieve • TF-IDF weighted word overlap score
 for DELETEANDRETREIVE and TEMPLETEBASED • Euclidian distance
 for RETRIEVEONLY • beamsize : 10
    
17. Result - Human Evaluation
    
18. Result - Automatic Evaluation
    
19. Result - Automatic Evaluation
    
20. Result - Output Example
    
21. Conclusion • Proposed simple models for style transfer • Process in 3 steps : Delete, Retrieve, Generate • Evaluated by Yelp, Amazon, Caption dataset • Best method is stronger than previous system