Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up for free
Retrieve_rerank_and_rewrite_Soft_template_based_neural_summarization.pdf
MARUYAMA
August 15, 2018
0
46
Retrieve_rerank_and_rewrite_Soft_template_based_neural_summarization.pdf
MARUYAMA
August 15, 2018
Tweet
Share
More Decks by MARUYAMA
See All by MARUYAMA
vampire.pdf
tmaru0204
0
68
Misspelling_Oblivious_Word_Embedding.pdf
tmaru0204
0
51
Simple_Unsupervised_Summarization_by_Contextual_Matching.pdf
tmaru0204
0
61
Controlling_Text_Complexity_in_Neural_Machine_Translation.pdf
tmaru0204
0
60
20191028_literature-review.pdf
tmaru0204
0
62
Hint-Based_Training_for_Non-Autoregressive_Machine_Translation.pdf
tmaru0204
0
45
Soft_Contextual_Data_Augmentation_for_Neural_Machine_Translation_.pdf
tmaru0204
0
60
An_Embarrassingly_Simple_Approach_for_Transfer_Learning_from_Pretrained_Language_Models_.pdf
tmaru0204
0
55
Addressing_Trobulesome_Words_in_Neural_Machine_Translation.pdf
tmaru0204
0
47
Featured
See All Featured
Fashionably flexible responsive web design (full day workshop)
malarkey
396
62k
Testing 201, or: Great Expectations
jmmastey
21
5.4k
The Straight Up "How To Draw Better" Workshop
denniskardys
225
120k
Building a Scalable Design System with Sketch
lauravandoore
447
30k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
151
12k
Designing for humans not robots
tammielis
241
23k
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
7
1k
How to name files
jennybc
39
58k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
349
27k
Bash Introduction
62gerente
596
210k
How New CSS Is Changing Everything About Graphic Design on the Web
jensimmons
212
11k
Large-scale JavaScript Application Architecture
addyosmani
499
110k
Transcript
Retrieve, Rerank and Rewrite: Soft Template Based Neural Summarization Ziqiang
Cao, Wenjie Li, Furu Wei, Sujian Li Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp.152-161, 2018 Nagaoka University of Technology Takumi Maruyama Literature review:
Ø template-based approachseq2seq Ø Informativeness, SOTA
Ø template ,
Ø “Seq2seq models tend to lose control sometimes” •
'# ( • '# )" Ø Template based summarization • -% • $+,template !* & Template: [REGION] shares [open/close] [NUMBER] percent [lower/higher]
Ø Soft-template based neural summarization • seq2seqtemplate •
… • Soft-templates
Ø Modules • Retrieve: • Rerank: Retrieve
• Rewrite: Rerank template
Ø Retrieve • Lucene (! ) • (over
3M) ! • "30 https://lucene.apache.org/
ØJointly Rerank and Rewrite • Rerank Bilinear networksaliency
• Rewrite x : , y*: , r : soft-templates (Retrieve ) Actual saliency : s*(r, y*) = ROUGE(r, y*) Saliency :
ØJointly Rerank and Rewrite • Learning
Ø Dataset Gigaword corpus
Ø Rerank -> Rewrite Rerank + Rewrite
Ø Soft-template"& Rondom ' First Retrieve ! $#
Max Retrieve %!$# , ROUGE$# Optimal %! ROUGE$# Rerank Rerank !$#
Ø Linguistic quality evaluation LEN_DIF !$& & &) LESS_3
3*(# & " COPY & & *( ' NEW_NE !$&& +% "
Ø template-based + seq2seq model Ø Soft-templates
,
None
None