Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Simple_Unsupervised_Summarization_by_Contextual...
Search
MARUYAMA
December 08, 2019
0
150
Simple_Unsupervised_Summarization_by_Contextual_Matching.pdf
MARUYAMA
December 08, 2019
Tweet
Share
More Decks by MARUYAMA
See All by MARUYAMA
vampire.pdf
tmaru0204
0
140
Misspelling_Oblivious_Word_Embedding.pdf
tmaru0204
0
170
Controlling_Text_Complexity_in_Neural_Machine_Translation.pdf
tmaru0204
0
140
20191028_literature-review.pdf
tmaru0204
0
130
Hint-Based_Training_for_Non-Autoregressive_Machine_Translation.pdf
tmaru0204
0
120
Soft_Contextual_Data_Augmentation_for_Neural_Machine_Translation_.pdf
tmaru0204
0
140
An_Embarrassingly_Simple_Approach_for_Transfer_Learning_from_Pretrained_Language_Models_.pdf
tmaru0204
0
130
Addressing_Trobulesome_Words_in_Neural_Machine_Translation.pdf
tmaru0204
0
130
Simple_Unsupervised_Keyphrase_Extraction_using_Sentence_Embeddings.pdf
tmaru0204
0
170
Featured
See All Featured
How GitHub (no longer) Works
holman
312
140k
Being A Developer After 40
akosma
89
590k
Building Applications with DynamoDB
mza
93
6.2k
4 Signs Your Business is Dying
shpigford
182
22k
Six Lessons from altMBA
skipperchong
27
3.6k
The Invisible Side of Design
smashingmag
299
50k
jQuery: Nuts, Bolts and Bling
dougneiner
62
7.6k
Learning to Love Humans: Emotional Interface Design
aarron
274
40k
Art, The Web, and Tiny UX
lynnandtonic
298
20k
The Language of Interfaces
destraynor
155
24k
How to Think Like a Performance Engineer
csswizardry
22
1.3k
VelocityConf: Rendering Performance Case Studies
addyosmani
327
24k
Transcript
4JNQMF6OTVQFSWJTFE4VNNBSJ[BUJPO CZ$POUFYUVBM.BUDIJOH จݙհ
1BQFS ɾ"$- ɾIUUQTXXXBDMXFCPSHBOUIPMPHZ1 ɾQBHFTr
"CTUSBDU ⾣ͭͷݴޠϞσϧͷΈͷγϯϓϧͳڭࢣͳ͠ੜܕཁ ⾣ੜܕཁɾநग़ܕཁͷํͰ༗༻ੑΛࣔͨ͠ ɾ$POUFYUVBMNBUDIJOHNPEFM ɾ%PNBJOqVFODZNPEFM
*OUSPEVDUJPO ⾣ڭࢣͳ͠ཁ ⾣ෳࡶͳϞσϧɾֶशΛඞཁͱ͠ͳ͍ϞσϧΛఏҊ MFOHUIDPOUSPMMFEWBSJBUJPOBMBVUPFODPEFS HFOFSBUJWFBEWFSTBSJBMOFUXPSL ʜ FH
.PEFM ⾣ཁจɺ࣍ͷͭͷಛੑΛຬ͍ͨͯ͠Δඞཁ͕͋Δ ਖ਼֬ੑ 'BJUIGVMOFTT ݪจͷҙຯͱಉ༷ͷҙຯΛ࣋ͭ ྲྀெੑ 'MVFODZ จ๏తʹਖ਼͘͠ཧղͰ͖Δ P(y|x) ∝
pcm (y|x)pfm (y|x)λ ೖྗจॻ x ཁจ y ਖ਼֬ੑ pcm (y|x) ྲྀெੑ pfm (y|x)
$POUFYUVBM.BUDIJOH.PEFM ⾣ਖ਼֬ੑೖྗςΩετͱग़ྗ୯ޠͷίαΠϯྨࣅ pcm (y|x) = N ∏ n=1 qcm (yn
|y<n , x) ͱͨ͠ͱ͖ sω = maxj≥1 Sim(x1:j , ω) ͜͜Ͱ ग़ྗީิޠͱೖྗςΩετͱͷྨࣅΛ ω x1:j qcm (y1 = ω|x) = softmax(s)
$POUFYUVBM.BUDIJOH.PEFM ⾣ॲཧखॱ sω = maxj≥zn−1 Sim(x1:j , ω) ࣍ࣜΑΓ ྨࣅΛܭࢉ
zn−1 ʹରԠ͘ҐஔҎ͔߱͠ߟྀ͠ͳ͍ zn−1 yn−1 ୯ௐੑͷԾఆ ग़ྗ ୯ޠҐஔΛܭࢉ qcm zn ͕ೖྗจॻඌͱͳΔ·Ͱ܁Γฦ͠ zn
%PNBJO'MVFODZ.PEFM ⾣ྲྀெੑݴޠϞσϧ֬ ೖྗจॻʹదͨ͠ޠΛબ͢ΔΑ͏ग़ྗޠኮΛ੍ ݴޠϞσϧͷग़ྗޠኮ7ΛϘϩϊΠׂʹΑΓ੍͖ޠኮ$ʹϚοϐϯά ͋Δڑ্ۭؒͷҙͷҐஔʹஔ͞Εͨෳݸͷʢʣʹରͯ͠ɺ ಉҰڑ্ۭؒͷଞͷ͕Ͳͷʹ͍͔ۙʹΑͬͯྖҬ͚͞Εͨਤͷ͜ͱɻ IUUQTKBXJLJQFEJBPSHXJLJϘϩϊΠਤ 8JLJQFEJBϘϩϊΠਤ
%PNBJO'MVFODZ.PEFM ⾣ྲྀெੑݴޠϞσϧ֬ pfm (y|x) = N ∏ n=1 ∑ ω′∈N(yn
) lm(ω′|y<n ) ϘϊϩΠׂͷʹ ೖྗจॻ୯ޠΛ༻͍Δ ͷϘϩϊΠྖҬΛͱͨ͠ͱ͖ ݴޠϞσϧ֬ yn N(yn )
.PEFM ⾣ཁจɺ࣍ͷͭͷಛੑΛຬ͍ͨͯ͠Δඞཁ͕͋Δ ਖ਼֬ੑ 'BJUIGVMOFTT ݪจͷҙຯͱಉ༷ͷҙຯΛ࣋ͭ ྲྀெੑ 'MVFODZ จ๏తʹਖ਼͘͠ཧղͰ͖Δ P(y|x) ∝
pcm (y|x)pfm (y|x)λ ೖྗจॻ x ཁจ y ਖ਼֬ੑ pcm (y|x) ྲྀெੑ pfm (y|x)
&YQFSJNFOUBMTFUVQ ⾣.PEFM ⾣%BUBTFU ɾੜܕཁ&OHMJTI(JHBXPSEEBUB 3VTI ɾநग़ܕཁ(PPHMFEBUBTFU 'JMJQQPWB
ɾGPSXBSEMBOHVBHFNPEFMPG&-.P ɾMBZFST-45.NPEFM pcm (y|x) pfm (y|x)
2VBOUJUBUJWF3FTVMUT 5BCMFੜܕཁͷ݁Ռ 5BCMFநग़ܕཁͷ݁Ռ
"OBMZTJT ⾣ೖྗจॻͷ୯ޠநग़͚ͩͰͳ͘ ੜͰ͖͍ͯΔ
4VNNBSZ ⾣ͭͷݴޠϞσϧͷΈͷγϯϓϧͳڭࢣͳ͠ੜܕཁ ⾣ੜܕཁɾநग़ܕཁͷํͰ༗༻ੑΛࣔͨ͠ ɾ$POUFYUVBMNBUDIJOHNPEFM ɾ%PNBJOqVFODZNPEFM
None