Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Simple_Unsupervised_Summarization_by_Contextual...
Search
MARUYAMA
December 08, 2019
200
0
Share
Simple_Unsupervised_Summarization_by_Contextual_Matching.pdf
MARUYAMA
December 08, 2019
More Decks by MARUYAMA
See All by MARUYAMA
vampire.pdf
tmaru0204
0
200
Misspelling_Oblivious_Word_Embedding.pdf
tmaru0204
0
210
Controlling_Text_Complexity_in_Neural_Machine_Translation.pdf
tmaru0204
0
180
20191028_literature-review.pdf
tmaru0204
0
160
Hint-Based_Training_for_Non-Autoregressive_Machine_Translation.pdf
tmaru0204
0
150
Soft_Contextual_Data_Augmentation_for_Neural_Machine_Translation_.pdf
tmaru0204
0
180
An_Embarrassingly_Simple_Approach_for_Transfer_Learning_from_Pretrained_Language_Models_.pdf
tmaru0204
0
160
Addressing_Trobulesome_Words_in_Neural_Machine_Translation.pdf
tmaru0204
0
180
Simple_Unsupervised_Keyphrase_Extraction_using_Sentence_Embeddings.pdf
tmaru0204
0
200
Featured
See All Featured
Between Models and Reality
mayunak
3
280
Building AI with AI
inesmontani
PRO
1
960
Highjacked: Video Game Concept Design
rkendrick25
PRO
1
350
Navigating the Design Leadership Dip - Product Design Week Design Leaders+ Conference 2024
apolaine
0
300
HDC tutorial
michielstock
2
650
Design in an AI World
tapps
1
210
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
32
2.9k
Docker and Python
trallard
47
3.8k
GitHub's CSS Performance
jonrohan
1032
470k
Evolving SEO for Evolving Search Engines
ryanjones
0
190
The Mindset for Success: Future Career Progression
greggifford
PRO
0
320
A Tale of Four Properties
chriscoyier
163
24k
Transcript
4JNQMF6OTVQFSWJTFE4VNNBSJ[BUJPO CZ$POUFYUVBM.BUDIJOH จݙհ
1BQFS ɾ"$- ɾIUUQTXXXBDMXFCPSHBOUIPMPHZ1 ɾQBHFTr
"CTUSBDU ⾣ͭͷݴޠϞσϧͷΈͷγϯϓϧͳڭࢣͳ͠ੜܕཁ ⾣ੜܕཁɾநग़ܕཁͷํͰ༗༻ੑΛࣔͨ͠ ɾ$POUFYUVBMNBUDIJOHNPEFM ɾ%PNBJOqVFODZNPEFM
*OUSPEVDUJPO ⾣ڭࢣͳ͠ཁ ⾣ෳࡶͳϞσϧɾֶशΛඞཁͱ͠ͳ͍ϞσϧΛఏҊ MFOHUIDPOUSPMMFEWBSJBUJPOBMBVUPFODPEFS HFOFSBUJWFBEWFSTBSJBMOFUXPSL ʜ FH
.PEFM ⾣ཁจɺ࣍ͷͭͷಛੑΛຬ͍ͨͯ͠Δඞཁ͕͋Δ ਖ਼֬ੑ 'BJUIGVMOFTT ݪจͷҙຯͱಉ༷ͷҙຯΛ࣋ͭ ྲྀெੑ 'MVFODZ จ๏తʹਖ਼͘͠ཧղͰ͖Δ P(y|x) ∝
pcm (y|x)pfm (y|x)λ ೖྗจॻ x ཁจ y ਖ਼֬ੑ pcm (y|x) ྲྀெੑ pfm (y|x)
$POUFYUVBM.BUDIJOH.PEFM ⾣ਖ਼֬ੑೖྗςΩετͱग़ྗ୯ޠͷίαΠϯྨࣅ pcm (y|x) = N ∏ n=1 qcm (yn
|y<n , x) ͱͨ͠ͱ͖ sω = maxj≥1 Sim(x1:j , ω) ͜͜Ͱ ग़ྗީิޠͱೖྗςΩετͱͷྨࣅΛ ω x1:j qcm (y1 = ω|x) = softmax(s)
$POUFYUVBM.BUDIJOH.PEFM ⾣ॲཧखॱ sω = maxj≥zn−1 Sim(x1:j , ω) ࣍ࣜΑΓ ྨࣅΛܭࢉ
zn−1 ʹରԠ͘ҐஔҎ͔߱͠ߟྀ͠ͳ͍ zn−1 yn−1 ୯ௐੑͷԾఆ ग़ྗ ୯ޠҐஔΛܭࢉ qcm zn ͕ೖྗจॻඌͱͳΔ·Ͱ܁Γฦ͠ zn
%PNBJO'MVFODZ.PEFM ⾣ྲྀெੑݴޠϞσϧ֬ ೖྗจॻʹదͨ͠ޠΛબ͢ΔΑ͏ग़ྗޠኮΛ੍ ݴޠϞσϧͷग़ྗޠኮ7ΛϘϩϊΠׂʹΑΓ੍͖ޠኮ$ʹϚοϐϯά ͋Δڑ্ۭؒͷҙͷҐஔʹஔ͞Εͨෳݸͷʢʣʹରͯ͠ɺ ಉҰڑ্ۭؒͷଞͷ͕Ͳͷʹ͍͔ۙʹΑͬͯྖҬ͚͞Εͨਤͷ͜ͱɻ IUUQTKBXJLJQFEJBPSHXJLJϘϩϊΠਤ 8JLJQFEJBϘϩϊΠਤ
%PNBJO'MVFODZ.PEFM ⾣ྲྀெੑݴޠϞσϧ֬ pfm (y|x) = N ∏ n=1 ∑ ω′∈N(yn
) lm(ω′|y<n ) ϘϊϩΠׂͷʹ ೖྗจॻ୯ޠΛ༻͍Δ ͷϘϩϊΠྖҬΛͱͨ͠ͱ͖ ݴޠϞσϧ֬ yn N(yn )
.PEFM ⾣ཁจɺ࣍ͷͭͷಛੑΛຬ͍ͨͯ͠Δඞཁ͕͋Δ ਖ਼֬ੑ 'BJUIGVMOFTT ݪจͷҙຯͱಉ༷ͷҙຯΛ࣋ͭ ྲྀெੑ 'MVFODZ จ๏తʹਖ਼͘͠ཧղͰ͖Δ P(y|x) ∝
pcm (y|x)pfm (y|x)λ ೖྗจॻ x ཁจ y ਖ਼֬ੑ pcm (y|x) ྲྀெੑ pfm (y|x)
&YQFSJNFOUBMTFUVQ ⾣.PEFM ⾣%BUBTFU ɾੜܕཁ&OHMJTI(JHBXPSEEBUB 3VTI ɾநग़ܕཁ(PPHMFEBUBTFU 'JMJQQPWB
ɾGPSXBSEMBOHVBHFNPEFMPG&-.P ɾMBZFST-45.NPEFM pcm (y|x) pfm (y|x)
2VBOUJUBUJWF3FTVMUT 5BCMFੜܕཁͷ݁Ռ 5BCMFநग़ܕཁͷ݁Ռ
"OBMZTJT ⾣ೖྗจॻͷ୯ޠநग़͚ͩͰͳ͘ ੜͰ͖͍ͯΔ
4VNNBSZ ⾣ͭͷݴޠϞσϧͷΈͷγϯϓϧͳڭࢣͳ͠ੜܕཁ ⾣ੜܕཁɾநग़ܕཁͷํͰ༗༻ੑΛࣔͨ͠ ɾ$POUFYUVBMNBUDIJOHNPEFM ɾ%PNBJOqVFODZNPEFM
None