Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
vampire.pdf
Search
MARUYAMA
February 25, 2020
0
180
vampire.pdf
MARUYAMA
February 25, 2020
Tweet
Share
More Decks by MARUYAMA
See All by MARUYAMA
Misspelling_Oblivious_Word_Embedding.pdf
tmaru0204
0
190
Simple_Unsupervised_Summarization_by_Contextual_Matching.pdf
tmaru0204
0
180
Controlling_Text_Complexity_in_Neural_Machine_Translation.pdf
tmaru0204
0
170
20191028_literature-review.pdf
tmaru0204
0
160
Hint-Based_Training_for_Non-Autoregressive_Machine_Translation.pdf
tmaru0204
0
140
Soft_Contextual_Data_Augmentation_for_Neural_Machine_Translation_.pdf
tmaru0204
0
170
An_Embarrassingly_Simple_Approach_for_Transfer_Learning_from_Pretrained_Language_Models_.pdf
tmaru0204
0
160
Addressing_Trobulesome_Words_in_Neural_Machine_Translation.pdf
tmaru0204
0
160
Simple_Unsupervised_Keyphrase_Extraction_using_Sentence_Embeddings.pdf
tmaru0204
0
190
Featured
See All Featured
Balancing Empowerment & Direction
lara
5
700
Into the Great Unknown - MozCon
thekraken
40
2.1k
Rails Girls Zürich Keynote
gr2m
95
14k
Why Our Code Smells
bkeepers
PRO
340
57k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
31
2.7k
How STYLIGHT went responsive
nonsquared
100
5.8k
KATA
mclloyd
PRO
32
15k
Optimising Largest Contentful Paint
csswizardry
37
3.5k
Thoughts on Productivity
jonyablonski
70
4.9k
4 Signs Your Business is Dying
shpigford
185
22k
Practical Orchestrator
shlominoach
190
11k
VelocityConf: Rendering Performance Case Studies
addyosmani
333
24k
Transcript
Variational Pretraining for Semi-supervised Text Classification จݙհ Suchin Gururangan, Tam
Dang, Dallas Card, Noah A. Smith ACL2019, pages 5880–5894
Abstract ⾣ܰྔͳࣄલֶशख๏ΛఏҊ ɾখنσʔλͰޮతʹֶशՄೳ ɾߴʹಈ࡞ ⾣ςΩετྨλεΫʹ͓͍ͯɺ ɹ&-.P #&35ʹඖఢ͢ΔੑೳΛୡ
Introduction ⾣&-.P #&35ͷࣄલֶशϞσϧ͕ ɹ͘ར༻͞Ε͍ͯΔ ⾣େنͳίʔύεɾܭࢉࢿݯ͕ඞཁ ⾣ܰྔ͔ͭޮՌతͳࣄલֶशํ๏ΛఏҊ
Model ⾣1SFUSBJOJOH ɾ7BSJBUJPOBM"VUP&ODPEFS
Model ⾣1SFUSBJOJOH ɾ7BSJBUJPOBM"VUP&ODPEFS
Model ⾣5FYUDMBTTJpDBUJPO
Model ⾣5FYUDMBTTJpDBUJPO ɾ7".1*3&FNCFEEJOH݁߹ ɾ%FFQ"WFSBHF/FUXPSLͰ Τϯίʔυ
Experimental setup ⾣%BUB ɾࣄલֶशσʔλ ɾྨثֶशςετ d FYBNQMFT JOEPNBJO d
FYBNQMFT
Experimental setup ⾣-PXSFTPVSDFTFUUJOH ɾ#BTFMJOFڭࢣσʔλͷΈͰֶश ɾ4FMGUSBJOJOHڭࢣ͋Γֶश ڭࢣσʔλ Ϟσϧͷ༧ଌ݁Ռ ɾ(-07& *%
JOEPNBJOσʔλͰֶश ɾ(-07& 0% .XPSETͷίʔύεͰֶश
Results ⾣-PXSFTPVSDFTFUUJOH
Results ⾣-PXSFTPVSDFTFUUJOH
Experimental setup ⾣)JHISFTPVSDFTFUUJOH ɾ5SBOTGPSNFSCBTFE&-.P ɾ#&35 GSP[FO pOFUVOJOH QSFUSBJOJOH JOEPNBJO GSP[FO
pOFUVOJOH
Results ⾣)JHISFTPVSDFTFUUJOH
Results ⾣)JHISFTPVSDFTFUUJOH
Results ⾣$PNQVUBUJPOBM3FRVJSFNFOUT
Conclusions ⾣ܰྔͳࣄલֶशख๏ΛఏҊ ɾখنσʔλͰޮతʹֶशՄೳ ɾߴʹಈ࡞ ⾣ςΩετྨλεΫʹ͓͍ͯɺ ɹ&-.P #&35ʹඖఢ͢ΔੑೳΛୡ