Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up for free
vampire.pdf
MARUYAMA
February 25, 2020
0
69
vampire.pdf
MARUYAMA
February 25, 2020
Tweet
Share
More Decks by MARUYAMA
See All by MARUYAMA
Misspelling_Oblivious_Word_Embedding.pdf
tmaru0204
0
52
Simple_Unsupervised_Summarization_by_Contextual_Matching.pdf
tmaru0204
0
62
Controlling_Text_Complexity_in_Neural_Machine_Translation.pdf
tmaru0204
0
61
20191028_literature-review.pdf
tmaru0204
0
63
Hint-Based_Training_for_Non-Autoregressive_Machine_Translation.pdf
tmaru0204
0
46
Soft_Contextual_Data_Augmentation_for_Neural_Machine_Translation_.pdf
tmaru0204
0
61
An_Embarrassingly_Simple_Approach_for_Transfer_Learning_from_Pretrained_Language_Models_.pdf
tmaru0204
0
56
Addressing_Trobulesome_Words_in_Neural_Machine_Translation.pdf
tmaru0204
0
48
Simple_Unsupervised_Keyphrase_Extraction_using_Sentence_Embeddings.pdf
tmaru0204
0
85
Featured
See All Featured
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
181
15k
Support Driven Design
roundedbygravity
86
8.5k
Clear Off the Table
cherdarchuk
79
280k
What's in a price? How to price your products and services
michaelherold
229
9.3k
Side Projects
sachag
449
37k
Typedesign – Prime Four
hannesfritz
33
1.3k
Building a Scalable Design System with Sketch
lauravandoore
447
30k
GitHub's CSS Performance
jonrohan
1020
410k
Adopting Sorbet at Scale
ufuk
63
7.5k
Streamline your AJAX requests with AmplifyJS and jQuery
dougneiner
125
8.5k
A Modern Web Designer's Workflow
chriscoyier
689
180k
A Tale of Four Properties
chriscoyier
149
20k
Transcript
Variational Pretraining for Semi-supervised Text Classification จݙհ Suchin Gururangan, Tam
Dang, Dallas Card, Noah A. Smith ACL2019, pages 5880–5894
Abstract ⾣ܰྔͳࣄલֶशख๏ΛఏҊ ɾখنσʔλͰޮతʹֶशՄೳ ɾߴʹಈ࡞ ⾣ςΩετྨλεΫʹ͓͍ͯɺ ɹ&-.P #&35ʹඖఢ͢ΔੑೳΛୡ
Introduction ⾣&-.P #&35ͷࣄલֶशϞσϧ͕ ɹ͘ར༻͞Ε͍ͯΔ ⾣େنͳίʔύεɾܭࢉࢿݯ͕ඞཁ ⾣ܰྔ͔ͭޮՌతͳࣄલֶशํ๏ΛఏҊ
Model ⾣1SFUSBJOJOH ɾ7BSJBUJPOBM"VUP&ODPEFS
Model ⾣1SFUSBJOJOH ɾ7BSJBUJPOBM"VUP&ODPEFS
Model ⾣5FYUDMBTTJpDBUJPO
Model ⾣5FYUDMBTTJpDBUJPO ɾ7".1*3&FNCFEEJOH݁߹ ɾ%FFQ"WFSBHF/FUXPSLͰ Τϯίʔυ
Experimental setup ⾣%BUB ɾࣄલֶशσʔλ ɾྨثֶशςετ d FYBNQMFT JOEPNBJO d
FYBNQMFT
Experimental setup ⾣-PXSFTPVSDFTFUUJOH ɾ#BTFMJOFڭࢣσʔλͷΈͰֶश ɾ4FMGUSBJOJOHڭࢣ͋Γֶश ڭࢣσʔλ Ϟσϧͷ༧ଌ݁Ռ ɾ(-07& *%
JOEPNBJOσʔλͰֶश ɾ(-07& 0% .XPSETͷίʔύεͰֶश
Results ⾣-PXSFTPVSDFTFUUJOH
Results ⾣-PXSFTPVSDFTFUUJOH
Experimental setup ⾣)JHISFTPVSDFTFUUJOH ɾ5SBOTGPSNFSCBTFE&-.P ɾ#&35 GSP[FO pOFUVOJOH QSFUSBJOJOH JOEPNBJO GSP[FO
pOFUVOJOH
Results ⾣)JHISFTPVSDFTFUUJOH
Results ⾣)JHISFTPVSDFTFUUJOH
Results ⾣$PNQVUBUJPOBM3FRVJSFNFOUT
Conclusions ⾣ܰྔͳࣄલֶशख๏ΛఏҊ ɾখنσʔλͰޮతʹֶशՄೳ ɾߴʹಈ࡞ ⾣ςΩετྨλεΫʹ͓͍ͯɺ ɹ&-.P #&35ʹඖఢ͢ΔੑೳΛୡ