vampire.pdf

Transcript

1. Variational Pretraining for Semi-supervised Text Classification จݙ঺հ Suchin Gururangan, Tam Dang, Dallas Card, Noah A. Smith ACL2019, pages 5880–5894
2. Abstract ⾣
   ܰྔͳࣄલֶशख๏ΛఏҊ ɾখن໛σʔλͰޮ཰తʹֶशՄೳ ɾߴ଎ʹಈ࡞ ⾣
   ςΩετ෼ྨλεΫʹ͓͍ͯɺ
   ɹ
   &-.P
   #&35ʹඖఢ͢ΔੑೳΛୡ੒
3. Introduction ⾣
   &-.P
   #&35౳ͷࣄલֶशϞσϧ͕
   ɹ
   ޿͘ར༻͞Ε͍ͯΔ ⾣
   େن໛ͳίʔύεɾܭࢉࢿݯ͕ඞཁ ⾣
   ܰྔ͔ͭޮՌతͳࣄલֶशํ๏ΛఏҊ
4. Model ⾣
   1SFUSBJOJOH ɾ7BSJBUJPOBM
   "VUP
   &ODPEFS
5. Model ⾣
   1SFUSBJOJOH ɾ7BSJBUJPOBM
   "VUP
   &ODPEFS
6. Model ⾣
   5FYU
   DMBTTJpDBUJPO
7. Model ⾣
   5FYU
   DMBTTJpDBUJPO ɾ7".1*3&
   FNCFEEJOH
   ݁߹ ɾ%FFQ
   "WFSBHF
   /FUXPSLͰ
   
   
   
   Τϯίʔυ
8. Experimental setup ⾣
   %BUB ɾࣄલֶशσʔλ ɾ෼ྨثֶश
   
   ςετ  
   d
    
   FYBNQMFT
   JOEPNBJO 
   d
    
   FYBNQMFT
9. Experimental setup ⾣
   -PX
   SFTPVSDF
   TFUUJOH ɾ#BTFMJOF
   ڭࢣσʔλͷΈͰֶश ɾ4FMGUSBJOJOH
   ൒ڭࢣ͋Γֶश
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   ڭࢣσʔλ
   
   Ϟσϧͷ༧ଌ݁Ռ ɾ(-07&
   *% 
   JOEPNBJOσʔλͰֶश ɾ(-07&
   0% 
   .
   XPSETͷίʔύεͰֶश
10. Results ⾣
    -PX
    SFTPVSDF
    TFUUJOH
11. Results ⾣
    -PX
    SFTPVSDF
    TFUUJOH
12. Experimental setup ⾣
    )JHI
    SFTPVSDF
    TFUUJOH ɾ5SBOTGPSNFSCBTFE
    &-.P ɾ#&35 
    GSP[FO 
    pOFUVOJOH 
    QSFUSBJOJOH
    JOEPNBJO 
    GSP[FO 
    pOFUVOJOH
13. Results ⾣
    )JHI
    SFTPVSDF
    TFUUJOH
14. Results ⾣
    )JHI
    SFTPVSDF
    TFUUJOH
15. Results ⾣
    $PNQVUBUJPOBM
    3FRVJSFNFOUT
    
16. Conclusions ⾣
    ܰྔͳࣄલֶशख๏ΛఏҊ ɾখن໛σʔλͰޮ཰తʹֶशՄೳ ɾߴ଎ʹಈ࡞ ⾣
    ςΩετ෼ྨλεΫʹ͓͍ͯɺ
    ɹ
    &-.P
    #&35ʹඖఢ͢ΔੑೳΛୡ੒