Visually Grounded Neural Syntax Acquisition

summarization of the paper presented at ACL 2019.
at the state-of-the-art NLP study group.

Transcript

1. 7JTVBMMZ
   (SPVOEFE
   /FVSBM
   4ZOUBY
   "DRVJTJUJPO ಡΉਓ
   দྛ ༏Ұ࿠ʢ౦๺େֶʣ  ࠷ઌ୺/-1ษڧձ )BPZVF
   4IJ +JBZVBO
   .BP ,FWJO
   (JNQFM ,BSFO
   -JWFTDV QSFTFOUFE
   BU
   "$-
   
2. ֓ཁ • ߏ੒ૉϕʔεͷߏจղੳΛڭࢣͳֶ͠श • 7JTVBMTFNBOUJD
   FNCFEEJOH
   TQBDF
   </HJBN FU
   BM
   > ͷख๏Λར༻͢Δ • ը૾ͱͦͷΩϟϓγϣϯΛ༻͍ɺςΩετ۠ؒͷ DPODSFUFOFTT BCTUSBDUOFTT
   είΞΛఆٛ͠ɺ ۠ؒͷ݁߹ΛΨΠυ͢Δ • ςΩετ୯ମͰͷֶशΑΓޮ཰Α͘ɺ҆ఆֶͨ͠श ͕ߦ͑Δ ࠷ઌ୺/-1ษڧձ 
3. ίϯηϓτ ࠷ઌ୺/-1ษڧձ  ը૾ͱϑϨʔζͷྨࣅੑ͕ΑΓΑ͘ܭࢉͰ͖Δ ྑ͍ߏ੒ૉͷ୯ҐΛߏ੒Ͱ͖Δ ʹ ͱ͍͏ԾఆΛஔ͖ɺ ͜ΕΛֶशͷγάφϧʹར༻͢Δ
4. ؔ࿈ݚڀ  -JOHVJTUJD
   TUSVDUVSF
   JOEVDUJPO
   GSPN
   UFYU • ೥୅͸΄ͱΜͲ඼ࢺ͔Βελʔτ – ݶքʹ͍ͭͯ͸࣋ڮ͞Μ͕ݴٴ • μ΢ϯετϦʔϜλεΫ͔Β EJTUBOU
   TVQFSWJTJPO
   Ͱؼೲ ͢Δ <
   ʹͨ͘͞Μݚڀ> – ݴޠֶతʹଥ౰ͱࢥ͑Δߏ଄ͷಋग़ʹ੒ޭͤͣ • WJB
   MBOHVBHF
   NPEFMJOH – l5IJT
   BQQSPBDI
   IBT
   BDIJFWFE
   SFNBSLBCMF QFSGPSNBODFz • 1BSTJOH3FBEJOH1SFEJDU
   /FUXPSL
   <4IFO
   FU
   BM
   B>
   • 0SEFSFE
   /FVSPO
   -45.
   <4IFO
   FU
   BM
   > ਫ໦͞Μ঺հ ࠷ઌ୺/-1ษڧձ  ͕࣌ؒͳ͍ͷͰ จݙϦετ͸࿦จΛݟ͍ͯͩ͘͞ ຊݚڀͰ͸ Language Modeling Ͱ͸ͳ͘ɺ ը૾ͱͷϚονϯάͰΨΠυ͢Δͱ͍͏ߟ͑Λಋೖ
5. ؔ࿈ݚڀ  (SPVOEFE
   MBOHVBHF
   BDRVJTJUJPO • ը૾΍ಈըͱͦͷΩϟϓγϣϯ͔Βͷؼೲ – ͍͍ͩͨ͸ WJTVBM
   BUUSJCVUFT
   ΍ BDUJPO
   ʹ͍ͭͯਓ खͷϥϕϧ΍ϧʔϧʹج͍ͮͯؼೲ͢Δ • 7JTVBMTFNBOUJD
   FNCFEEJOH
   TQBDF
   </HJBN FU
   BM
   > – ը૾ͱςΩετͷϖΞΛѻ͏ηοςΟϯάͰ ͨ͘͞Μͷݚڀ͋Γ • JNBHFDBQUJPO
   SFUSJFWBM
   JNBHF
   DBQUJPO
   HFOFSBUJPO WJTVBM RVFTUJPO
   BOTXFSJOH ࠷ઌ୺/-1ษڧձ  ͕࣌ؒͳ͍ͷͰ จݙϦετ͸࿦จΛݟ͍ͯͩ͘͞ ຊݚڀ͸׬શͳڭࢣͳ͠ ͜ͷΞΠσΞΛआΓΔ
6. ख๏ͷ֓ཁ ࠷ઌ୺/-1ษڧձ  [Ngiam et al., 2011] REINFORCE [Williams, 1992] ResNet-101 Bottom-up binary tree parsing Φ: ℝ$%&' → ℝ)*$ ͦΕͧΕͷߏ੒ૉʹ ϕΫτϧදݱ͕Ͱ͖Δ ಉۭؒ͡Ͱֶश
7. 1BSTJOH TUFQ ࠷ઌ୺/-1ษڧձ  The selected pair is combined to form a single new constituent two-layer feedforward network
8. 5SBJOJOH WJTVBM
   SFQSFTFOUBUJPO
   NBQQJOH ࠷ઌ୺/-1ษڧձ  ͋Δը૾ͷΩϟϓγϣϯ಺ͷͦΕͧΕͷߏ੒ૉ͸ ผͷΩϟϓγϣϯͷߏ੒ૉΑΓ͜ͷը૾ͱࣅ͍ͯΔ΂͖ ͜ͷը૾͸ɺผͷը૾ͱൺ΂ͯɺରԠ͢Δ Ωϟϓγϣϯ಺ͷߏ੒ૉͱࣅ͍ͯΔ΂͖ ը૾ͱߏ੒ૉͷྨࣅ౓
9. 5SBJOJOH 5FYUVBM
   4USVDUVSF
   3FQSFTFOUBUJPOT ࠷ઌ୺/-1ษڧձ  ใुؔ਺ Captionͷߏ੒ૉ͸ ը૾ͱͯ͠ө͍ͬͯΔ ΋ͷͱࣅ͍ͯΔ΂͖ ͬͪ͜͸ใुͳͷͰූ߸͕ٯ ͬͪ͜͸Ұͭͷߏ੒ૉͷ෼͚ͩܭࢉ
10. 5SBJOJOH 5FYUVBM
    4USVDUVSF
    3FQSFTFOUBUJPOT ࠷ઌ୺/-1ษڧձ  Head-Initial Inductive Bias ޙΖଆ͕ functional wordΛؚΉͳΒ ͳΔ΂͘ޙ·Ͱ͚ͬͭ͘ͳ͍Ͱ͍͍ͨ a white on the lawn cat º the where … … desk º • ୯ମͰ͚ͬͭͨ͘͘ͳ͍ • ۟΍અΛ࡞͔ͬͯΒ͚͍ͬͭͨ͘ ⋅,⋅ ͱਖ਼൓ରͷείΞɻ ͭ·Γը૾ͱؔ࿈͕ബ͍ߏ੒ૉ ޙΖଆͷந৅౓ΛଌΔ
11. ࣮ݧ • σʔλ .4$0$0 – USBJOEFWUFTU
        – #FOFQBS <,JUBFW  ,MFJO >
    Λ࢖ͬͯQBSTJOH
    ͨ݁͠ՌΛ (0-%ͷ໦ͩͱࢥ͏ • 
    ' PO
    
    SBOEBNMZ TBNQMFE
    DBQUJPOT ࠷ઌ୺/-1ษڧձ  ͑ͬ…
12. ݁Ռ ࠷ઌ୺/-1ษڧձ 
13. ݁Ռ̎ ࠷ઌ୺/-1ษڧձ  The high correlation between VG-NSL and the concreteness scores produced by Turney et al. (2011) and Brysbaert et al. (2014) supports the argument that the linguistic concept of concreteness can be acquired in an unsupervised way Compared to PRPN trained on the full training set, VG-NSL and VG-NSL+HI reach comparable performance using only 20% of thedata. VG-NSL tends to quickly become more stable as the amount of data increases, while PRPN and ON-LSTM remain less stable.
14. ·ͱΊ • ը૾ͱΩϟϓγϣϯͷϖΞΛ࢖ͬͯจͷ໦ߏ଄Λ׬શ ڭࢣͳ͠Ͱֶश • ݴޠϞσϧΛݩʹ͢Δख๏ʹൺ΂ͯޮ཰Α҆͘ఆͨ͠ ֶश͕Ͱ͖Δ • ը૾શମͰ͸ͳ͘ɺը૾ʹ΋෦෼ߏ଄Λ༩͑ͯ FH
    -V
    FU
    BM 
    8V
    FU
    BM
    
    ΞϥΠϯϝϯτ͢Ε͹΋ͬͱ͍͍͔΋Ͷ • ಋೖͨ͠ Head-Initial Inductive Bias ͷΑ͏ͳ΋ͷΛ ࣗಈతʹखʹೖΕΔʹ͸Ͳ͏ͨ͠Β͍͍ΜͩΖ͏ ࠷ઌ୺/-1ษڧձ