文献紹介：Treat the Word As a Whole or Look Inside? Subword Embeddings Model Language Change and Typology

Transcript

1. จݙ঺հʢʣ
   Treat the Word As a Whole or Look Inside? Subword Embeddings Model Language Change and Typology Yang Xu, Jiasheng Zhang, David Reitter 1st International Workshop on Computational Approaches to Historical Language Change, ACL2019 ௕Ԭٕज़Պֶେֶ
   ࣗવݴޠॲཧݚڀࣨɹ
   ૬ాɹଠҰ
2. Abstract • ݴޠֶతͳԾઆΛௐ΂ΔͨΊʹ subword Λߟྀͨ͠୯ޠ෼ࢄදݱΛఏҊ • Indo-European ͷݴޠ͸৽͍͠୯ޠ΄Ͳ subword ʹର͢ΔॏΈ͕૿͑ɺɹ ٯʹதࠃޠ͸ subword ʹର͢ΔॏΈ͕ݮΓɺ୯ޠʹର͢ΔॏΈ͕૿͑ͨ !2
3. Motivation w ݴޠֶతͳ݁࿦
   ʮதࠃޠʹ͓͍ͯɺ࣌ؒͱͱ΋ʹ༏Ґੑ͕୯ԻઅˠೋԻઅʹҠͬͨʯ
   w Ծઆ
   ʮݱ୅ͷதࠃޠʹ͓͍ͯɺ୯ޠʹؚ·ΕΔ׽ࣈจࣈʢTVCXPSEʣ ͸ҙຯతͳ໾ׂ͕গͳ͍ʯ !3
4. Related Work w $#08ʢDPOUFYU
   
   
   ͔Β
   UBSHFU
   
   
   Λ༧ଌʣ
   w $IBSBDUFSFOIBODFE
   XPSE
   FNCFEEJOH
   $8&
   w 4LJQHSBNʢUBSHFU
   
   
   ͔Β
   DPOUFYU
   
   
   Λ༧ଌʣ
   w GBTU5FYU vc ui vc ui !4 ୯ޠͱจࣈΛಉ͡ॏཁ౓Ͱѻ͏
5. Method w %ZOBNJD
   TVCXPSEJODPSQPSBUFE
   FNCFEEJOH
   NPEFM %4&
   w %4&$#08
   w %4&4(
   w ୯ޠʹ͸୯ޠͷॏΈ
   ͰɺTVCXPSEʹ͸
   ͰॏΈ෇͚͢Δ hw i 1 − hw i !5
6. Method !6
7. Experiment w %BUBTFUT
   w 5SBJOJOH
   XPSE
   FNCFEEJOH
   8JLJQFEJB
   EBUBCBTF
   EVNQT
   w $IJOFTF
   &OHMJTI
   'SFODI
   (FSNBO
   *UBMJBO
   4QBOJTI
   w .PEFM
   w %4&$#08
   %4&4(ʢఏҊख๏ʣ
   w $8&
   GBTU5FYU !7
8. Experiment w ࣮ݧ߲໨
   
   ͱ୯ޠͷൃੜ࣌ظͱͷ૬ؔ
   w ൃੜ࣌ظɿ͋Δޠ͕
   (PPHMF
   #PPLT
   /HSBN
   ʹॳΊͯొ৔ͨ͠೥
    ޠͷҙຯλεΫ
   w &NCFEEJOH
   ͷੑೳΛଌΔ
   w 4JNJMBSJUZ
   ͱ
   "OBMPHZ
   Λ࢖༻ hw i !8
9. Result ୯ޠͷॏΈ
   ͱൃੜ࣌ظͱͷ૬ؔɿ*OEP&VSPQFBO
   ͱ
   தࠃ
   Ͱਖ਼൓ର
   w hw i !9
10. Result ୯ޠͷॏΈ
    ͱൃੜ࣌ظͱͷ૬ؔɿ*OEP&VSPQFBO
    ͱ
    தࠃ
    Ͱਖ਼൓ର
    w hw i !10 ࣌୅͕ਐΉͱ
    ୯ޠʹର͢ΔॏΈ͕ݮগ
    ˣ
    ୯ޠ
    ΑΓ
    4VCXPSE
    Λॏࢹ
11. Result ୯ޠͷॏΈ
    ͱൃੜ࣌ظͱͷ૬ؔɿ*OEP&VSPQFBO
    ͱ
    தࠃ
    Ͱਖ਼൓ର
    w hw i !11 ࣌୅͕ਐΉͱ
    ୯ޠʹର͢ΔॏΈ͕૿Ճ
    ˣ
    4VCXPSE
    ΑΓ
    ୯ޠ
    Λॏࢹ
    ʢԾઆ͕͔֬ΊΒΕͨʣ
12. Result w ͦΕͧΕͷάϧʔϓͰൺֱ
    w $#08ܥʢ%4&$#08
    $8&ʣ
    w 4LJQHSBNܥʢ%4&4(
    GBTUUFYUʣ
    w %4&4(
    Ͱੑೳͷ޲্Λ֬ೝ !12
13. Conclusion w ԾઆΛݕূ͢ΔҝʹɺTVCXPSE
    Λߟྀ͢Δ୯ޠ෼ࢄදݱΛఏҊͨ͠
    w *OEP&VSPQFBO
    ͷݴޠͰ͸৽͘͠ੜ·ΕΔ୯ޠ΄Ͳ
    TVCXPSE
    ʹҙຯͷ ॏΈ͕ॏࢹ͞ΕɺதࠃޠͰ͸ٯʹ
    TVCXPSE
    ΁ͷॏΈ͕ݮΓɺ୯ޠͦͷ΋ ͷʹରͯ͠ॏΈ͕ͭ͘Α͏ʹͳͬͨʢԾઆΛݕূͨ͠ʣ !13
15. Discussion w ࣮ݧʹରͯ͠۩ମతͳൺֱΛߦͬͨ
    w தࠃɿ೥୅ͷۙ୅ԽͰٕज़΍Պֶ͕ൃలͨ͜͠ͱʹΑΓɺ৽͍͠୯ ޠ͕ೖ͖ͬͯͨʁ !15