文献紹介:Treat the Word As a Whole or Look Inside? Subword Embeddings Model Language Change and Typology

9e650916f36300d64c9c61eeb4ab697e?s=47 Taichi Aida
September 16, 2019

文献紹介:Treat the Word As a Whole or Look Inside? Subword Embeddings Model Language Change and Typology

9e650916f36300d64c9c61eeb4ab697e?s=128

Taichi Aida

September 16, 2019
Tweet

Transcript

  1. จݙ঺հʢʣ Treat the Word As a Whole or Look Inside?

    Subword Embeddings Model Language Change and Typology Yang Xu, Jiasheng Zhang, David Reitter 1st International Workshop on Computational Approaches to Historical Language Change, ACL2019 ௕Ԭٕज़Պֶେֶ ࣗવݴޠॲཧݚڀࣨɹ ૬ాɹଠҰ
  2. Abstract • ݴޠֶతͳԾઆΛௐ΂ΔͨΊʹ subword Λߟྀͨ͠୯ޠ෼ࢄදݱΛఏҊ • Indo-European ͷݴޠ͸৽͍͠୯ޠ΄Ͳ subword ʹର͢ΔॏΈ͕૿͑ɺɹ

    ٯʹதࠃޠ͸ subword ʹର͢ΔॏΈ͕ݮΓɺ୯ޠʹର͢ΔॏΈ͕૿͑ͨ !2
  3. Motivation w ݴޠֶతͳ݁࿦ ʮதࠃޠʹ͓͍ͯɺ࣌ؒͱͱ΋ʹ༏Ґੑ͕୯ԻઅˠೋԻઅʹҠͬͨʯ w Ծઆ ʮݱ୅ͷதࠃޠʹ͓͍ͯɺ୯ޠʹؚ·ΕΔ׽ࣈจࣈʢTVCXPSEʣ ͸ҙຯతͳ໾ׂ͕গͳ͍ʯ !3

  4. Related Work w $#08ʢDPOUFYU ͔ΒUBSHFU Λ༧ଌʣ w $IBSBDUFSFOIBODFEXPSEFNCFEEJOH $8& 

    w 4LJQHSBNʢUBSHFU ͔ΒDPOUFYU Λ༧ଌʣ w GBTU5FYU vc ui vc ui !4 ୯ޠͱจࣈΛಉ͡ॏཁ౓Ͱѻ͏
  5. Method w %ZOBNJDTVCXPSEJODPSQPSBUFEFNCFEEJOHNPEFM %4&  w %4&$#08 w %4&4( w

    ୯ޠʹ͸୯ޠͷॏΈ ͰɺTVCXPSEʹ͸ ͰॏΈ෇͚͢Δ hw i 1 − hw i !5
  6. Method !6

  7. Experiment w %BUBTFUT w 5SBJOJOHXPSEFNCFEEJOH8JLJQFEJBEBUBCBTFEVNQT w $IJOFTF &OHMJTI 'SFODI (FSNBO

    *UBMJBO 4QBOJTI w .PEFM w %4&$#08 %4&4(ʢఏҊख๏ʣ w $8& GBTU5FYU !7
  8. Experiment w ࣮ݧ߲໨   ͱ୯ޠͷൃੜ࣌ظͱͷ૬ؔ w ൃੜ࣌ظɿ͋Δޠ͕(PPHMF#PPLT/HSBNʹॳΊͯొ৔ͨ͠೥  ޠͷҙຯλεΫ

    w &NCFEEJOHͷੑೳΛଌΔ w 4JNJMBSJUZͱ"OBMPHZΛ࢖༻ hw i !8
  9. Result ୯ޠͷॏΈ ͱൃੜ࣌ظͱͷ૬ؔɿ*OEP&VSPQFBOͱதࠃͰਖ਼൓ର w hw i !9

  10. Result ୯ޠͷॏΈ ͱൃੜ࣌ظͱͷ૬ؔɿ*OEP&VSPQFBOͱதࠃͰਖ਼൓ର w hw i !10 ࣌୅͕ਐΉͱ ୯ޠʹର͢ΔॏΈ͕ݮগ ˣ

    ୯ޠΑΓ 4VCXPSEΛॏࢹ
  11. Result ୯ޠͷॏΈ ͱൃੜ࣌ظͱͷ૬ؔɿ*OEP&VSPQFBOͱதࠃͰਖ਼൓ର w hw i !11 ࣌୅͕ਐΉͱ ୯ޠʹର͢ΔॏΈ͕૿Ճ ˣ

    4VCXPSEΑΓ ୯ޠΛॏࢹ ʢԾઆ͕͔֬ΊΒΕͨʣ
  12. Result w ͦΕͧΕͷάϧʔϓͰൺֱ w $#08ܥʢ%4&$#08 $8&ʣ w 4LJQHSBNܥʢ%4&4( GBTUUFYUʣ w

    %4&4(Ͱੑೳͷ޲্Λ֬ೝ !12
  13. Conclusion w ԾઆΛݕূ͢ΔҝʹɺTVCXPSEΛߟྀ͢Δ୯ޠ෼ࢄදݱΛఏҊͨ͠ w *OEP&VSPQFBOͷݴޠͰ͸৽͘͠ੜ·ΕΔ୯ޠ΄ͲTVCXPSEʹҙຯͷ ॏΈ͕ॏࢹ͞ΕɺதࠃޠͰ͸ٯʹTVCXPSE΁ͷॏΈ͕ݮΓɺ୯ޠͦͷ΋ ͷʹରͯ͠ॏΈ͕ͭ͘Α͏ʹͳͬͨʢԾઆΛݕূͨ͠ʣ !13

  14. None
  15. Discussion w ࣮ݧʹରͯ͠۩ମతͳൺֱΛߦͬͨ w தࠃɿ೥୅ͷۙ୅ԽͰٕज़΍Պֶ͕ൃలͨ͜͠ͱʹΑΓɺ৽͍͠୯ ޠ͕ೖ͖ͬͯͨʁ !15