จݙ঺հʢʣ Treat the Word As a Whole or Look Inside? Subword Embeddings Model Language Change and Typology Yang Xu, Jiasheng Zhang, David Reitter 1st International Workshop on Computational Approaches to Historical Language Change, ACL2019 ௕Ԭٕज़Պֶେֶ ࣗવݴޠॲཧݚڀࣨɹ ૬ాɹଠҰ

Abstract • ݴޠֶతͳԾઆΛௐ΂ΔͨΊʹ subword Λߟྀͨ͠୯ޠ෼ࢄදݱΛఏҊ • Indo-European ͷݴޠ͸৽͍͠୯ޠ΄Ͳ subword ʹର͢ΔॏΈ͕૿͑ɺɹ ٯʹதࠃޠ͸ subword ʹର͢ΔॏΈ͕ݮΓɺ୯ޠʹର͢ΔॏΈ͕૿͑ͨ !2

Motivation w ݴޠֶతͳ݁࿦ ʮதࠃޠʹ͓͍ͯɺ࣌ؒͱͱ΋ʹ༏Ґੑ͕୯ԻઅˠೋԻઅʹҠͬͨʯ w Ծઆ ʮݱ୅ͷதࠃޠʹ͓͍ͯɺ୯ޠʹؚ·ΕΔ׽ࣈจࣈʢTVCXPSEʣ ͸ҙຯతͳ໾ׂ͕গͳ͍ʯ !3

Related Work w $#08ʢDPOUFYU ͔ΒUBSHFU Λ༧ଌʣ w $IBSBDUFSFOIBODFEXPSEFNCFEEJOH $8& w 4LJQHSBNʢUBSHFU ͔ΒDPOUFYU Λ༧ଌʣ w GBTU5FYU vc ui vc ui !4 ୯ޠͱจࣈΛಉ͡ॏཁ౓Ͱѻ͏

Method w %ZOBNJDTVCXPSEJODPSQPSBUFEFNCFEEJOHNPEFM %4& w %4&$#08 w %4&4( w ୯ޠʹ͸୯ޠͷॏΈ ͰɺTVCXPSEʹ͸ ͰॏΈ෇͚͢Δ hw i 1 − hw i !5

Method !6

Experiment w %BUBTFUT w 5SBJOJOHXPSEFNCFEEJOH8JLJQFEJBEBUBCBTFEVNQT w $IJOFTF &OHMJTI 'SFODI (FSNBO *UBMJBO 4QBOJTI w .PEFM w %4&$#08 %4&4(ʢఏҊख๏ʣ w $8& GBTU5FYU !7

Experiment w ࣮ݧ߲໨ ͱ୯ޠͷൃੜ࣌ظͱͷ૬ؔ w ൃੜ࣌ظɿ͋Δޠ͕(PPHMF#PPLT/HSBNʹॳΊͯొ৔ͨ͠೥ ޠͷҙຯλεΫ w &NCFEEJOHͷੑೳΛଌΔ w 4JNJMBSJUZͱ"OBMPHZΛ࢖༻ hw i !8

Result ୯ޠͷॏΈ ͱൃੜ࣌ظͱͷ૬ؔɿ*OEP&VSPQFBOͱதࠃͰਖ਼൓ର w hw i !9

Result ୯ޠͷॏΈ ͱൃੜ࣌ظͱͷ૬ؔɿ*OEP&VSPQFBOͱதࠃͰਖ਼൓ର w hw i !10 ࣌୅͕ਐΉͱ ୯ޠʹର͢ΔॏΈ͕ݮগ ˣ ୯ޠΑΓ 4VCXPSEΛॏࢹ

Result ୯ޠͷॏΈ ͱൃੜ࣌ظͱͷ૬ؔɿ*OEP&VSPQFBOͱதࠃͰਖ਼൓ର w hw i !11 ࣌୅͕ਐΉͱ ୯ޠʹର͢ΔॏΈ͕૿Ճ ˣ 4VCXPSEΑΓ ୯ޠΛॏࢹ ʢԾઆ͕͔֬ΊΒΕͨʣ

Result w ͦΕͧΕͷάϧʔϓͰൺֱ w $#08ܥʢ%4&$#08 $8&ʣ w 4LJQHSBNܥʢ%4&4( GBTUUFYUʣ w %4&4(Ͱੑೳͷ޲্Λ֬ೝ !12

Conclusion w ԾઆΛݕূ͢ΔҝʹɺTVCXPSEΛߟྀ͢Δ୯ޠ෼ࢄදݱΛఏҊͨ͠ w *OEP&VSPQFBOͷݴޠͰ͸৽͘͠ੜ·ΕΔ୯ޠ΄ͲTVCXPSEʹҙຯͷ ॏΈ͕ॏࢹ͞ΕɺதࠃޠͰ͸ٯʹTVCXPSE΁ͷॏΈ͕ݮΓɺ୯ޠͦͷ΋ ͷʹରͯ͠ॏΈ͕ͭ͘Α͏ʹͳͬͨʢԾઆΛݕূͨ͠ʣ !13

No content

Discussion w ࣮ݧʹରͯ͠۩ମతͳൺֱΛߦͬͨ w தࠃɿ೥୅ͷۙ୅ԽͰٕज़΍Պֶ͕ൃలͨ͜͠ͱʹΑΓɺ৽͍͠୯ ޠ͕ೖ͖ͬͯͨʁ !15