Upgrade to Pro — share decks privately, control downloads, hide ads and more …

文献紹介：Treat the Word As a Whole or Look Inside? Subword Embeddings Model Language Change and Typology

9e650916f36300d64c9c61eeb4ab697e?s=47 Taichi Aida
September 16, 2019
Technology
0
280

文献紹介：Treat the Word As a Whole or Look Inside? Subword Embeddings Model Language Change and Typology

9e650916f36300d64c9c61eeb4ab697e?s=128

Taichi Aida

September 16, 2019
Tweet

More Decks by Taichi Aida

See All by Taichi Aida
新入生向けチュートリアル：文献のサーベイ
9e650916f36300d64c9c61eeb4ab697e?s=48 a1da4
0
21
文献紹介：Temporal Attention for Language Models
9e650916f36300d64c9c61eeb4ab697e?s=48 a1da4
0
24
文献紹介：Dynamic Contextualized Word Embeddings
9e650916f36300d64c9c61eeb4ab697e?s=48 a1da4
2
160
文献紹介：Learning Lexical Subspaces in a Distributional Vector Space
9e650916f36300d64c9c61eeb4ab697e?s=48 a1da4
0
170
文献紹介：Unsupervised Word Polysemy Quantification with Multiresolution Grids of Contextual Embeddings
9e650916f36300d64c9c61eeb4ab697e?s=48 a1da4
0
26
新入生向けチュートリアル：tmux
9e650916f36300d64c9c61eeb4ab697e?s=48 a1da4
0
83
文献紹介：Fake it Till You Make it: Self-Supervised Semantic Shifts for Monolingual Word Embedding Tasks
9e650916f36300d64c9c61eeb4ab697e?s=48 a1da4
1
36
文献紹介：Living Machines: A study of atypical animacy
9e650916f36300d64c9c61eeb4ab697e?s=48 a1da4
0
29
文献紹介：Sequential Modelling of the Evolution of Word Representations for Semantic Change Detection
9e650916f36300d64c9c61eeb4ab697e?s=48 a1da4
0
28

Other Decks in Technology

See All in Technology
Cloud Foundryの移行先はどこか？ オープンソースPaaS探し
49844569246a6d183a5551bb74895e2a?s=48 kolinz
0
350
Backlog × RPAでいろいろ捗った話
87470304247051028bed32895e4816cd?s=48 z_tetsu
0
370
Step-by-Step MLOps and Microsoft Products
0fa6038e477031fcca028bab0efe07e3?s=48 shisyu_gaku
1
530
最先端の生成AIから考える、ビジネスにおける10年後のパラダイムシフト
61c616e83bef28a1cd2666439ebf334b?s=48 sbtechnight
1
310
金融領域のマルチプロダクトを効率よく開発・運用するためのシステム基盤と組織設計について / 2022-07-28-multi-product-platform
0251532f4fb413ea1a026efe6eb16b87?s=48 stajima
0
140
Red Hat Enterprise Linux 9のリリースノートを読む前に知りたい最近のキーワードをまとめて復習
79c2f7db29ee6df3e1ceb85c6a0126d3?s=48 moriwaka
0
360
DBRE 活動と information_schema
9513a9cc8ee9b5f2a8b4a5a914da8411?s=48 _awache
0
250
ここが好きだよAWS管理ポリシー_devio2022/i_am_iam_lover
325ce6fcd0a74ff78990b8632817da55?s=48 yukihirochiba
0
3.1k
データをコネコネ！メール配信用データ生成の仕組み
Cdb7fe48c050935995f8a6a58a5ceba9?s=48 kappezoro
0
110
Autonomous Database Cloud 技術詳細 / adb-s_technical_detail_jp
3115a782126be714b5f94d24073c957d?s=48 oracle4engineer
PRO
10
19k
品質特性のすすめ
95d0e8ddf4a989d9734c25bd6b1af295?s=48 honamin09
0
160
ログ集約基盤をCloudWatchからOpenSearchに変えてみた
098ad475722e3697ec2fba28c8654f9f?s=48 yuhta28
0
130

Featured

See All Featured
Let's Do A Bunch of Simple Stuff to Make Websites Faster
8081b26e05bb4354f7d65ffc34cbbd67?s=48 chriscoyier
498
130k
Java REST API Framework Comparison - PWX 2021
72a2082c6a4dd79ad68befb3db911616?s=48 mraible
PRO
11
4.8k
Design by the Numbers
027d1ebf66cc039a0bd3b55eeadbe75d?s=48 sachag
271
17k
Dealing with People You Can't Stand - Big Design 2015
44b54d2ffcb7857004071cc6795dde11?s=48 cassininazir
351
21k
Building Better People: How to give real-time feedback that sticks.
9952dfcd5d338f8a8e7175c8a8f65fb5?s=48 wjessup
344
17k
Building an army of robots
5f2da528927a2ec9ba4fec2069cbc958?s=48 kneath
298
40k
KATA
E9221b172e78e888355fa2fe4541b595?s=48 mclloyd
7
8.8k
Fashionably flexible responsive web design (full day workshop)
433acaea1012b25d97ae66da9b998dad?s=48 malarkey
396
62k
Become a Pro
828ace851b606e1206900f26459f55ad?s=48 speakerdeck
PRO
3
900
The Straight Up "How To Draw Better" Workshop
Aff5641764408271f7bc398f2097edd0?s=48 denniskardys
225
120k
YesSQL, Process and Tooling at Scale
D09fad3e36ae6841f54820be0e907f7a?s=48 rocio
157
12k
How New CSS Is Changing Everything About Graphic Design on the Web
D83926c323d4f9289f947b4b4e76b939?s=48 jensimmons
213
11k

Transcript

  1. จݙ঺հʢʣ Treat the Word As a Whole or Look Inside?

    Subword Embeddings Model Language Change and Typology Yang Xu, Jiasheng Zhang, David Reitter 1st International Workshop on Computational Approaches to Historical Language Change, ACL2019 ௕Ԭٕज़Պֶେֶ ࣗવݴޠॲཧݚڀࣨɹ ૬ాɹଠҰ

  2. Abstract • ݴޠֶతͳԾઆΛௐ΂ΔͨΊʹ subword Λߟྀͨ͠୯ޠ෼ࢄදݱΛఏҊ • Indo-European ͷݴޠ͸৽͍͠୯ޠ΄Ͳ subword ʹର͢ΔॏΈ͕૿͑ɺɹ

    ٯʹதࠃޠ͸ subword ʹର͢ΔॏΈ͕ݮΓɺ୯ޠʹର͢ΔॏΈ͕૿͑ͨ !2

  3. Motivation w ݴޠֶతͳ݁࿦ ʮதࠃޠʹ͓͍ͯɺ࣌ؒͱͱ΋ʹ༏Ґੑ͕୯ԻઅˠೋԻઅʹҠͬͨʯ w Ծઆ ʮݱ୅ͷதࠃޠʹ͓͍ͯɺ୯ޠʹؚ·ΕΔ׽ࣈจࣈʢTVCXPSEʣ ͸ҙຯతͳ໾ׂ͕গͳ͍ʯ !3

  4. Related Work w $#08ʢDPOUFYU ͔ΒUBSHFU Λ༧ଌʣ w $IBSBDUFSFOIBODFEXPSEFNCFEEJOH $8& 

    w 4LJQHSBNʢUBSHFU ͔ΒDPOUFYU Λ༧ଌʣ w GBTU5FYU vc ui vc ui !4 ୯ޠͱจࣈΛಉ͡ॏཁ౓Ͱѻ͏

  5. Method w %ZOBNJDTVCXPSEJODPSQPSBUFEFNCFEEJOHNPEFM %4&  w %4&$#08 w %4&4( w

    ୯ޠʹ͸୯ޠͷॏΈ ͰɺTVCXPSEʹ͸ ͰॏΈ෇͚͢Δ hw i 1 − hw i !5

  6. Method !6

  7. Experiment w %BUBTFUT w 5SBJOJOHXPSEFNCFEEJOH8JLJQFEJBEBUBCBTFEVNQT w $IJOFTF &OHMJTI 'SFODI (FSNBO

    *UBMJBO 4QBOJTI w .PEFM w %4&$#08 %4&4(ʢఏҊख๏ʣ w $8& GBTU5FYU !7

  8. Experiment w ࣮ݧ߲໨   ͱ୯ޠͷൃੜ࣌ظͱͷ૬ؔ w ൃੜ࣌ظɿ͋Δޠ͕(PPHMF#PPLT/HSBNʹॳΊͯొ৔ͨ͠೥  ޠͷҙຯλεΫ

    w &NCFEEJOHͷੑೳΛଌΔ w 4JNJMBSJUZͱ"OBMPHZΛ࢖༻ hw i !8

  9. Result ୯ޠͷॏΈ ͱൃੜ࣌ظͱͷ૬ؔɿ*OEP&VSPQFBOͱதࠃͰਖ਼൓ର w hw i !9

  10. Result ୯ޠͷॏΈ ͱൃੜ࣌ظͱͷ૬ؔɿ*OEP&VSPQFBOͱதࠃͰਖ਼൓ର w hw i !10 ࣌୅͕ਐΉͱ ୯ޠʹର͢ΔॏΈ͕ݮগ ˣ

    ୯ޠΑΓ 4VCXPSEΛॏࢹ

  11. Result ୯ޠͷॏΈ ͱൃੜ࣌ظͱͷ૬ؔɿ*OEP&VSPQFBOͱதࠃͰਖ਼൓ର w hw i !11 ࣌୅͕ਐΉͱ ୯ޠʹର͢ΔॏΈ͕૿Ճ ˣ

    4VCXPSEΑΓ ୯ޠΛॏࢹ ʢԾઆ͕͔֬ΊΒΕͨʣ

  12. Result w ͦΕͧΕͷάϧʔϓͰൺֱ w $#08ܥʢ%4&$#08 $8&ʣ w 4LJQHSBNܥʢ%4&4( GBTUUFYUʣ w

    %4&4(Ͱੑೳͷ޲্Λ֬ೝ !12

  13. Conclusion w ԾઆΛݕূ͢ΔҝʹɺTVCXPSEΛߟྀ͢Δ୯ޠ෼ࢄදݱΛఏҊͨ͠ w *OEP&VSPQFBOͷݴޠͰ͸৽͘͠ੜ·ΕΔ୯ޠ΄ͲTVCXPSEʹҙຯͷ ॏΈ͕ॏࢹ͞ΕɺதࠃޠͰ͸ٯʹTVCXPSE΁ͷॏΈ͕ݮΓɺ୯ޠͦͷ΋ ͷʹରͯ͠ॏΈ͕ͭ͘Α͏ʹͳͬͨʢԾઆΛݕূͨ͠ʣ !13

  14. None

  15. Discussion w ࣮ݧʹରͯ͠۩ମతͳൺֱΛߦͬͨ w தࠃɿ೥୅ͷۙ୅ԽͰٕज़΍Պֶ͕ൃలͨ͜͠ͱʹΑΓɺ৽͍͠୯ ޠ͕ೖ͖ͬͯͨʁ !15