Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
注意機構を用いた言語創発ゲーム
Search
Ryokan RI
August 19, 2023
Research
0
220
注意機構を用いた言語創発ゲーム
2023 JSAI オーガナイズドセッション
言語とコミュニケーションの創発 ~記号創発システムから共創的言語進化まで~
Ryokan RI
August 19, 2023
Tweet
Share
More Decks by Ryokan RI
See All by Ryokan RI
[論文紹介] Intuitive Fine-Tuning
ryou0634
0
120
[論文紹介] Language is primarily a tool for communication rather than thought
ryou0634
4
1k
マルチリンガルな言語モデル入門:これまでとこれから
ryou0634
4
4.4k
What Are You Token About? Dense Retrieval as Distributions Over the Vocabulary
ryou0634
0
680
人工言語を使った事前訓練:言語間転移が可能なエンコーダの持っている知識とは何か?
ryou0634
0
930
MIROSTAT で意外さを コントロールした文章生成
ryou0634
1
970
Other Decks in Research
See All in Research
Minimax and Bayes Optimal Best-arm Identification: Adaptive Experimental Design for Treatment Choice
masakat0
0
180
20250605_新交通システム推進議連_熊本都市圏「車1割削減、渋滞半減、公共交通2倍」から考える地方都市交通政策
trafficbrain
0
840
問いを起点に、社会と共鳴する知を育む場へ
matsumoto_r
PRO
0
640
A scalable, annual aboveground biomass product for monitoring carbon impacts of ecosystem restoration projects
satai
4
330
最適決定木を用いた処方的価格最適化
mickey_kubo
4
1.9k
日本語新聞記事を用いた大規模言語モデルの暗記定量化 / LLMC2025
upura
0
230
世界の人気アプリ100個を分析して見えたペイウォール設計の心得
akihiro_kokubo
PRO
62
30k
Submeter-level land cover mapping of Japan
satai
3
400
Mechanistic Interpretability:解釈可能性研究の新たな潮流
koshiro_aoki
1
440
大学見本市2025 JSTさきがけ事業セミナー「顔の見えないセンシング技術:多様なセンサにもとづく個人情報に配慮した人物状態推定」
miso2024
0
160
GPUを利用したStein Particle Filterによる点群6自由度モンテカルロSLAM
takuminakao
0
340
IMC の細かすぎる話 2025
smly
2
660
Featured
See All Featured
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
33
2.5k
How GitHub (no longer) Works
holman
315
140k
A designer walks into a library…
pauljervisheath
209
24k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
49
3.1k
Optimising Largest Contentful Paint
csswizardry
37
3.4k
Imperfection Machines: The Place of Print at Facebook
scottboms
269
13k
Thoughts on Productivity
jonyablonski
70
4.9k
Keith and Marios Guide to Fast Websites
keithpitt
411
22k
Principles of Awesome APIs and How to Build Them.
keavy
127
17k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
31
2.5k
Faster Mobile Websites
deanohume
310
31k
It's Worth the Effort
3n
187
28k
Transcript
ҙػߏΛ༻͍ͨ ݴޠൃήʔϜ ཥ ྇פɹ্ా ྄ɹJason Naradowsky ౦ژେֶ
ࢦࣔήʔϜͱʁ - ίϛϡχέʔγϣϯ͔Βࢦࣔͷଆ໘͚ͩΛऔΓग़ͨ͠ήʔϜɻ - ൃίϛϡχέʔγϣϯͷݚڀͰΑ͘ΘΕΔɻ ࢦࣔήʔϜ (referential game)
ࢦࣔήʔϜ (referential game) Speaker 3 14 2 25 ඪΦϒδΣΫτ Listener
ΦϒδΣΫτީิू߹
ࢦࣔήʔϜ (referential game) Speaker 3 14 2 25 Listener ඪΦϒδΣΫτ
ΦϒδΣΫτީิू߹ ᶃ Speaker ͕ ඪΦϒδΣΫτΛݟͯ ϝοηʔδΛੜ
ࢦࣔήʔϜ (referential game) Speaker 3 14 2 25 Listener ඪΦϒδΣΫτ
ΦϒδΣΫτީิू߹ ᶄ Listener ͕ϝοηʔδΛݟͯ ΦϒδΣΫτީิू߹͔Β ඪΦϒδΣΫτΛ༧ଌ͢Δ
ࢦࣔήʔϜ (referential game) Speaker 3 14 2 25 ඪΦϒδΣΫτ Listener
ΦϒδΣΫτީิू߹ ᶅ Listener ͷ༧ଌͱ ඪΦϒδΣΫτ͕Ұக͍ͯ͠Ε ใु͕༩͑ΒΕΔɻ
ࢦࣔήʔϜͷΤʔδΣϯτʹҙػߏΛ࣮͢Δ ຊݚڀͰͬͨ͜ͱ Speaker Listener
ͳͥҙػߏ͔ʁ - ൃݴޠͷߏੑ্͕͕Δͣ - ൃݴޠͷղऍੑ্͕͕Δͣ ຊݚڀͰͬͨ͜ͱ
ͳͥҙػߏ͔ʁ - ൃݴޠͷߏੑ্͕͕Δͣ - ൃݴޠͷղऍੑ্͕͕Δͣ ຊݚڀͰͬͨ͜ͱ
ߏੑͱɿશମ͕෦ͷ૯͔ΒͳΔ͜ͱ ҙػߏൃݴޠͷߏੑΛ্͛Δʁ ߏੑ ߴ🔺 ߏੑ 🔻 5 3 2 3
2 1 6 4
ҙػߏൃݴޠͷߏੑΛ্͛Δʁ 5 Speaker ҙػߏ୯ޠͱΦϒδΣΫτͷ෦ͷ݁ͼ͖ͭ Λֶश͢Δͷʹཱͪͦ͏
ͳͥҙػߏ͔ʁ - ൃݴޠͷߏੑ্͕͕Δͣ - ൃݴޠͷղऍੑ্͕͕Δͣ ຊݚڀͰͬͨ͜ͱ
ҙػߏൃݴޠͷղऍੑΛ্͛Δʁ 5 Speaker ͜ͷ୯ޠ͜ͷҙຯͳͷͩͱΘ͔Δ
࣮ݧઃఆɿࢦࣔήʔϜ Speaker 3 14 2 25 ඪΦϒδΣΫτ Listener ΦϒδΣΫτީิू߹ Fashion
MNIST Λ༻͍ͨࢦࣔήʔϜ ʢεχʔΧʔͱϓϧΦʔόʔʣ
࣮ݧઃఆɿࢦࣔήʔϜ Speaker 3 14 2 25 ඪΦϒδΣΫτ Listener ΦϒδΣΫτީิू߹ Fashion
MNIST Λ༻͍ͨࢦࣔήʔϜ ʢεχʔΧʔͱϓϧΦʔόʔʣ
FashionMNIST - 10 छྨͷΞΠςϜͷը૾ΛؚΉσʔληοτ ࣮ݧઃఆɿࢦࣔήʔϜ
FashionMNIST - 10 छྨͷΞΠςϜͷը૾ΛؚΉσʔληοτ ࣮ݧઃఆɿࢦࣔήʔϜ ̎छྨͷΞΠςϜΛબΜͰ ̍ͭͷΦϒδΣΫτΛ࡞Δ
࣮ݧઃఆɿࢦࣔήʔϜ FashionMNIST Λ༻͍ͨࢦࣔήʔϜ - FashionMNIST த͔Βը૾Λαϯϓϧͯ͠ଟ༷ͳΦϒδΣΫ τΛ࡞ɻ - ಉ͡छྨͷΞΠςϜؚ͕·Ε͍ͯΕಉ͡ΦϒδΣΫτͰ͋Δ ͱΈͳ͢ɻ
- ΦϒδΣΫτͷछྨ - ܇࿅ηοτʹ30छྨɺධՁηοτʹ15छྨ༻ 10 C2 = 45
࣮ݧઃఆɿࢦࣔήʔϜ ਓؒʹ͍ۙݴޠΛֶश͢ΔͨΊʹ ಉ͡छྨͷҟͳΔΦϒδΣΫτΛ Speaker ͱ Listener ʹݟͤΔ ඪΦϒδΣΫτ ΦϒδΣΫτީิू߹
ҙػߏͳ͠ͷ Speaker ҙػߏ͋Γͷ Speaker ࣮ݧઃఆɿϞσϧ ҙػߏͳ͠ͷ Listner ҙػߏ͋Γͷ Listener ×
ͦΕͧΕͷΈ߹ΘͤͰ࣮ݧ ͞ΒʹΤʔδΣϯτΛ࣮͢ΔωοτϫʔΫͱͯ͠ LSTM ͱ Transformer ͦΕͧΕͰ࣮ݧ
࣮ݧઃఆɿϞσϧ ϕʔεϥΠϯɾΤʔδΣϯτ otgt o1 %FDPEFS &ODPEFS m1 m2 m3 ϝοηʔδΦϒδΣΫτ
߹கείΞ m0 m1 m2 m1 m2 m3 Speaker Listener - ҙػߏͳ͠ - ΦϒδΣΫτ୯ҰͷϕΫτϧͰදݱ͞ΕΔ
࣮ݧઃఆɿϞσϧ ҙػߏ͋ΓΤʔδΣϯτ otgt o1 %FDPEFS &ODPEFS m1 m2 m3 "UUFOUJPO
"UUFOUJPO m0 m1 m2 m1 m2 m3 Speaker Listener ϝοηʔδΦϒδΣΫτ ߹கείΞ - ҙػߏ͋Γ - ΦϒδΣΫτෳͷϕΫτϧͰදݱ͞ΕΔ
ҙػߏ͋Γ/ͳ͠ΤʔδΣϯτͷൺֱ - ҙػߏ͋Γͩͱߏੑͷई (TopSim) ্͕͢Δ ҙػߏ͋ΓΤʔδΣϯτͷݴޠͷௐࠪ ࣮ݧͨ͜͠ͱ
ҙػߏ͋Γ/ͳ͠ΤʔδΣϯτͷൺֱ ܇࿅ηοτͷਖ਼ղ ධՁηοτͷਖ਼ղ TopSimʢߏੑͷईʣ - ҙػߏ͕͋Δํ͕ਖ਼ղ͕ߴ͘ͳΔʹ͋Δ - ҙػߏ͕͋Δํ͕ɺTopSim ͕ߴ͘ͳΔʹ͋Δ
ҙػߏ͋ΓΤʔδΣϯτͷ ҙػߏͷՄࢹԽ Speaker Listener Speaker Listener Speaker Listener ޭͨ͠ύλʔϯ ޭͨ͠ύλʔϯ
ࣦഊͨ͠ύλʔϯ
ҙػߏ͋ΓΤʔδΣϯτͷ ൃͨ͠ݴޠͷγϯϘϧͱΦϒδΣΫτͷରԠ Speaker Listener
ҙػߏ͋ΓΤʔδΣϯτͷ ൃͨ͠ݴޠͷγϯϘϧͱΦϒδΣΫτͷରԠ Speaker Listener ্ணܥͷΦϒδΣΫτ ۺܥͷΦϒδΣΫτ
ҙػߏ͋ΓSpeaker ͱ Listener ͷ ҙൣғͷͣΕͱਖ਼ղͷؔ ͣΕ͕গͳ͍΄Ͳਖ਼ղ͍͢͠
ͬͨ͜ͱ - ࢦࣔήʔϜͷΤʔδΣϯτʹҙػߏΛ͚ͭΔ - FashionMNIST ͷը૾ΛͬͨࢦࣔήʔϜͰ࣮ݧ ͔ͬͨ͜ͱ - ҙػߏ͕͋Δͱߏੑ͕ߴ͍ݴޠ͕ൃ͍͢͠
- ҙػߏΛ࣋ͬͨΤʔδΣϯτղऍੑͷ͋ΔݴޠΛൃͤ͞ΒΕΔ ·ͱΊ
࣮͏ͪΐͬͱࣗવͰෳࡶͳը૾ΛͬͨࢦࣔήʔϜͰ ͏·͍ͬͯ͘΄͔͕ͬͨ͠ɺֶश͕͔ͬͨ͠ - ෳࡶͳը૾ͩͱղऍෆೳͳҙͷύλʔϯ͔͠؍͞Εͣ ➡︎ ҙػߏ͕ਓؒͷΑ͏ͳબੑΛ͍࣋ͬͯͳ͍͔Βʁ ਓؒͷ࣋ͭೝػೳͱݴޠͷੑ࣭ͷؔ࿈ͳͲ͕ௐΒΕΔͱ ָͦ͠͏ʁ ࠓޙͷൃల