Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
注意機構を用いた言語創発ゲーム
Search
Ryokan RI
August 19, 2023
Research
0
140
注意機構を用いた言語創発ゲーム
2023 JSAI オーガナイズドセッション
言語とコミュニケーションの創発 ~記号創発システムから共創的言語進化まで~
Ryokan RI
August 19, 2023
Tweet
Share
More Decks by Ryokan RI
See All by Ryokan RI
Language is primarily a tool for communication rather than thought
ryou0634
4
740
マルチリンガルな言語モデル入門:これまでとこれから
ryou0634
4
3.7k
What Are You Token About? Dense Retrieval as Distributions Over the Vocabulary
ryou0634
0
600
人工言語を使った事前訓練:言語間転移が可能なエンコーダの持っている知識とは何か?
ryou0634
0
690
MIROSTAT で意外さを コントロールした文章生成
ryou0634
1
670
Other Decks in Research
See All in Research
20240918 交通くまもとーく 未来の鉄道網編(太田恒平)
trafficbrain
0
220
テキストマイニングことはじめー基本的な考え方からメディアディスコース研究への応用まで
langstat
1
120
Whoisの闇
hirachan
3
140
2024/10/30 産総研AIセミナー発表資料
keisuke198619
1
330
最近のVisual Odometryと Depth Estimation
sgk
1
270
Weekly AI Agents News! 8月号 プロダクト/ニュースのアーカイブ
masatoto
1
190
20240820: Minimum Bayes Risk Decoding for High-Quality Text Generation Beyond High-Probability Text
de9uch1
0
120
marukotenant01/tenant-20240916
marketing2024
0
500
言語処理学会30周年記念事業留学支援交流会@YANS2024:「学生のための短期留学」
a1da4
1
240
工学としてのSRE再訪 / Revisiting SRE as Engineering
yuukit
19
11k
LLM時代にLabは何をすべきか聞いて回った1年間
hargon24
1
490
[CV勉強会@関東 CVPR2024] Visual Layout Composer: Image-Vector Dual Diffusion Model for Design Layout Generation / kantocv 61th CVPR 2024
shunk031
1
450
Featured
See All Featured
How to Think Like a Performance Engineer
csswizardry
20
1.1k
Designing for humans not robots
tammielis
250
25k
Bootstrapping a Software Product
garrettdimon
PRO
305
110k
Unsuck your backbone
ammeep
668
57k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
44
6.8k
Optimizing for Happiness
mojombo
376
70k
Designing the Hi-DPI Web
ddemaree
280
34k
Speed Design
sergeychernyshev
24
610
Visualization
eitanlees
145
15k
Being A Developer After 40
akosma
86
590k
The Pragmatic Product Professional
lauravandoore
31
6.3k
Agile that works and the tools we love
rasmusluckow
327
21k
Transcript
ҙػߏΛ༻͍ͨ ݴޠൃήʔϜ ཥ ྇פɹ্ా ྄ɹJason Naradowsky ౦ژେֶ
ࢦࣔήʔϜͱʁ - ίϛϡχέʔγϣϯ͔Βࢦࣔͷଆ໘͚ͩΛऔΓग़ͨ͠ήʔϜɻ - ൃίϛϡχέʔγϣϯͷݚڀͰΑ͘ΘΕΔɻ ࢦࣔήʔϜ (referential game)
ࢦࣔήʔϜ (referential game) Speaker 3 14 2 25 ඪΦϒδΣΫτ Listener
ΦϒδΣΫτީิू߹
ࢦࣔήʔϜ (referential game) Speaker 3 14 2 25 Listener ඪΦϒδΣΫτ
ΦϒδΣΫτީิू߹ ᶃ Speaker ͕ ඪΦϒδΣΫτΛݟͯ ϝοηʔδΛੜ
ࢦࣔήʔϜ (referential game) Speaker 3 14 2 25 Listener ඪΦϒδΣΫτ
ΦϒδΣΫτީิू߹ ᶄ Listener ͕ϝοηʔδΛݟͯ ΦϒδΣΫτީิू߹͔Β ඪΦϒδΣΫτΛ༧ଌ͢Δ
ࢦࣔήʔϜ (referential game) Speaker 3 14 2 25 ඪΦϒδΣΫτ Listener
ΦϒδΣΫτީิू߹ ᶅ Listener ͷ༧ଌͱ ඪΦϒδΣΫτ͕Ұக͍ͯ͠Ε ใु͕༩͑ΒΕΔɻ
ࢦࣔήʔϜͷΤʔδΣϯτʹҙػߏΛ࣮͢Δ ຊݚڀͰͬͨ͜ͱ Speaker Listener
ͳͥҙػߏ͔ʁ - ൃݴޠͷߏੑ্͕͕Δͣ - ൃݴޠͷղऍੑ্͕͕Δͣ ຊݚڀͰͬͨ͜ͱ
ͳͥҙػߏ͔ʁ - ൃݴޠͷߏੑ্͕͕Δͣ - ൃݴޠͷղऍੑ্͕͕Δͣ ຊݚڀͰͬͨ͜ͱ
ߏੑͱɿશମ͕෦ͷ૯͔ΒͳΔ͜ͱ ҙػߏൃݴޠͷߏੑΛ্͛Δʁ ߏੑ ߴ🔺 ߏੑ 🔻 5 3 2 3
2 1 6 4
ҙػߏൃݴޠͷߏੑΛ্͛Δʁ 5 Speaker ҙػߏ୯ޠͱΦϒδΣΫτͷ෦ͷ݁ͼ͖ͭ Λֶश͢Δͷʹཱͪͦ͏
ͳͥҙػߏ͔ʁ - ൃݴޠͷߏੑ্͕͕Δͣ - ൃݴޠͷղऍੑ্͕͕Δͣ ຊݚڀͰͬͨ͜ͱ
ҙػߏൃݴޠͷղऍੑΛ্͛Δʁ 5 Speaker ͜ͷ୯ޠ͜ͷҙຯͳͷͩͱΘ͔Δ
࣮ݧઃఆɿࢦࣔήʔϜ Speaker 3 14 2 25 ඪΦϒδΣΫτ Listener ΦϒδΣΫτީิू߹ Fashion
MNIST Λ༻͍ͨࢦࣔήʔϜ ʢεχʔΧʔͱϓϧΦʔόʔʣ
࣮ݧઃఆɿࢦࣔήʔϜ Speaker 3 14 2 25 ඪΦϒδΣΫτ Listener ΦϒδΣΫτީิू߹ Fashion
MNIST Λ༻͍ͨࢦࣔήʔϜ ʢεχʔΧʔͱϓϧΦʔόʔʣ
FashionMNIST - 10 छྨͷΞΠςϜͷը૾ΛؚΉσʔληοτ ࣮ݧઃఆɿࢦࣔήʔϜ
FashionMNIST - 10 छྨͷΞΠςϜͷը૾ΛؚΉσʔληοτ ࣮ݧઃఆɿࢦࣔήʔϜ ̎छྨͷΞΠςϜΛબΜͰ ̍ͭͷΦϒδΣΫτΛ࡞Δ
࣮ݧઃఆɿࢦࣔήʔϜ FashionMNIST Λ༻͍ͨࢦࣔήʔϜ - FashionMNIST த͔Βը૾Λαϯϓϧͯ͠ଟ༷ͳΦϒδΣΫ τΛ࡞ɻ - ಉ͡छྨͷΞΠςϜؚ͕·Ε͍ͯΕಉ͡ΦϒδΣΫτͰ͋Δ ͱΈͳ͢ɻ
- ΦϒδΣΫτͷछྨ - ܇࿅ηοτʹ30छྨɺධՁηοτʹ15छྨ༻ 10 C2 = 45
࣮ݧઃఆɿࢦࣔήʔϜ ਓؒʹ͍ۙݴޠΛֶश͢ΔͨΊʹ ಉ͡छྨͷҟͳΔΦϒδΣΫτΛ Speaker ͱ Listener ʹݟͤΔ ඪΦϒδΣΫτ ΦϒδΣΫτީิू߹
ҙػߏͳ͠ͷ Speaker ҙػߏ͋Γͷ Speaker ࣮ݧઃఆɿϞσϧ ҙػߏͳ͠ͷ Listner ҙػߏ͋Γͷ Listener ×
ͦΕͧΕͷΈ߹ΘͤͰ࣮ݧ ͞ΒʹΤʔδΣϯτΛ࣮͢ΔωοτϫʔΫͱͯ͠ LSTM ͱ Transformer ͦΕͧΕͰ࣮ݧ
࣮ݧઃఆɿϞσϧ ϕʔεϥΠϯɾΤʔδΣϯτ otgt o1 %FDPEFS &ODPEFS m1 m2 m3 ϝοηʔδΦϒδΣΫτ
߹கείΞ m0 m1 m2 m1 m2 m3 Speaker Listener - ҙػߏͳ͠ - ΦϒδΣΫτ୯ҰͷϕΫτϧͰදݱ͞ΕΔ
࣮ݧઃఆɿϞσϧ ҙػߏ͋ΓΤʔδΣϯτ otgt o1 %FDPEFS &ODPEFS m1 m2 m3 "UUFOUJPO
"UUFOUJPO m0 m1 m2 m1 m2 m3 Speaker Listener ϝοηʔδΦϒδΣΫτ ߹கείΞ - ҙػߏ͋Γ - ΦϒδΣΫτෳͷϕΫτϧͰදݱ͞ΕΔ
ҙػߏ͋Γ/ͳ͠ΤʔδΣϯτͷൺֱ - ҙػߏ͋Γͩͱߏੑͷई (TopSim) ্͕͢Δ ҙػߏ͋ΓΤʔδΣϯτͷݴޠͷௐࠪ ࣮ݧͨ͜͠ͱ
ҙػߏ͋Γ/ͳ͠ΤʔδΣϯτͷൺֱ ܇࿅ηοτͷਖ਼ղ ධՁηοτͷਖ਼ղ TopSimʢߏੑͷईʣ - ҙػߏ͕͋Δํ͕ਖ਼ղ͕ߴ͘ͳΔʹ͋Δ - ҙػߏ͕͋Δํ͕ɺTopSim ͕ߴ͘ͳΔʹ͋Δ
ҙػߏ͋ΓΤʔδΣϯτͷ ҙػߏͷՄࢹԽ Speaker Listener Speaker Listener Speaker Listener ޭͨ͠ύλʔϯ ޭͨ͠ύλʔϯ
ࣦഊͨ͠ύλʔϯ
ҙػߏ͋ΓΤʔδΣϯτͷ ൃͨ͠ݴޠͷγϯϘϧͱΦϒδΣΫτͷରԠ Speaker Listener
ҙػߏ͋ΓΤʔδΣϯτͷ ൃͨ͠ݴޠͷγϯϘϧͱΦϒδΣΫτͷରԠ Speaker Listener ্ணܥͷΦϒδΣΫτ ۺܥͷΦϒδΣΫτ
ҙػߏ͋ΓSpeaker ͱ Listener ͷ ҙൣғͷͣΕͱਖ਼ղͷؔ ͣΕ͕গͳ͍΄Ͳਖ਼ղ͍͢͠
ͬͨ͜ͱ - ࢦࣔήʔϜͷΤʔδΣϯτʹҙػߏΛ͚ͭΔ - FashionMNIST ͷը૾ΛͬͨࢦࣔήʔϜͰ࣮ݧ ͔ͬͨ͜ͱ - ҙػߏ͕͋Δͱߏੑ͕ߴ͍ݴޠ͕ൃ͍͢͠
- ҙػߏΛ࣋ͬͨΤʔδΣϯτղऍੑͷ͋ΔݴޠΛൃͤ͞ΒΕΔ ·ͱΊ
࣮͏ͪΐͬͱࣗવͰෳࡶͳը૾ΛͬͨࢦࣔήʔϜͰ ͏·͍ͬͯ͘΄͔͕ͬͨ͠ɺֶश͕͔ͬͨ͠ - ෳࡶͳը૾ͩͱղऍෆೳͳҙͷύλʔϯ͔͠؍͞Εͣ ➡︎ ҙػߏ͕ਓؒͷΑ͏ͳબੑΛ͍࣋ͬͯͳ͍͔Βʁ ਓؒͷ࣋ͭೝػೳͱݴޠͷੑ࣭ͷؔ࿈ͳͲ͕ௐΒΕΔͱ ָͦ͠͏ʁ ࠓޙͷൃల