Share
Embed
「人間にAIはどのように辿り着けばよいのか?ー 系統的汎化からの第一歩 ー」@第22回 Language and Robotics研究会
第22回 Language and Robotics研究会にて発表した「人間にAIはどのように辿り着けばよいのか?ー 系統的汎化からの第一歩 ー」の発表資料になります.
概要:大規模言語モデルの台頭により,人類が夢見ていた汎用人工知能は単なる夢物語ではなく,現実のものとして近づいてきた.汎用人工知能の先には,人間の知能を超えた人工超知能があり,人工超知能の実現も少しずつ現実味を帯び始めている.本講演では,まずは人間の知能に辿り着くという目標の上で,どのような道筋を辿っていくのがよいのかについてを論じる.そして,具体的な第一歩としての系統的汎化について,これまでの歴史を振り返りながら最新のベンチマークと手法,現状の課題を紹介する.
maguro27
June 28, 2024
Transcript
55 slides
-
ࣗݾհ 2 ߝౡ लथ uॴଐ ૣҴాେֶ ത࢜5ʵౡൟੜݚڀࣨ uݚڀςʔϚ म࢜՝ఔɿਂը૾ੜϞσϧͷܭࢉྔɾύϥϝʔλݮ ത࢜՝ఔ1ɿෳମΛରͱͨ͠ڭࢣແ͠લܠഎܠղ
ത࢜՝ఔ2ɿEmbodied AIؔ࿈ ത࢜՝ఔ3ʙɿৗࣝ֫ಘɼܥ౷త൚Խ ࢈૯ݚʢݩʣɿͱਓͷϖΞσʔλΛඞཁͱ͠ͳ͍Ծࢼண uझຯ ےτϨɼΞϝϑτɼಡॻɼԻָήʔϜɼ'14ɼ ΰϧϑɼҿञɼμʔπɼϏϦϠʔυɼࣸਅɼFUD -
ೳ 5 <>ਓೳͱ lIUUQTXXXJCNDPNKQKBUPQJDTBSUJGJDJBMTVQFSJOUFMMJHFODFz ݄Ӿཡ <>ڧ͍"*ͱ lIUUQTXXXJCNDPNKQKBUPQJDTTUSPOHBJz ݄Ӿཡ ਓೳʢ"SUJGJDJBM 4VQFS
*OUFMMJHFODF "4*ʣਓؒͷೳΛ͑ͨతൣғΛͭ ਓೳγεςϜͷҰͭʢԾతଘࡏʣ<> ݱࡏͷ"*ऑ͍"*ͱݺΕɼಛఆͷλεΫʹ͓͍ͯ༏ΕͨੑೳΛൃش͢Δ"* <> "(*ʢ"SUJGJDJBM (FOFSBM *OUFMMJHFODFʣڧ͍"*ͱݺΕɼਓؒͱಉͷҙࣝΛ࣋ͪɼਓؒͱಉͷ ॊೈੑ൚ԽੑΛ࣋ͪ߹ΘͤΔ"*ʢఆٛͱͯ͠·ͩ֬ఆ͍ͯ͠ͳ͍ʣ<> ྫ͑Ͱݴ͏ͱɼλʔϛωʔλʔʹొ͢ΔϚβʔ"*ͷʮεΧΠωοτʯ͕"4*ͷҰྫ ʢͨͩɼޙड़ͷཧ༝ʹΑΓएׯ"(*دΓ͔ʁʣ -
Contents l --.ͰطʹΔ͜ͱऴΘͬͯ͠·͍ͬͯΔʁ l ͲͷΑ͏ʹਓʹḷΓண͘ͷ͔ʁ l ߝౡͷݚڀऀͱͯ͠ͷىݯɼ͜Ε·Ͱͷݚڀ l ܥ౷త൚Խ l
ܥ౷త൚ԽΤʔδΣϯτͷϕϯνϚʔΫɼख๏܈ɼ՝ 9 -
Contents l --.ͰطʹΔ͜ͱऴΘͬͯ͠·͍ͬͯΔʁ l ͲͷΑ͏ʹਓʹḷΓண͘ͷ͔ʁ l ߝౡͷݚڀऀͱͯ͠ͷىݯɼ͜Ε·Ͱͷݚڀ l ܥ౷త൚Խ l
ܥ౷త൚ԽΤʔδΣϯτͷϕϯνϚʔΫɼख๏܈ɼ՝ 10 -
Q. ͱ͜ΖͰɼLLMͰશͯऴΘΓʁ 11 A. ͦΜͳ͜ͱͳ͍ େنݴޠϞσϧʢLLMʣ֬తΦϜͱᎏ᎐͞ΕΔΑ͏ʹɼͦΕͬΆ͍୯ޠΛฦ͍ͯ͠Δ͚ͩ [1] ʢΠϝʔδͱͯ͠ૹͷϑϦʔϨϯͷຐͷΑ͏ͳײ͡ʣ GPT-4Ͱ͢Β͜ͷ͋Γ [1]
E. M. Bender et al., “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜”, FAccT, 2021. [2] ૹͷϑϦʔϨϯXެࣜΞΧϯτ, https://twitter.com/FRIEREN_PR/status/1772020566293111290/photo/1, 20244݄18Ӿཡ. ૹͷϑϦʔϨϯɼຐʮஅ಄ͷΞϥʯҰ෦վม <> -
Q. ͱ͜ΖͰɼLLMͰશͯऴΘΓʁ 12 A. ͦΜͳ͜ͱͳ͍ LLM͕େنʹͳΕͳΔ΄Ͳੑೳ͕Լ͢ΔInverse Scaling Prize [1] ɼ
ΦʔϓϯϘΩϟϒϥϦʔͷମΛ࣋ͬͨΤʔδΣϯτʹΑΔ࣭ԠͷOpenEQA [2]ͳͲɼ ·ͩ·ͩLLMෳࡶͳਪ͕ۤखͰ͋ͬͨΓɼLLMಛ༗ͷؒҧ͍Λͨ͠Γ͢Δ [1] Inverse Scaling Prize, https://huggingface.co/inverse-scaling. [2] A. Majumdar et al., “OpenEQA: Embodied Question Answering in the Era of Foundation Models”, Preprint, 2024. OpenEQAͷਓͱLLMͷਖ਼ൺֱ [2] -
Contents l --.ͰطʹΔ͜ͱऴΘͬͯ͠·͍ͬͯΔʁ l ͲͷΑ͏ʹਓʹḷΓண͘ͷ͔ʁ l ߝౡͷݚڀऀͱͯ͠ͷىݯɼ͜Ε·Ͱͷݚڀ l ܥ౷త൚Խ l
ܥ౷త൚ԽΤʔδΣϯτͷϕϯνϚʔΫɼख๏܈ɼ՝ 13 -
Contents l --.ͰطʹΔ͜ͱऴΘͬͯ͠·͍ͬͯΔʁ l ͲͷΑ͏ʹਓʹḷΓண͘ͷ͔ʁ l ߝౡͷݚڀऀͱͯ͠ͷىݯɼ͜Ε·Ͱͷݚڀ l ܥ౷త൚Խ l
ܥ౷త൚ԽΤʔδΣϯτͷϕϯνϚʔΫɼख๏܈ɼ՝ 14 -
ਓͷߏΛਅࣅΔʁೳྗΛਅࣅΔʁ 15 ߏͱಉ͡Ͱಉ͡ೳྗൃݱ͠ͳ͍ʢֶशํ๏ͱσʔλΛἧ͑ͯൃݱ͠ͳ͍ʣ ߏΛ฿ͨ͠NN [1] [1] R. P. Rane et
al., “PredNet and Predictive Coding: A Critical Review”, ICMR, 2020. [2] D. Lee et al., “Difference Target Propagation”, ECML/PKDD. 2015. [3] J. Sullivan et al., “SAYCam: A Large, Longitudinal Audiovisual Dataset Recorded From the Infant’s Perspective”, Open Mind, 2021. [4] A. E. Orhan et al., “Self-supervised learning through the eyes of a child”, NeurIPS, 2020. ੜֶతʹଥͳֶशํ๏ [2] ༮ࣇͷҰਓশࢹಈը [3, 4] -
ਓͷߏΛਅࣅΔʁೳྗΛਅࣅΔʁ 16 ਓͷೝػೳೳྗΛਅࣅΔ͜ͱ͕λεΫղܾʹ݁͢ΔʢखஈΘͳ͍ʣ [1] Y. Zhu et al., “Dark, Beyond
Deep: A Paradigm Shift to Cognitive AI with Humanlike Common Sense”, Engineering, 2020. ৗੜ׆Ͱਓ͕ؒར༻͍ͯ͠Δৗࣝతਪ [1] -
Contents l --.ͰطʹΔ͜ͱऴΘͬͯ͠·͍ͬͯΔʁ l ͲͷΑ͏ʹਓʹḷΓண͘ͷ͔ʁ l ߝౡͷݚڀऀͱͯ͠ͷىݯɼ͜Ε·Ͱͷݚڀ l ܥ౷త൚Խ l
ܥ౷త൚ԽΤʔδΣϯτͷϕϯνϚʔΫɼख๏܈ɼ՝ 17 -
Contents l --.ͰطʹΔ͜ͱऴΘͬͯ͠·͍ͬͯΔʁ l ͲͷΑ͏ʹਓʹḷΓண͘ͷ͔ʁ l ߝౡͷݚڀऀͱͯ͠ͷىݯɼ͜Ε·Ͱͷݚڀ l ܥ౷త൚Խ l
ܥ౷త൚ԽΤʔδΣϯτͷϕϯνϚʔΫɼख๏܈ɼ՝ 18 -
͜Ε·Ͱͷݚڀ༰ 23 uੜϞσϧͷܭࢉίετݮ ü ͷߦྻܭࢉΛݮ uڭࢣͳ͠Ծࢼண uڭࢣͳ͠લܠഎܠ uৗࣝΛ֫ಘͨ͠"*ͷ։ൃ %"--&Ͱੜ <>
NJEKPVSOFZͰੜ <> <>%"--&, “https://openai.com/index/dall-e-2/”, ݄Ӿཡ <>NJEKPVSOFZ lIUUQTXXXNJEKPVSOFZDPNIPNFz ݄Ӿཡ ੲʹੜͨ͠ͷͰɼใͱͯ͠ݹ͍Ͱ͢ɼɼɼ -
Contents l --.ͰطʹΔ͜ͱऴΘͬͯ͠·͍ͬͯΔʁ l ͲͷΑ͏ʹਓʹḷΓண͘ͷ͔ʁ l ߝౡͷݚڀऀͱͯ͠ͷىݯɼ͜Ε·Ͱͷݚڀ l ܥ౷త൚Խ l
ܥ౷త൚ԽΤʔδΣϯτͷϕϯνϚʔΫɼख๏܈ɼ՝ 28 -
Contents l --.ͰطʹΔ͜ͱऴΘͬͯ͠·͍ͬͯΔʁ l ͲͷΑ͏ʹਓʹḷΓண͘ͷ͔ʁ l ߝౡͷݚڀऀͱͯ͠ͷىݯɼ͜Ε·Ͱͷݚڀ l ܥ౷త൚Խ l
ܥ౷త൚ԽΤʔδΣϯτͷϕϯνϚʔΫɼख๏܈ɼ՝ 29 -
·ͣະͷγφϦΦͰڧ͍ਓ͔ؒΒֶͿ 30 ਓ͕ؒະͷγφϦΦʹڧ͍ͷ “the infinite use of finite means” [1]
ͷೳྗ͕͋ΔͨΊ ݩʑݴޠֶͰChomskyઌੜ͓ͬ͠Όͬͨ͜ͱͰɼ༗ݶͷޠኮ͔ΒແݶͷจষΛ࡞ΕΔ͜ͱ͔Β ͜ͷೳྗࣗମܥ౷త൚ԽʢSystematic Generalizationʣ[2] ͱݺΕ͍ͯΔ [1] N. Chomsky. “Aspect of the Theory of Syntax”, The MIT Press, 1965. [2] B. M. Lake and M. Baroni. “Human-like systematic generalization through a meta-learning neural network”, Nature, 2023. ܥ౷త൚Խͷྫ [2] -
ܥ౷త൚Խͱߏੑ 31 ܥ౷త൚ԽΛࢧ͍͑ͯΔͷߏੑʢCompositionalityʣͱ͍͏ݪଇʢPrincipleʣ ߏੑ3ͭͷݪଇͱ2ͭͷධՁج४͔Β͍ͬͯΔ [1] ʻݪଇʼ 1. ܥ౷ੑʢSystematicityʣ 2. ੜ࢈ੑʢProductivityʣ
3. ସੑʢSubstitutivityʣ ʻධՁج४ʼ 1. ہॴੑʢLocalityʣ 2. ա൚ԽʢOvergeneralizationʣ [1] D. Hupkes et al., “Compositionality Decomposed: How do Neural Networks Generalise?”, JAIR, 2020. ߏੑͷཁૉ [1] -
ܥ౷త൚Խͱߏੑ 32 ʻݪଇʼ 1. ܥ౷ੑʢSystematicityʣɿଐੑͷΈ߹Θͤͷ൚Խ ߏੑͷཁૉ [1] [1] D. Hupkes
et al., “Compositionality Decomposed: How do Neural Networks Generalise?”, JAIR, 2020. -
ܥ౷త൚Խͱߏੑ 33 ʻݪଇʼ 1. ܥ౷ੑʢSystematicityʣɿଐੑͷΈ߹Θͤͷ൚Խ ʁʁʁʁʁʁʁ ʻطʼ ʻطʼ ʻະʼ ʻະʼ
[1] D. Hupkes et al., “Compositionality Decomposed: How do Neural Networks Generalise?”, JAIR, 2020. -
ܥ౷త൚Խͱߏੑ 34 ʻݪଇʼ 2. ੜ࢈ੑʢProductivityʣɿະͷ͞ͷγʔέϯεͷ൚Խ ߏੑͷཁૉ [1] [1] D. Hupkes
et al., “Compositionality Decomposed: How do Neural Networks Generalise?”, JAIR, 2020. -
ܥ౷త൚Խͱߏੑ 35 ʻݪଇʼ 3. ସੑʢSubstitutivityʣɿྨࣅͷݴ༿ʢه߸ʣͷஔͷ൚Խ ͋Δจষʹ͓͍ͯɼྨࣅͷݴ༿͕ஔ͖͑ΒΕͯ จষͷҙຯ͕มԽ͠ͳ͍ ίϯςΩετΛؚΊ্ͨͰͷҙຯཧղ͕Ͱ͖͍ͯΔ ຊޠͰͷྫɿʮΑ͘ߟ͑Δʯͱʮख़ྀʯ ߏੑͷཁૉ
[1] [1] D. Hupkes et al., “Compositionality Decomposed: How do Neural Networks Generalise?”, JAIR, 2020. -
ܥ౷త൚Խͱߏੑ 36 ʻධՁج४ʼ 1. ہॴੑʢLocalityʣɿॲཧ͕ہॴత͔େҬత͔ʢϞσϧͷಛੑΛΔͨΊͷධՁج४ʣ จষʹΑͬͯɼہॴతେҬతͲͪΒͰॲཧͯ͠ ݁Ռ͕ಉ͡ʹͳΔͷɼͳΒͳ͍ͷ͋Δ จষͷྫɿʮJohn loves MaryʯͱʮMary
loves Johnʯ จষߏʢSVOʣͱ͍͏େҬతʹݟΔͱಉҰ จষͷҙຯͱ͍͏ہॴతʹݟΔͱҧ͏ ֶͷྫɿʮ14 – (2 + 3)ʯ ʮ14 – (2 + 3)ʯͱେҬతʹܭࢉͯ͠ ʮ14 – 5ʯͱہॴతʹܭࢉͯ݁͠ՌಉҰ ߏੑͷཁૉ [1] [1] D. Hupkes et al., “Compositionality Decomposed: How do Neural Networks Generalise?”, JAIR, 2020. -
ܥ౷త൚Խͱߏੑ 37 ʻධՁج४ʼ 2. ա൚ԽʢOvergeneraliationʣɿྫ֎ʹطଘͷϧʔϧΛద༻͢Δ͔ ݴޠͷྫɿʮgoʯͱʮgoedʯͱʮwentʯ ӳޠجຊతʹաڈܗͷ߹-edΛ͚ͭΔ͕ɼ goͷྫ֎έʔεͰɼgoedͰͳ͘wentʹͳΔ ֶशதʹgo͕ग़͖ͯͨ߹ʹطଘͷϧʔϧͷ-edΛ ͚ͭͯgoedʹͨ͠ͳΒɼϧʔϧΛ֫ಘͰ͖͍ͯΔ
ͱղऍͰ͖Δ ߏੑͷཁૉ [1] [1] D. Hupkes et al., “Compositionality Decomposed: How do Neural Networks Generalise?”, JAIR, 2020. -
ܥ౷త൚Խͱߏੑ 38 ʻݪଇʼ 1. ܥ౷ੑʢSystematicityʣ 2. ੜ࢈ੑʢProductivityʣ 3. ସੑʢSubstitutivityʣ ʻධՁج४ʼ
1. ہॴੑʢLocalityʣ 2. ա൚ԽʢOvergeneralizationʣ 3ͭͷݪଇͱ2ͭͷධՁج४Λͱʹɼ Ϟσϧ͕ͲͷΑ͏ͳੑ࣭Λ͍࣋ͬͯΔͷ͔Λ ଟ໘తʹධՁ͢Δ [1] D. Hupkes et al., “Compositionality Decomposed: How do Neural Networks Generalise?”, JAIR, 2020. ߏੑͷཁૉ [1] -
Contents l --.ͰطʹΔ͜ͱऴΘͬͯ͠·͍ͬͯΔʁ l ͲͷΑ͏ʹਓʹḷΓண͘ͷ͔ʁ l ߝౡͷݚڀऀͱͯ͠ͷىݯɼ͜Ε·Ͱͷݚڀ l ܥ౷త൚Խ l
ܥ౷త൚ԽΤʔδΣϯτͷϕϯνϚʔΫɼख๏܈ɼ՝ ʢφΠʔϒͳܥ౷త൚Խ࣌ؒͷؔͰলུʣ 39 -
Contents l --.ͰطʹΔ͜ͱऴΘͬͯ͠·͍ͬͯΔʁ l ͲͷΑ͏ʹਓʹḷΓண͘ͷ͔ʁ l ߝౡͷݚڀऀͱͯ͠ͷىݯɼ͜Ε·Ͱͷݚڀ l ܥ౷త൚Խ l
ܥ౷త൚ԽΤʔδΣϯτͷϕϯνϚʔΫɼख๏܈ɼ՝ ʢφΠʔϒͳܥ౷త൚Խ࣌ؒͷؔͰলུʣ 40 -
ܥ౷త൚ԽΤʔδΣϯτͷͨΊͷϕϯνϚʔΫ 41 [1] B. M. Lake and M. Baroni, “Generalization
without Systematicity: On the Compositional Skills of Sequence-to-Sequence Recurrent Networks”, ICML, 2018. [2] B. M. Lake and M. Baroni. “Human-like systematic generalization through a meta-learning neural network”, Nature, 2023. 4$"/ <>ɿݴޠࢦࣔΛೖྗͱͯ͠ɼߦಈΛग़ྗ 4$"/ϕϯνϚʔΫͷ֓೦ਤ [2] 4$"/ϕϯνϚʔΫͷೖग़ྗྫ <> -
ܥ౷త൚ԽΤʔδΣϯτͷͨΊͷϕϯνϚʔΫ 42 [1] B. M. Lake and M. Baroni, “Generalization
without Systematicity: On the Compositional Skills of Sequence-to-Sequence Recurrent Networks”, ICML, 2018. [2] B. M. Lake and M. Baroni. “Human-like systematic generalization through a meta-learning neural network”, Nature, 2023. 4$"/ <>ɿݴޠࢦࣔΛೖྗͱͯ͠ɼߦಈΛग़ྗ εϓϦοτͭ ܥ౷ੑ ੜ࢈ੑ KVNQͳͲͷ୯ҰͷϓϦϛςΟϒͷΈΛݟ͍ͯΔ ͷͰͷܥ౷ੑ ӳޠ ϑϥϯεޠͷػց༁ʹ͓͚Δܥ౷ੑ 4$"/ϕϯνϚʔΫͷ֓೦ਤ [2] 4$"/ϕϯνϚʔΫͷೖग़ྗྫ <> -
ܥ౷త൚ԽΤʔδΣϯτͷͨΊͷϕϯνϚʔΫ 43 [1] L. Ruis et al., “A Benchmark for
Systematic Generalization in Grounded Language Understanding”, NeurIPS, 2020. gSCANσʔληοτ [1] H4$"/ HSPVOEFE 4$"/ <>ɿঢ়ଶ˞ͱݴޠࢦࣔΛೖྗͱͯ͠ɼߦಈΛग़ྗ ˞ ঢ়ଶͱɼڧԽֶश༻ޠͷʮঢ়ଶʢ4UBUFʣʯͰ͋Γɼਅͷ؍ଌใʢਤͰ͍͏൫໘ใʣ -
ܥ౷త൚ԽΤʔδΣϯτͷͨΊͷϕϯνϚʔΫ 44 [1] L. Ruis et al., “A Benchmark for
Systematic Generalization in Grounded Language Understanding”, NeurIPS, 2020. gSCANσʔληοτ [1] H4$"/ HSPVOEFE 4$"/ <>ɿঢ়ଶ˞ͱݴޠࢦࣔΛೖྗͱͯ͠ɼߦಈΛग़ྗ ˞ ঢ়ଶͱɼڧԽֶश༻ޠͷʮঢ়ଶʢ4UBUFʣʯͰ͋Γɼਅͷ؍ଌใʢਤͰ͍͏൫໘ใʣ εϓϦοτͭ ମଐੑͷܥ౷ੑ ମଐੑͷܥ౷ੑύʔτ ະͷҠಈํ ະͷΦϒδΣΫτͷ૬ରతαΠζ ମͱߦಈؒͷܥ౷ੑ ະͷ෭ࢺͷGFXTIPU MFBSOJOH ෭ࢺΛಈࢺʹஔ ੜ࢈ੑ -
ܥ౷త൚ԽΤʔδΣϯτͷͨΊͷϕϯνϚʔΫ 45 [1] Z. Wu et al., “ReaSCAN: Compositional Reasoning
in Language Grounding”, NeurIPS Datasets and Benchmarks, 2021. ReaSCANσʔληοτ [1] 3FB4$"/ 3FBTPOJOHCBTFE 4$"/ <>ɿঢ়ଶͱݴޠࢦࣔΛೖྗͱͯ͠ɼߦಈΛग़ྗ -
ܥ౷త൚ԽΤʔδΣϯτͷͨΊͷϕϯνϚʔΫ 46 [1] Z. Wu et al., “ReaSCAN: Compositional Reasoning
in Language Grounding”, NeurIPS Datasets and Benchmarks, 2021. ReaSCANσʔληοτ [1] 3FB4$"/ 3FBTPOJOHCBTFE 4$"/ <>ɿঢ়ଶͱݴޠࢦࣔΛೖྗͱͯ͠ɼߦಈΛग़ྗ H4$"/ͭͷ͕͋ͬͨ p ݴޠࢦࣔͷΈ߹Θ͕ͤγϯϓϧ͗ͯ͢ จ຺ແࢹͷ#P8ʢ#BHPG8PSETʣͰेͩͬͨ p ΦϒδΣΫτ͕ܥ౷త൚ԽͷͨΊͷ ਖ਼֬ͳཧղʹ΄ͱΜͲد༩͍ͯ͠ͳ͔ͬͨ p ݴޠࢦࣔͷܗ༰ࢺ͕φϏήʔγϣϯʹ ͍Βͳ͍͜ͱ͕ଟ͔ͬͨ -
ܥ౷త൚ԽΤʔδΣϯτͷͨΊͷϕϯνϚʔΫ 47 [1] Z. Wu et al., “ReaSCAN: Compositional Reasoning
in Language Grounding”, NeurIPS Datasets and Benchmarks, 2021. ReaSCANσʔληοτ [1] 3FB4$"/ 3FBTPOJOHCBTFE 4$"/ <>ɿঢ়ଶͱݴޠࢦࣔΛೖྗͱͯ͠ɼߦಈΛग़ྗ େ͖͚ͯ͘εϓϦοτͭʢࡉ͔͍ͱͭʣ ମଐੑͷܥ౷ੑ ঢ়ଶͱݴޠͷܥ౷ੑʢ৽نମΈ߹Θͤʣ ৽ن۟ߏʢੜ࢈ੑͱ۟ͷೖΕସ͑ʣ -
ܥ౷త൚ԽΤʔδΣϯτͷख๏ 49 l -45. TFRTFR l $POW TFRTFR l 5SBOTGPSNFS
l ($/-45. l .FUB 5SBOTGPSNFS -
ܥ౷త൚ԽΤʔδΣϯτͷख๏ 50 l -45. TFRTFRɿγʔέϯεΛॱʑʹೖྗͯ͠ɼग़ྗΛ࠶ىతʹೖྗͱͯ͠ར༻ l $POW TFRTFR l 5SBOTGPSNFS
l ($/-45. l .FUB 5SBOTGPSNFS -45. $POW 5SBOTGPSNFS<> 4FR4FRʢ4FRVFODFUP4FRVFODFʣ<> [1] D. Hupkes et al., “Compositionality Decomposed: How do Neural Networks Generalise?”, JAIR, 2020. [2] B. M. Lake and M. Baroni, “Generalization without Systematicity: On the Compositional Skills of Sequence-to-Sequence Recurrent Networks”, ICML, 2018. -
ܥ౷త൚ԽΤʔδΣϯτͷख๏ 51 l -45. TFRTFRɿγʔέϯεΛॱʑʹೖྗͯ͠ɼग़ྗΛ࠶ىతʹೖྗͱͯ͠ར༻ l $POW TFRTFRɿγʔέϯεͰඃΔΑ͏ʹΈࠐΈʢͻͱੲલʹݴޠॲཧͰྲྀߦʣ l 5SBOTGPSNFS
l ($/-45. l .FUB 5SBOTGPSNFS -45. $POW 5SBOTGPSNFS<> [1] D. Hupkes et al., “Compositionality Decomposed: How do Neural Networks Generalise?”, JAIR, 2020. [2] B. M. Lake and M. Baroni, “Generalization without Systematicity: On the Compositional Skills of Sequence-to-Sequence Recurrent Networks”, ICML, 2018. -
ܥ౷త൚ԽΤʔδΣϯτͷख๏ 52 l -45. TFRTFRɿγʔέϯεΛॱʑʹೖྗͯ͠ɼग़ྗΛ࠶ىతʹೖྗͱͯ͠ར༻ l $POW TFRTFRɿγʔέϯεͰඃΔΑ͏ʹΈࠐΈʢͻͱੲલʹݴޠॲཧͰྲྀߦʣ l 5SBOTGPSNFSɿγʔέϯεΛ.-1ͱ"UUFOUJPOͰॲཧ͠ɼԕ͍γʔέϯεͰߟྀ͕༻ҙ
l ($/-45. l .FUB 5SBOTGPSNFS -45. $POW 5SBOTGPSNFS<> 5SBOTGPSNFSΞʔΩςΫνϟ <> [1] D. Hupkes et al., “Compositionality Decomposed: How do Neural Networks Generalise?”, JAIR, 2020. [2] B. M. Lake and M. Baroni, “Generalization without Systematicity: On the Compositional Skills of Sequence-to-Sequence Recurrent Networks”, ICML, 2018. [3] A. Vaswani et al., “Attention Is All You Need”, NIPS, 2017. -
ܥ౷త൚ԽΤʔδΣϯτͷख๏ 53 [1] D. Hupkes et al., “Compositionality Decomposed: How
do Neural Networks Generalise?”, JAIR, 2020. [2] B. M. Lake and M. Baroni, “Generalization without Systematicity: On the Compositional Skills of Sequence-to-Sequence Recurrent Networks”, ICML, 2018. [3] A. Vaswani et al., “Attention Is All You Need”, NIPS, 2017. [4] T. Gao et al., “Systematic Generalization on gSCAN with Language Conditioned Embedding”, AACL, 2020. l -45. TFRTFRɿγʔέϯεΛॱʑʹೖྗͯ͠ɼग़ྗΛ࠶ىతʹೖྗͱͯ͠ར༻ l $POW TFRTFRɿγʔέϯεͰඃΔΑ͏ʹΈࠐΈʢͻͱੲલʹݴޠॲཧͰྲྀߦʣ l 5SBOTGPSNFSɿγʔέϯεΛ.-1ͱ"UUFOUJPOͰॲཧ͠ɼԕ͍γʔέϯεͰߟྀ͕༻ҙ l ($/-45.ɿάϥϑΈࠐΈΛ༻͍ͯݴޠͱঢ়ଶͷؔੑߟྀ͢ΔH4$"/Ͱͷ405" l .FUB 5SBOTGPSNFS ($/-45.ͷΞʔΩςΫνϟ <> -
ܥ౷త൚ԽΤʔδΣϯτͷख๏ 54 [1] D. Hupkes et al., “Compositionality Decomposed: How
do Neural Networks Generalise?”, JAIR, 2020. [2] B. M. Lake and M. Baroni, “Generalization without Systematicity: On the Compositional Skills of Sequence-to-Sequence Recurrent Networks”, ICML, 2018. [3] A. Vaswani et al., “Attention Is All You Need”, NIPS, 2017. [4] T. Gao et al., “Systematic Generalization on gSCAN with Language Conditioned Embedding”, AACL, 2020. [5] B. M. Lake and M. Baroni. “Human-like systematic generalization through a meta-learning neural network”, Nature, 2023. l -45. TFRTFRɿγʔέϯεΛॱʑʹೖྗͯ͠ɼग़ྗΛ࠶ىతʹೖྗͱͯ͠ར༻ l $POW TFRTFRɿγʔέϯεͰඃΔΑ͏ʹΈࠐΈʢͻͱੲલʹݴޠॲཧͰྲྀߦʣ l 5SBOTGPSNFSɿγʔέϯεΛ.-1ͱ"UUFOUJPOͰॲཧ͠ɼԕ͍γʔέϯεͰߟྀ͕༻ҙ l ($/-45.ɿάϥϑΈࠐΈΛ༻͍ͯݴޠͱঢ়ଶͷؔੑߟྀ͢ΔH4$"/Ͱͷ405" l .FUB 5SBOTGPSNFSɿGFXTIPU FYBNQMFTͷઃఆͷ5SBOTGPSNFS ݱঢ়ɼ5SBOTGPSNFSܥ͕࠷༗ .FUB5SBOTGPSNFS<> -
ܥ౷త൚ԽΤʔδΣϯτͷ՝ͱࠓޙͷల 55 [1] D. Bahdanau et al., “Systematic Generalization: What
Is Required and Can It Be Learned?”, ICLR, 2018. [2] R. Csordás et al., “The Devil is in the Detail: Simple Tricks Improve Systematic Generalization of Transformers”, EMNLP, 2021. [3] R. Kirk et al., “A Survey of Zero-shot Generalisation in Deep Reinforcement Learning”, JAIR, 2023. l طଘͷϕϯνϚʔΫݱ࣮ੈքͷઃఆʹଈ͍ͯ͠ͳ͍ʢਅͷঢ়ଶऔಘͰ͖ͳ͍ʣ l طଘͷϕϯνϚʔΫͰఏҊ͞Εͨख๏Ͱ͢Βɼܥ౷త൚ԽΤʔδΣϯτͷ࣮ݱʹఔԕ͍ l ϕϯνϚʔΫʹ͓͍ͯɼ܇࿅ޡࠩͱςετੑೳʹ૬͕ؔݟΒΕͳ͍ [1, 2] Ø ܇࿅ޡࠩͱݕূޡ͕ࠩখ͘͞ͳ͔ͬͨΒͱ͍ͬͯςετੑೳ͕ߴ͍ͱݶΒͳ͍ l ܥ౷త൚Խͷઌθϩγϣοτ൚ԽʢZSG; Zero-shot Generalizationʣ[3] Ø ZSGݪཧతʹղ͚ͳ͍ͣͰɼڧ͍ؼೲόΠΞεΠϯλϥΫγϣϯ͕ඞཁ ZSGͷ֓೦ਤ [3]