Upgrade to Pro — share decks privately, control downloads, hide ads and more …

「人間にAIはどのように辿り着けばよいのか?ー 系統的汎化からの第一歩 ー」@第22回 Lan...

maguro27
June 28, 2024

「人間にAIはどのように辿り着けばよいのか?ー 系統的汎化からの第一歩 ー」@第22回 Language and Robotics研究会

第22回 Language and Robotics研究会にて発表した「人間にAIはどのように辿り着けばよいのか?ー 系統的汎化からの第一歩 ー」の発表資料になります.

概要:大規模言語モデルの台頭により,人類が夢見ていた汎用人工知能は単なる夢物語ではなく,現実のものとして近づいてきた.汎用人工知能の先には,人間の知能を超えた人工超知能があり,人工超知能の実現も少しずつ現実味を帯び始めている.本講演では,まずは人間の知能に辿り着くという目標の上で,どのような道筋を辿っていくのがよいのかについてを論じる.そして,具体的な第一歩としての系統的汎化について,これまでの歴史を振り返りながら最新のベンチマークと手法,現状の課題を紹介する.

maguro27

June 28, 2024
Tweet

More Decks by maguro27

Other Decks in Research

Transcript

  1. ࣗݾ঺հ 2 ߝౡ लथ uॴଐ ૣҴాେֶ ത࢜5೥ʵ৿ౡൟੜݚڀࣨ uݚڀςʔϚ म࢜՝ఔɿਂ૚ը૾ੜ੒Ϟσϧͷܭࢉྔɾύϥϝʔλ࡟ݮ ത࢜՝ఔ1೥ɿෳ਺෺ମΛର৅ͱͨ͠ڭࢣແ͠લܠഎܠ෼ղ

    ത࢜՝ఔ2೥ɿEmbodied AIؔ࿈ ത࢜՝ఔ3೥ʙɿৗࣝ֫ಘɼܥ౷త൚Խ ࢈૯ݚʢݩʣɿ෰ͱਓͷϖΞσʔλΛඞཁͱ͠ͳ͍Ծ૝ࢼண uझຯ ےτϨɼΞϝϑτɼಡॻɼԻָήʔϜɼ'14ɼ ΰϧϑɼҿञɼμʔπɼϏϦϠʔυɼࣸਅɼFUD
  2. ௒஌ೳ 5 <>ਓ޻௒஌ೳͱ͸ lIUUQTXXXJCNDPNKQKBUPQJDTBSUJGJDJBMTVQFSJOUFMMJHFODFz ೥݄೔Ӿཡ <>ڧ͍"*ͱ͸ lIUUQTXXXJCNDPNKQKBUPQJDTTUSPOHBJz ೥݄೔Ӿཡ ਓ޻௒஌ೳʢ"SUJGJDJBM 4VQFS

    *OUFMMJHFODF "4*ʣ͸ਓؒͷ஌ೳΛ௒͑ͨ஌తൣғΛ΋ͭ ਓ޻஌ೳγεςϜͷҰͭʢԾ૝తଘࡏʣ<> ݱࡏͷ"*͸ऑ͍"*ͱݺ͹ΕɼಛఆͷλεΫʹ͓͍ͯ༏ΕͨੑೳΛൃش͢Δ"* <> "(*ʢ"SUJGJDJBM (FOFSBM *OUFMMJHFODFʣ͸ڧ͍"*ͱݺ͹Εɼਓؒͱಉ౳ͷҙࣝΛ࣋ͪɼਓؒͱಉ౳ͷ ॊೈੑ΍൚ԽੑΛ࣋ͪ߹ΘͤΔ"*ʢఆٛͱͯ͠·ͩ֬ఆ͸͍ͯ͠ͳ͍ʣ<> ྫ͑Ͱݴ͏ͱɼλʔϛωʔλʔʹొ৔͢ΔϚβʔ"*ͷʮεΧΠωοτʯ͕"4*ͷҰྫ ʢͨͩɼޙड़ͷཧ༝ʹΑΓएׯ"(*دΓ͔΋ʁʣ
  3. Q. ͱ͜ΖͰɼLLMͰશͯऴΘΓʁ 11 A. ͦΜͳ͜ͱ͸ͳ͍ େن໛ݴޠϞσϧʢLLMʣ͸֬཰తΦ΢Ϝͱᎏ᎐͞ΕΔΑ͏ʹɼͦΕͬΆ͍୯ޠΛฦ͍ͯ͠Δ͚ͩ [1] ʢΠϝʔδͱͯ͠͸૴ૹͷϑϦʔϨϯͷຐ଒ͷΑ͏ͳײ͡ʣ GPT-4Ͱ͢Β͜ͷ܏޲͋Γ [1]

    E. M. Bender et al., “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜”, FAccT, 2021. [2] ૴ૹͷϑϦʔϨϯXެࣜΞΧ΢ϯτ, https://twitter.com/FRIEREN_PR/status/1772020566293111290/photo/1, 2024೥4݄18೔Ӿཡ. ૴ૹͷϑϦʔϨϯɼຐ଒ʮஅ಄୆ͷΞ΢ϥʯҰ෦վม <>
  4. Q. ͱ͜ΖͰɼLLMͰશͯऴΘΓʁ 12 A. ͦΜͳ͜ͱ͸ͳ͍ LLM͕େن໛ʹͳΕ͹ͳΔ΄Ͳੑೳ͕௿Լ͢ΔInverse Scaling Prize [1] ΍ɼ

    ΦʔϓϯϘΩϟϒϥϦʔͷ਎ମΛ࣋ͬͨΤʔδΣϯτʹΑΔ࣭໰Ԡ౴ͷOpenEQA [2]ͳͲɼ ·ͩ·ͩLLM͸ෳࡶͳਪ࿦͕ۤखͰ͋ͬͨΓɼLLMಛ༗ͷؒҧ͍Λͨ͠Γ͢Δ [1] Inverse Scaling Prize, https://huggingface.co/inverse-scaling. [2] A. Majumdar et al., “OpenEQA: Embodied Question Answering in the Era of Foundation Models”, Preprint, 2024. OpenEQAͷਓͱLLMͷਖ਼౴཰ൺֱ [2]
  5. ਓͷ೴ߏ଄ΛਅࣅΔʁೳྗΛਅࣅΔʁ 15 ೴ߏ଄ͱಉ͡Ͱ΋ಉ͡ೳྗ͸ൃݱ͠ͳ͍ʢֶशํ๏ͱσʔλΛἧ͑ͯ΋ൃݱ͠ͳ͍ʣ ೴ߏ଄Λ໛฿ͨ͠NN [1] [1] R. P. Rane et

    al., “PredNet and Predictive Coding: A Critical Review”, ICMR, 2020. [2] D. Lee et al., “Difference Target Propagation”, ECML/PKDD. 2015. [3] J. Sullivan et al., “SAYCam: A Large, Longitudinal Audiovisual Dataset Recorded From the Infant’s Perspective”, Open Mind, 2021. [4] A. E. Orhan et al., “Self-supervised learning through the eyes of a child”, NeurIPS, 2020. ੜ෺ֶతʹଥ౰ͳֶशํ๏ [2] ༮ࣇͷҰਓশࢹ఺ಈը [3, 4]
  6. ਓͷ೴ߏ଄ΛਅࣅΔʁೳྗΛਅࣅΔʁ 16 ਓͷೝ஌ػೳ΍ೳྗΛਅࣅΔ͜ͱ͕λεΫղܾʹ௚݁͢Δʢखஈ͸໰Θͳ͍ʣ [1] Y. Zhu et al., “Dark, Beyond

    Deep: A Paradigm Shift to Cognitive AI with Humanlike Common Sense”, Engineering, 2020. ೔ৗੜ׆Ͱਓ͕ؒར༻͍ͯ͠Δৗࣝతਪ࿦ [1]
  7. ͜Ε·Ͱͷݚڀ಺༰ 23 uੜ੒Ϟσϧͷܭࢉίετ࡟ݮ ü ͷߦྻܭࢉΛ࡟ݮ uڭࢣͳ͠Ծ૝ࢼண uڭࢣͳ͠લܠഎܠ෼཭ uৗࣝΛ֫ಘͨ͠"*ͷ։ൃ %"--&Ͱੜ੒ <>

    NJEKPVSOFZͰੜ੒ <> <>%"--&, “https://openai.com/index/dall-e-2/”, ೥݄೔Ӿཡ <>NJEKPVSOFZ lIUUQTXXXNJEKPVSOFZDPNIPNFz ೥݄೔Ӿཡ ੲʹੜ੒ͨ͠ͷͰɼ৘ใͱͯ͠͸ݹ͍Ͱ͢ɼɼɼ
  8. ·ͣ͸ະ஌ͷγφϦΦͰڧ͍ਓ͔ؒΒֶͿ 30 ਓ͕ؒະ஌ͷγφϦΦʹڧ͍ͷ͸ “the infinite use of finite means” [1]

    ͷೳྗ͕͋ΔͨΊ ݩʑ͸ݴޠֶͰChomskyઌੜ͓ͬ͠Όͬͨ͜ͱͰɼ༗ݶͷޠኮ͔ΒແݶͷจষΛ࡞ΕΔ͜ͱ͔Β ͜ͷೳྗࣗମ͸ܥ౷త൚ԽʢSystematic Generalizationʣ[2] ͱݺ͹Ε͍ͯΔ [1] N. Chomsky. “Aspect of the Theory of Syntax”, The MIT Press, 1965. [2] B. M. Lake and M. Baroni. “Human-like systematic generalization through a meta-learning neural network”, Nature, 2023. ܥ౷త൚Խͷྫ [2]
  9. ܥ౷త൚Խͱߏ੒ੑ 31 ܥ౷త൚ԽΛࢧ͍͑ͯΔͷ͸ߏ੒ੑʢCompositionalityʣͱ͍͏ݪଇʢPrincipleʣ ߏ੒ੑ͸3ͭͷݪଇͱ2ͭͷධՁج४͔Β੒͍ͬͯΔ [1] ʻݪଇʼ 1. ܥ౷ੑʢSystematicityʣ 2. ੜ࢈ੑʢProductivityʣ

    3. ୅ସੑʢSubstitutivityʣ ʻධՁج४ʼ 1. ہॴੑʢLocalityʣ 2. ա൚ԽʢOvergeneralizationʣ [1] D. Hupkes et al., “Compositionality Decomposed: How do Neural Networks Generalise?”, JAIR, 2020. ߏ੒ੑͷཁૉ [1]
  10. ܥ౷త൚Խͱߏ੒ੑ 32 ʻݪଇʼ 1. ܥ౷ੑʢSystematicityʣɿଐੑͷ૊Έ߹Θͤ΁ͷ൚Խ ߏ੒ੑͷཁૉ [1] [1] D. Hupkes

    et al., “Compositionality Decomposed: How do Neural Networks Generalise?”, JAIR, 2020.
  11. ܥ౷త൚Խͱߏ੒ੑ 33 ʻݪଇʼ 1. ܥ౷ੑʢSystematicityʣɿଐੑͷ૊Έ߹Θͤ΁ͷ൚Խ ʁʁʁʁʁʁʁ ʻط஌ʼ ʻط஌ʼ ʻະ஌ʼ ʻະ஌ʼ

    [1] D. Hupkes et al., “Compositionality Decomposed: How do Neural Networks Generalise?”, JAIR, 2020.
  12. ܥ౷త൚Խͱߏ੒ੑ 34 ʻݪଇʼ 2. ੜ࢈ੑʢProductivityʣɿະ஌ͷ௕͞ͷγʔέϯε΁ͷ൚Խ ߏ੒ੑͷཁૉ [1] [1] D. Hupkes

    et al., “Compositionality Decomposed: How do Neural Networks Generalise?”, JAIR, 2020.
  13. ܥ౷త൚Խͱߏ੒ੑ 36 ʻධՁج४ʼ 1. ہॴੑʢLocalityʣɿॲཧ͕ہॴత͔େҬత͔ʢϞσϧͷಛੑΛ஌ΔͨΊͷධՁج४ʣ จষʹΑͬͯ͸ɼہॴతେҬతͲͪΒͰॲཧͯ͠΋ ݁Ռ͕ಉ͡ʹͳΔ΋ͷ΍ɼͳΒͳ͍΋ͷ΋͋Δ จষͷྫɿʮJohn loves MaryʯͱʮMary

    loves Johnʯ จষߏ଄ʢSVOʣͱ͍͏େҬతʹݟΔͱಉҰ จষͷҙຯͱ͍͏ہॴతʹݟΔͱҧ͏ ਺ֶͷྫɿʮ14 – (2 + 3)ʯ ʮ14 – (2 + 3)ʯͱେҬతʹܭࢉͯ͠΋ ʮ14 – 5ʯͱہॴతʹܭࢉͯ͠΋݁Ռ͸ಉҰ ߏ੒ੑͷཁૉ [1] [1] D. Hupkes et al., “Compositionality Decomposed: How do Neural Networks Generalise?”, JAIR, 2020.
  14. ܥ౷త൚Խͱߏ੒ੑ 38 ʻݪଇʼ 1. ܥ౷ੑʢSystematicityʣ 2. ੜ࢈ੑʢProductivityʣ 3. ୅ସੑʢSubstitutivityʣ ʻධՁج४ʼ

    1. ہॴੑʢLocalityʣ 2. ա൚ԽʢOvergeneralizationʣ 3ͭͷݪଇͱ2ͭͷධՁج४Λ΋ͱʹɼ Ϟσϧ͕ͲͷΑ͏ͳੑ࣭Λ͍࣋ͬͯΔͷ͔Λ ଟ໘తʹධՁ͢Δ [1] D. Hupkes et al., “Compositionality Decomposed: How do Neural Networks Generalise?”, JAIR, 2020. ߏ੒ੑͷཁૉ [1]
  15. Contents l --.Ͱطʹ΍Δ͜ͱ͸ऴΘͬͯ͠·͍ͬͯΔʁ l ͲͷΑ͏ʹਓʹḷΓண͘ͷ͔ʁ l ߝౡͷݚڀऀͱͯ͠ͷىݯɼ͜Ε·Ͱͷݚڀ l ܥ౷త൚Խ l

    ܥ౷త൚ԽΤʔδΣϯτͷϕϯνϚʔΫɼख๏܈ɼ՝୊ ʢφΠʔϒͳܥ౷త൚Խ͸࣌ؒͷؔ܎Ͱলུʣ 39
  16. Contents l --.Ͱطʹ΍Δ͜ͱ͸ऴΘͬͯ͠·͍ͬͯΔʁ l ͲͷΑ͏ʹਓʹḷΓண͘ͷ͔ʁ l ߝౡͷݚڀऀͱͯ͠ͷىݯɼ͜Ε·Ͱͷݚڀ l ܥ౷త൚Խ l

    ܥ౷త൚ԽΤʔδΣϯτͷϕϯνϚʔΫɼख๏܈ɼ՝୊ ʢφΠʔϒͳܥ౷త൚Խ͸࣌ؒͷؔ܎Ͱলུʣ 40
  17. ܥ౷త൚ԽΤʔδΣϯτͷͨΊͷϕϯνϚʔΫ 41 [1] B. M. Lake and M. Baroni, “Generalization

    without Systematicity: On the Compositional Skills of Sequence-to-Sequence Recurrent Networks”, ICML, 2018. [2] B. M. Lake and M. Baroni. “Human-like systematic generalization through a meta-learning neural network”, Nature, 2023. 4$"/ <>ɿݴޠࢦࣔΛೖྗͱͯ͠ɼߦಈΛग़ྗ 4$"/ϕϯνϚʔΫͷ֓೦ਤ [2] 4$"/ϕϯνϚʔΫͷೖग़ྗྫ <>
  18. ܥ౷త൚ԽΤʔδΣϯτͷͨΊͷϕϯνϚʔΫ 42 [1] B. M. Lake and M. Baroni, “Generalization

    without Systematicity: On the Compositional Skills of Sequence-to-Sequence Recurrent Networks”, ICML, 2018. [2] B. M. Lake and M. Baroni. “Human-like systematic generalization through a meta-learning neural network”, Nature, 2023. 4$"/ <>ɿݴޠࢦࣔΛೖྗͱͯ͠ɼߦಈΛग़ྗ εϓϦοτ͸ͭ  ܥ౷ੑ  ੜ࢈ੑ  KVNQͳͲͷ୯ҰͷϓϦϛςΟϒͷΈΛݟ͍ͯΔ ΋ͷͰͷܥ౷ੑ  ӳޠ  ϑϥϯεޠͷػց຋༁ʹ͓͚Δܥ౷ੑ 4$"/ϕϯνϚʔΫͷ֓೦ਤ [2] 4$"/ϕϯνϚʔΫͷೖग़ྗྫ <>
  19. ܥ౷త൚ԽΤʔδΣϯτͷͨΊͷϕϯνϚʔΫ 43 [1] L. Ruis et al., “A Benchmark for

    Systematic Generalization in Grounded Language Understanding”, NeurIPS, 2020. gSCANσʔληοτ [1] H4$"/ HSPVOEFE 4$"/ <>ɿঢ়ଶ˞ͱݴޠࢦࣔΛೖྗͱͯ͠ɼߦಈΛग़ྗ ˞ ঢ়ଶͱ͸ɼڧԽֶश༻ޠͷʮঢ়ଶʢ4UBUFʣʯͰ͋Γɼਅͷ؍ଌ৘ใʢਤͰ͍͏൫໘৘ใʣ
  20. ܥ౷త൚ԽΤʔδΣϯτͷͨΊͷϕϯνϚʔΫ 44 [1] L. Ruis et al., “A Benchmark for

    Systematic Generalization in Grounded Language Understanding”, NeurIPS, 2020. gSCANσʔληοτ [1] H4$"/ HSPVOEFE 4$"/ <>ɿঢ়ଶ˞ͱݴޠࢦࣔΛೖྗͱͯ͠ɼߦಈΛग़ྗ ˞ ঢ়ଶͱ͸ɼڧԽֶश༻ޠͷʮঢ়ଶʢ4UBUFʣʯͰ͋Γɼਅͷ؍ଌ৘ใʢਤͰ͍͏൫໘৘ใʣ εϓϦοτ͸ͭ  ෺ମଐੑͷܥ౷ੑ  ෺ମଐੑͷܥ౷ੑύʔτ  ະ஌ͷҠಈํ޲  ະ஌ͷΦϒδΣΫτͷ૬ରతαΠζ  ෺ମͱߦಈؒͷܥ౷ੑ  ະ஌ͷ෭ࢺͷGFXTIPU MFBSOJOH  ෭ࢺΛಈࢺʹஔ׵  ੜ࢈ੑ
  21. ܥ౷త൚ԽΤʔδΣϯτͷͨΊͷϕϯνϚʔΫ 45 [1] Z. Wu et al., “ReaSCAN: Compositional Reasoning

    in Language Grounding”, NeurIPS Datasets and Benchmarks, 2021. ReaSCANσʔληοτ [1] 3FB4$"/ 3FBTPOJOHCBTFE 4$"/ <>ɿঢ়ଶͱݴޠࢦࣔΛೖྗͱͯ͠ɼߦಈΛग़ྗ
  22. ܥ౷త൚ԽΤʔδΣϯτͷͨΊͷϕϯνϚʔΫ 46 [1] Z. Wu et al., “ReaSCAN: Compositional Reasoning

    in Language Grounding”, NeurIPS Datasets and Benchmarks, 2021. ReaSCANσʔληοτ [1] 3FB4$"/ 3FBTPOJOHCBTFE 4$"/ <>ɿঢ়ଶͱݴޠࢦࣔΛೖྗͱͯ͠ɼߦಈΛग़ྗ H4$"/͸ͭͷ໰୊͕͋ͬͨ p ݴޠࢦࣔͷ૊Έ߹Θ͕ͤγϯϓϧ͗ͯ͢ จ຺ແࢹͷ#P8ʢ#BHPG8PSETʣͰे෼ͩͬͨ p ๦֐ΦϒδΣΫτ͕ܥ౷త൚ԽͷͨΊͷ ਖ਼֬ͳཧղʹ΄ͱΜͲد༩͍ͯ͠ͳ͔ͬͨ p ݴޠࢦࣔͷܗ༰ࢺ͕φϏήʔγϣϯʹ ͍Βͳ͍͜ͱ͕ଟ͔ͬͨ
  23. ܥ౷త൚ԽΤʔδΣϯτͷͨΊͷϕϯνϚʔΫ 47 [1] Z. Wu et al., “ReaSCAN: Compositional Reasoning

    in Language Grounding”, NeurIPS Datasets and Benchmarks, 2021. ReaSCANσʔληοτ [1] 3FB4$"/ 3FBTPOJOHCBTFE 4$"/ <>ɿঢ়ଶͱݴޠࢦࣔΛೖྗͱͯ͠ɼߦಈΛग़ྗ େ͖͘෼͚ͯεϓϦοτ͸ͭʢࡉ͔͍ͱͭʣ  ෺ମଐੑͷܥ౷ੑ  ঢ়ଶͱݴޠͷܥ౷ੑʢ৽ن෺ମ૊Έ߹Θͤʣ  ৽ن۟ߏ଄ʢੜ࢈ੑͱ۟ͷೖΕସ͑ʣ
  24. ܥ౷త൚ԽΤʔδΣϯτͷख๏ 50 l -45. TFRTFRɿγʔέϯεΛॱʑʹೖྗͯ͠ɼग़ྗΛ࠶ىతʹೖྗͱͯ͠ར༻ l $POW TFRTFR l 5SBOTGPSNFS

    l ($/-45. l .FUB 5SBOTGPSNFS -45. $POW 5SBOTGPSNFS<> 4FR4FRʢ4FRVFODFUP4FRVFODFʣ<> [1] D. Hupkes et al., “Compositionality Decomposed: How do Neural Networks Generalise?”, JAIR, 2020. [2] B. M. Lake and M. Baroni, “Generalization without Systematicity: On the Compositional Skills of Sequence-to-Sequence Recurrent Networks”, ICML, 2018.
  25. ܥ౷త൚ԽΤʔδΣϯτͷख๏ 51 l -45. TFRTFRɿγʔέϯεΛॱʑʹೖྗͯ͠ɼग़ྗΛ࠶ىతʹೖྗͱͯ͠ར༻ l $POW TFRTFRɿγʔέϯεͰඃΔΑ͏ʹ৞ΈࠐΈʢͻͱੲલʹݴޠॲཧͰྲྀߦʣ l 5SBOTGPSNFS

    l ($/-45. l .FUB 5SBOTGPSNFS -45. $POW 5SBOTGPSNFS<> [1] D. Hupkes et al., “Compositionality Decomposed: How do Neural Networks Generalise?”, JAIR, 2020. [2] B. M. Lake and M. Baroni, “Generalization without Systematicity: On the Compositional Skills of Sequence-to-Sequence Recurrent Networks”, ICML, 2018.
  26. ܥ౷త൚ԽΤʔδΣϯτͷख๏ 52 l -45. TFRTFRɿγʔέϯεΛॱʑʹೖྗͯ͠ɼग़ྗΛ࠶ىతʹೖྗͱͯ͠ར༻ l $POW TFRTFRɿγʔέϯεͰඃΔΑ͏ʹ৞ΈࠐΈʢͻͱੲલʹݴޠॲཧͰྲྀߦʣ l 5SBOTGPSNFSɿγʔέϯεΛ.-1ͱ"UUFOUJPOͰॲཧ͠ɼԕ͍γʔέϯεͰ΋ߟྀ͕༻ҙ

    l ($/-45. l .FUB 5SBOTGPSNFS -45. $POW 5SBOTGPSNFS<> 5SBOTGPSNFSΞʔΩςΫνϟ <> [1] D. Hupkes et al., “Compositionality Decomposed: How do Neural Networks Generalise?”, JAIR, 2020. [2] B. M. Lake and M. Baroni, “Generalization without Systematicity: On the Compositional Skills of Sequence-to-Sequence Recurrent Networks”, ICML, 2018. [3] A. Vaswani et al., “Attention Is All You Need”, NIPS, 2017.
  27. ܥ౷త൚ԽΤʔδΣϯτͷख๏ 53 [1] D. Hupkes et al., “Compositionality Decomposed: How

    do Neural Networks Generalise?”, JAIR, 2020. [2] B. M. Lake and M. Baroni, “Generalization without Systematicity: On the Compositional Skills of Sequence-to-Sequence Recurrent Networks”, ICML, 2018. [3] A. Vaswani et al., “Attention Is All You Need”, NIPS, 2017. [4] T. Gao et al., “Systematic Generalization on gSCAN with Language Conditioned Embedding”, AACL, 2020. l -45. TFRTFRɿγʔέϯεΛॱʑʹೖྗͯ͠ɼग़ྗΛ࠶ىతʹೖྗͱͯ͠ར༻ l $POW TFRTFRɿγʔέϯεͰඃΔΑ͏ʹ৞ΈࠐΈʢͻͱੲલʹݴޠॲཧͰྲྀߦʣ l 5SBOTGPSNFSɿγʔέϯεΛ.-1ͱ"UUFOUJPOͰॲཧ͠ɼԕ͍γʔέϯεͰ΋ߟྀ͕༻ҙ l ($/-45.ɿάϥϑ৞ΈࠐΈΛ༻͍ͯݴޠͱঢ়ଶͷؔ܎ੑߟྀ͢ΔH4$"/Ͱͷ405" l .FUB 5SBOTGPSNFS ($/-45.ͷΞʔΩςΫνϟ <>
  28. ܥ౷త൚ԽΤʔδΣϯτͷख๏ 54 [1] D. Hupkes et al., “Compositionality Decomposed: How

    do Neural Networks Generalise?”, JAIR, 2020. [2] B. M. Lake and M. Baroni, “Generalization without Systematicity: On the Compositional Skills of Sequence-to-Sequence Recurrent Networks”, ICML, 2018. [3] A. Vaswani et al., “Attention Is All You Need”, NIPS, 2017. [4] T. Gao et al., “Systematic Generalization on gSCAN with Language Conditioned Embedding”, AACL, 2020. [5] B. M. Lake and M. Baroni. “Human-like systematic generalization through a meta-learning neural network”, Nature, 2023. l -45. TFRTFRɿγʔέϯεΛॱʑʹೖྗͯ͠ɼग़ྗΛ࠶ىతʹೖྗͱͯ͠ར༻ l $POW TFRTFRɿγʔέϯεͰඃΔΑ͏ʹ৞ΈࠐΈʢͻͱੲલʹݴޠॲཧͰྲྀߦʣ l 5SBOTGPSNFSɿγʔέϯεΛ.-1ͱ"UUFOUJPOͰॲཧ͠ɼԕ͍γʔέϯεͰ΋ߟྀ͕༻ҙ l ($/-45.ɿάϥϑ৞ΈࠐΈΛ༻͍ͯݴޠͱঢ়ଶͷؔ܎ੑߟྀ͢ΔH4$"/Ͱͷ405" l .FUB 5SBOTGPSNFSɿGFXTIPU FYBNQMFTͷઃఆͷ5SBOTGPSNFS ݱঢ়ɼ5SBOTGPSNFSܥ͕࠷΋༗๬ .FUB5SBOTGPSNFS<>
  29. ܥ౷త൚ԽΤʔδΣϯτͷ՝୊ͱࠓޙͷల๬ 55 [1] D. Bahdanau et al., “Systematic Generalization: What

    Is Required and Can It Be Learned?”, ICLR, 2018. [2] R. Csordás et al., “The Devil is in the Detail: Simple Tricks Improve Systematic Generalization of Transformers”, EMNLP, 2021. [3] R. Kirk et al., “A Survey of Zero-shot Generalisation in Deep Reinforcement Learning”, JAIR, 2023. l طଘͷϕϯνϚʔΫ͸ݱ࣮ੈքͷઃఆʹଈ͍ͯ͠ͳ͍ʢਅͷঢ়ଶ͸औಘͰ͖ͳ͍ʣ l طଘͷϕϯνϚʔΫͰఏҊ͞Εͨख๏Ͱ͢Βɼܥ౷త൚ԽΤʔδΣϯτͷ࣮ݱʹ͸ఔԕ͍ l ϕϯνϚʔΫʹ͓͍ͯɼ܇࿅ޡࠩͱςετੑೳʹ૬͕ؔݟΒΕͳ͍ [1, 2] Ø ܇࿅ޡࠩͱݕূޡ͕ࠩখ͘͞ͳ͔ͬͨΒͱ͍ͬͯςετੑೳ͕ߴ͍ͱ͸ݶΒͳ͍ l ܥ౷త൚Խͷઌ͸θϩγϣοτ൚ԽʢZSG; Zero-shot Generalizationʣ[3] Ø ZSG͸ݪཧతʹղ͚ͳ͍͸ͣͰɼڧ͍ؼೲόΠΞε΍ΠϯλϥΫγϣϯ͕ඞཁ ZSGͷ֓೦ਤ [3]