「人間にAIはどのように辿り着けばよいのか？ー 系統的汎化からの第一歩 ー」＠第22回 Language and Robotics研究会

第22回 Language and Robotics研究会にて発表した「人間にAIはどのように辿り着けばよいのか？ー 系統的汎化からの第一歩 ー」の発表資料になります．

概要：大規模言語モデルの台頭により，人類が夢見ていた汎用人工知能は単なる夢物語ではなく，現実のものとして近づいてきた．汎用人工知能の先には，人間の知能を超えた人工超知能があり，人工超知能の実現も少しずつ現実味を帯び始めている．本講演では，まずは人間の知能に辿り着くという目標の上で，どのような道筋を辿っていくのがよいのかについてを論じる．そして，具体的な第一歩としての系統的汎化について，これまでの歴史を振り返りながら最新のベンチマークと手法，現状の課題を紹介する．

Transcript

1. ૣҴాେֶ ৿ౡൟੜݚڀࣨ D5 ߝౡलथ ਓؒʹ"*͸ͲͷΑ͏ʹḷΓண͚͹Α͍ͷ͔ʁ ʔ ܥ౷త൚Խ͔ΒͷୈҰา ʔ
2. ࣗݾ঺հ 2 ߝౡ लथ uॴଐ ૣҴాେֶ ത࢜5೥ʵ৿ౡൟੜݚڀࣨ uݚڀςʔϚ म࢜՝ఔɿਂ૚ը૾ੜ੒Ϟσϧͷܭࢉྔɾύϥϝʔλ࡟ݮ ത࢜՝ఔ1೥ɿෳ਺෺ମΛର৅ͱͨ͠ڭࢣແ͠લܠഎܠ෼ղ ത࢜՝ఔ2೥ɿEmbodied AIؔ࿈ ത࢜՝ఔ3೥ʙɿৗࣝ֫ಘɼܥ౷త൚Խ ࢈૯ݚʢݩʣɿ෰ͱਓͷϖΞσʔλΛඞཁͱ͠ͳ͍Ծ૝ࢼண uझຯ ےτϨɼΞϝϑτɼಡॻɼԻָήʔϜɼ'14ɼ ΰϧϑɼҿञɼμʔπɼϏϦϠʔυɼࣸਅɼFUD
3. ະ஌γφϦΦͰ൚Խ͢ΔΤʔδΣϯτΛͲ͏࣮ݱ͢Δ͔
4. ਓΛ௒͑ͨ"*ͱ͸ʁ
5. ௒஌ೳ 5 <>
   ਓ޻௒஌ೳͱ͸
   lIUUQTXXXJCNDPNKQKBUPQJDTBSUJGJDJBMTVQFSJOUFMMJHFODFz
   ೥݄೔Ӿཡ <>
   ڧ͍"*ͱ͸
   lIUUQTXXXJCNDPNKQKBUPQJDTTUSPOHBJz
   ೥݄೔Ӿཡ ਓ޻௒஌ೳʢ"SUJGJDJBM 4VQFS *OUFMMJHFODF "4*ʣ͸ਓؒͷ஌ೳΛ௒͑ͨ஌తൣғΛ΋ͭ ਓ޻஌ೳγεςϜͷҰͭʢԾ૝తଘࡏʣ<> ݱࡏͷ"*͸ऑ͍"*ͱݺ͹ΕɼಛఆͷλεΫʹ͓͍ͯ༏ΕͨੑೳΛൃش͢Δ"* <> "(*ʢ"SUJGJDJBM (FOFSBM *OUFMMJHFODFʣ͸ڧ͍"*ͱݺ͹Εɼਓؒͱಉ౳ͷҙࣝΛ࣋ͪɼਓؒͱಉ౳ͷ ॊೈੑ΍൚ԽੑΛ࣋ͪ߹ΘͤΔ"*ʢఆٛͱͯ͠·ͩ֬ఆ͸͍ͯ͠ͳ͍ʣ<> ྫ͑Ͱݴ͏ͱɼλʔϛωʔλʔʹొ৔͢ΔϚβʔ"*ͷʮεΧΠωοτʯ͕"4*ͷҰྫ ʢͨͩɼޙड़ͷཧ༝ʹΑΓएׯ"(*دΓ͔΋ʁʣ
6. ௒஌ೳ 6 ᷿ͰᷚͷγϯΪϡϥϦςΟʢٕज़తಛҟ఺ʣͱ͸ɼਓؒͷ஌ೳΛ௒ӽͨ͠"4*͕ɼࣗ਎Λ௒͑Δ "44*ʢ"SUJGJDJBM 4VQFS 4VQFS *OUFMMJHFODF˞ʣΛੜΈग़ࣗ͢ݾվળϧʔϓ͕࢝·Δ࣌ ˞ ߝౡͷ଄ޠ ͨͩɼ͜Ε͸·ͩເ෺ޠʹ͗͢ͳ͍ ͯ͞ɼզʑ͕ࠓ໨ࢦ͢ઌ͸ʁ
7. ະ஌γφϦΦͰ൚Խ͢ΔΤʔδΣϯτΛͲ͏࣮ݱ͢Δ͔
8. ະ஌γφϦΦͰ൚Խ͢ΔΤʔδΣϯτΛͲ͏࣮ݱ͢Δ͔ ·ͣ͸ɼਓʹḷΓண͘
9. Contents l --.Ͱطʹ΍Δ͜ͱ͸ऴΘͬͯ͠·͍ͬͯΔʁ l ͲͷΑ͏ʹਓʹḷΓண͘ͷ͔ʁ l ߝౡͷݚڀऀͱͯ͠ͷىݯɼ͜Ε·Ͱͷݚڀ l ܥ౷త൚Խ l ܥ౷త൚ԽΤʔδΣϯτͷϕϯνϚʔΫɼख๏܈ɼ՝୊ 9
10. Contents l --.Ͱطʹ΍Δ͜ͱ͸ऴΘͬͯ͠·͍ͬͯΔʁ l ͲͷΑ͏ʹਓʹḷΓண͘ͷ͔ʁ l ߝౡͷݚڀऀͱͯ͠ͷىݯɼ͜Ε·Ͱͷݚڀ l ܥ౷త൚Խ l ܥ౷త൚ԽΤʔδΣϯτͷϕϯνϚʔΫɼख๏܈ɼ՝୊ 10
11. Q. ͱ͜ΖͰɼLLMͰશͯऴΘΓʁ 11 A. ͦΜͳ͜ͱ͸ͳ͍ େن໛ݴޠϞσϧʢLLMʣ͸֬཰తΦ΢Ϝͱᎏ᎐͞ΕΔΑ͏ʹɼͦΕͬΆ͍୯ޠΛฦ͍ͯ͠Δ͚ͩ [1] ʢΠϝʔδͱͯ͠͸૴ૹͷϑϦʔϨϯͷຐ଒ͷΑ͏ͳײ͡ʣ GPT-4Ͱ͢Β͜ͷ܏޲͋Γ [1] E. M. Bender et al., “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜”, FAccT, 2021. [2] ૴ૹͷϑϦʔϨϯXެࣜΞΧ΢ϯτ, https://twitter.com/FRIEREN_PR/status/1772020566293111290/photo/1, 2024೥4݄18೔Ӿཡ. ૴ૹͷϑϦʔϨϯɼຐ଒ʮஅ಄୆ͷΞ΢ϥʯҰ෦վม <>
12. Q. ͱ͜ΖͰɼLLMͰશͯऴΘΓʁ 12 A. ͦΜͳ͜ͱ͸ͳ͍ LLM͕େن໛ʹͳΕ͹ͳΔ΄Ͳੑೳ͕௿Լ͢ΔInverse Scaling Prize [1] ΍ɼ ΦʔϓϯϘΩϟϒϥϦʔͷ਎ମΛ࣋ͬͨΤʔδΣϯτʹΑΔ࣭໰Ԡ౴ͷOpenEQA [2]ͳͲɼ ·ͩ·ͩLLM͸ෳࡶͳਪ࿦͕ۤखͰ͋ͬͨΓɼLLMಛ༗ͷؒҧ͍Λͨ͠Γ͢Δ [1] Inverse Scaling Prize, https://huggingface.co/inverse-scaling. [2] A. Majumdar et al., “OpenEQA: Embodied Question Answering in the Era of Foundation Models”, Preprint, 2024. OpenEQAͷਓͱLLMͷਖ਼౴཰ൺֱ [2]
13. Contents l --.Ͱطʹ΍Δ͜ͱ͸ऴΘͬͯ͠·͍ͬͯΔʁ l ͲͷΑ͏ʹਓʹḷΓண͘ͷ͔ʁ l ߝౡͷݚڀऀͱͯ͠ͷىݯɼ͜Ε·Ͱͷݚڀ l ܥ౷త൚Խ l ܥ౷త൚ԽΤʔδΣϯτͷϕϯνϚʔΫɼख๏܈ɼ՝୊ 13
14. Contents l --.Ͱطʹ΍Δ͜ͱ͸ऴΘͬͯ͠·͍ͬͯΔʁ l ͲͷΑ͏ʹਓʹḷΓண͘ͷ͔ʁ l ߝౡͷݚڀऀͱͯ͠ͷىݯɼ͜Ε·Ͱͷݚڀ l ܥ౷త൚Խ l ܥ౷త൚ԽΤʔδΣϯτͷϕϯνϚʔΫɼख๏܈ɼ՝୊ 14
15. ਓͷ೴ߏ଄ΛਅࣅΔʁೳྗΛਅࣅΔʁ 15 ೴ߏ଄ͱಉ͡Ͱ΋ಉ͡ೳྗ͸ൃݱ͠ͳ͍ʢֶशํ๏ͱσʔλΛἧ͑ͯ΋ൃݱ͠ͳ͍ʣ ೴ߏ଄Λ໛฿ͨ͠NN [1] [1] R. P. Rane et al., “PredNet and Predictive Coding: A Critical Review”, ICMR, 2020. [2] D. Lee et al., “Difference Target Propagation”, ECML/PKDD. 2015. [3] J. Sullivan et al., “SAYCam: A Large, Longitudinal Audiovisual Dataset Recorded From the Infant’s Perspective”, Open Mind, 2021. [4] A. E. Orhan et al., “Self-supervised learning through the eyes of a child”, NeurIPS, 2020. ੜ෺ֶతʹଥ౰ͳֶशํ๏ [2] ༮ࣇͷҰਓশࢹ఺ಈը [3, 4]
16. ਓͷ೴ߏ଄ΛਅࣅΔʁೳྗΛਅࣅΔʁ 16 ਓͷೝ஌ػೳ΍ೳྗΛਅࣅΔ͜ͱ͕λεΫղܾʹ௚݁͢Δʢखஈ͸໰Θͳ͍ʣ [1] Y. Zhu et al., “Dark, Beyond Deep: A Paradigm Shift to Cognitive AI with Humanlike Common Sense”, Engineering, 2020. ೔ৗੜ׆Ͱਓ͕ؒར༻͍ͯ͠Δৗࣝతਪ࿦ [1]
17. Contents l --.Ͱطʹ΍Δ͜ͱ͸ऴΘͬͯ͠·͍ͬͯΔʁ l ͲͷΑ͏ʹਓʹḷΓண͘ͷ͔ʁ l ߝౡͷݚڀऀͱͯ͠ͷىݯɼ͜Ε·Ͱͷݚڀ l ܥ౷త൚Խ l ܥ౷త൚ԽΤʔδΣϯτͷϕϯνϚʔΫɼख๏܈ɼ՝୊ 17
18. Contents l --.Ͱطʹ΍Δ͜ͱ͸ऴΘͬͯ͠·͍ͬͯΔʁ l ͲͷΑ͏ʹਓʹḷΓண͘ͷ͔ʁ l ߝౡͷݚڀऀͱͯ͠ͷىݯɼ͜Ε·Ͱͷݚڀ l ܥ౷త൚Խ l ܥ౷త൚ԽΤʔδΣϯτͷϕϯνϚʔΫɼख๏܈ɼ՝୊ 18
19. ݚڀऀΛࢤ͖͔͚ͨͬ͠ 19 ιʔυΞʔτɾΦϯϥΠϯʢ4"0ʣ [1] ΞχϝʰιʔυΞʔτɾΦϯϥΠϯʱ5XJUUFSϑΥϩϫʔສਓه೦σδίϯϓϨθϯτʂ, “https://sao10th.net/digicon_present/”, ೥݄೔Ӿཡ ιʔυΞʔτɾΦϯϥΠϯ <>
20. ݚڀऀΛࢤ͖͔͚ͨͬ͠ 20 ιʔυΞʔτɾΦϯϥΠϯʢ4"0ʣ ήʔϜͷੈքͰͷࢮ͕ɼݱ࣮ੈքͰͷࢮͱ ͳΔσεήʔϜʹด͡ࠐΊΒΕΔͱ͜Ζ͔Β ࢝·Δ͓࿩ 4"0ͷதͷਓ޻஌ೳͷʮϢΠʯͱ͍͏Ωϟϥ ͕͓Γɼਓؒͱ"*ͷڞ૑ʹՄೳੑΛײͨ͡ ʢߴߍ೥ੜ ೥ʣ
21. ݚڀऀΛࢤ͖͔͚ͨͬ͠ 21 ιʔυΞʔτɾΦϯϥΠϯʢ4"0ʣ 4"0ͷதͷਓ޻஌ೳͷʮϢΠʯͱ͍͏Ωϟϥ ͕͓Γɼਓؒͱ"*ͷڞ૑ʹՄೳੑΛײͨ͡ ʢߴߍ೥ੜ ೥ʣ ಉ೥ʹ(PPHMFͷೣʢਂ૚ֶशʹΑͬͯೣͷ ೝ͕ࣝͰ͖ΔΑ͏ʹͳͬͨ࿩ʣ΋ొ৔͠ɼ ΑΓҰ૚"*΁ͷಌΕ͕ڧ͘ͳͬͨ
22. ͜Ε·Ͱͷݚڀ಺༰ 22 uੜ੒Ϟσϧͷܭࢉίετ࡟ݮ uڭࢣͳ͠Ծ૝ࢼண uڭࢣͳ͠લܠഎܠ෼཭ uৗࣝΛ֫ಘͨ͠"*ͷ։ൃ
23. ͜Ε·Ͱͷݚڀ಺༰ 23 uੜ੒Ϟσϧͷܭࢉίετ࡟ݮ ü ͷߦྻܭࢉΛ࡟ݮ uڭࢣͳ͠Ծ૝ࢼண uڭࢣͳ͠લܠഎܠ෼཭ uৗࣝΛ֫ಘͨ͠"*ͷ։ൃ %"--&
    Ͱੜ੒ <> NJEKPVSOFZͰੜ੒ <> <>
    %"--&
    , “https://openai.com/index/dall-e-2/”, ೥݄೔Ӿཡ <>
    NJEKPVSOFZ
    lIUUQTXXXNJEKPVSOFZDPNIPNFz
    ೥݄೔Ӿཡ ੲʹੜ੒ͨ͠ͷͰɼ৘ใͱͯ͠͸ݹ͍Ͱ͢ɼɼɼ
24. ͜Ε·Ͱͷݚڀ಺༰ 24 uੜ੒Ϟσϧͷܭࢉίετ࡟ݮ uڭࢣͳ͠Ծ૝ࢼண ü ڭࢣσʔλΛ༻͍ͣɼڭࢣ͋Γख๏Λ྇կ uڭࢣͳ͠લܠഎܠ෼཭ uৗࣝΛ֫ಘͨ͠"*ͷ։ൃ
25. ͜Ε·Ͱͷݚڀ಺༰ 25 uੜ੒Ϟσϧͷܭࢉίετ࡟ݮ uڭࢣͳ͠Ծ૝ࢼண uڭࢣͳ͠લܠഎܠ෼཭ ü ֶश҆ఆੑͷ޲্ uৗࣝΛ֫ಘͨ͠"*ͷ։ൃ
26. ͜Ε·Ͱͷݚڀ಺༰ 26 uੜ੒Ϟσϧͷܭࢉίετ࡟ݮ uڭࢣͳ͠Ծ૝ࢼண uڭࢣͳ͠લܠഎܠ෼཭ uৗࣝΛ֫ಘͨ͠"*ͷ։ൃ
27. ͜Ε·Ͱͷݚڀ಺༰ 27 uੜ੒Ϟσϧͷܭࢉίετ࡟ݮ uڭࢣͳ͠Ծ૝ࢼண uڭࢣͳ͠લܠഎܠ෼཭ uৗࣝΛ֫ಘͨ͠"*ͷ։ൃ ҰݟͲΕ΋ؔ܎ͳͦ͞͏ͳݚڀʹݟ͑Δ͕ɼɼɼ ӡಈ੍ޚʹؔ͢Δ಺෦ϞσϧͰ͋Δੜ੒ϞσϧΛ༗͢Δখ೴Λϕʔεʹ͍ͯ͠Δ ʢݩʑ͸೴Λ࠶ݱ͢Δશ೴ΞʔΩςΫνϟతࢥ૝ʣ
28. Contents l --.Ͱطʹ΍Δ͜ͱ͸ऴΘͬͯ͠·͍ͬͯΔʁ l ͲͷΑ͏ʹਓʹḷΓண͘ͷ͔ʁ l ߝౡͷݚڀऀͱͯ͠ͷىݯɼ͜Ε·Ͱͷݚڀ l ܥ౷త൚Խ l ܥ౷త൚ԽΤʔδΣϯτͷϕϯνϚʔΫɼख๏܈ɼ՝୊ 28
29. Contents l --.Ͱطʹ΍Δ͜ͱ͸ऴΘͬͯ͠·͍ͬͯΔʁ l ͲͷΑ͏ʹਓʹḷΓண͘ͷ͔ʁ l ߝౡͷݚڀऀͱͯ͠ͷىݯɼ͜Ε·Ͱͷݚڀ l ܥ౷త൚Խ l ܥ౷త൚ԽΤʔδΣϯτͷϕϯνϚʔΫɼख๏܈ɼ՝୊ 29
30. ·ͣ͸ະ஌ͷγφϦΦͰڧ͍ਓ͔ؒΒֶͿ 30 ਓ͕ؒະ஌ͷγφϦΦʹڧ͍ͷ͸ “the infinite use of finite means” [1] ͷೳྗ͕͋ΔͨΊ ݩʑ͸ݴޠֶͰChomskyઌੜ͓ͬ͠Όͬͨ͜ͱͰɼ༗ݶͷޠኮ͔ΒແݶͷจষΛ࡞ΕΔ͜ͱ͔Β ͜ͷೳྗࣗମ͸ܥ౷త൚ԽʢSystematic Generalizationʣ[2] ͱݺ͹Ε͍ͯΔ [1] N. Chomsky. “Aspect of the Theory of Syntax”, The MIT Press, 1965. [2] B. M. Lake and M. Baroni. “Human-like systematic generalization through a meta-learning neural network”, Nature, 2023. ܥ౷త൚Խͷྫ [2]
31. ܥ౷త൚Խͱߏ੒ੑ 31 ܥ౷త൚ԽΛࢧ͍͑ͯΔͷ͸ߏ੒ੑʢCompositionalityʣͱ͍͏ݪଇʢPrincipleʣ ߏ੒ੑ͸3ͭͷݪଇͱ2ͭͷධՁج४͔Β੒͍ͬͯΔ [1] ʻݪଇʼ 1. ܥ౷ੑʢSystematicityʣ 2. ੜ࢈ੑʢProductivityʣ 3. ୅ସੑʢSubstitutivityʣ ʻධՁج४ʼ 1. ہॴੑʢLocalityʣ 2. ա൚ԽʢOvergeneralizationʣ [1] D. Hupkes et al., “Compositionality Decomposed: How do Neural Networks Generalise?”, JAIR, 2020. ߏ੒ੑͷཁૉ [1]
32. ܥ౷త൚Խͱߏ੒ੑ 32 ʻݪଇʼ 1. ܥ౷ੑʢSystematicityʣɿଐੑͷ૊Έ߹Θͤ΁ͷ൚Խ ߏ੒ੑͷཁૉ [1] [1] D. Hupkes et al., “Compositionality Decomposed: How do Neural Networks Generalise?”, JAIR, 2020.
33. ܥ౷త൚Խͱߏ੒ੑ 33 ʻݪଇʼ 1. ܥ౷ੑʢSystematicityʣɿଐੑͷ૊Έ߹Θͤ΁ͷ൚Խ ʁʁʁʁʁʁʁ ʻط஌ʼ ʻط஌ʼ ʻະ஌ʼ ʻະ஌ʼ [1] D. Hupkes et al., “Compositionality Decomposed: How do Neural Networks Generalise?”, JAIR, 2020.
34. ܥ౷త൚Խͱߏ੒ੑ 34 ʻݪଇʼ 2. ੜ࢈ੑʢProductivityʣɿະ஌ͷ௕͞ͷγʔέϯε΁ͷ൚Խ ߏ੒ੑͷཁૉ [1] [1] D. Hupkes et al., “Compositionality Decomposed: How do Neural Networks Generalise?”, JAIR, 2020.
35. ܥ౷త൚Խͱߏ੒ੑ 35 ʻݪଇʼ 3. ୅ସੑʢSubstitutivityʣɿྨࣅͷݴ༿ʢه߸ʣͷஔ׵΁ͷ൚Խ ͋Δจষʹ͓͍ͯɼྨࣅͷݴ༿͕ஔ͖׵͑ΒΕͯ΋ จষͷҙຯ͕มԽ͠ͳ͍ ίϯςΩετΛؚΊ্ͨͰͷҙຯཧղ͕Ͱ͖͍ͯΔ ೔ຊޠͰͷྫɿʮΑ͘ߟ͑Δʯͱʮख़ྀʯ ߏ੒ੑͷཁૉ [1] [1] D. Hupkes et al., “Compositionality Decomposed: How do Neural Networks Generalise?”, JAIR, 2020.
36. ܥ౷త൚Խͱߏ੒ੑ 36 ʻධՁج४ʼ 1. ہॴੑʢLocalityʣɿॲཧ͕ہॴత͔େҬత͔ʢϞσϧͷಛੑΛ஌ΔͨΊͷධՁج४ʣ จষʹΑͬͯ͸ɼہॴతେҬతͲͪΒͰॲཧͯ͠΋ ݁Ռ͕ಉ͡ʹͳΔ΋ͷ΍ɼͳΒͳ͍΋ͷ΋͋Δ จষͷྫɿʮJohn loves MaryʯͱʮMary loves Johnʯ จষߏ଄ʢSVOʣͱ͍͏େҬతʹݟΔͱಉҰ จষͷҙຯͱ͍͏ہॴతʹݟΔͱҧ͏ ਺ֶͷྫɿʮ14 – (2 + 3)ʯ ʮ14 – (2 + 3)ʯͱେҬతʹܭࢉͯ͠΋ ʮ14 – 5ʯͱہॴతʹܭࢉͯ͠΋݁Ռ͸ಉҰ ߏ੒ੑͷཁૉ [1] [1] D. Hupkes et al., “Compositionality Decomposed: How do Neural Networks Generalise?”, JAIR, 2020.
37. ܥ౷త൚Խͱߏ੒ੑ 37 ʻධՁج४ʼ 2. ա৒൚ԽʢOvergeneraliationʣɿྫ֎ʹطଘͷϧʔϧΛద༻͢Δ͔ ݴޠͷྫɿʮgoʯͱʮgoedʯͱʮwentʯ ӳޠ͸جຊతʹաڈܗͷ৔߹͸-edΛ͚ͭΔ͕ɼ goͷྫ֎έʔεͰ͸ɼgoedͰ͸ͳ͘wentʹͳΔ ֶशதʹgo͕ग़͖ͯͨ৔߹ʹطଘͷϧʔϧͷ-edΛ ͚ͭͯgoedʹͨ͠ͳΒ͹ɼϧʔϧΛ֫ಘͰ͖͍ͯΔ ͱղऍͰ͖Δ ߏ੒ੑͷཁૉ [1] [1] D. Hupkes et al., “Compositionality Decomposed: How do Neural Networks Generalise?”, JAIR, 2020.
38. ܥ౷త൚Խͱߏ੒ੑ 38 ʻݪଇʼ 1. ܥ౷ੑʢSystematicityʣ 2. ੜ࢈ੑʢProductivityʣ 3. ୅ସੑʢSubstitutivityʣ ʻධՁج४ʼ 1. ہॴੑʢLocalityʣ 2. ա൚ԽʢOvergeneralizationʣ 3ͭͷݪଇͱ2ͭͷධՁج४Λ΋ͱʹɼ Ϟσϧ͕ͲͷΑ͏ͳੑ࣭Λ͍࣋ͬͯΔͷ͔Λ ଟ໘తʹධՁ͢Δ [1] D. Hupkes et al., “Compositionality Decomposed: How do Neural Networks Generalise?”, JAIR, 2020. ߏ੒ੑͷཁૉ [1]
39. Contents l --.Ͱطʹ΍Δ͜ͱ͸ऴΘͬͯ͠·͍ͬͯΔʁ l ͲͷΑ͏ʹਓʹḷΓண͘ͷ͔ʁ l ߝౡͷݚڀऀͱͯ͠ͷىݯɼ͜Ε·Ͱͷݚڀ l ܥ౷త൚Խ l ܥ౷త൚ԽΤʔδΣϯτͷϕϯνϚʔΫɼख๏܈ɼ՝୊ ʢφΠʔϒͳܥ౷త൚Խ͸࣌ؒͷؔ܎Ͱলུʣ 39
40. Contents l --.Ͱطʹ΍Δ͜ͱ͸ऴΘͬͯ͠·͍ͬͯΔʁ l ͲͷΑ͏ʹਓʹḷΓண͘ͷ͔ʁ l ߝౡͷݚڀऀͱͯ͠ͷىݯɼ͜Ε·Ͱͷݚڀ l ܥ౷త൚Խ l ܥ౷త൚ԽΤʔδΣϯτͷϕϯνϚʔΫɼख๏܈ɼ՝୊ ʢφΠʔϒͳܥ౷త൚Խ͸࣌ؒͷؔ܎Ͱলུʣ 40
41. ܥ౷త൚ԽΤʔδΣϯτͷͨΊͷϕϯνϚʔΫ 41 [1] B. M. Lake and M. Baroni, “Generalization without Systematicity: On the Compositional Skills of Sequence-to-Sequence Recurrent Networks”, ICML, 2018. [2] B. M. Lake and M. Baroni. “Human-like systematic generalization through a meta-learning neural network”, Nature, 2023. 4$"/ <>ɿݴޠࢦࣔΛೖྗͱͯ͠ɼߦಈΛग़ྗ 4$"/ϕϯνϚʔΫͷ֓೦ਤ [2] 4$"/ϕϯνϚʔΫͷೖग़ྗྫ <>
42. ܥ౷త൚ԽΤʔδΣϯτͷͨΊͷϕϯνϚʔΫ 42 [1] B. M. Lake and M. Baroni, “Generalization without Systematicity: On the Compositional Skills of Sequence-to-Sequence Recurrent Networks”, ICML, 2018. [2] B. M. Lake and M. Baroni. “Human-like systematic generalization through a meta-learning neural network”, Nature, 2023. 4$"/ <>ɿݴޠࢦࣔΛೖྗͱͯ͠ɼߦಈΛग़ྗ εϓϦοτ͸ͭ  ܥ౷ੑ  ੜ࢈ੑ  KVNQͳͲͷ୯ҰͷϓϦϛςΟϒͷΈΛݟ͍ͯΔ ΋ͷͰͷܥ౷ੑ  ӳޠ  ϑϥϯεޠͷػց຋༁ʹ͓͚Δܥ౷ੑ 4$"/ϕϯνϚʔΫͷ֓೦ਤ [2] 4$"/ϕϯνϚʔΫͷೖग़ྗྫ <>
43. ܥ౷త൚ԽΤʔδΣϯτͷͨΊͷϕϯνϚʔΫ 43 [1] L. Ruis et al., “A Benchmark for Systematic Generalization in Grounded Language Understanding”, NeurIPS, 2020. gSCANσʔληοτ [1] H4$"/ HSPVOEFE 4$"/ <>ɿঢ়ଶ˞ͱݴޠࢦࣔΛೖྗͱͯ͠ɼߦಈΛग़ྗ ˞ ঢ়ଶͱ͸ɼڧԽֶश༻ޠͷʮঢ়ଶʢ4UBUFʣʯͰ͋Γɼਅͷ؍ଌ৘ใʢਤͰ͍͏൫໘৘ใʣ
44. ܥ౷త൚ԽΤʔδΣϯτͷͨΊͷϕϯνϚʔΫ 44 [1] L. Ruis et al., “A Benchmark for Systematic Generalization in Grounded Language Understanding”, NeurIPS, 2020. gSCANσʔληοτ [1] H4$"/ HSPVOEFE 4$"/ <>ɿঢ়ଶ˞ͱݴޠࢦࣔΛೖྗͱͯ͠ɼߦಈΛग़ྗ ˞ ঢ়ଶͱ͸ɼڧԽֶश༻ޠͷʮঢ়ଶʢ4UBUFʣʯͰ͋Γɼਅͷ؍ଌ৘ใʢਤͰ͍͏൫໘৘ใʣ εϓϦοτ͸ͭ  ෺ମଐੑͷܥ౷ੑ  ෺ମଐੑͷܥ౷ੑύʔτ  ະ஌ͷҠಈํ޲  ະ஌ͷΦϒδΣΫτͷ૬ରతαΠζ  ෺ମͱߦಈؒͷܥ౷ੑ  ະ஌ͷ෭ࢺͷGFXTIPU MFBSOJOH  ෭ࢺΛಈࢺʹஔ׵  ੜ࢈ੑ
45. ܥ౷త൚ԽΤʔδΣϯτͷͨΊͷϕϯνϚʔΫ 45 [1] Z. Wu et al., “ReaSCAN: Compositional Reasoning in Language Grounding”, NeurIPS Datasets and Benchmarks, 2021. ReaSCANσʔληοτ [1] 3FB4$"/ 3FBTPOJOHCBTFE 4$"/ <>ɿঢ়ଶͱݴޠࢦࣔΛೖྗͱͯ͠ɼߦಈΛग़ྗ
46. ܥ౷త൚ԽΤʔδΣϯτͷͨΊͷϕϯνϚʔΫ 46 [1] Z. Wu et al., “ReaSCAN: Compositional Reasoning in Language Grounding”, NeurIPS Datasets and Benchmarks, 2021. ReaSCANσʔληοτ [1] 3FB4$"/ 3FBTPOJOHCBTFE 4$"/ <>ɿঢ়ଶͱݴޠࢦࣔΛೖྗͱͯ͠ɼߦಈΛग़ྗ H4$"/͸ͭͷ໰୊͕͋ͬͨ p ݴޠࢦࣔͷ૊Έ߹Θ͕ͤγϯϓϧ͗ͯ͢ จ຺ແࢹͷ#P8ʢ#BHPG8PSETʣͰे෼ͩͬͨ p ๦֐ΦϒδΣΫτ͕ܥ౷త൚ԽͷͨΊͷ ਖ਼֬ͳཧղʹ΄ͱΜͲد༩͍ͯ͠ͳ͔ͬͨ p ݴޠࢦࣔͷܗ༰ࢺ͕φϏήʔγϣϯʹ ͍Βͳ͍͜ͱ͕ଟ͔ͬͨ
47. ܥ౷త൚ԽΤʔδΣϯτͷͨΊͷϕϯνϚʔΫ 47 [1] Z. Wu et al., “ReaSCAN: Compositional Reasoning in Language Grounding”, NeurIPS Datasets and Benchmarks, 2021. ReaSCANσʔληοτ [1] 3FB4$"/ 3FBTPOJOHCBTFE 4$"/ <>ɿঢ়ଶͱݴޠࢦࣔΛೖྗͱͯ͠ɼߦಈΛग़ྗ େ͖͘෼͚ͯεϓϦοτ͸ͭʢࡉ͔͍ͱͭʣ  ෺ମଐੑͷܥ౷ੑ  ঢ়ଶͱݴޠͷܥ౷ੑʢ৽ن෺ମ૊Έ߹Θͤʣ  ৽ن۟ߏ଄ʢੜ࢈ੑͱ۟ͷೖΕସ͑ʣ
48. ܥ౷త൚ԽΤʔδΣϯτͷͨΊͷϕϯνϚʔΫ 48 ओͳͷ͸Ҏ্ͷͭ l 4$"/ l H4$"/ l 3FB4$"/
49. ܥ౷త൚ԽΤʔδΣϯτͷख๏ 49 l -45. TFRTFR l $POW TFRTFR l 5SBOTGPSNFS l ($/-45. l .FUB 5SBOTGPSNFS
50. ܥ౷త൚ԽΤʔδΣϯτͷख๏ 50 l -45. TFRTFRɿγʔέϯεΛॱʑʹೖྗͯ͠ɼग़ྗΛ࠶ىతʹೖྗͱͯ͠ར༻ l $POW TFRTFR l 5SBOTGPSNFS l ($/-45. l .FUB 5SBOTGPSNFS -45.
    $POW
    5SBOTGPSNFS
    <> 4FR4FRʢ4FRVFODFUP4FRVFODFʣ<> [1] D. Hupkes et al., “Compositionality Decomposed: How do Neural Networks Generalise?”, JAIR, 2020. [2] B. M. Lake and M. Baroni, “Generalization without Systematicity: On the Compositional Skills of Sequence-to-Sequence Recurrent Networks”, ICML, 2018.
51. ܥ౷త൚ԽΤʔδΣϯτͷख๏ 51 l -45. TFRTFRɿγʔέϯεΛॱʑʹೖྗͯ͠ɼग़ྗΛ࠶ىతʹೖྗͱͯ͠ར༻ l $POW TFRTFRɿγʔέϯεͰඃΔΑ͏ʹ৞ΈࠐΈʢͻͱੲલʹݴޠॲཧͰྲྀߦʣ l 5SBOTGPSNFS l ($/-45. l .FUB 5SBOTGPSNFS -45.
    $POW
    5SBOTGPSNFS
    <> [1] D. Hupkes et al., “Compositionality Decomposed: How do Neural Networks Generalise?”, JAIR, 2020. [2] B. M. Lake and M. Baroni, “Generalization without Systematicity: On the Compositional Skills of Sequence-to-Sequence Recurrent Networks”, ICML, 2018.
52. ܥ౷త൚ԽΤʔδΣϯτͷख๏ 52 l -45. TFRTFRɿγʔέϯεΛॱʑʹೖྗͯ͠ɼग़ྗΛ࠶ىతʹೖྗͱͯ͠ར༻ l $POW TFRTFRɿγʔέϯεͰඃΔΑ͏ʹ৞ΈࠐΈʢͻͱੲલʹݴޠॲཧͰྲྀߦʣ l 5SBOTGPSNFSɿγʔέϯεΛ.-1ͱ"UUFOUJPOͰॲཧ͠ɼԕ͍γʔέϯεͰ΋ߟྀ͕༻ҙ l ($/-45. l .FUB 5SBOTGPSNFS -45.
    $POW
    5SBOTGPSNFS
    <> 5SBOTGPSNFSΞʔΩςΫνϟ <> [1] D. Hupkes et al., “Compositionality Decomposed: How do Neural Networks Generalise?”, JAIR, 2020. [2] B. M. Lake and M. Baroni, “Generalization without Systematicity: On the Compositional Skills of Sequence-to-Sequence Recurrent Networks”, ICML, 2018. [3] A. Vaswani et al., “Attention Is All You Need”, NIPS, 2017.
53. ܥ౷త൚ԽΤʔδΣϯτͷख๏ 53 [1] D. Hupkes et al., “Compositionality Decomposed: How do Neural Networks Generalise?”, JAIR, 2020. [2] B. M. Lake and M. Baroni, “Generalization without Systematicity: On the Compositional Skills of Sequence-to-Sequence Recurrent Networks”, ICML, 2018. [3] A. Vaswani et al., “Attention Is All You Need”, NIPS, 2017. [4] T. Gao et al., “Systematic Generalization on gSCAN with Language Conditioned Embedding”, AACL, 2020. l -45. TFRTFRɿγʔέϯεΛॱʑʹೖྗͯ͠ɼग़ྗΛ࠶ىతʹೖྗͱͯ͠ར༻ l $POW TFRTFRɿγʔέϯεͰඃΔΑ͏ʹ৞ΈࠐΈʢͻͱੲલʹݴޠॲཧͰྲྀߦʣ l 5SBOTGPSNFSɿγʔέϯεΛ.-1ͱ"UUFOUJPOͰॲཧ͠ɼԕ͍γʔέϯεͰ΋ߟྀ͕༻ҙ l ($/-45.ɿάϥϑ৞ΈࠐΈΛ༻͍ͯݴޠͱঢ়ଶͷؔ܎ੑߟྀ͢ΔH4$"/Ͱͷ405" l .FUB 5SBOTGPSNFS ($/-45.ͷΞʔΩςΫνϟ <>
54. ܥ౷త൚ԽΤʔδΣϯτͷख๏ 54 [1] D. Hupkes et al., “Compositionality Decomposed: How do Neural Networks Generalise?”, JAIR, 2020. [2] B. M. Lake and M. Baroni, “Generalization without Systematicity: On the Compositional Skills of Sequence-to-Sequence Recurrent Networks”, ICML, 2018. [3] A. Vaswani et al., “Attention Is All You Need”, NIPS, 2017. [4] T. Gao et al., “Systematic Generalization on gSCAN with Language Conditioned Embedding”, AACL, 2020. [5] B. M. Lake and M. Baroni. “Human-like systematic generalization through a meta-learning neural network”, Nature, 2023. l -45. TFRTFRɿγʔέϯεΛॱʑʹೖྗͯ͠ɼग़ྗΛ࠶ىతʹೖྗͱͯ͠ར༻ l $POW TFRTFRɿγʔέϯεͰඃΔΑ͏ʹ৞ΈࠐΈʢͻͱੲલʹݴޠॲཧͰྲྀߦʣ l 5SBOTGPSNFSɿγʔέϯεΛ.-1ͱ"UUFOUJPOͰॲཧ͠ɼԕ͍γʔέϯεͰ΋ߟྀ͕༻ҙ l ($/-45.ɿάϥϑ৞ΈࠐΈΛ༻͍ͯݴޠͱঢ়ଶͷؔ܎ੑߟྀ͢ΔH4$"/Ͱͷ405" l .FUB 5SBOTGPSNFSɿGFXTIPU FYBNQMFTͷઃఆͷ5SBOTGPSNFS ݱঢ়ɼ5SBOTGPSNFSܥ͕࠷΋༗๬ .FUB
    5SBOTGPSNFS
    <>
55. ܥ౷త൚ԽΤʔδΣϯτͷ՝୊ͱࠓޙͷల๬ 55 [1] D. Bahdanau et al., “Systematic Generalization: What Is Required and Can It Be Learned?”, ICLR, 2018. [2] R. Csordás et al., “The Devil is in the Detail: Simple Tricks Improve Systematic Generalization of Transformers”, EMNLP, 2021. [3] R. Kirk et al., “A Survey of Zero-shot Generalisation in Deep Reinforcement Learning”, JAIR, 2023. l طଘͷϕϯνϚʔΫ͸ݱ࣮ੈքͷઃఆʹଈ͍ͯ͠ͳ͍ʢਅͷঢ়ଶ͸औಘͰ͖ͳ͍ʣ l طଘͷϕϯνϚʔΫͰఏҊ͞Εͨख๏Ͱ͢Βɼܥ౷త൚ԽΤʔδΣϯτͷ࣮ݱʹ͸ఔԕ͍ l ϕϯνϚʔΫʹ͓͍ͯɼ܇࿅ޡࠩͱςετੑೳʹ૬͕ؔݟΒΕͳ͍ [1, 2] Ø ܇࿅ޡࠩͱݕূޡ͕ࠩখ͘͞ͳ͔ͬͨΒͱ͍ͬͯςετੑೳ͕ߴ͍ͱ͸ݶΒͳ͍ l ܥ౷త൚Խͷઌ͸θϩγϣοτ൚ԽʢZSG; Zero-shot Generalizationʣ[3] Ø ZSG͸ݪཧతʹղ͚ͳ͍͸ͣͰɼڧ͍ؼೲόΠΞε΍ΠϯλϥΫγϣϯ͕ඞཁ ZSGͷ֓೦ਤ [3]