Share
Embed
論文紹介/Towards understanding Chain-of-Thought prompting: An empirical study of what matters
第15回最先端NLP勉強会のスライドです
https://sites.google.com/view/snlp-jp/home/2023?authuser=0
Shota Kato
August 21, 2023
Transcript
34 slides
-
հऀɿՃ౻ ↅଠʢژେֶʣ ͱ͘ʹऍ͕ͳ͍ݶΓɼਤදࣄྫհจ͔ΒͷҾ༻Ͱ͢ ˞հऀͷίϝϯτͰ͢ Towards Understanding Chain-of-Thought Prompting: An Empirical
Study of What Matters Boshi Wang, Sewon Min, Xiang Deng, Jiaming Shen, You Wu, Luke Zettlemoyer, Huan Sun ACL 2023 https://aclanthology.org/2023.acl-long.153/ !ୈճ࠷ઌ/-1ษڧձ -
·ͱΊ • త • Chain-of-thought (CoT) ϓϩϯϓτʹد༩͢ΔΛղ໌͢Δɽ • ख๏ •
CoT ϓϩϯϓτΛͭͷཁૉʹׂͯ͠ ablation study Λߦͬͨɽ • ؔ࿈ੑͱҰ؏ੑΛධՁͨ͠ɽ • ಘΒΕͨݟ • ਪͷଥੑੑೳʹ΄ͱΜͲӨڹ͠ͳ͍ɽ • CoT ͰɼೖྗΫΤϦͱͷؔ࿈ੑɾਪաఔͷҰ؏ੑ͕ॏཁɽ • LLM ࣄલֶशʹΑͬͯਪํ๏Λֶश͍ͯ͠Δɽ 1 -
A: The cafeteria had 23 apples originally. They used 20
to make lunch. So they had 23 - 20 = 3. They bought 6 more apples, so they have 3 + 6 = 9. The answer is 9. Chain-of-Thought Prompting Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each can has 3 tennis balls. How many tennis balls does he have now? A: The answer is 11. Q: The cafeteria had 23 apples. If they used 20 to make lunch and bought 6 more, how many apples do they have? A: The answer is 27. Standard Prompting Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each can has 3 tennis balls. How many tennis balls does he have now? A: Roger started with 5 balls. 2 cans of 3 tennis balls each is 6 tennis balls. 5 + 6 = 11. The answer is 11. Q: The cafeteria had 23 apples. If they used 20 to make lunch and bought 6 more, how many apples do they have? Model Input Model Output Model Output Model Input େنݴޠϞσϧͷϓϩϯϓτ େنݴޠϞσϧʢ--.ʣͷೖྗʢϓϩϯϓτʣΛ͢Δͱ ৽͍͠λεΫͰߴ͍ੑೳΛୡͰ͖Δɽ • ຊจதֶशʢIn-Context Learning; ICLʣ[Brown+,20] • ࢥߟ࿈ܕʢChain-of-Thought; CoTʣϓϩϯϓτ [Wei+,22] [Wei+,22] 2 -
A: The cafeteria had 23 apples originally. They used 20
to make lunch. So they had 23 - 20 = 3. They bought 6 more apples, so they have 3 + 6 = 9. The answer is 9. Chain-of-Thought Prompting Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each can has 3 tennis balls. How many tennis balls does he have now? A: The answer is 11. Q: The cafeteria had 23 apples. If they used 20 to make lunch and bought 6 more, how many apples do they have? A: The answer is 27. Standard Prompting Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each can has 3 tennis balls. How many tennis balls does he have now? A: Roger started with 5 balls. 2 cans of 3 tennis balls each is 6 tennis balls. 5 + 6 = 11. The answer is 11. Q: The cafeteria had 23 apples. If they used 20 to make lunch and bought 6 more, how many apples do they have? Model Input Model Output Model Output Model Input Chain-of-Thought (CoT) ϓϩϯϓτ େنݴޠϞσϧʢ--.ʣͷೖྗʢϓϩϯϓτʣΛ͢Δͱ ৽͍͠λεΫͰߴ͍ੑೳΛୡͰ͖Δɽ • ຊจதֶशʢIn-Context Learning; ICLʣ[Brown+,20] • ࢥߟ࿈ܕʢChain-of-Thought; CoTʣϓϩϯϓτ [Wei+,22] CoT ϓϩϯϓτΛ༻͍Δͱɺ ੑೳ্͕͢Δ ྫɿࢉज़ਪλεΫ [Cobbe+,21] Accuracy: 15.4 → 48.5 (InstructGPT-175B text-davinci-002 [Ouyang+,22;Brown+,20]) CoT ϓϩϯϓτ͕ߴ͍ੑೳΛ ୡͰ͖Δͷͳ͔ͥʁ [Wei+,22] 3 -
CoT ϓϩϯϓτͷߏཁૉ Bridging objects ਖ਼͍͠༧ଌΛ͢ΔͨΊʹඞཁͳΦϒδΣΫτ ࢉज़ਪɿਪʹؚ·ΕΔ෦ʢɾࣜʣ ࣄ࣮ܕ࣭Ԡɿओମͱ٬ମͷΤϯςΟςΟ Language templates bridging
objects Λิ͢Δ෦ Research question ਖ਼֬ͳ bridging objects ͱ language templates ඞཁ͔ʁ ͷճ͕ No ͳΒɼ LLM ͕దʹਪ͢ΔͨΊʹॏཁͳཁૉԿ͔ʁ 5 -
࣮ݧઃఆ • ༻͢ΔݴޠϞσϧ • InstructGPT-175B [Ouyang+,22;Brown+,20] • text-davinci-002ʢϝΠϯʣ, text-davinci-003 •
PaLM [Chowdhery+,22] • Flan-PaLM [Chung+,22]: PaLM + instruction tuning • λεΫɿCoT Ͱੑೳ্͕ͨ͠ଟஈਪΛཁ͢ΔλεΫ • ࢉज़ਪɿGSM8K [Cobbe+,21] • ࣄ࣮ܕϚϧνϗοϓ2"ɿBamboogle [Press+,22] • ϕʔεϥΠϯʢCoT ϓϩϯϓτʣ GSM8K ͱ Bamboogle Ͱ༻͍ΒΕͨϓϩϯϓτΛमਖ਼͢Δɽ ʢGSM8K: 8-shotɼBamboogle: 4-shotʣ 7 -
࣮ݧ̍ʛख๏ • Invalid reasoning ͷϓϩϯϓτΛख࡞ۀͰ࡞͢Δɽ CoT ͷ bridging objects ͱ
language templates Λมߋͨ͠ϓϩϯϓτ • Invalid reasoning ͱ CoT ͱͷੑೳͷࠩΛଌΔɽ ճʹཱͭ෦ͷΈ Λมߋ͢Δ 9 -
ධՁํ๏ • ࠷ऴग़ྗͷධՁʢ֎ࡏతධՁʣ • GSM8K: Accuracy • BamboogleɿF1 • ਪաఔͷධՁʢࡏతධՁʣ
Bridging objects ͷ Recall / F1 Λଌఆ͢Δɽ • GSM8KɿਪաఔதͷࣈͷRecall / F1ʢInter. Recall / F1ʣ GSM8K தͷਪաఔͷϥϕϧ͖σʔλΛ༻͍Δɽ • Bamboogleɿओମɼ٬ମΤϯςΟςΟͷ Recall ʢInter. Recallʣ ϥϕϧ͖σʔλΛखಈͰ࡞ͯ͠༻͍Δɽ ਪաఔΛධՁͰ͖ͳ͍ɽ 10 -
࣮ݧ̍ʛ݁Ռ Flan-PaLM PaLM text-davinci-002 text-davinci-003 • 全モデルで invalid reasoning の性能は
CoT の約90%. • おそらく事前学習で多段推論能力を獲得している. ਪաఔͷଥੑͱճͷ࣭ʹڧ͍ؔ࿈ੑແ͍ 12 -
࣮ݧ̍ʛ·ͱΊ 2ਖ਼֬ͳ bridging objects/language templates ඞཁ͔ʁ "ඞཁͰͳ͍ɽ ײɿCoT ͱͯ͠ଥͰͳ͍ਪաఔ͕༩͑ΒΕͨΒɼ LLM
ਖ਼͍͠ਪ͕Ͱ͖ͳ͍ͣ… Research question ਖ਼֬ͳ bridging objects ͱ language templates ඞཁ͔ʁ ͷճ͕ No ͳΒɼ LLM ͕దʹਪ͢ΔͨΊʹॏཁͳཁૉԿ͔ʁ 13 -
࣮ݧ̍ͷख๏Λݟฦ͢ͱʜ Invalid reasoning ʹɼਪʹཱͭใ͕͍ͬͯΔɽ ΫΤϦʹؔ͢ΔใΛؚΉɽ Bridging objects ಉ͡ࣈ͕ΫΤϦʹؚ·ΕΔɽ Language templates
͕ΫΤϦͱಉ͡ɽ จ͕ܨ͕͍ͬͯͯے͕௨͍ͬͯΔɽ ྫ͑ɼલͷจʹग़͖ͯͨࣈ ͕࣍ͷจͰΘΕ͍ͯΔɽ Relevanceʢؔ࿈ੑʣ CoherenceʢҰ؏ੑʣ 16 -
࣮ݧ̎ cख๏ CoT ϓϩϯϓτΛมߋͯ͠ɼ ύλʔϯͷϓϩϯϓτΛ৽ͨʹ࡞͢Δɽ • No coherence for bridging
objects • No relevance for bridging objects • No coherence for language templates • No relevance for language templates • No coherence • No relevance 17 -
࣮ݧ̎ cϓϩϯϓτͷྫ ߏཁૉͷॱংΛϥϯμϜʹ ೖΕସ͑Δɽ ࣈΛΫΤϦ͔ΒϥϯμϜʹ αϯϓϦϯάͯ͠ஔ͢Δɽ ˞No relevance for bridging
objects Ͱ ࠷ޙͷ͕͑ෆਖ਼ղʹͳ͍ͬͯΔɽ 18 -
࣮ݧ̎ʛ݁Ռ • ؔ࿈ੑͱҰ؏ੑ CoT ʹඞཁɽ • ؔ࿈ੑಛʹॏཁɽ • αϯϓϧΛਓखͰධՁɽͰग़ྗͱΫΤϦͷؔ࿈ੑͳ͠ɽ
• ແؔͷग़ྗࣅͨʢcats and dogs” "passengers and buses”ʣɽ ͓ͦΒ͘ࣄલֶशίʔύεதͷֶؔ࿈෦Ͱසग़ͷɽ text-davinci-002 ˞No relevance for bridging objects ͱ No relevance ͷ͕͑ෆਖ਼ղͰ͋Δ͜ͱͷӨڹʜʁ 19 -
࣮ݧ̎ʛ݁Ռ • Bridging objects Ͱؔ࿈ੑ͕ΑΓॏཁɽ • ؔ࿈ੑͳ͠ ᶅ ᶉ
ͷग़ྗͱΫΤϦͱͷ bridging objects ͷҰக ҎԼͰɼଞͷ߹ʢʣΑΓ͔ͬͨɽ • Language templates ͰҰ؏ੑ͕ΑΓॏཁɽ • αϯϓϧͷग़ྗ͕Ұ؏ੑͳ͠ɽ text-davinci-002 ˞No relevance for bridging objects ͱ No relevance ͷ͕͑ෆਖ਼ղͰ͋Δ͜ͱͷӨڹʜʁ 20 -
LLMCoT͔Βਪํ๏ΛֶΜͰ͍Δʁ • LLM ͕ CoT ͔ΒֶͿਪํ๏ݶఆతɽ üLLM ɼࣄલֶशʹΑͬͯෳࡶͳਪೳྗΛ֫ಘ͍ͯ͠Δɽ üCoT ͷׂؔ࿈ੑͱҰ؏ੑΛ࣋ͭΑ͏ʹग़ྗΛ੍ޚ͢Δ͜ͱɽ
• λεΫͷ͕ࣝଟ͍ͱ ablations ʹΑΔੑೳԼখ͍͞ɽ üλεΫΛղ͘ࡍʹࣄલࣝΛ׆༻Ͱ͖Δɽ ✘ଥͰͳ͍ਪաఔΛੜ͢ΔλεΫͷ࣮ߦ͍͠ɽ Flan-PaLM 21 -
LLMCoT͔Βਪํ๏ΛֶΜͰ͍Δʁ • LLM ͕ CoT ͔ΒֶͿਪํ๏ݶఆతɽ üLLM ɼࣄલֶशʹΑͬͯෳࡶͳਪೳྗΛ֫ಘ͍ͯ͠Δɽ üCoT ͷׂؔ࿈ੑͱҰ؏ੑΛ࣋ͭΑ͏ʹग़ྗΛ੍ޚ͢Δ͜ͱɽ
• λεΫͷ͕ࣝଟ͍ͱ ablations ʹΑΔੑೳԼখ͍͞ɽ üλεΫΛղ͘ࡍʹࣄલࣝΛ׆༻Ͱ͖Δɽ ✘ଥͰͳ͍ਪաఔΛੜ͢ΔλεΫͷ࣮ߦ͍͠ɽ LLM CoT ͔Βਪํ๏ΛֶΔ͔ʁ • ݁Λग़͢ʹݱঢ়ͷ݁Ռෆेɽ • CoT ͷओͳׂࣄલֶशͰಘͨਪεΩϧΛҾ͖ग़͢͜ͱɽ 22 -
[Wei+,22] ՝ • ଞͷਪλεΫʹద༻Մೳͳ࣮ݧͷઃܭ • ຊݚڀͰͬͨख๏൚༻తͰͳ͘ɼ CoT ϓϩϯϓτͷߏཁૉ͕Ұ༷ͩͱద༻Ͱ͖ͳ͍ɽ ྫɿLast letter
concatenation task • Invalid reasoning ͷϓϩϯϓτ࡞ํ๏ͷࣗಈԽ • ࡏతධՁͷվળ • ධՁʹ༻͍ͨ bridging objects ͷਖ਼ղ͍ͭͰར༻ՄೳͰͳ͍ • แׅత͔ͭࢀরෆཁͳධՁํ๏ͷ։ൃ͕՝ɽؔ࿈[Golovneva+,23] 23 -
·ͱΊ • త • Chain-of-thought (CoT) ϓϩϯϓτʹد༩͢ΔΛղ໌͢Δɽ • ख๏ •
CoT ϓϩϯϓτΛͭͷཁૉʹׂͯ͠ ablation study Λߦͬͨɽ • ؔ࿈ੑͱҰ؏ੑΛධՁͨ͠ɽ • ಘΒΕͨݟ • ਪͷଥੑੑೳʹ΄ͱΜͲӨڹ͠ͳ͍ɽ • CoT ͰɼೖྗΫΤϦͱͷؔ࿈ੑɾਪաఔͷҰ؏ੑ͕ॏཁɽ • LLM ࣄલֶशʹΑͬͯਪํ๏Λֶश͍ͯ͠Δɽ ஶऀ࣮ɿhttps://github.com/sunlab-osu/Understanding-CoT 24 -
ࢀߟจݙ [Brown+,20] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah,
Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901. [Wei+,22] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed Chi, Quoc V Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems, volume 35, pages 24824–24837. Curran Associates, Inc. [Cobbe+,21] Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168. [Ouyang+,22] Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. arXiv preprint arXiv:2203.02155. 25 -
ࢀߟจݙ [Chowdhery+,22] Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma,
Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2022. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311. [Chung+,22] Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. 2022. Scaling instruction- finetuned language models. arXiv preprint arXiv:2210.11416. [Press+,22] Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A Smith, and Mike Lewis. 2022. Measuring and narrowing the compositionality gap in language models. arXiv preprint arXiv:2210.03350. [Golovneva+,23] Olga Golovneva, Moya Peng Chen, Spencer Poff, Martin Corredor, Luke Zettlemoyer, Maryam Fazel-Zarandi, and Asli Celikyilmaz. 2023. ROSCOE: A suite of metrics for scoring step-by-step reasoning. In The Eleventh International Conference on Learning Representations. 26 -
࣮ݧ̍ʛ݁Ռ text-davinci-002 ͷྔతղੳ) • invalid reasoning ͷੑೳ CoT ͷɽ •
қͷҟͳΔαϯϓϧؒͰ ύϑΥʔϚϯεͷԼҰ༷ɽ • CoTͷΈෆਖ਼ղยํͷΈෆਖ਼ղ GSM8K: 62/196, Bamboogle: 6/20 ਪաఔͷଥੑͱճͷ࣭ʹڧ͍ؔ࿈ੑແ͍ text-davinci-002 28 -
ߟ • few-shot ਪͷϕϯνϚʔΫʹ͍ͭͯ • ຊݚڀ LLM ͷଟஈਪʹؔ͢ΔࣄલࣝͷఆྔԽํ๏ͱΈͳͤΔɽ • few-shot
͔Βਪํ๏Λֶश͢Δ LLM ͷೳྗΛධՁ͢ΔͨΊʹɼ LLM ʹؚ·Ε͍ͯΔ͕ࣝগͳ͍ϕϯνϚʔΫ͕ඞཁɽ 30