Share
Embed
[輪講資料] LoRA: Low-Rank Adaptation of Large Language Models
パラメータを固定した事前学習済みモデルに対して、ごく少数のパラメータからなる低ランク行列を導入・学習することで、モデル全体のfine-tuningと同等の性能を発揮できる手法であるLoRAと、その論文について解説した資料です。
深層学習を用いた自然言語処理の歴史的な変遷と周辺技術から、LoRAが必要とされるに至った背景まで丁寧に解説します。
Hayato Tsukagoshi
April 18, 2023
Transcript
66 slides
-
LoRA: Low-Rank Adaptation of Large Language Models Graduate School of
Informatics, Nagoya University, Japan. ൃදऀ: Hayato Tsukagoshi Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen ICLR 2022 https://arxiv.org/abs/2106.09685 -
•ࣄલֶशࡁΈϞσϧʹରͯ͠Ұ෦ͷ ύϥϝʔλͷΈΛඍௐ͢Δख๏ΛఏҊ •ೋͭͷϥϯΫߦྻA, BΛಋೖ • A = (d × r)
ߦྻ, B = (r × d) ߦྻ • r2ͳͲ͘͝খ͍͞ࣈͰOK •શͳ fi ne-tuningʹରͯ͠ • ֶशύϥϝʔλΛܶతʹݮ • ಉੑೳ •ਪίετΛ૿Ճͤͣ͞ʹϞσϧͷඍௐ͕Մೳ จ֓ཁ 2 -
•ۙඇৗʹར༻ऀ͕૿͍͑ͯΔϞσϧ • ը૾ɾΠϥετੜͰͷΓ্͕ΓʹΑͬͯೝ͕ߴ·ͬͨ •େنݴޠϞσϧ࣌ͷσϑΝΫτελϯμʔυʹͳΓ͏Δख๏ • ͢ͰʹLLMͷద༻͕ۃΊͯΜ • MetaͷLLaMAʹର͢ΔAlpacaͱAlpaca-LoRAͱ͔ •ਂֶशϞσϧͷֶशख๏ͷྺ࢙Λ၆ᛌ͢ΔͷʹஸΑ͔ͬͨ •
BERTҎલ: શͯࣗͰֶश • ࣄલֶश (pre-training) → ඍௐ ( fi ne-tuning) • ෦తͳඍௐख๏ or Few-shot / Zero-shot learning બఆཧ༝ 3 -
ࣄલࣝ •ਂֶशΛ༻͍ͨࣗવݴޠॲཧͷجૅ •ҙػߏɺTransformerɺBERT, … LoRA •େنݴޠϞσϧͷඍௐʹ͓͚Δͱղܾࡦ •ؔ࿈ݚڀ •ఏҊख๏ɾධՁ࣮ݧ LoRAͷ͍ํ •PEFT:
Parameter-E ff i cient Fine-Tuning ࣍ 4 -
ࣄલࣝ •ਂֶशΛ༻͍ͨࣗવݴޠॲཧͷجૅ •ҙػߏɺTransformerɺBERT, … LoRA •େنݴޠϞσϧͷඍௐʹ͓͚Δͱղܾࡦ •ؔ࿈ݚڀ •ఏҊख๏ɾධՁ࣮ݧ LoRAͷ͍ํ •PEFT:
Parameter-E ff i cient Fine-Tuning ࣍ 5 -
•ਂֶशϞσϧΛ༻͍ͨݴޠϞσϧ͕த৺తଘࡏ •ʮࣗવݴޠͷ୯ޠจষ͕ੜ͞ΕΔ֬ΛϞσϧԽͨ͠ͷʯ* • ͕ͩɺ࠷ۙʮݴޠΛѻ͑ΔϞσϧʯ͘Β͍ͷҙຯͰΘΕ͕ͪ •ਂֶशΛ༻͍ͨࣗવݴޠॲཧͷجૅٕज़ • Ϟσϧྫ: BERT, GPT-2, GPT-3,
GPT-4, LLaMA, Alpaca, Vicuna, Dolly, … • ͜ΕΒج൫Ϟσϧ (Foundation Models) ͱݺΕΔ ݴޠϞσϧͷΘΕํ •จΛ୯ޠϕΫτϧͷྻɾจϕΫτϧʹม •͋Δจʹଓ࣍͘ͷ୯ޠɾจΛ༧ଌ ਂֶशΛ༻͍ͨࣗવݴޠॲཧͷجૅ 7 *IT Text ʰࣗવݴޠॲཧͷجૅʱΑΓҾ༻ -
•ۙͷϞσϧͷଟ͘ҙػߏ(Attention Mechanism)ʹجͮ͘ TransformerͰߏ •͍Ζ͍Ζͳछྨ͕ଘࡏ ࣗݾճؼܕݴޠϞσϧ (Causal LM) •ࠨ͔Βӈʹ୯ޠΛ༧ଌͯ͠܇࿅ •ྫ: GPT,
GPT-2, GPT-3, … ϚεΫݴޠϞσϧ (Masked LM) •จதͷҰ෦ΛӅ͢ɾ༧ଌͯ͠܇࿅ •ྫ: BERT, RoBERTa, DeBERTa, … ݴޠϞσϧ: Language Models 9 ଞʹݴޠϞσϧʹ͞·͟·ͳछྨ͕ଘࡏɻྫ: XLNet, ELECTRA, UL2, … BERTͷ֓ཁਤ -
•ϕΫτϧྻΛೖྗʹϕΫτϧྻΛग़ྗ͢Δػߏ •ೖྗΛQ (Query), K (Key), V (Value)ʹ͚ͯܭࢉ • K, V:
nݸͷd࣍ݩϕΫτϧ • Q: mݸͷd࣍ݩϕΫτϧ ҙػߏ (Attention Mechanism) 10 ਤ Jaegle et al., Perceiver IO: A General Architecture for Structured Inputs & Outputs, ICLR 2022. ΑΓҾ༻ Θ͔Γ͍͢ղઆ: ʲਂֶशʳAttention - શྖҬʹԠ༻͞Ε࠷ߴਫ਼Λୟ͖ग़͢ҙػߏͷΈʲσΟʔϓϥʔχϯάͷੈք vol. 24ʳ -
•ϕΫτϧྻΛೖྗʹϕΫτϧྻΛग़ྗ͢Δػߏ •ೖྗΛQ (Query), K (Key), V (Value)ʹ͚ͯܭࢉ • K, V:
nݸͷd࣍ݩϕΫτϧ • Q: mݸͷd࣍ݩϕΫτϧ •Qͷ֤ϕΫτϧʹର͢ΔKͷ֤ϕΫτϧͷॏཁΛܭࢉ • Attention Weights: ܭࢉͷ݁ՌಘΒΕΔ(m × n)ߦྻ •Self Attention (ࣗݾҙػߏ): Q, K, VΛಉ͡ϕΫτϧྻ͔Βߏ (i.e. n=m) •Cross Attention: ʮQʯͱʮK, VʯΛҟͳΔϕΫτϧྻ͔Βߏ ҙػߏ (Attention Mechanism) 11 ਤ Jaegle et al., Perceiver IO: A General Architecture for Structured Inputs & Outputs, ICLR 2022. ΑΓҾ༻ Θ͔Γ͍͢ղઆ: ʲਂֶशʳAttention - શྖҬʹԠ༻͞Ε࠷ߴਫ਼Λୟ͖ग़͢ҙػߏͷΈʲσΟʔϓϥʔχϯάͷੈք vol. 24ʳ -
•ҙػߏͷΈͰߏ͞ΕͨϞσϧߏ • ͦΕ·ͰNLPͰΑ͘ར༻͞Ε͍ͯͨ RNN, LSTMCNNΛഉআ •ϕΫτϧྻΛೖྗʹϕΫτϧྻΛग़ྗ͢Δػߏ • ೖྗϕΫτϧಉ࢜ͷ૬ޓ࡞༻Λߟྀ Transformer 12
Vaswani etl al., Attention Is All You Need, NeurIPS 2017. Θ͔Γ͍͢ղઆ: ʲਂֶशʳTransformer - Multi-Head AttentionΛཧղͯ͠Ζ͏͡Όͳ͍ͷʲσΟʔϓϥʔχϯάͷੈքvol.28ʳ ֓ཁਤ Encoder Decoder -
•ҙػߏͷΈͰߏ͞ΕͨϞσϧߏ • ͦΕ·ͰNLPͰΑ͘ར༻͞Ε͍ͯͨ RNN, LSTMCNNΛഉআ •ϕΫτϧྻΛೖྗʹϕΫτϧྻΛग़ྗ͢Δػߏ • ೖྗϕΫτϧಉ࢜ͷ૬ޓ࡞༻Λߟྀ •EncoderͱDecoderͷೋछྨ͕ଘࡏ •
EncoderͷΈ: BERT, LUKE, … • DecoderͷΈ: GPT, GPT-2, GPT-3, … • Encoder-Decoder: BART, T5, UL2, … Transformer 13 Vaswani etl al., Attention Is All You Need, NeurIPS 2017. Θ͔Γ͍͢ղઆ: ʲਂֶशʳTransformer - Multi-Head AttentionΛཧղͯ͠Ζ͏͡Όͳ͍ͷʲσΟʔϓϥʔχϯάͷੈքvol.28ʳ ֓ཁਤ Encoder Decoder -
•Transformer EncoderΛෳॏͶͯେنʹࣄલֶशͨ͠Ϟσϧ • base12 (1.1ԯύϥϝʔλ)ɺlarge24 (3.3ԯύϥϝʔλ) •ࣄલֶश (pre-training) → ඍௐ
( fi ne-tuning) ͱ͍͏ύϥμΠϜ͕ීٴ BERT: Bidirectional Encoder Representations from Transformers 14 Devlin et al., BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, NAACL 2019. -
•BERTҎલ: ݸผλεΫͷσʔληοτͰϞσϧΛؤுͬͯ܇࿅ •BERTΛར༻: ݸผλεΫͷσʔληοτͰBERTΛඍௐ( fi ne-tuning) ϙΠϯτ •BERTݴޠͱԿ͔ͱ͍͏ࣝΛࣄલֶशʹΑͬͯ֫ಘ • ͦͷࣝΛ͏ͷͰɺݸผλεΫʹରͯ͠
“গ͠” ௐ͢Δ͚ͩͰ͍͍ •܇࿅σʔλ͕ൺֱతগྔͰߴ͍ੑೳΛಘΒΕΔΑ͏ʹ (10ສ~ →1000~) • ܇࿅ίετ (σʔληοτऩूɾ܇࿅࣌ؒ) ͕ܶతʹݮগ BERTΛ༻͍ͨࣗવݴޠॲཧ 15 Devlin et al., BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, NAACL 2019. -
•175B (1750ԯ) ύϥϝʔλͱ͍͏ඇৗʹڊେͳࣗݾճؼܕݴޠϞσϧΛ܇࿅ • ࢀߟ: BERT-base 110M (=0.11B) ύϥϝʔλ •Few-shot
/ Zero-shot learningͱݺΕΔೳྗΛ֫ಘ • ͘͝গ(0~100)ͷࣄྫΛݟΔ͚ͩͰλεΫΛ͋Δఔղ͚ΔΑ͏ʹ GPT-3 16 -
•175B (1750ԯ) ύϥϝʔλͱ͍͏ඇৗʹڊେͳࣗݾճؼܕݴޠϞσϧΛ܇࿅ • ࢀߟ: BERT-base 110M (=0.11B) ύϥϝʔλ •Few-shot
/ Zero-shot learningͱݺΕΔೳྗΛ֫ಘ • ͘͝গ(0~100)ͷࣄྫΛݟΔ͚ͩͰλεΫΛ͋Δఔղ͚ΔΑ͏ʹ Scaling law •ϞσϧɾσʔλɾܭࢉྔΛσΧ͘͢Ε͢Δ΄Ͳੑೳ্͕͕Δͱ͍͏ܦݧଇ •**·ͩݶք͕ݟ͍͑ͯͳ͍** • ܭࢉྔΛ૿͢͜ͱͷΈͰਓྨͷతೳྗΛ͑ΒΕΔʁ • ਓྨ͕ເݟͨʮ൚༻ਓೳʯͷҰา͔ʁ GPT-3 17 -
•େنݴޠϞσϧΛ༻͍ͨਪख๏ͷҰͭ • In-context learning (จ຺ֶश) ͱ •λεΫͷʮೖग़ྗྫʯΛݟͤΔ͚ͩͰ ͋Δఔਖ਼͘͠λεΫ͕ղ͚Δ • λεΫΛղ͚ΔΑ͏ʹݴޠϞσϧΛ
͏·͘ʮ͚݅ʯ͢Δ •ʮ͚݅ʯͷͨΊͷࣄલೖྗΛ ϓϩϯϓτ (prompt)ͱ͍͏ Few-shot / Zero-shot learning 18 ਤGPT-3จΑΓҾ༻ Zero-shot learning Few-shot learning -
•ଟ͘ͷ߹promptͨͩͷจࣈྻ • ࿈ଓతͳϕΫτϧΛpromptʹ͢Δൊ • Soft Promptsͱ͍͏ • ޙड़͢ΔPre fi x-TuningPrompt
Tuning •͋ΔλεΫʹର͢Δੑೳpromptʹେ͖͘ґଘ • promptΛ͍͍ײ͡ʹվળ͢Δඞཁ • ͜ͷϓϩηεΛPrompt Engineeringͱ͍͏ • Chain-of-ThoughtͳͲͷํ๏͕ଘࡏ •Prompt Engineeringͷ·ͱΊ Prompt / Prompt Engineering 19 ਤ Kojima et al. Large Language Models are Zero-Shot Reasoners, NeurIPS 2022. ΑΓҾ༻ -
•OpenAIͷChatGPT APIͱGPT-4, MetaͷLLaMAͷެ։ʹΑͬͯେྲྀߦ • GPT-4͕͑Δ ChatGPT Plus ΠΠκ •LLMͷద༻ൣғͷ֦େͱLLMΛखݩͰಈ͔ͨ͢Ίͷٕज़։ൃ͕രൃతʹྲྀߦ •
llama.cppLangChain, LlamaIndexͳͲपลٕज़ͷ։ൃΞπ͍ • Alpaca-LoRAͳͲLLMΛLoRAͰௐ͢Δଟ •ϑϦʔͷେنݴޠϞσϧΛެ։͢Δಈ͖׆ൃ • OPT, Alpaca, Vicuna, Dolly, RWKV, ChatRWKV, … େنݴޠϞσϧ (Large Language Models; LLMs) 20 RNNϕʔεͷṖͷLLM (จະެ։) -
•ຊޠʹಛԽͯ͠ࣄલֶश͞ΕͨϞσϧଟଘࡏ • ౦େBERT • ૣҴాେRoBERTa • Studio OusiaຊޠLUKE • ژେDeBERTa-v2
• rinnaࣾ GPT-2 (1.3B) • ૣҴాେ GPT-2 (1.5B) • ABEJAࣾ GPT-NeoX-Japanese (2.7B) ࢀߟ: ຊޠࣄલֶशࡁΈݴޠϞσϧ 21 ࢀߟจݙ: ϑϦʔͰ͑ΔຊޠͷओͳେنݴޠϞσϧʢLLMʣ·ͱΊ ݁ہຊޠେنݴޠϞσϧʢLLMʣͬͯͲΕΛ͍͍͑ͷʁJGLUEϕϯνϚʔΫඇެࣜ·ͱΊ -
ࣄલࣝ •ਂֶशΛ༻͍ͨࣗવݴޠॲཧͷجૅ •ҙػߏɺTransformerɺBERT, … LoRA •େنݴޠϞσϧͷඍௐʹ͓͚Δͱղܾࡦ •ؔ࿈ݚڀ •ఏҊख๏ɾධՁ࣮ݧ LoRAͷ͍ํ •PEFT:
Parameter-E ff i cient Fine-Tuning ࣍ 22 -
•ඍௐޙͷϞσϧͷอଘ͕ͱͯେม • ྫ: GPT-3ͷ߹ɺඍௐͷͨͼʹ175BͷϞσϧ͕ग़དྷ্͕Δ • 175BͷϞσϧҰͭͰ350GBఔͷετϨʔδ༰ྔ* •େ͖ͳϞσϧGPUʹࡌͤΔͷ͕͍͠ • ྫ: GPT-3
fi ne-tuningͷͨΊʹGPUͷϝϞϦ͕1.2TBඞཁ େنݴޠϞσϧͷඍௐʹ͓͚Δ 24 *LoRAจͷهड़ A100 80GB (150ສԁ) × 15 🙄 -
•ඍௐޙͷϞσϧͷอଘ͕ͱͯେม • ྫ: GPT-3ͷ߹ɺඍௐͷͨͼʹ175BͷϞσϧ͕ग़དྷ্͕Δ • 175BͷϞσϧҰͭͰ350GBఔͷετϨʔδ༰ྔ* •େ͖ͳϞσϧGPUʹࡌͤΔͷ͕͍͠ • ྫ: GPT-3
fi ne-tuningͷͨΊʹGPUͷϝϞϦ͕1.2TBඞཁ •Few-shot / Zero-shot learningͳΒϞσϧͷߋ৽ෆཁ͕ͩ… • ҰൠʹͪΌΜͱϞσϧΛௐͯ͋͛͠Δํ͕ੑೳߴ͍ •ͳΒͤΊͯอଘ/࠷దԽ͢ΔύϥϝʔλΛݮΒ͍ͨ͠ • ϞσϧͷҰ෦ͷΈͷߋ৽ͰλεΫΛղ͚ͨΒخ͍͠ େنݴޠϞσϧͷඍௐʹ͓͚Δ 25 *LoRAจͷهड़ A100 80GB (150ສԁ) × 15 🙄 -
•ج൫ϞσϧͷོʹΑΓɺ෦తͳඍௐख๏ͷݚڀ͕Μʹ • ຊࢿྉͰհ͢ΔLoRA͜ͷྨͷख๏ •͍ͣΕͷख๏ʮج൫ϞσϧΛগ͚ͩ͠ௐ͢Δʯͱ͍͏ίϯηϓτ طଘݚڀ •BitFit: ϞσϧͷόΠΞε߲ͷΈߋ৽ •Adapter: ϞσϧʹλεΫઐ༻ͷΛՃ •Pre
fi x-Tuning: λεΫઐ༻ͷೖྗϕΫτϧ+ϞσϧͷҰ෦ͷΈߋ৽ •Prompt Tuning: λεΫઐ༻ͷೖྗϕΫτϧͷΈߋ৽ ෦తͳඍௐ 26 -
•TransformerͷதʹֶशՄೳͳ MLPΛࠩ͠ࠐΉख๏ • MLPඇઢܗมΛඋ͑Δ •MLPͷࠩ͠ࠐΉॴʹΑͬͯ ͍͔ͭ͘มछ͕ଘࡏ •ຊࢿྉͰհ͢ΔLoRA AdapterͷҰछ Adapter 27
Houlsby et al., Parameter-E ff i cient Transfer Learning for NLP, ICML 2019. Adapterͷࠩ͠ࠐΈํ Adapterͷߏ γϯϓϧͳMLP -
•ೖྗͷઌ಄ʹλεΫ༻ͷϕΫτϧ(Soft Prompts)Λ༻ҙ • λεΫ༻ϕΫτϧ (+தؒͷҰ෦)Λ࠷దԽ •ಉ࣌ظʹఏҊ͞Εͨख๏ • Prompt TuningSoft PromptsͷΈߋ৽
Pre fi x-Tuning / Prompt Tuning 28 Li et al., Pre fi x-Tuning: Optimizing Continuous Prompts for Generation, ACL-IJCNLP 2021. Lester et al., The Power of Scale for Parameter-E ff i cient Prompt Tuning, EMNLP 2021. LLMͰ༻͍ΒΕΔ “ࢄతͳ” promptͱରরత Prompt Tuning Pre fi x-Tuning -
ਪίετͷ૿Ճ •طଘͷAdapterΛྻతʹՃ • “ܭࢉͪ”ʹΑΓGPUͷฒྻॲཧੑೳΛ͏·͘׆͔ͤͳ͍ • ϕʔεϞσϧʹ༨ͳܭࢉίετΛಋೖͯ͠͠·͏ ࠷దԽͷ͠͞ •(ಛʹ) Pre fi
x-Tuning࠷దԽ͕ࠔͰੑೳ༧ଌ͕͖ͭͮΒ͍ • ܇࿅ՄೳͳύϥϝʔλΛ૿ͯ͠ੑೳ্͕͕Βͳ͍͜ͱ͕͋Δ(ޙड़) •ੑೳΛग़͢ʹ128 token΄ͲೖΕΔඞཁ͕͋Δ • ೖྗܥྻΛ͔ͳΓѹഭ͢Δ طଘख๏ͷ 32 -
LoRA: Low-Rank Adaptation of Large Language Models 33 •ϕʔεϞσϧͷઢܗͷྡʹ ࠩߦྻ
ΛՃ •ೋͭͷϥϯΫߦྻA, BΛಋೖ • ߦྻA: (d × r) • ߦྻB: (r × d) • r2ͳͲ͘͝খ͍͞ࣈͰOK ΔW = BA -
LoRA: Low-Rank Adaptation of Large Language Models 34 •ϕʔεϞσϧͷઢܗͷྡʹ ࠩߦྻ
ΛՃ •ೋͭͷϥϯΫߦྻA, BΛಋೖ • ߦྻA: (d × r) • ߦྻB: (r × d) • r2ͳͲ͘͝খ͍͞ࣈͰOK •શͳ fi ne-tuningͱൺֱͯ͠ • ֶशύϥϝʔλΛܶతʹݮ • ಉੑೳ ΔW = BA ϕʔεϞσϧதͷ ઢܗΛݻఆ -
LoRA: Low-Rank Adaptation of Large Language Models 35 •ϕʔεϞσϧͷઢܗͷྡʹ ࠩߦྻ
ΛՃ •ೋͭͷϥϯΫߦྻA, BΛಋೖ • ߦྻA: (d × r) • ߦྻB: (r × d) • r2ͳͲ͘͝খ͍͞ࣈͰOK •શͳ fi ne-tuningͱൺֱͯ͠ • ֶशύϥϝʔλΛܶతʹݮ • ಉੑೳ ΔW = BA ઢܗมͯ͠ ͚ͩ͢ Transformerதͷҙػߏͷ ઢܗ ͳͲ Wq , Wv ϕʔεϞσϧதͷ ઢܗΛݻఆ ҙ: ϞσϧࣗମͰͳ͘ ϞσϧதͷҰ෦ͷઢܗ -
λεΫͷΓସ͕͑؆୯ •λεΫ͝ͱʹLoRAΛ༻ҙ •ϕʔεϞσϧͦͷ··ʹLoRAͷΈΛࠩ͠ସ͑Δ͚ͩ อଘ༰ྔ͕খ͍͞ •ϕʔεϞσϧมߋෆཁɺLoRA͚ͩอଘ͢Εྑ͍ ਪίετ͕૿Ճ͠ͳ͍ •LoRAઢܗมΛ͍ͯ͠Δ͚ͩ → ΦϦδφϧͷઢܗͱϚʔδͰ͖Δ •ਪ࣌ͷલॲཧͱͯ͠ϚʔδΛઌʹ͓͚ͯ͠OK
LoRAԿ͕خ͍͠ͷ͔ʁ 36 h = W0 x + ΔWx = W0 x + BAx = (W0 + BA)x = W′  x ϕʔεϞσϧGPUʹ ࡌͤͬͺͳ͠ͰOK -
GPT-3ద༻࣌ͷྫ (r=4) •܇࿅࣌ͷVRAM (GPUͷϝϞϦ)Λ 1.2TB → 350GB ʹݮ • Gradient
CheckpointingͳͲଞͷͰ͞ΒʹݮͰ͖ͦ͏ •อଘ͢Δύϥϝʔλͷ༰ྔΛ 350GB → 35MB ʹݮ • ༰ྔ͕ݮͬͯอଘɾI/Oίετ͕ܶతʹখ͘͞ • ݸผϢʔβɾλεΫͷͨΊʹΧελϚΠζͨ͠Ϟσϧͷ࡞͕༰қʹ •܇࿅͕25%ఔߴԽ • ͦ͜·Ͱ͘ͳ͍ͬͯͳ͍ͷผͷ෦͕ϘτϧωοΫ͔ͩΒʁ LoRAͷ༗༻ੑ 37 ܇࿅༻σʔλͷసૹͱ͔ϊʔυؒ௨৴ʁ -
ॳظԽͷ •Aฏۉ0ͷਖ਼نɺBྵߦྻͰॳظԽ • ࠷ॳԿ͠ͳ͍ • LoRA͕ݩͷϞσϧΛअຐ͠ͳ͍ ϞσϧߏΛมԽͤ͞ͳ͍ •ֶशՄೳύϥϝʔλΛ૿͍ͯ͘͠ͱ… • LoRA:
r͕ݩͷߦྻͱಉ͡ = ݩͷϞσϧͱ΄΅ಉ͡ • Adapter: MLPΛՃͨ͠ϞσϧΛFull FT͢Δͷͱಉ • Prompt Tuning: ೖྗ͕গ͘͠ͳͬͨϞσϧΛFull FT͢Δͷͱಉ ΔW = BA LoRAͳͥ͏·͍͘͘ͷ͔ʁ 38 -
•LoRAࣗମͦ͜·Ͱ৽͍͠Ͱͳ͍͕… • Adapter͔ΒඇઢܗੑΛൈ͍͚ͨͩͱ͍͏ؾ͢Δ ͓͠ΖϙΠϯτ •AdapterΑΓগͳ͍ύϥϝʔλͰੑೳग़ͤΔ •AdapterͷΑ͏ʹඇઢܗੑΛಋೖ͠ͳͯ͘ੑೳग़ͤΔ ҙ •10BҎ্ͷϞσϧͩͱLoRAͰ͢Β20MҎ্ͷύϥϝʔλʹͳΔ͜ͱ •Ϟσϧ͕ेେ͖͘ͳ͍ͱ͏·͘ಈ࡞͠ͳ͍͔ •
6.7Bͩͱ͏·͍͕͘͘ɺ1BϞσϧͩͱ͏·͘ඍௐͰ͖ͳ͍ͱ͍͏ࣄྫ LoRAͷ৽نੑɾҙ 40 -
•LoRAࣗମͦ͜·Ͱ৽͍͠Ͱͳ͍͕… • Adapter͔ΒඇઢܗੑΛൈ͍͚ͨͩͱ͍͏ؾ͢Δ ͓͠ΖϙΠϯτ •AdapterΑΓগͳ͍ύϥϝʔλͰੑೳग़ͤΔ •AdapterͷΑ͏ʹඇઢܗੑΛಋೖ͠ͳͯ͘ੑೳग़ͤΔ ҙ •10BҎ্ͷϞσϧͩͱLoRAͰ͢Β20MҎ্ͷύϥϝʔλʹͳΔ͜ͱ •Ϟσϧ͕ेେ͖͘ͳ͍ͱ͏·͘ಈ࡞͠ͳ͍͔ •
6.7Bͩͱ͏·͍͕͘͘ɺ1BϞσϧͩͱ͏·͘ඍௐͰ͖ͳ͍ͱ͍͏ࣄྫ LoRAͷ৽نੑɾҙ 41 ͦΜͳʹখ͘͞ͳ͍… ಉ͡όονʹҟͳΔ λεΫΛೖΕΔͷ গ͍͠͠ খ͞ΊͷϞσϧͩͱͲͷ͘Β͍ ϋΠύϥௐ͕ඞཁ / sensitive ͔एׯෆ໌ -
•LoRAAdapterͷҰछ • ඇઢܗੑΛഉͨ͠Adapterͷಛघܥͱଊ͑ΒΕΔ •Pre fi x- / Prompt TuningΑΓֶशύϥϝʔλଟ͘ͳΔ LoRAͱطଘख๏ͱͷؔ
42 ख๏໊ fi ne-tuning Adapter LoRA Pre fi x-Tuning / Prompt Tuning few-shot / zero-shot learning ࠷దԽର ͯ͢ MLP (ଟύʔηϓτϩϯ) ϥϯΫߦྻ Soft Prompts + α - ਪίετ ૿Ճ͢Δʁ No Yes No Yes (ܥྻ͕૿Ճ) - -
•ਂֶशͰߦྻܭࢉ͕ඞཁෆՄܽ • ઢܗ(ॏΈߦྻ + όΠΞε)ಛʹසग़ •ײతʹΨϦόʔτϯωϧΈ͍ͨͳͷ • ϕΫτϧΛೖΕΔͱग़ޱͷܗʹͳͬͯग़ͯ͘Δ • ϕΫτϧΛҙͷܗʹมߋͰ͖Δػߏ
• ྫ: ྨ(ϕΫτϧ࣍ݩ, Ϋϥε)Ͱมܗ ؓٳ: ߦྻ (ςϯιϧ) ܭࢉͷײతཧղ 43 ը૾ ςϨϏே ʰͻΈͭಓ۩ΧλϩάʱΑΓҾ༻ ΨϦόʔτϯωϧ nn.Linear(5, 3) ग़ޱͷܗʹ֦େɾॖখ͢ΔͻΈͭಓ۩ -
•LoRAͷ༗༻ੑΛ fi ne-tuning, AdapterͳͲͱൺֱͯ͠ݕূ •ෳछྨͷλεΫ • ࣗવݴޠཧղ (Natural Language Understanding;
NLU) ܥλεΫ • ࣗવݴޠੜ (Natural Language Generation; NLG) ܥλεΫ •ෳͷϞσϧ • RoBERTa base (125M) / large (355M) • DeBERTa XXL (1.5B) • GPT-2 medium (355M) / large (774M) • GPT-3 (175B) ධՁ࣮ݧ 44 -
•WikiSQLMNLI, SAMSumͰධՁɺLoRA͕ߴ͍ੑೳ • ෳͷrͰ fi ne-tuningΛ্ճΔੑೳΛࣔ͢͜ͱ NLU & NLG: GPT-3
175B 47 ࣗવݴޠˠSQL ձཁ ࣗવݴޠਪ -
•WikiSQLMNLI, SAMSumͰධՁɺLoRA͕ߴ͍ੑೳ • fi ne-tuningfew-shotΑΓ ֨ஈʹੑೳ͕ߴ͍ˠ • fi ne-tuning͕ऑ͍Θ͚Ͱͳ͍ ͳͥLoRA
fi ne-tuningͱಉʁ •ݩͷϞσϧʹඞཁͳ͕ࣝଘࡏ •LoRAͦΕΛڧௐ͢Δ͚ͩ* • ࣮ͦΕ͚ͩͰेͩͬͨʁ NLU & NLG: GPT-3 175B 48 *ݩจͷ7અΛࢀর ࣗવݴޠˠSQL ձཁ ࣗવݴޠਪ fi ne-tuningࣗମ༗ޮ -
•GPT-3ΛLoRAνϡʔχϯάɺֶशՄೳύϥϝʔλͱੑೳͷؔΛ؍ •LoRAൺֱతੑೳ͕҆ఆ͍ͯͯ͠ѻ͍ͦ͢͏ • Pre fi x Tuningੑೳ͕ෆ҆ఆ ֶशՄೳύϥϝʔλͱੑೳͷؔ 50 256
tokens 256 tokens 32 tokensҎԼ 32 tokensҎԼ LoRA҆ఆ -
•ϞσϧͷύϥϝʔλΛ෦తʹඍௐ͢Δख๏ΛఏҊ •ೋͭͷϥϯΫߦྻA, BΛಋೖ •ֶशύϥϝʔλΛܶతʹݮɺ fi ne-tuningͱಉੑೳ ॴײ •ݚڀతͳ৽نੑΑΓɺૉͰศརͳઃܭʹྗΛஔ͍ͨख๏ (ҹ) •Appendix͔ΒϋΠύϥௐͷͨΊͷਘৗͳΒ͟Δྗͷ͕Ӑ͑Δ
• ͦ͜·ͰϋΠύϥʹsensitiveͳख๏Ͱͳͦ͞͏͕ͩ…(ମײ) •ઢܗมͷΈͷߏʹΑΓϕʔεϞσϧʹ༥߹ͤ͞ΒΕΔͷ໘ന͍ •ͱͯγϯϓϧͳख๏ͳͷͰద༻ൣғ͔ͳΓͦ͏ ·ͱΊɾॴײ 53 -
ࣄલࣝ •ਂֶशΛ༻͍ͨࣗવݴޠॲཧͷجૅ •ҙػߏɺTransformerɺBERT, … LoRA •େنݴޠϞσϧͷඍௐʹ͓͚Δͱղܾࡦ •ؔ࿈ݚڀ •ఏҊख๏ɾධՁ࣮ݧ LoRAͷ͍ํ •PEFT:
Parameter-E ff i cient Fine-Tuning ࣍ 54 -
•HuggingFace͕ఏڙ͢ΔϥΠϒϥϦ • https://github.com/huggingface/peft • LoRAPrompt Tuning, AdaLoRA (LoRAͷޙଓख๏) ͕࣮ PEFT:
Parameter-E ffi cient Fine-Tuning 🤗 57 -
•HuggingFace͕ఏڙ͢ΔϥΠϒϥϦ • https://github.com/huggingface/peft • LoRAPrompt Tuning, AdaLoRA (LoRAͷޙଓख๏) ͕࣮ •PEFTΛ༻͍ͨLoRAʹΑΔඍௐͷྲྀΕ
1. ϕʔεͱͳΔϞσϧΛ༻ҙ͢Δ 2. ϕʔεϞσϧͷͷҰ෦ΛLoRAͷʹஔ 3. LoRAͷҎ֎ͷΛݻఆ(freeze) 4. LoRA෦ͷΈΛֶश •อଘ͢ΔͷՃͨ͠ͷΈ (ϕʔεϞσϧͱൺֱͯ͠ۃΊͯখ͍͞) PEFT: Parameter-E ffi cient Fine-Tuning 🤗 58 -
PEFT: ࣮ࡍͷίʔυ 63 LoRAͷద༻ରΛ Ҿ(target_modules)Ͱ ࢦఆ͢Δ͜ͱՄೳ ͜Ε͕ͳ͍ͱಈ͔ͳ͍͜ͱ͕ ͋ΔͷͰҙ ਖ਼نදݱʹΑΔ ϞδϡʔϧࢦఆՄೳ
ྫ: .*Attention.(q|k|v|o)$ -
•GoogleͷFlan-UL2 (20B)ʹରͯ͠PEFTΛ༻͍ͯखݩͷλεΫʹద༻ • ੑೳྑɺอଘ༰ྔখ͘͞܇࿅ૣ͍ • ྫ (r=16): ܇࿅ύϥϝʔλ 25MɺVRAM༻ྔ 55GiB
PEFT: ࣮ࡍʹ৮ͬͨײ 64 Batch size 32, BF16, Gradient Checkpointing -
•GoogleͷFlan-UL2 (20B)ʹରͯ͠PEFTΛ༻͍ͯखݩͷλεΫʹద༻ • ੑೳྑɺอଘ༰ྔখ͘͞܇࿅ૣ͍ • ྫ (r=16): ܇࿅ύϥϝʔλ 25MɺVRAM༻ྔ 55GiB
༨ஊ •Flan-T5 XXL (11B) ͳͲFlan instruction tuning͞ΕͨϞσϧۃΊͯڧྗ • खݩͷλεΫͰۃΊͯߴ͍Zero-shotੑೳɺLoRAνϡʔχϯάྑ • ͜ͷลΓɺInstructGPTChatGPTʹ௨ͣΔͷ͕͋Γͦ͏ •ӳޠλεΫͱΓ͋͑ͣFlan-T5 XXL / UL2 + LoRAΛߟ͑ͯྑͦ͞͏ •Flan-UL2Flan-T5Encoder-DecoderϞσϧͷT5ϕʔεͳͷͰࣗ༝ࣗࡏ • BERTͷΑ͏ʹEncoderͷΈͬͯྑ͍͠ɺੜͤͯ͞ྑ͍ PEFT: ࣮ࡍʹ৮ͬͨײ 65 Batch size 32, BF16, Gradient Checkpointing -
•GoogleͷFlan-UL2 (20B)ʹରͯ͠PEFTΛ༻͍ͯखݩͷλεΫʹద༻ • ੑೳྑɺอଘ༰ྔখ͘͞܇࿅ૣ͍ • ྫ (r=16): ܇࿅ύϥϝʔλ 25MɺVRAM༻ྔ 55GiB
༨ஊ •Flan-T5 XXL (11B) ͳͲFlan instruction tuning͞ΕͨϞσϧۃΊͯڧྗ • खݩͷλεΫͰۃΊͯߴ͍Zero-shotੑೳɺLoRAνϡʔχϯάྑ • ͜ͷลΓɺInstructGPTChatGPTʹ௨ͣΔͷ͕͋Γͦ͏ •ӳޠλεΫͱΓ͋͑ͣFlan-T5 XXL / UL2 + LoRAΛߟ͑ͯྑͦ͞͏ •Flan-UL2Flan-T5Encoder-DecoderϞσϧͷT5ϕʔεͳͷͰࣗ༝ࣗࡏ • BERTͷΑ͏ʹEncoderͷΈͬͯྑ͍͠ɺੜͤͯ͞ྑ͍ PEFT: ࣮ࡍʹ৮ͬͨײ 66 Batch size 32, BF16, Gradient Checkpointing T5 (11B) ͳΒVRAM 40GBͰ܇࿅ՄೳɺA6000Ͱಈ͘ʂ BERTͷସͱ͙ͯ͢͠ಈ͔ͤΔϞσϧײ