【ACL読み会2026@名古屋大学】Find Your Optimal Teacher: Personalized Data Synthesis via Router-Guided Multi-Teacher Distillation

知識蒸留において生徒モデルによく合う教師モデルをクエリごとに選択する方法
対象論文：https://aclanthology.org/2026.acl-long.666/

Transcript

1. ACL https://aclanthology.org/2026.acl-long.666/ Find Your Optimal Teacher Personalized Data Synthesis via Router Guided Multi Teacher Distillation Hengyuan Zhang¹ Shiping Yang Xiao Liang Chenming Shang⁴ Yuxuan Jiang⁵ Chaofan Tao¹ Jing Xiong¹ Hayden Kwok Hay So¹ Ruobing Xie⁶ Angel X Chang ⁷ Ngai Wong¹ ¹The University of Hong Kong Simon Fraser University University of California Los Angeles ⁴Dartmouth College ⁵University of Maryland Baltimore County ⁶Tencent ⁷Canada CIFAR AI Chair Amii hengyuan zhang88 gmail com
2. 1 LLM SFT 1 2 1 2
3. 1 4 Mix CAR Small Models Struggle to Learn from Strong Reasoners Mix Li et al 2025b Findings ACL 2025 CoT PerSyn CAR Xu et al 2025b NAACL 2025 Stronger Models are Not Always Stronger Teachers for Instruction Tuning CAR Li et al Mix Compatibility Adjusted Reward Findings ACL 2025 Xu et al NAACL 2025 PerSyn 1 4 A 1 3
4. 2 2 2 Generate then Select Route then Generate 1 1 3 3 1 1 PerSyn 2 1 2 2 1 PDF p 3 3 4
5. 2 2 2 PerSyn 1 2 2 PDF p 3 5
6. 2 2 2 SFT 1 PDF p 3 6
7. 𝑟 𝑦, 𝜃 = yt 1 𝑦 log 𝑝 𝑦 ∣ 𝑦 ,𝑥 y<t
8. 𝑟 𝑦, 𝜃 = 1 − 𝛼 𝑟 𝑦 + 𝛼𝑟 𝑦, 𝜃
9. o = π(x) = (oA, oB, …)
10. 2 2.3 2,500 r B A x C=1 C=0 𝑝=𝑃 𝐵 ≻𝐴 ∣𝑥 =𝜎 𝑜 −𝑜 Bradley–Terry oA oB p x 0 B 1 A ℒ 𝑝, 𝐶 = − 𝐶 log 𝑝 + 1 − 𝐶 log 1 − 𝑝 L C=1 B p p p 1 C C=0 A 0 C 2.3 (3)(4) A.2 Router Training 10
11. – –
12. 3 3 1 Magpie Zoo 50K PerSyn Math 19 15 6 IFEval TruthfulQA LiveBench Qwen2 5 0 5B 3 1 5 1 5B Table 10 Table 11 10K 7 GSM8K MATH SVAMP 0 5B 3B 3B Gemma 2 2B Llama 3 2 3B 7B 14B 12
13. 3 3 1 A 1 Strong Mix Family Strong CAR PerSyn 3 1 Appendix A 1 Table 6 7 13
14. 3 3 2 6 6 Strong Mix Family CAR PerSyn Qwen2 5 0 5B 28 51 30 75 32 03 32 77 34 13 Qwen2 5 1 5B 46 82 47 79 48 64 49 21 50 63 Gemma 2 2B 28 45 29 76 29 56 31 41 32 85 Qwen2 5 3B 55 38 55 39 56 59 57 17 58 09 Llama 3 2 3B 31 37 31 75 30 85 32 99 34 81 Family Family Strong 6 30 28 Table 2 Avg 14
15. 3 3 2 PerSyn Qwen2 5 3B IFEval Strong 40 61 44 16 Llama 3 2 3B MATH CAR 22 80 24 50 Qwen2 5 3B MATH CAR 58 10 57 80 Gemma 2 2B LiveBench CAR 12 80 12 40 2 2 3 55 8 7 1 70 7 5 seed Table 2 3 2 15
16. 3 3 3 2 4 0 4 6 3 0 6 2 4 3 3 PDF p 6 0 4 0 4 16
17. 3 3 3 5 K Hit 3 3 0 5B 1 5B 3B Qwen2 5 3B 500K 1 5B 5 3 3 PDF p 7 9 p 15 17
18. 3 3 3 Oracle Oracle PerSyn Oracle Qwen2 5 0 5B 27 18 27 63 Qwen2 5 1 5B 33 72 34 36 Gemma 2 2B 29 17 28 84 Qwen2 5 3B 40 53 41 02 Llama 3 2 3B 30 35 30 18 Oracle PerSyn Oracle Oracle Table 3 Table 9 18
19. 3 3 3 6 7 Qwen2 5 72B Qwen2 5 3B DeepSeek R1 37B 6 7 3 3 PDF p 8 19
20. 3 3 3 Long CoT Long CoT Long CoT Long CoT Qwen2 5 Math 7B 1 3 7 4 Qwen2 5 3B Short CoT 1 3 3 3 Figure 7 Finding 4 20
21. 5 Limitations 14B 5 Limitations 21
22. • • • • LLM • LLM 8 CAR P.25 Find Your Optimal Teacher Personalized Data Synthesis via Router Guided Multi Teacher Distillation 22
23. N N K K K n K N n n 1 N PerSyn n 1 50K 1 19 2 5K 10K 950K NK 95K 15 2 5K 150K 1 10 45K 1 3 33 Strong 2 2 A 2–A 3 23
24. 2 500 19 427 500 171 190 500K ID seed 2 1 3 2 A 1–A 2 GPU 24
25. 3 3 A 1 7B 14B 4 Qwen2 5 7B 14B Llama 3 1 8B 8B 3B PerSyn CAR 3 Gemma 2 9B 3B 8 32B 3 3 3 12 A 1 8 25
26. A 1 A 3 SFT OpenR1 10K 0 6 2 4 1 15 Math Verify 1 2 epoch SFT 2 10⁻⁵ 4 096 32 16 384 Gemma IFEval strict accuracy GSM8K 5 shot zero shot LiveBench 2024 11 25 72B 2 2 Long CoT A 1 4 2 A 3 10 4 PDF 8 192 greedy A 3 26