[輪講資料] One Embedder, Any Task: Instruction-Finetuned Text Embeddings

指示に基づいた文埋め込みを生成するモデル InstructOR について解説した輪講資料です。

元論文: https://aclanthology.org/2023.findings-acl.71/

Transcript

1. One Embedder, Any Task:
 Instruction-Finetuned Text Embeddings D1, Graduate School of Informatics, Nagoya University, Japan Hayato Tsukagoshi Hongjin Su, Weijia Shi, Jungo Kasai, Yizhong Wang, Yushi Hu, Mari Ostendorf, Wen-tau Yih
 Noah A. Smith, Luke Zettlemoyer, Tao Yu ACL 2023 Findings
 https://aclanthology.org/2023. fi ndings-acl.71/
2. •จຒΊࠐΈ͸Out Of Domain (OOD) ʹऑ͍ •ࢦࣔʹैͬͯจΛຒΊࠐΉϞσϧ
 InstructORΛఏҊ • λεΫɾυϝΠϯ͝ͱʹҟͳΔࢦࣔ
 ͰจຒΊࠐΈΛੜ੒Մೳ •ଟ༷ͳσʔληοτऩू
 →ࢦࣔΛΞϊςʔτ (MEDI dataset)
 →ࢦࣔ+จΛຒΊࠐΈදݱʹ
 →ରরֶशͰϞσϧΛ܇࿅ •MTEBͳͲෳ਺ͷϕϯνϚʔΫͰฏۉͯ͠ੑೳ޲্ ֓ཁ 2
3. •࠷ۙจΛquery, passageͰ৔߹෼͚ͯ͠ຒΊࠐΉϞσϧ͕ڧ͍(E5) • LLM+ICLͷ༂ਐ΋͋ΓɺInstructOR͸ͦͷࣗવͳൃలʹࢥ͑Δ • ʮͲ͏΍ͬͯจΛຒΊࠐΉ͔ʯΛϢʔβ͕ૢ࡞Ͱ͖Δ࣌୅͕དྷͨ •ݸਓతʹ͔ͳΓ޷͖ͳ࿩ ໔੹ࣄ߲ •εϥΠυதͷਤද͸֤εϥΠυͰݴٴ͞Ε͍ͯΔ࿦จ͔ΒͷҾ༻Ͱ͢ •࿦จதͷ਺ࣜͱ͸ҟͳΔจࣈΛ࢖͍ͬͯΔ৔߹͕͋Γ·͢ બఆཧ༝ 3
4. •܇࿅ख๏ •MEDI dataset •ධՁ࣮ݧ ໨࣍ 4
5. •ࣗવݴޠจͷີϕΫτϧදݱ •ϕΫτϧͷڑ཭͕จͷҙຯͷۙ͞Λදݱ ಋೖ: จຒΊࠐΈ / Sentence embedding 5 ͜Ͳ΋͕Ոʹ޲͔͍ͬͯΔɻ ͜Ͳ΋ֶ͕ߍ͔ΒՈʹ޲͔͍ͬͯΔɻ ͜Ͳ΋͕ਤॻؗʹ͍Δɻ ͜Ͳ΋͕ޕޙʹา͍͍ͯΔɻ จຒΊࠐΈۭؒ [0.1, 0.2, ...] [0.1, 0.3, ...] [0.9, 0.8, ...] [0.5, 0.7, ...]
6. •ࣗવݴޠจͷີϕΫτϧදݱ •ϕΫτϧͷڑ཭͕จͷҙຯͷۙ͞Λදݱ ಋೖ: จຒΊࠐΈ / Sentence embedding 6 ͜Ͳ΋͕Ոʹ޲͔͍ͬͯΔɻ ͜Ͳ΋ֶ͕ߍ͔ΒՈʹ޲͔͍ͬͯΔɻ ͜Ͳ΋͕ਤॻؗʹ͍Δɻ ͜Ͳ΋͕ޕޙʹา͍͍ͯΔɻ จຒΊࠐΈۭؒ [0.1, 0.2, ...] [0.1, 0.3, ...] [0.9, 0.8, ...] [0.5, 0.7, ...] ҙຯతʹྨࣅ ͍ۙҙຯΛ࣋ͭจ͸ ۙ͘ʹ෼෍ ϕΫτϧؒͷڑ཭͕
 ҙຯతͳؔ܎Λදݱ
7. •จຒΊࠐΈ͸Out Of Domain (OOD) ʹऑ͍ •ࢦࣔʹैͬͯจΛຒΊࠐΉϞσϧ
 InstructORΛఏҊ • Instruction-based Omnifarious Representations • จ͝ͱʹࢦࣔΛม͑ଟ༷ͳຒΊࠐΈΛੜ੒ •ࢦ͕ࣔ͋Δ͜ͱͰOOD΁ͷੑೳ΋޲্ ख๏֓ཁ •ଟ༷ͳσʔληοτऩू
 →ࢦࣔΛΞϊςʔτ (MEDI dataset)
 →ࢦࣔ+จΛຒΊࠐΈදݱʹ
 →ରরֶशͰϞσϧΛ܇࿅ ख๏֓ཁ 7
8. •ൺֱతγϯϓϧͳରরֶशʹΑΔ܇࿅ •ࢦࣔ+จΛ·Δ͝ͱϞσϧʹೖྗ͢Δ • ࢦࣔΛߟྀͨ͠จຒΊࠐΈΛੜ੒ ܇࿅खॱ 1. ࢦࣔͱจͷϖΞ(x, Ix, y, Iy)Λ༻ҙ 2. Ex = E(Ix⊕x), Ey = E(Iy⊕y)ͷΑ͏ʹຒΊࠐΉ 3. ਖ਼ྫಉ͕࢜ۙͮ͘Α͏ʹରরֶश •ෛྫͱͯ͠in-batch negativesΛར༻ InstructOR: ܇࿅खॱ 8 Ix⊕x Iy⊕y Model Model ਖ਼ྫͷຒΊࠐΈΛ͚ۙͮΔ ଛࣦؔ਺
9. •܇࿅ख๏ •MEDI dataset •ධՁ࣮ݧ ໨࣍ 9
10. •InstructORͷ܇࿅ʹ͸
 ࢦࣔͱจ͕ϖΞʹͳͬͨσʔληοτ͕ඞཁ •طଘσʔληοτΛ܇࿅༻ʹ౷߹ (ܭ300ݸ) • Super-NaturalInstructions (super-NI) • Sentence Transformers embedding data MEDI: Multitask Embedding Data with Instructions 10
11. •InstructORͷ܇࿅ʹ͸
 ࢦࣔͱจ͕ϖΞʹͳͬͨσʔληοτ͕ඞཁ •طଘσʔληοτΛ܇࿅༻ʹ౷߹ (ܭ300ݸ) • Super-NaturalInstructions (super-NI) • Sentence Transformers embedding data •super-NI͸ࢦࣔͱจ͕ϖΞʹͳ͍ͬͯΔ͕
 ਖ਼ྫɾෛྫ͕ଘࡏ͠ͳ͍ → Sentence-T5ͰจຒΊࠐΈΛੜ੒ (ࢦࣔͳ͠)
 → ྨࣅ౓Λ࢖ͬͯਖ਼ෛྫϖΞΛࣗಈੜ੒ MEDI: Multitask Embedding Data with Instructions 11
12. •ςΩετ෼ྨܥͷσʔληοτ͔Βͷ࡞੒͸݁ߏָ खॱ 1. σʔληοτ͔Βೖྗจಉ࢜ͷྨࣅ౓͕ߴ͍จϖΞΛબͿ 2. ςΩετ෼ྨʹ͓͚Δϥϕϧ͕ಉ͡ͳΒਖ਼ྫɾҧ͏ͳΒෛྫʹ͢Δ MEDI: ਖ਼ྫɾෛྫϖΞͷࣗಈੜ੒ख๏ 12
13. •Seq2Seqܥͷσʔληοτ͸ϥϕϧ͕ͳ͍ͷͰ޻෉͕͍Δ •ਖ਼ྫ༻ͷείΞSpos ͱෛྫ༻ͷείΞSneg Λ༻ҙ •࠷΋Spos ͕ߴ͍ϖΞΛਖ਼ྫɺSneg ͕ߴ͍ϖΞΛෛྫʹ MEDI: ਖ਼ྫɾෛྫϖΞͷࣗಈੜ੒ख๏ 13 ೖྗจͷྨࣅ౓ ग़ྗจͷྨࣅ౓ ೖྗͱग़ྗ͕ڞʹྨࣅ͍ͯ͠ΔͱߴείΞ ೖྗจͷྨࣅ౓ ग़ྗจͷྨࣅ౓ ೖྗ͸ࣅ͍ͯΔ͕ग़ྗ͸ࣅ͍ͯͳ͍ͱߴείΞ
14. •Seq2Seqܥͷσʔληοτ͸ϥϕϧ͕ͳ͍ͷͰ޻෉͕͍Δ •ਖ਼ྫ༻ͷείΞSpos ͱෛྫ༻ͷείΞSneg Λ༻ҙ •࠷΋Spos ͕ߴ͍ϖΞΛਖ਼ྫɺSneg ͕ߴ͍ϖΞΛෛྫʹ MEDI: ਖ਼ྫɾෛྫϖΞͷࣗಈੜ੒ख๏ 14 ೖྗจͷྨࣅ౓ ग़ྗจͷྨࣅ౓ ೖྗͱग़ྗ͕ڞʹྨࣅ͍ͯ͠ΔͱߴείΞ ೖྗจͷྨࣅ౓ ग़ྗจͷྨࣅ౓ ೖྗ͸ࣅ͍ͯΔ͕ग़ྗ͸ࣅ͍ͯͳ͍ͱߴείΞ
15. •Seq2Seqܥͷσʔληοτ͸ϥϕϧ͕ͳ͍ͷͰ޻෉͕͍Δ •ਖ਼ྫ༻ͷείΞSpos ͱෛྫ༻ͷείΞSneg Λ༻ҙ •࠷΋Spos ͕ߴ͍ϖΞΛਖ਼ྫɺSneg ͕ߴ͍ϖΞΛෛྫʹ MEDI: ਖ਼ྫɾෛྫϖΞͷࣗಈੜ੒ख๏ 15 ೖྗจͷྨࣅ౓ ग़ྗจͷྨࣅ౓ ೖྗͱग़ྗ͕ڞʹྨࣅ͍ͯ͠ΔͱߴείΞ ೖྗจͷྨࣅ౓ ग़ྗจͷྨࣅ౓ ೖྗ͸ࣅ͍ͯΔ͕ग़ྗ͸ࣅ͍ͯͳ͍ͱߴείΞ
16. •Seq2Seqܥͷσʔληοτ͸ϥϕϧ͕ͳ͍ͷͰ޻෉͕͍Δ •ਖ਼ྫ༻ͷείΞSpos ͱෛྫ༻ͷείΞSneg Λ༻ҙ •࠷΋Spos ͕ߴ͍ϖΞΛਖ਼ྫɺSneg ͕ߴ͍ϖΞΛෛྫʹ MEDI: ਖ਼ྫɾෛྫϖΞͷࣗಈੜ੒ख๏ 16 ೖྗจͷྨࣅ౓ ग़ྗจͷྨࣅ౓ ೖྗͱग़ྗ͕ڞʹྨࣅ͍ͯ͠ΔͱߴείΞ ೖྗจͷྨࣅ౓ ग़ྗจͷྨࣅ౓ ೖྗ͸ࣅ͍ͯΔ͕ग़ྗ͸ࣅ͍ͯͳ͍ͱߴείΞ hard negative
 తͳཱͪҐஔ
17. •ࢦࣔʹجͮ͘จຒΊࠐΈͷͨΊʹ͸ɺࢦ͕ࣔඞཁ(౰વ) •ҎԼͷ৘ใͱςϯϓϨʔτʹج͍ͮͯσʔληοτ͝ͱʹࢦࣔΛ࡞੒ •Text Type • ຒΊࠐΉจ͕Ͳ͏͍͏΋ͷ͔ • QAͷqueryͳΒʮ࣭໰จʯ • passageͳΒʮจॻʯ •Task Objective (optional) • ຒΊࠐΈ͕Կʹ࢖ΘΕΔ͔ •Domain (optional) • ʮχϡʔεʯͳͲจͷυϝΠϯ MEDI: ࢦࣔͷߏங 17
18. •ࢦࣔ͸ҎԼͷςϯϓϨʔτ͔Βߏங •“Represent The (DOMAIN) TEXT TYPE for TASK OBJECTIVE:.” •࣮ࡍͷࢦ͕ࣔҎԼ • ॏཁ: จͷλΠϓʹΑͬͯࢦࣔͰຒΊࠐΈํΛม͑ΒΕΔ MEDI: ࢦࣔͷߏங 18 ্ද͸࿦จதͷද͔Βൈਮͨ͠΋ͷ
19. •ଟ༷ͳσʔληοτͷจϖΞ͔Β
 ͳΔେن໛ͳσʔληοτ •ྨࣅ౓͕ରশ/ඇରশͳ΋ͷ͕
 ͍ࠞͬͯ͟Δ • ྫ: ݕࡧͷquery—passage •ΞελϦεΫ(*)͕͍͍ͭͯΔ
 σʔληοτ͸test setΛؚΉ • ஫ҙ:
 NQ, HotPotQA͸ޙड़͢ΔMTEB
 ʹؚ·Ε͓ͯΓɺϦʔΫ͍ͯ͠Δ MEDI: σʔληοτͷ಺༁ 19
20. •ࢦࣔʹैͬͯλεΫΛղ͚ΔΑ͏ʹLMΛtuning (Instruction Tuning) •zero-shotͰ৭ʑͳੜ੒ܥλεΫ͕͏·͘ղ͚Δ •InstructOR͸FLANͷຒΊࠐΈ൛ͱ΋ݴ͑Δ •FLAN͸Finetuned Language Net ͷུΒ͍͠🧐 Wei+: Finetuned Language Models Are Zero-Shot Learners, ICLR ’22 ؔ࿈ݚڀ: FLAN 20 Flan-T5, Flan-UL2 ͸ΊͪΌͪ͘Όڧ͍ϞσϧͳͷͰ͓͢͢Ίʂ(͜ͷ࣌୅Ͱ΋·ͩڧ͍)
21. •܇࿅ख๏ •MEDI dataset •ධՁ࣮ݧ ໨࣍ 21
22. •ෳ਺ͷϕϯνϚʔΫͰ޿ൣʹੑೳΛධՁ • MTEB: จຒΊࠐΈͷେن໛ϕϯνϚʔΫ • Billboard (ʹΑΔධՁ): ੜ੒͞Εͨཁ໿ͱਖ਼ղཁ໿ͱͷྨࣅ౓ͷ૬ؔ • Prompt Retrieval: ੜ੒ʹ࠷దͳϓϩϯϓτͷݕࡧ ධՁ࣮ݧ 22
23. •58ͷσʔληοτɺ112ͷݴޠ͔Β
 ͳΔจຒΊࠐΈͷϕϯνϚʔΫ •STSΛ͸͡Ί8छྨͷλεΫ͕͋Δ •(༨ஊ) MTEBͷ࣮ݧ݁Ռ͔Β
 จຒΊࠐΈͰ΋σΧ͍Ϟσϧ
 ͷํ͕ڧ͍܏޲͕ݟ͑Δ •ධՁ؍఺: InstructOR͕
 ͲΕ͘Β͍ྑ͍จຒΊࠐΈΛੜ੒Ͱ͖Δ͔ •ಛʹɺࢦࣔΛ෇Ճ͢Δ͜ͱͰ
 طଘख๏ΑΓੑೳ͕޲্͢Δ͔͕ؾʹͳΔ Muennigho ff +: MTEB: Massive Text Embedding Benchmark, arXiv ’22 ධՁ࣮ݧ: Massive Text Embedding Benchmark (MTEB) 23 Sentence-BERT ͷஶऀͷ Nils Reimers ͕last authorͷproject
24. •ੜ੒ϞσϧɾධՁࢦඪͷ૒ํΛ
 ొ࿥Ͱ͖ΔϦʔμʔϘʔυ(ϑϨʔϜϫʔΫ) • ֤ࢦඪΛΞϯαϯϒϧͨ͠ࢦඪ΋ࣗಈࢉग़ • ੜ੒Ϟσϧͷ։ൃͱධՁΛ૬ޓʹଅਐ •ධՁ؍఺: ੜ੒จ—ਖ਼ղཁ໿ͷྨࣅ౓͕
 ͲΕ͘Β͍ਓؒධՁͱ͍͔ۙ •ੜ੒จͱ֤ਖ਼ղཁ໿ͱͷcosྨࣅ౓ͷ
 ࠷େ஋ͱਓखධՁͱͷϐΞιϯ૬ؔ •3ͭͷσʔληοτͰͦΕͧΕ૬ؔ܎਺Λࢉग़ɺฏۉΛධՁ஋ʹ Kasai+: Bidimensional Leaderboards: Generate and Evaluate Language Hand in Hand, NAACL ’22 ධՁ࣮ݧ: Billboard (Ͱͷཁ໿ͷྨࣅ౓ධՁ) 24
25. •In-Context Learning͸ςετ࣌ͷfew-shotࣄྫΛબͿඞཁ͕͋Δ • ࣄલʹগ਺ʹߜΔͷ͸େมɾͲΕ͕͍͍͔Θ͔Βͳ͍ •few-shotࣄྫΛຒΊࠐΈʹม׵ˠςετࣄྫʹ͍ۙࣄྫΛೖΕͯੑೳ޲্ •ධՁ؍఺: ੜ੒͕ͲΕ͚ͩ͏·͘ग़དྷΔ͔ • ੑೳͷد༩͢Δྑ͍few-shotࣄྫΛ࣋ͬͯདྷΕΔ͔Ͳ͏͔͕ධՁई౓ Su+: Selective Annotation Makes Language Models Better Few-Shot Learners, arXiv ’22 ධՁ࣮ݧ: Prompt Retrieval 25 ίϝϯτ: LLM࣌୅ͷSentEvalతͳཱͪҐஔʹݟ͑Δ (SentEval: ຒΊࠐΈϕʔεͷઢܗ෼ྨثΛֶशɾͦͷਫ਼౓ͰධՁ) ੜ੒͸GPT-J
26. SimCSE: ҟͳΔDropoutΛద༻ͨ͠ಉ͡จΛਖ਼ྫ or ؚҙؔ܎ͷจϖΞΛਖ਼ྫͱͨ͠ରরֶश Contriever: ϕΫτϧݕࡧʹ͓͚ΔରরֶशͷݶքΛௐࠪɼMS MARCOͰͷ fi ne-tuningͰੑೳ޲্֬ೝ GTR: ScalingͰจຒΊࠐΈͰ΋ϕΫτϧݕࡧੑೳ & ൚Խੑೳ޲্Λ֬ೝ coCondenser: ࣄલ܇࿅࣌ʹίʔύεґଘͷଛࣦΛՃ͑Retrieverͷ܇࿅Λؤ݈ʹ&ੑೳ޲্ [00] Sentence-T5: T5ΛNLI/QAσʔλͰରরֶशʹΑΔ fi ne-tuning → scaling lawͷௐࠪ + SentGLUEͰධՁ SGPT: GPTΛ৘ใݕࡧλεΫʹར༻ɼCross-EncoderܗࣜͱBi-Encoderܗࣜͷ྆ํΛBEIRͰ࣮ݧ Gao+: SimCSE: Simple Contrastive Learning of Sentence Embeddings, EMNLP ’21 Izacard+: Unsupervised Dense Information Retrieval with Contrastive Learning, TMLR ’23 Ni+: Large Dual Encoders Are Generalizable Retrievers, EMNLP ’22 Gao+: Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrieval, ACL ’22 Ni+: Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Models, CoRR ’21 Muennigho ff : SGPT: GPT Sentence Embeddings for Semantic Search, arXiv ’22 ධՁ࣮ݧ: ൺֱख๏ 26
27. •GTRΛ fi ne-tuning • Թ౓ύϥϝʔλ: 0.01 • ֶश཰: 2e-5 • Optimizer: AdamW ϛχόονͷαϯϓϦϯάख๏ •֤ϛχόον͸୯Ұͷσʔληοτ͔ΒͳΔ • λεΫ΍σʔληοτͷҧ͍Λֶश͠ͳ͍Α͏ʹ͢Δ ධՁ࣮ݧ: InstructORͷ܇࿅ઃఆ 27 in-batch negatives
 ʹΑΔѱӨڹͷ௿ݮ
28. •InstructOR͸λεΫʹΑΒͣߴੑೳ • SimCSE͸STSͰ͸ߴੑೳ͕ͩଞͷλεΫͰ௿Ίͷੑೳ • GTR΍Contrierver͸ݕࡧͰߴੑೳ͕ͩSTS͕௿Ί •ࢦࣔΛ༩͑Δ͜ͱͰ֤λεΫʹదͨ͠ຒΊࠐΈΛੜ੒Ͱ͖͍ͯΔ ࣮ݧ݁Ռ: খ͞ΊͷϞσϧ 28 ্ද͸࿦จதͷදͷҰ෦Λൈਮͨ͠΋ͷ
29. •େ͖ΊͷϞσϧͰ΋ࢦࣔΛ༻͍ͨඍௐ੔Ͱੑೳ޲্ •GTR, InstructOR͸335M→1.5BͰੑೳ͕গ্͔͕͍ͬͯ͠͠ͳ͍ (58.4→58.8) • InstructORͷϕʔεϞσϧͷGTR͕ͪΐ͍ऑͦ͏ʁ ࣮ݧ݁Ռ: େ͖ΊͷϞσϧ 29 ্ද͸࿦จதͷදͷҰ෦Λൈਮͨ͠΋ͷ
30. •ࢦࣔͳ͠ͰඇରশͳจϖΞ(query & passageͳͲ)Λֶश͢Δͱੑೳ௿Լ • ࢦࣔΛೖΕΔ͜ͱͰΉ͠Ζੑೳ͕޲্͢Δ •super-NI͕͋Δ͜ͱͰࢦࣔʹର͢Δؤ݈ੑ͕޲্ ෼ੳ: ࢦࣔͷ༗ແʹΑΔੑೳͷมԽ & ؤ݈ੑ 30
31. •InstructOR͸ࢦࣔͷৄࡉ౓Λ্͛Δ΄Ͳੑೳ޲্ •ϞσϧύϥϝʔλΛ૿΍͢΄Ͳੑೳ޲্ • ৳ͼํ͸େ͖͘͸ͳ͍ ෼ੳ: ࢦࣔͷৄࡉ౓ & ϞσϧαΠζʹΑΔੑೳͷมԽ 31 ৳ͼํ͕ऑ͍ͷͰScalingͱ
 ߹Θͤͯผͷ޻෉͕͍Δ͔΋
32. •ࢦ͕ࣔ͋Δ͜ͱͰOODʹର͢Δੑೳ΋޲্ •ൃදऀίϝϯτ • ࢦࣔͳ͠Ͱ fi ne-tuningͨ͠GTRͱͷൺֱ͕ཉ͍͠ • ࢦࣔͷޮՌ͔ɺMEDIͷଟ༷ੑͷ͓ӄ͔֬৴͕࣋ͯͳ͍ ࢦࣔͷޮՌ: Domain Shift 32 ܇࿅࣌ʹΈ͍ͯͳ͍
 υϝΠϯͷσʔληοτ
33. •ຒΊࠐΈΛT-SNEͰ2࣍ݩʹ •Ͳͷࢦ͔ࣔ͸ෆ໌ɺଟ෼ˣ •ࢦࣔΛ࢖͏͜ͱͰʮจߏ଄ʯ
 Ͱ͸ͳ͘ʮײ৘ʯΛදͨ͠
 ҐஔʹҠಈ͍ͯ͠Δ • ʮͲͷ؍఺ͰจΛຒΊࠐΉ͔ʯ
 ΛϢʔβ͕ૢ࡞Ͱ͖͍ͯΔ ఆੑධՁ: ՄࢹԽ 33 ࢦࣔ͋Γ ࢦࣔͳ͠
34. •ࢦࣔʹैͬͯจΛຒΊࠐΉϞσϧInstructORΛఏҊ •ࢦࣔΛՃ͑Δ͜ͱͰطଘख๏ΑΓߴੑೳ ࢥͬͨ͜ͱɾؾʹͳΔ͜ͱ •ࢦࣔνϡʔχϯά͞ΕͨϞσϧ(Flan-T5ͱ͔)Λ
 ϕʔεʹͨ࣌͠ͷੑೳͷมԽ • ࢦࣔʹै͏ೳྗ͕͋ΔϞσϧͷํ͕ڧ͘ͳΓͦ͏ • GTR΍InstructOR͸ຊ౰ʹࢦࣔʹै͍͑ͯΔʁ(ICL΍ΔͳΒDecoderܕ͸ʁ) •MEDI datasetͷࢦ͕ࣔςϯϓϨʔτʹج͍͍ͮͯΔͷͰunnatural • ࢦࣔΛ΋ͬͱଟ༷͔ͭࣗવʹͰ͖ͨΒΑΓຒΊࠐΈͷ࣭Λ্͛ΒΕͦ͏ ·ͱΊ 34