修論発表.pdf - Speaker Deck

Slide 1

Slide 1 text

ҟͳΔڭࢣ৴߸͔Βߏஙͨ͠  จϕΫτϧͷൺֱͱ౷߹ ਖ਼ࢦಋڭһ: ෢ా ߒҰ ෭ࢦಋڭһ: ࡫໺ ྒྷฏ 252106192 ௩ӽ ॣ

Slide 2

Slide 2 text

จϕΫτϧ: ࣗવݴޠจͷϕΫτϧදݱ 2 จϕΫτϧۭؒ ͜Ͳ΋͕Ոʹ޲͔͍ͬͯΔɻ ͜Ͳ΋ֶ͕ߍ͔ΒՈʹ޲͔͍ͬͯΔɻ ͜Ͳ΋͕ਤॻؗʹ͍Δɻ ͜Ͳ΋͕ޕޙʹา͍͍ͯΔɻ จಉ࢜ͷҙຯ͕͍ۙ   ϕΫτϧಉ͕͍࢜ۙ

Slide 3

Slide 3 text

จϕΫτϧ: ࣗવݴޠจͷϕΫτϧදݱ 3 จϕΫτϧۭؒ ͜Ͳ΋͕Ոʹ޲͔͍ͬͯΔɻ ͜Ͳ΋ֶ͕ߍ͔ΒՈʹ޲͔͍ͬͯΔɻ ͜Ͳ΋͕ਤॻؗʹ͍Δɻ ͜Ͳ΋͕ޕޙʹา͍͍ͯΔɻ จಉ࢜ͷҙຯ͕͍ۙ   ϕΫτϧಉ͕͍࢜ۙ

Slide 4

Slide 4 text

จϕΫτϧ: ࣗવݴޠจͷϕΫτϧදݱ 4 จϕΫτϧۭؒ ͜Ͳ΋͕Ոʹ޲͔͍ͬͯΔɻ ͜Ͳ΋ֶ͕ߍ͔ΒՈʹ޲͔͍ͬͯΔɻ ͜Ͳ΋͕ਤॻؗʹ͍Δɻ ͜Ͳ΋͕ޕޙʹา͍͍ͯΔɻ จಉ࢜ͷҙຯ͕͍ۙ   ϕΫτϧಉ͕͍࢜ۙ • ྨࣅจݕࡧ • ΫϥελϦϯά ɹͳͲ෯޿͍Ԡ༻

Slide 5

Slide 5 text

จϕΫτϧ: ࣗવݴޠจͷϕΫτϧදݱ 5 จϕΫτϧۭؒ ͜Ͳ΋͕Ոʹ޲͔͍ͬͯΔɻ ͜Ͳ΋ֶ͕ߍ͔ΒՈʹ޲͔͍ͬͯΔɻ ͜Ͳ΋͕ਤॻؗʹ͍Δɻ ͜Ͳ΋͕ޕޙʹา͍͍ͯΔɻ จಉ࢜ͷҙຯ͕͍ۙ   ϕΫτϧಉ͕͍࢜ۙ • ྨࣅจݕࡧ • ΫϥελϦϯά ɹͳͲ෯޿͍Ԡ༻ • ϕΫτϧͷ඼࣭޲্ • ϕΫτϧͷੑ࣭ཧղ ɹ͕༗༻ੑ޲্ʹ௚݁ จϕΫτϧʹ͍ͭͯͷཧղΛਂΊΔ  ͜ͱ͕ࠓޙͷൃలͷͨΊʹॏཁ

Slide 6

Slide 6 text

طଘݚڀͷ՝୊ จϕΫτϧݚڀͷݱঢ় •Ұ෦ͷϕϯνϚʔΫλεΫͰͷੑೳධՁ͕ओ •ͦΕͧΕͷख๏ͷཧղ͸ͦ͜·ͰਐΜͰ͍ͳ͍ 6

Slide 7

Slide 7 text

طଘݚڀͷ՝୊ จϕΫτϧݚڀͷݱঢ় •Ұ෦ͷϕϯνϚʔΫλεΫͰͷੑೳධՁ͕ओ •ͦΕͧΕͷख๏ͷཧղ͸ͦ͜·ͰਐΜͰ͍ͳ͍ ࣮Ԡ༻ͱͷΪϟοϓ •จϕΫτϧʹٻΊΒΕΔੑ࣭͸Ԡ༻ࣄྫ͝ͱʹมԽ͢Δ • ݕࡧͰ͸จҙΑΓ΋τϐοΫ΍ΧςΰϦ͕ॏཁ • ࣭໰Ԡ౴Ͱ͸࣭໰ͱճ౴͕ۙ͘ʹ෼෍͢Δ͜ͱ͕ॏཁ •Ԡ༻ࣄྫ͝ͱʹద੾ͳੑ࣭Λ࣋ͭจϕΫτϧΛ࢖͍͍ͨ •ʮख๏͝ͱʹͲͷΑ͏ͳੑ࣭͕͋Δ͔ʁʯ΋ۃΊͯॏཁͳ໰͍ 7

Slide 8

Slide 8 text

ڭࢣ৴߸ʹண໨ͨ͠จϕΫτϧͷ෼ੳ •ػցֶशϞσϧͷৼΔ෣͍͸܇࿅σʔλɾख๏ʹେ͖͘ґଘ •ҟͳΔڭࢣ৴߸ʹج͍ͮͯߏங͞ΕͨจϕΫτϧͷੑ࣭Λ෼ੳ •ͦΕͧΕͷख๏ʹΑΔจϕΫτϧͷੑ࣭Λ෼ੳ 8

Slide 9

Slide 9 text

ڭࢣ৴߸ʹண໨ͨ͠จϕΫτϧͷ෼ੳ •ػցֶशϞσϧͷৼΔ෣͍͸܇࿅σʔλɾख๏ʹେ͖͘ґଘ •ҟͳΔڭࢣ৴߸ʹج͍ͮͯߏங͞ΕͨจϕΫτϧͷੑ࣭Λ෼ੳ •ͦΕͧΕͷख๏ʹΑΔจϕΫτϧͷੑ࣭Λ෼ੳ ੑ࣭ൺֱʹదͨ͠จϕΫτϧख๏ •ྨࣅͨ͠ΞʔΩςΫνϟΛ͕࣋ͭҟͳΔڭࢣ৴߸Λ༻͍Δ •ϕϯνϚʔΫλεΫʹ͓͍ͯಉ౳ͷੑೳΛࣔ͢ 9

Slide 10

Slide 10 text

Slide 11

Slide 11 text

ࣄલֶशࡁΈݴޠϞσϧ •େن໛ͳςΩετΛ༻͍ͨࣄલֶशʹΑͬͯݴޠ஌ࣝΛ֫ಘ •୅දྫ: BERT, RoBERTa, GPT-2, GPT-3 11 BERTͷ֓ཁਤ

Slide 12

Slide 12 text

SBERT: ࣗવݴޠਪ࿦λεΫʹجͮ͘ख๏ • ࣗવݴޠਪ࿦λεΫͰ  จϕΫτϧϞσϧΛ܇࿅͢Δख๏ • ࣗવݴޠਪ࿦λεΫ:  จϖΞͷҙຯؔ܎Λ༧ଌ 12 จB จA BERT BERT ໃ६ ؚҙ ͦͷଞ ϥϕϧ༧ଌ૚ Pooling Pooling

Slide 13

Slide 13 text

SBERT: ࣗવݴޠਪ࿦λεΫʹجͮ͘ख๏ • ࣗવݴޠਪ࿦λεΫͰ  จϕΫτϧϞσϧΛ܇࿅͢Δख๏ • ࣗવݴޠਪ࿦λεΫ:  จϖΞͷҙຯؔ܎Λ༧ଌ SBERTʹΑΔﬁne-tuningͷखॱ 0. ࣄલֶशࡁΈݴޠϞσϧΛ༻ҙ 1. จϖΞΛͦΕͧΕจϕΫτϧʹ 2. ಘΒΕͨจϕΫτϧͷϖΞ͔Β  จϖΞͷҙຯؔ܎Λ༧ଌ 3. ਖ਼͍͠ҙຯؔ܎Λ༧ଌͰ͖Δ  Α͏ʹϞσϧΛ܇࿅ 13 จB จA BERT BERT ໃ६ ؚҙ ͦͷଞ ϥϕϧ༧ଌ૚ Pooling Pooling

Slide 14

Slide 14 text

DefSent: ఆٛจˠ୯ޠ༧ଌλεΫʹجͮ͘ख๏ • ఆٛจˠ୯ޠ༧ଌλεΫʹΑͬͯ  จϕΫτϧϞσϧΛ܇࿅͢Δख๏ 14 ఆٛจ จB จA w |V| w1 w2 w3 ... BERT BERT ໃ६ ؚҙ ͦͷଞ ϥϕϧ༧ଌ૚ BERT ୯ޠ༧ଌ૚ Pooling Pooling Pooling

Slide 15

Slide 15 text

DefSent: ఆٛจˠ୯ޠ༧ଌλεΫʹجͮ͘ख๏ • ఆٛจˠ୯ޠ༧ଌλεΫʹΑͬͯ  จϕΫτϧϞσϧΛ܇࿅͢Δख๏ DefSentʹΑΔﬁne-tuningͷखॱ 0. ࣄલֶशࡁΈݴޠϞσϧΛ༻ҙ 1. ఆٛจΛBERTʹೖྗͯ͠  จϕΫτϧΛ֫ಘ 2. ಘΒΕͨϕΫτϧ͔Βఆٛจ  ʹରԠ͢Δ୯ޠΛ༧ଌ 3. ఆٛจ͕ද͢୯ޠͷ֬཰஋Λ  ࠷େԽ͢ΔΑ͏ʹ܇࿅ 15 ఆٛจ จB จA w |V| w1 w2 w3 ... BERT BERT ໃ६ ؚҙ ͦͷଞ ϥϕϧ༧ଌ૚ BERT ୯ޠ༧ଌ૚ Pooling Pooling Pooling

Slide 16

Slide 16 text

ຊݚڀͷ֓ཁ 16 ڭࢣ৴߸ͷҧ͍ʹ  ண໨ͨ͠จϕΫτϧͷ  ൺֱɾ౷߹ SBERT DefSent BERT ؚҙؔ܎ೝࣝͰ ﬁne-tuning ఆٛจ→୯ޠ  ༧ଌͰﬁne-tuning จϕΫτϧ  Ϟσϧ

Slide 17

Slide 17 text

ຊݚڀͷ֓ཁ 17 Semantic Textual Similarity (STS) ᶃ จͷιʔε ᶄ จϖΞͷද૚తྨࣅ౓ SentEval ᶅ ײ৘ɾ੍࣌෼ྨͳͲͷԼྲྀλεΫ ᶆ ݴޠֶత৘ใͷ෼ྨλεΫ ൺֱ •SBERT→DefSent •DefSent→SBERT •ϚϧνλεΫֶश •Average •Concat ౷߹ ڭࢣ৴߸ͷҧ͍ʹ  ண໨ͨ͠จϕΫτϧͷ  ൺֱɾ౷߹ SBERT DefSent BERT ؚҙؔ܎ೝࣝͰ ﬁne-tuning ఆٛจ→୯ޠ  ༧ଌͰﬁne-tuning จϕΫτϧ  Ϟσϧ

Slide 18

Slide 18 text

ҟͳΔڭࢣ৴߸͔Βߏஙͨ͠  จϕΫτϧͷൺֱ

Slide 19

Slide 19 text

จϕΫτϧͷੑ࣭ൺֱ: STS 19 Semantic Textual Similarity (STS) ᶃ จͷιʔε ᶄ จϖΞͷද૚తྨࣅ౓ จϖΞ (จϕΫτϧͷϖΞ) ೖྗ ൺֱ؍఺ ਓؒධՁͱจϕΫτϧಉ࢜ͷ  ྨࣅ౓ͱͷॱҐ૬ؔ܎਺ ධՁࢦඪ STSͷධՁखॱ ᶃ จϕΫτϧϞσϧΛ༻ҙ ᶄ จϖΞͦΕͧΕΛจϕΫτϧʹม׵ ᶅ จϕΫτϧͷϖΞͷྨࣅ౓Λܭࢉ ᶆ ਓؒධՁͱͷ૬ؔ܎਺Λܭࢉ จA จB จϕΫτϧϞσϧ ਓखධՁͱͷ  ૬ؔ܎਺ͰධՁ จྨࣅ౓ ᶄ ᶃ ᶅ ᶆ

Slide 20

Slide 20 text

จϕΫτϧͷੑ࣭ൺֱ: STS 20 Semantic Textual Similarity (STS) ᶃ จͷιʔε ᶄ จϖΞͷද૚తྨࣅ౓ จϖΞ (จϕΫτϧͷϖΞ) ೖྗ ൺֱ؍఺ ਓؒධՁͱจϕΫτϧಉ࢜ͷ  ྨࣅ౓ͱͷॱҐ૬ؔ܎਺ ධՁࢦඪ

Slide 21

Slide 21 text

จϕΫτϧͷੑ࣭ൺֱ: STS 21 Semantic Textual Similarity (STS) ᶃ จͷιʔε ᶄ จϖΞͷද૚తྨࣅ౓ จϖΞ (จϕΫτϧͷϖΞ) ೖྗ ൺֱ؍఺ ਓؒධՁͱจϕΫτϧಉ࢜ͷ  ྨࣅ౓ͱͷॱҐ૬ؔ܎਺ ධՁࢦඪ • 2ͭͷ؍఺ͰσʔληοτΛ෼ׂ • ੑೳͷมԽΛ؍࡯

Slide 22

Slide 22 text

จϕΫτϧͷੑ࣭ൺֱ: STS 22 Semantic Textual Similarity (STS) ᶃ จͷιʔε ᶄ จϖΞͷද૚తྨࣅ౓ จϖΞ (จϕΫτϧͷϖΞ) ೖྗ ൺֱ؍఺ ਓؒධՁͱจϕΫτϧಉ࢜ͷ  ྨࣅ౓ͱͷॱҐ૬ؔ܎਺ ධՁࢦඪ • จͷιʔεʹΑͬͯੑೳʹࠩ • ֤ख๏ͷ܇࿅σʔληοτʹ͍ۙ จͷํ͕͏·͘ྨࣅ౓ΛଌΕΔ • 2ͭͷ؍఺ͰσʔληοτΛ෼ׂ • ੑೳͷมԽΛ؍࡯

Slide 23

Slide 23 text

จϕΫτϧͷੑ࣭ൺֱ: STS 23 ද૚తྨࣅ౓ͱੑೳͷؔ܎ SBERT DefSent Semantic Textual Similarity (STS) ᶃ จͷιʔε ᶄ จϖΞͷද૚తྨࣅ౓ จϖΞ (จϕΫτϧͷϖΞ) ೖྗ ൺֱ؍఺ ਓؒධՁͱจϕΫτϧಉ࢜ͷ  ྨࣅ౓ͱͷॱҐ૬ؔ܎਺ ධՁࢦඪ • ද૚తྨࣅ౓ʹΑͬͯੑೳࠩ • SBERT (ؚҙؔ܎)͸ද૚తྨ ࣅ౓ͷӨڹΛड͚ͮΒ͍ • DefSent (ఆٛจ)͸ද૚తʹྨ ࣅ͍ͯ͠ͳ͍จͷྨࣅ౓Λ  ൺֱతਖ਼͘͠ਪఆͰ͖Δ • จͷιʔεʹΑͬͯੑೳʹࠩ • ֤ख๏ͷ܇࿅σʔληοτʹ͍ۙ จͷํ͕͏·͘ྨࣅ౓ΛଌΕΔ • 2ͭͷ؍఺ͰσʔληοτΛ෼ׂ • ੑೳͷมԽΛ؍࡯

Slide 24

Slide 24 text

จϕΫτϧͷੑ࣭ൺֱ: SentEval 24 SentEval ᶅ ԼྲྀλεΫ͝ͱͷੑೳ ᶆ ݴޠֶత৘ใͷ෼ྨੑೳ จϕΫτϧ ೖྗ ൺֱ؍఺ จϕΫτϧΛೖྗͱ͢Δ  ઢܗ෼ྨثͷ෼ྨੑೳ ධՁࢦඪ จ ෼ྨੑೳ͔Β  จຒΊࠐΈͷ඼࣭ΛධՁ จϕΫτϧϞσϧ ෼ྨث ᶄ ᶃ ᶅ ᶆ SentEvalͷධՁखॱ ᶃ จຒΊࠐΈϞσϧΛ༻ҙ ᶄ ֤จΛจϕΫτϧʹม׵ ᶅ จϕΫτϧΛೖྗͱ͢Δ෼ྨثΛ܇࿅ ᶆ ෼ྨثͷੑೳ͔ΒจϕΫτϧͷ඼࣭ΛධՁ

Slide 25

Slide 25 text

จϕΫτϧͷੑ࣭ൺֱ: SentEval 25 SentEval ᶅ ԼྲྀλεΫ͝ͱͷੑೳ ᶆ ݴޠֶత৘ใͷ෼ྨੑೳ จϕΫτϧ ೖྗ ൺֱ؍఺ จϕΫτϧΛೖྗͱ͢Δ  ઢܗ෼ྨثͷ෼ྨੑೳ ධՁࢦඪ

Slide 26

Slide 26 text

จϕΫτϧͷੑ࣭ൺֱ: SentEval 26 SentEval ᶅ ԼྲྀλεΫ͝ͱͷੑೳ ᶆ ݴޠֶత৘ใͷ෼ྨੑೳ จϕΫτϧ ೖྗ ൺֱ؍఺ จϕΫτϧΛೖྗͱ͢Δ  ઢܗ෼ྨثͷ෼ྨੑೳ ධՁࢦඪ • SBERT͸ҙຯతͳ৘ใΛ๛෋ ʹຒΊࠐΜͰ͍Δ • DefSent͸ද૚త৘ใ͕๛෋ • ϑϨʔζͷߏ੒΋ಘҙ

Slide 27

Slide 27 text

Length WordContent Tense SubjNumber จϕΫτϧͷੑ࣭ൺֱ: SentEval 27 SentEval ᶅ ԼྲྀλεΫ͝ͱͷੑೳ ᶆ ݴޠֶత৘ใͷ෼ྨੑೳ จϕΫτϧ ೖྗ ൺֱ؍఺ จϕΫτϧΛೖྗͱ͢Δ  ઢܗ෼ྨثͷ෼ྨੑೳ ධՁࢦඪ • SBERT͸ҙຯతͳ৘ใΛ๛෋ ʹຒΊࠐΜͰ͍Δ • DefSent͸ද૚త৘ใ͕๛෋ • ϑϨʔζͷߏ੒΋ಘҙ • DefSent͸੍࣌΍จத୯ޠͷ৘ใ  ͳͲද૚తͳ৘ใ͕ൺֱత๛෋ 50 60 70 80 90 Length WordContent Tense จ௕༧ଌ 50 60 70 80 90 Length WordContent Tense จத୯ޠ༧ଌ ੍࣌༧ଌ • SBERT͸จͷද૚৘ใ͕ॏཁͳλεΫ ͷੑೳ͕௿͍ • จத୯ޠͳͲͷ৘ใ͸গͳΊ

Slide 28

Slide 28 text

จϕΫτϧͷੑ࣭ൺֱ: ·ͱΊ 28 Semantic Textual Similarity (STS) ᶃ จͷιʔε ᶄ จϖΞͷද૚తྨࣅ౓ จϖΞ (จϕΫτϧͷϖΞ) ೖྗ ൺֱ؍఺ ਓؒධՁͱจϕΫτϧಉ࢜ͷ  ྨࣅ౓ͱͷॱҐ૬ؔ܎਺ ධՁࢦඪ SentEval ᶅ ԼྲྀλεΫ͝ͱͷੑೳ ᶆ ݴޠֶత৘ใͷ෼ྨੑೳ จϕΫτϧ ೖྗ ൺֱ؍఺ จϕΫτϧΛೖྗͱ͢Δ  ઢܗ෼ྨثͷ෼ྨੑೳ ධՁࢦඪ

Slide 29

Slide 29 text

จϕΫτϧͷੑ࣭ൺֱ: ·ͱΊ 29 ද૚తྨࣅ౓ͱੑೳͷؔ܎ SBERT DefSent Semantic Textual Similarity (STS) ᶃ จͷιʔε ᶄ จϖΞͷද૚తྨࣅ౓ จϖΞ (จϕΫτϧͷϖΞ) ೖྗ ൺֱ؍఺ ਓؒධՁͱจϕΫτϧಉ࢜ͷ  ྨࣅ౓ͱͷॱҐ૬ؔ܎਺ ධՁࢦඪ SentEval ᶅ ԼྲྀλεΫ͝ͱͷੑೳ ᶆ ݴޠֶత৘ใͷ෼ྨੑೳ จϕΫτϧ ೖྗ ൺֱ؍఺ จϕΫτϧΛೖྗͱ͢Δ  ઢܗ෼ྨثͷ෼ྨੑೳ ධՁࢦඪ SBERT • λεΫ: ࣗવݴޠਪ࿦ • ײ৘ۃੑͳͲҙຯత৘ใ͕๛෋ • ද૚త৘ใ͸গͳΊ DefSent • λεΫ: ఆٛจˠ୯ޠ༧ଌ • จ௕΍੍࣌ͳͲද૚త৘ใ͕๛෋ • ϑϨʔζͷߏ੒΋ಘҙ

Slide 30

Slide 30 text

ҟͳΔڭࢣ৴߸͔Βߏஙͨ͠  จϕΫτϧͷ౷߹

Slide 31

Slide 31 text

จϕΫτϧͷ౷߹ ੑ࣭ൺֱͷ݁Ռ •ಉ͡ϞσϧΛϕʔεͱ͠ɺҟͳΔڭࢣ৴߸Λ༻͍Δख๏Λൺֱ • SBERTͱDefSent͸ҟͳΔੑ࣭Λ࣋ͭ͜ͱ͕Θ͔ͬͨ •ҟͳΔৼΔ෣͍Λ͢ΔϞσϧͷΞϯαϯϒϧ͸༗๬ͳखཱͯ • จϕΫτϧʹ͓͍ͯ΋ҟͳΔϞσϧͷ౷߹͸༗༻͔ʁ 31

Slide 32

Slide 32 text

จϕΫτϧͷ౷߹ ੑ࣭ൺֱͷ݁Ռ •ಉ͡ϞσϧΛϕʔεͱ͠ɺҟͳΔڭࢣ৴߸Λ༻͍Δख๏Λൺֱ • SBERTͱDefSent͸ҟͳΔੑ࣭Λ࣋ͭ͜ͱ͕Θ͔ͬͨ •ҟͳΔৼΔ෣͍Λ͢ΔϞσϧͷΞϯαϯϒϧ͸༗๬ͳखཱͯ • จϕΫτϧʹ͓͍ͯ΋ҟͳΔϞσϧͷ౷߹͸༗༻͔ʁ ҟͳΔੑ࣭Λ࣋ͭจϕΫτϧͷ౷߹ •5ͭͷ౷߹ख๏ʹ͍࣮ͭͯݧ • S+D, D+S • Multi • Average • Concat 32 • ౷߹ʹΑͬͯੑೳ͸޲্͢Δ͔ʁ • ࠓޙͷจϕΫτϧݚڀʹ͓͍ͯෳ਺  ͷڭࢣ৴߸ͷ૊Έ߹Θͤ͸༗๬͔ʁ

Slide 33

Slide 33 text

จϕΫτϧͷ౷߹: ୯ҰϞσϧ΁ͷ౷߹ 33 • S+D: ୯ҰϞσϧʹSBERT, DefSentʹΑΔfine-tuningΛॱʹ࣮ࢪ • D+S: ୯ҰϞσϧʹDefSent, SBERTʹΑΔfine-tuningΛॱʹ࣮ࢪ • Multi: SBERTͱDefSentʹΑΔfine-tuningΛަޓʹ࣮ࢪ

Slide 34

Slide 34 text

จϕΫτϧͷ౷߹: ୯ҰϞσϧ΁ͷ౷߹ 34 • S+D: ୯ҰϞσϧʹSBERT, DefSentʹΑΔfine-tuningΛॱʹ࣮ࢪ • D+S: ୯ҰϞσϧʹDefSent, SBERTʹΑΔfine-tuningΛॱʹ࣮ࢪ • Multi: SBERTͱDefSentʹΑΔfine-tuningΛަޓʹ࣮ࢪ

Slide 35

Slide 35 text

จϕΫτϧͷ౷߹: ୯ҰϞσϧ΁ͷ౷߹ 35 • S+D: ୯ҰϞσϧʹSBERT, DefSentʹΑΔfine-tuningΛॱʹ࣮ࢪ • D+S: ୯ҰϞσϧʹDefSent, SBERTʹΑΔfine-tuningΛॱʹ࣮ࢪ • Multi: SBERTͱDefSentʹΑΔfine-tuningΛަޓʹ࣮ࢪ

Slide 36

Slide 36 text

จϕΫτϧͷ౷߹: ෳ਺Ϟσϧͷ౷߹ 36 • Average: SBERT, DefSentʹΑΔจϕΫτϧΛฏۉ • Concat: SBERT, DefSentʹΑΔจϕΫτϧΛ࿈݁

Slide 37

Slide 37 text

จϕΫτϧͷ౷߹: ෳ਺Ϟσϧͷ౷߹ 37 • Average: SBERT, DefSentʹΑΔจϕΫτϧΛฏۉ • Concat: SBERT, DefSentʹΑΔจϕΫτϧΛ࿈݁

Slide 38

Slide 38 text

จϕΫτϧͷ౷߹: ධՁ࣮ݧ 38 •౷߹ख๏͝ͱʹϞσϧΛ܇࿅ɾධՁ •STSͰ10ճ, SentEvalͰ3ճϞσϧΛ  ܇࿅ͯ͠ฏۉੑೳΛใࠂ ࣮ݧઃఆ ධՁର৅ •SBERT •DefSent •S+D (SBERT→DefSent) •D+S (DefSent→SBERT) •Multi •Average •Concat ୯Ұख๏ͱੑೳΛൺֱ ධՁλεΫ •STS •SentEval

Slide 39

Slide 39 text

จϕΫτϧͷ౷߹: ධՁ࣮ݧ 39 BERT-base STS SentEval SBERT 73.19 86.49 DefSent 75.20 86.61 S+D 78.45 86.80 D+S 72.89 86.09 Multi 72.89 86.23 Average 77.82 87.47 Concat 76.03 87.93 ֤౷߹ख๏͝ͱͷSTSͱSentEvalͷฏۉੑೳ (%) SBERT→DefSentͱ Average͕ߴੑೳ • SentEvalͰ͸Concatͷੑೳ͕  ྑ͍͕ɺจϕΫτϧͷ࣍ݩ͕ େ͖͘༗རͳͷͰ஫ҙ

Slide 40

Slide 40 text

จϕΫτϧͷ౷߹: ධՁ࣮ݧ 40 BERT-base STS SentEval SBERT 73.19 86.49 DefSent 75.20 86.61 S+D 78.45 86.80 D+S 72.89 86.09 Multi 72.89 86.23 Average 77.82 87.47 Concat 76.03 87.93 ֤౷߹ख๏͝ͱͷSTSͱSentEvalͷฏۉੑೳ (%) • ౷߹ख๏͕୯Ұख๏  ΛԼճΔ৔߹΋ • ഁ໓త๨٫ͷӨڹ͔ SBERT→DefSentͱ Average͕ߴੑೳ DefSent→SBERT͸ ੑೳ͕ѱԽ • SentEvalͰ͸Concatͷੑೳ͕  ྑ͍͕ɺจϕΫτϧͷ࣍ݩ͕ େ͖͘༗རͳͷͰ஫ҙ

Slide 41

Slide 41 text

จϕΫτϧͷ౷߹: ධՁ࣮ݧ 41 BERT-base STS SentEval SBERT 73.19 86.49 DefSent 75.20 86.61 S+D 78.45 86.80 D+S 72.89 86.09 Multi 72.89 86.23 Average 77.82 87.47 Concat 76.03 87.93 ֤౷߹ख๏͝ͱͷSTSͱSentEvalͷฏۉੑೳ (%) • SentEvalͰ͸Concatͷੑೳ͕  ྑ͍͕ɺจϕΫτϧͷ࣍ݩ͕ େ͖͘༗རͳͷͰ஫ҙ • ౷߹ख๏͕୯Ұख๏  ΛԼճΔ৔߹΋ • ഁ໓త๨٫ͷӨڹ͔ • จϕΫτϧͷ୯७ฏۉ͕Α͍ੑೳ • ౷߹ख๏ʹΑΔϕΫτϧͷੑ࣭෼ੳ ͸ࠓޙͷ՝୊ SBERT→DefSentͱ Average͕ߴੑೳ DefSent→SBERT͸ ੑೳ͕ѱԽ

Slide 42

Slide 42 text

·ͱΊɾࠓޙͷ՝୊ 42 ౷߹ ڭࢣ৴߸ͷҧ͍ʹண໨͠จϕΫτϧͷ  ੑ࣭Λൺֱ෼ੳɾ౷߹ ൺֱ • จͷιʔεʹΑΔख๏͝ͱͷੑೳ͕ࠩݦஶ SBERT • ײ৘ۃੑͳͲҙຯత৘ใ͕๛෋ • ද૚తྨࣅ౓ͷӨڹΛड͚ͮΒ͍ DefSent • จ௕΍੍࣌ͳͲද૚త৘ใ͕๛෋ • ϑϨʔζͷߏ੒΋ಘҙ • ද૚తྨࣅ౓͕௿͍จϖΞʹڧ͍ • ౷߹ʹΑͬͯੑೳ޲্ • SBERT→DefSent΍ Average͕ߴੑೳ • ഁ໓త๨٫ͷӨڹͰੑ ೳ͕௿Լ͢Δ৔߹΋ ෼ੳର৅ ɹSBERT: ࣗવݴޠਪ࿦ϕʔε ɹDefSent: ఆٛจ→୯ޠ༧ଌϕʔε ࠓޙͷ՝୊ 1. ΑΓ޿൚ͳϞσϧɾจϕΫτϧख๏ͷௐࠪ 2. ౷߹ख๏Ͱߏ੒͞ΕͨϕΫτϧͷੑ࣭෼ੳ 3. ΑΓΑ͍౷߹ख๏ͷ։ൃ

Slide 43

Slide 43 text

·ͱΊɾࠓޙͷ՝୊ 43 ౷߹ ڭࢣ৴߸ͷҧ͍ʹண໨͠จϕΫτϧͷ  ੑ࣭Λൺֱ෼ੳɾ౷߹ ൺֱ • จͷιʔεʹΑΔख๏͝ͱͷੑೳ͕ࠩݦஶ SBERT • ײ৘ۃੑͳͲҙຯత৘ใ͕๛෋ • ද૚తྨࣅ౓ͷӨڹΛड͚ͮΒ͍ DefSent • จ௕΍੍࣌ͳͲද૚త৘ใ͕๛෋ • ϑϨʔζͷߏ੒΋ಘҙ • ද૚తྨࣅ౓͕௿͍จϖΞʹڧ͍ • ౷߹ʹΑͬͯੑೳ޲্ • SBERT→DefSent΍ Average͕ߴੑೳ • ഁ໓త๨٫ͷӨڹͰੑ ೳ͕௿Լ͢Δ৔߹΋ ෼ੳର৅ ɹSBERT: ࣗવݴޠਪ࿦ϕʔε ɹDefSent: ఆٛจ→୯ޠ༧ଌϕʔε ࠓޙͷ՝୊ 1. ΑΓ޿൚ͳϞσϧɾจϕΫτϧख๏ͷௐࠪ 2. ౷߹ख๏Ͱߏ੒͞ΕͨϕΫτϧͷੑ࣭෼ੳ 3. ΑΓΑ͍౷߹ख๏ͷ։ൃ

Slide 44

Slide 44 text

ത࢜ޙظ՝ఔͷల๬

Slide 45

Slide 45 text

ത࢜ޙظ՝ఔͷల๬ •จϕΫτϧͷকདྷతͳൃలͷͨΊͷॏཁͳ՝୊ • طଘख๏͸จͷݶΒΕͨଆ໘ʹ͔͠ண໨͍ͯ͠ͳ͍ • طଘͷࣄલֶशࡁΈݴޠϞσϧ͸จϕΫτϧͷදݱྗ͕ෆ ଍ɺܭࢉྔ΋େ͖͍ •͋ΒΏΔจϕΫτϧख๏ͷج൫ͱͯ͠༻͍Δ͜ͱ͕Ͱ͖Δɺ൚ ༻ੑʹ༏ΕͨϞσϧ͕ඞཁ จϕΫτϧͷͨΊͷج൫Ϟσϧͷ։ൃ •ݚڀ1: จϕΫτϧͷੑ࣭෼ੳ •ݚڀ2: จϕΫτϧʹ͓͚Δج൫ϞσϧͷఏҊ •ݚڀ3: จϕΫτϧʹ͓͚Δج൫ϞσϧͷԠ༻ 45

Slide 46

Slide 46 text

ത࢜ޙظ՝ఔͷݚڀ 46

Slide 47

Slide 47 text

ത࢜ޙظ՝ఔͷݚڀ 47

Slide 48

Slide 48 text

ത࢜ޙظ՝ఔͷݚڀ 48

Slide 49

Slide 49 text

ݚڀܭը 49 . % % % ੑ࣭෼ੳɾ ৽نධՁఏҊ ج൫Ϟσϧͷ։ൃ ج൫ϞσϧͷԠ༻ ത࢜࿦จ

Slide 50

Slide 50 text

ݚڀۀ੷ ࠃ಺࿦จࢽ (ࠪಡ͋Γ) • ௩ӽॣ, ࡫໺ྒྷฏ, ෢ాߒҰ. ఆٛจΛ༻͍ͨจຒΊࠐΈߏ੒๏, ࣗવݴޠॲཧ Vol. 30 No. 1 (ൃߦ༧ఆ). ࠃࡍձٞ (ࠪಡ͋Γ) • Hayato Tsukagoshi, Ryohei Sasano, Koichi Takeda. Comparison and Combination of Sentence Embeddings Derived from Diﬀerent Supervision Signals, in Proceedings of the 11th Joint Conference on Lexical and Computational Semantics (*SEM 2022). • Hayato Tsukagoshi, Ryohei Sasano, Koichi Takeda. DefSent: Sentence Embeddings using Deﬁnition Sentences, in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL-IJCNLP 2021). ࠃ಺ձٞ (ࠪಡͳ͠) • ཅాᠳฏ, ௩ӽॣ, ࡫໺ྒྷฏ, ෢ాߒҰ. Ψ΢εຒΊࠐΈʹجͮ͘จදݱੜ੒, ݴޠॲཧֶձ ୈ29ճ೥࣍େձ (NLP2023) ൃද༧ఆ. • ௩ӽॣ, ฏඌ౒, ৿Լກ, ஽ࠤࠀݾ, ࡫໺ྒྷฏ, ෢ాߒҰ. ࣗવݴޠਪ࿦ͱ࠶ݱثΛ༻͍ͨSplit and Rephraseʹ ͓͚Δੜ੒จͷ඼࣭޲্, ݴޠॲཧֶձ ୈ28ճ೥࣍େձ (NLP2022). • ௩ӽॣ, ࡫໺ྒྷฏ, ෢ాߒҰ. ఆٛจΛ༻͍ͨจຒΊࠐΈߏ੒๏, ݴޠॲཧֶձ ୈ27ճ೥࣍େձ (NLP2021). ͦͷଞ • 2023೥౓ ೔ຊֶज़ৼڵձ ಛผݚڀһ-DC1 ࠾༻಺ఆ • 2023೥౓ ໊ݹ԰େֶ༥߹ϑϩϯςΟΞϑΣϩʔೝఆ 50