ハンドメイド作品を扱うECサイトに特化したBERTを用いた言語モデル構築に向けた取り組み/ipsj-NL250-05

Slide 1

Slide 1 text

ञҪහ඙(GMOϖύϘגࣜձࣾ ϖύϘݚڀॴ/۝भେֶେֶӃ),   ࡾ୐༔հ(GMOϖύϘגࣜձࣾ ϖύϘݚڀॴ), ܀ྛ݈ଠ࿠(GMOϖύϘגࣜձࣾ ϖύϘݚڀॴ) ൃදऀɿञҪහ඙ / Pepabo R&D Institute, GMO Pepabo, Inc. / Kyushu University 2021.09.28 ৘ใॲཧֶձ ୈ250ճࣗવݴޠॲཧݚڀձ ϋϯυϝΠυ࡞඼Λѻ͏ECαΠτʹಛԽͨ͠ BERTΛ༻͍ͨݴޠϞσϧߏஙʹ޲͚ͨऔΓ૊Έ

Slide 2

Slide 2 text

1. ݚڀͷഎܠ 2. ՝୊ͱղܾࡦ 3. ࣮ݧ 4. ࣮ݧ݁Ռͱߟ࡯ 5. ·ͱΊͱࠓޙͷ՝୊ 2 ໨࣍

Slide 3

Slide 3 text

1. ݚڀͷഎܠ

Slide 4

Slide 4 text

• ฐࣾͰ͸ɺϋϯυϝΠυ࡞඼Λର৅ͱͨ͠CtoCͷECαΠτ(ʮϋϯυϝΠυ࡞ ඼Λѻ͏ECαΠτʯͱݺͿ)Ͱ͋ΔminneΛӡӦ 4 ݚڀͷഎܠ(1/3) ˞NJOOF࡞Ոొ࿥ϚχϡΞϧΑΓ ࡞඼Λൢച ࡞඼Λߪೖ

Slide 5

Slide 5 text

• ֤࡞඼ʹ͸ɺλΠτϧ΍આ໌จͳͲ ͷςΩετ৘ใ͕෇༩͞Ε͍ͯΔ • ֤छλεΫʹར༻͍ͨ͠ • ࡞඼ͷΧςΰϦ෼ྨ • λΠτϧɾઆ໌จͷࣗಈੜ੒ • ࣭໰Ԡ౴ ͳͲ 5 ݚڀͷഎܠ(2/3) େখΧςΰϦ ࡞඼λΠτϧ ࡞඼આ໌จ ˙࡞඼ͷྫ

Slide 6

Slide 6 text

• ͜ΕΒͷλεΫΛਓखͰղ͘ͷ͸ࠔ೉ͳͨΊػցతͳΞϓϩʔν͕ඞཁͱͳΔ • ҰํɺʮϋϯυϝΠυ࡞඼Λѻ͏ECαΠτʯ͸ҎԼͷΑ͏ͳಛ௃Λ࣋ͭ • ࡞඼͕ଟ༷Ͱ͋ΔͨΊɺ֤࡞඼Λਖ਼͘͠ಛ௃͚ͮΔ͜ͱ͕ࠔ೉ • αʔϏεͷมԽʹԠͨ͡ߏ଄తͳมԽʹ௥ै͢Δ͜ͱ͸ࠔ೉ → ্هͷಛ௃Λଊ͑ͯɺ໰୊ΛղܾͰ͖Δख๏͕ඞཁ 6 ݚڀͷഎܠ(3/3)

Slide 7

Slide 7 text

• ࣗવݴޠॲཧͰ͸ɺBERT[1]Λ࢖͏͜ͱ͕ओྲྀʹͳ͖͍ͬͯͯΔ • BERTͰ͸ɺࣄલֶशࡁΈϞσϧΛ࢖͏͜ͱͰɺ൚༻తͳ஌ࣝΛ࣋ͭݴޠϞσ ϧΛར༻͢Δ͜ͱ͕Ͱ͖Δ • ࣄલֶशࡁΈϞσϧΛɺλεΫʹԠͨ͡গͳ͍ڭࢣ༗ΓσʔλͰ fi ne-tuning͢ Δ͜ͱͰɺߴ͍ੑೳΛಘΔ͜ͱ͕Մೳ(ຊݚڀͰ͸ɺBERT+ fi ne-tuningͱݺͿ) • ௥ՃֶशʢBERTͷࣄલֶशࡁΈϞσϧ͔Βେن໛ͳίʔύεͰ࠶ࣄલֶशʣ Λߦ͏͜ͱͰɺίʔύεʹಛԽͨ͠஌ࣝ֫ಘ͕Մೳ [4],[6] 7 BERT [1] %FWMJO + $IBOH .8 -FF ,BOE5PVUBOPWB ,#&351SFUSBJOJOHPG%FFQ#JEJSFDUJPOBM5SBOTGPSNFSTGPS-BOHVBHF6OEFSTUBOEJOH <>-FF + :PPO 8 ,JN 4 ,JN % ,JN 4 4P $)BOE,BOH +#JP#&35BQSFUSBJOFECJPNFEJDBMMBOHVBHFSFQSFTFOUBUJPONPEFMGPSCJPNFEJDBMUFYUNJOJOH #JPJOGPSNBUJDT 7PM /P QQr <>/55 デ ʔλۚ༥ۀք޲͚ࣗવݴޠॲཧٕज़ͷݕূ։࢝dۚ༥൛#&35Ϟ デ ϧͷ։ൃdɼIUUQTXXXOUUEBUBDPNKQKBOFXTSFMFBTF

Slide 8

Slide 8 text

• ʮϋϯυϝΠυ࡞඼Λѻ͏ECαΠτʯͰͷࣗવݴޠॲཧͷλεΫΛղ͖͍ͨ → BERT+ fi ne-tuningΛ࢖͏͜ͱͰɺ֤՝୊ΛղܾͰ͖ΔͷͰ͸ͳ͍͔ • ࣗવݴޠॲཧͷλεΫͷ಺ɺࠓճ͸ɺ࡞඼ͷΧςΰϦ෼ྨʹऔΓ૊Ή • ຊݚڀͰ͸ɺ࡞඼ͷΧςΰϦ෼ྨʹ͓͍ͯϕʔεϥΠϯख๏ͱBERT+ fi ne- tuningͷϞσϧΛൺֱɾධՁ͢Δɻ݁ՌΛ౿·͑ͯɺࠓޙͷํ޲ੑΛܾΊ͍ͨ 8 ຊݚڀͷશମ૾ ˞ϕʔεϥΠϯख๏͸จॻϕΫτϧԽख๏ɿ5'*%'ɺ෼ྨثɿ47.ɺϩδεςΟοΫճؼͱͨ͠

Slide 9

Slide 9 text

2. ՝୊ͱղܾࡦ

Slide 10

Slide 10 text

• ࡞඼Λग़඼͢Δࡍʹ͸ɺ࡞Ո͕࡞඼ͷେ/খΧςΰϦΛొ࿥͍ͯ͠Δ         → ొ࿥͢ΔࡍʹɺࣗಈͰେ/খΧςΰϦΛਪଌ͠ɺ࡞Ո΁ఏ͍ࣔͨ͠   ɹɹࣄલʹΧςΰϦΛఏࣔ͢Δ͜ͱͰɺΧςΰϦઃఆޡΓΛ๷ࢭ͍ͨ͠ • ͔͠͠ɺ࡞඼ͷΧςΰϦ෼ྨΛӡӦଆ͕ਓखͰߦ͏͜ͱ͸ݱ࣮తͰͳ͍   → ՝୊ᶃʹର͢Δղܾࡦɿػցతʹ෼ྨ͢Δ͜ͱ͕ඞཁ 10 ՝୊ᶃɿਓखͰλεΫΛղ͘ͷ͸ࠔ೉ ˙େΧςΰϦͷҰཡ ΞΫηαϦʔ ࢦྠɾϦϯά ϐΞε ɾɾɾ ɾɾɾ ˙େখΧςΰϦͷྫ

Slide 11

Slide 11 text

• ʮϋϯυϝΠυ࡞඼Λѻ͏ECαΠτʯͰ͸࡞Ո͕࡞ͬͨ࡞඼Λߪೖऀ͕ߪೖ   → ࡞Ո͕ҟͳΔ৔߹ɺಉ͡࡞඼͸ଘࡏ͠ͳ͍   ಉ͡࡞඼͕ଘࡏ͠ͳ͍͜ͱ͕େ͖ͳಛ௃Ͱ͋Δ • औΓѻ͏࡞඼ͷछྨ͕ଟ༷ͱͳΓɺ࡞඼Λద੾ͳΧςΰϦ΁෼ྨ͢ΔͨΊʹ͸ ෯޿͍࡞඼஌͕ࣝඞཁͱͳΔ   → ѻ͏࡞඼͕ଟ༷Ͱ͋ͬͯ΋ɺ࡞඼ͷಛ௃Λଊ͑ΒΕΔख๏͕ඞཁ 11 ՝୊ᶄɿʮϋϯυϝΠυ࡞඼Λѻ͏ECαΠτʯͷ࡞඼͕ଟ༷

Slide 12

Slide 12 text

• BERTͰ͸ɺࣄલֶशࡁΈϞσϧΛར༻͢Δ͜ͱ͕Ͱ͖ɺ൚༻తͳ஌ࣝΛ࣋ͭ ݴޠϞσϧͷύϥϝʔλΛར༻Ͱ͖Δ • λεΫʹԠͨ͡গͳ͍ڭࢣ༗ΓσʔλͰ fi ne-tuning͢Δ͜ͱͰɺߴ͍ੑೳΛಘ Δ͜ͱ͕Մೳ • ͞ΒʹɺBERT௥ՃֶशʹΑΓɺଟ༷ͳ࡞඼ͷ஌ࣝ֫ಘ΋ظ଴Ͱ͖Δ → BERT(ࣄલֶशࡁΈ or ௥Ճֶश)+ fi ne-tuningʹΑΓɺ   ɹ࡞඼ͷଟ༷ੑΛଊ͑ΒΕΔͱߟ͑Δ 12 ՝୊ᶄʹର͢Δղܾࡦ

Slide 13

Slide 13 text

• ߏ଄తͳมԽͷྫɿΧςΰϦͷ৽ن࡞੒΍࡟আɺ౷ഇ߹ͳͲ • ࡞඼ͷ෼ྨΛߦ͏৔߹ɺ࡞඼จॻʹରͯ͠ɺจॻϕΫτϧԽख๏ɾલॲཧɾద ੾ͳ෼ྨثͷબఆ͕ඞཁʹͳͬͯ͘Δ   → ߏ଄తͳมԽ͕ى͖ͨࡍɺ౎౓ߟྀ͠ͳ͚Ε͹ͳΒͣɺ௥ै͢Δͷ͸ࠔ೉ 13 ՝୊ᶅɿʮϋϯυϝΠυ࡞඼Λѻ͏ECαΠτʯͷߏ଄తͳมԽ΁ͷ௥ै͕ࠔ೉ จॻϕΫτϧԽख๏ લॲཧ ෼ྨثʹΑΔֶश ࠷దͳจॻϕΫτϧख๏͸Կʁ ࠷దͳલॲཧ͸Կʁ ࠷దͳ෼ྨث͸Կʁ ه߸Λআ͘ɺ୯ޠස౓͕ߴස ౓ɺ௿ස౓͸আ͘ͳͲ 47. ϩδεςΟοΫճؼ   ϥϯμϜϑΥϨετ ܾఆ໦ͳͲ #P8 5'*%' %PD7FD   48&. 4$%7ɹͳͲ

Slide 14

Slide 14 text

• BERTͷ৔߹ɺ൚༻తͳϞσϧ͔Β fi ne-tuningͷΈΛ࣮ࢪ͢Δ͜ͱͰ௥ैՄ   ྫʣΧςΰϦͷߏ଄͕มΘͬͨͱͯ͠΋ɺมΘͬͨΧςΰϦϥϕϧͷΈΛ fi ne-tuning͢Ε͹ରԠͰ͖Δ 14 ՝୊ᶅʹର͢Δղܾࡦ #&35ʢࣄલֶशࡁΈϞσϧʣPS#&35௥Ճֶश 'JOFUVOJOH จॻϕΫτϧԽख๏ લॲཧ ෼ྨثʹΑΔֶश ه߸Λআ͘ɺ୯ޠස౓͕ߴස ౓ɺ௿ස౓͸আ͘ͳͲ 47. ϩδεςΟοΫճؼ   ϥϯμϜϑΥϨετ ܾఆ໦ͳͲ #P8 5'*%' %PD7FD   48&. 4$%7ɹͳͲ ϙΠϯτᶃɿࣄલʹ൚༻తͳϞσϧΛ࡞੒Ͱ͖Δ ϙΠϯτᶄɿߏ଄ͷมԽ͕ى͖ͨͱͯ͠΋ɺ fi OFUVOJOHͷΈͰ௥ैՄೳ

Slide 15

Slide 15 text

• ຊݚڀͰ͸ɺ࡞඼෼ྨʹ͓͍ͯϕʔεϥΠϯख๏ͱBERT+ fi ne-tuningͷϞσϧ ΛൺֱɾධՁ͢Δɻ • ՝୊ͱղܾࡦٴͼޮՌͷԾઆΛҎԼʹࣔ͢ 15 ຊݚڀͷ՝୊ͱղܾࡦͷରԠ ϕʔεϥΠϯख๏˞ #&35 fi OFUVOJOH #&35௥Ճֶश fi OF UVOJOH ՝୊ᶃɿਓखͰ͸ࠔ೉ ˓ ˓ ˓ ՝୊ᶄɿ࡞඼͕ଟ༷ ˚ ˓ ˕ ՝୊ᶅɿมԽʹ௥ै º ˓ ˓ ຌྫɹºɿѱ͍ɹ˓ɿྑ͍ɹ˕ɿ͔ͳΓྑ͍ ˞ϕʔεϥΠϯख๏͸จॻϕΫτϧԽख๏ɿ5'*%'ɺ෼ྨثɿ47.ɺϩδεςΟοΫճؼ ຊݚڀͰऔΓ૊Ήൣғ

Slide 16

Slide 16 text

3. ࣮ݧ

Slide 17

Slide 17 text

• ຊݚڀͷλεΫ͸ɺ֤࡞඼ʹ෇༩͞Ε͍ͯΔେ/খΧςΰϦΛ෼ྨ͢Δ͜ͱͱ ͨ͠ɻ࡞඼จॻ͸λΠτϧɺઆ໌จΛ࢖༻ͨ͠ɻ • จॻ෼ྨੑೳΛҎԼ3छྨͷख๏Ͱൺֱ • ϕʔεϥΠϯख๏ᶃɿ࡞඼จॻΛTF-IDFʹΑΔϕΫτϧԽɺ෼ྨثɿSVM • ϕʔεϥΠϯख๏ᶄɿ࡞඼จॻΛTF-IDFʹΑΔϕΫτϧԽɺ෼ྨثɿϩδε ςΟοΫճؼ • BERT+ fi ne-tuningख๏ɿࣄલֶशࡁΈBERTΛ fi ne-tuning͢Δ 17 ࣮ݧ֓ཁ(1/2)

Slide 18

Slide 18 text

• σʔληοτ͸֤ΧςΰϦʹ͓͍ͯਫ਼౓ͷ֬อ͕े෼ɺ͔ͭɺܭࢉϦιʔε಺ͰֶशՄೳ ͳ݅਺ΛԾͱͯ͠10ສ݅ͱઃఆ   → minneͷ࡞඼܈͔Β104,161݅Λநग़(2021೥6݄29೔࣌఺) • খΧςΰϦͷΫϥε਺͸239, େΧςΰϦͷΫϥε਺͸19 • લॲཧͱͯ͠ɺׅހه߸ɺશۭ֯നɺURLΛ࡟আ • ֶशσʔλͱςετσʔλͷൺ཰͸9:1ͱͨ͠ • ධՁࢦඪ͸Accuracy, Precision, Recall, F1-scoreΛ༻͍ͨ • ຊݚڀͰ͸ɺ෼ྨੑೳΛ૯߹తʹධՁ͢ΔͨΊɺF1-scoreΛॏࢹ͢Δ 18 ࣮ݧ֓ཁ(2/2)

Slide 19

Slide 19 text

• BERTͷࣄલֶशࡁΈϞσϧ͸ɺ౦๺େֶ͕ެ։͍ͯ͠Δ೔ຊޠֶशϞσϧ (Tohoku-BERTͱݺͿ)Λ༻͍ͨ[11] • Tokenizer͸Tohoku-BERT(MeCab<10>+NEologd<9>)Λ༻͍ɺ fi ne-tuningʹ͸ BertForSequenceClassi fi cation<3>Λ࢖༻ͨ͠ • ֶशύϥϝʔλ͸ɺଛࣦؔ਺͸ަࠩΤϯτϩϐʔޡࠩɺ࠷దԽؔ਺͸ AdamW<5>ɺֶश཰͸2e-5 19 BERT+ fi ne-tuning(1/2) [3] Huggingface, Transformers library, https://huggingface.co/transformers [5] Loshchilov, I. and Hutter, F.: Decoupled weight decay regularization, arXiv preprint arXiv:1711.05101 (2017). [9] Toshinori, S.: Neologism dictionary based on the language resources on the Web for Mecab (2015). [10] ޻౻୓ : MeCab, https://taku910.github.io/   [11] ౦๺େֶެ։ͷ೔ຊޠࣄલֶशࡁΈBERT: https://github.com/cl-tohoku/bert-japanese

Slide 20

Slide 20 text

• จॻͷ࠷େ௕͕512Λ௒͑ͨ৔߹͸ɺઆ໌จͷઌ಄ͱ຤ඌΛ࢒͢ํࣜΛద༻[8] • fine-tuningͷֶशঢ়گΛ౿·͑ɺࠓճ͸epoch10·Ͱͱͨ͠ɻ෼ྨʹ͸F1- score͕Ұ൪ߴ͔ͬͨepochͷϞσϧΛ༻͍ͨ 20 BERT+ fi ne-tuning(2/2) <8> Sun, C., Qiu, X., Xu, Y. and Huang, X.: How to fine-tune bert for text classification?, China National Conference on Chinese Computational Linguistics, Springer, pp. 194–206 (2019) ਤ1ɹখΧςΰϦ෼ྨλεΫʹ͓͚ΔBERT+fine-tuningͷֶशঢ়گ(ೖྗ͸λΠτϧ) ਤɹখΧςΰϦ෼ྨλεΫʹ͓͚ΔBERT+fine-tuningͷֶशঢ়گ(ೖྗ͸λΠτϧઆ໌จ)

Slide 21

Slide 21 text

• จॻϕΫτϧ࡞੒ʹ͓͍ͯɺϕʔεϥΠϯख๏ͱͯ͠ɺTF-IDFΛ༻͍ͨʢ඼ࢺ ͸໊ࢺͷΈʣ • ֤࡞඼จॻΛscikit-learnͷt fi dfvectorizerʹΑΓϕΫτϧԽ   ೖྗ͕ʮλΠτϧ+આ໌จʯͷޠኮ਺ΛʮλΠτϧʯͱ߹Θͤͨ • 3෼ׂަࠩݕূͰ࠷దͳϋΠύʔύϥϝʔλΛٻΊͨޙɺςετσʔλΛධՁ 21 ϕʔεϥΠϯख๏ɿTF-IDF

Slide 22

Slide 22 text

4. ࣮ݧ݁Ռͱߟ࡯

Slide 23

Slide 23 text

• ᶃ PrecisionҎ֎ɿ   BERT+ fi ne-tuning͕ྑ͍݁Ռ • ᶄ Precisionɿ   TF-IDF, SVM͕ྑ͍݁Ռ • ೖྗจॻ͸ʮλΠτϧʯΑΓʮλ Πτϧ+આ໌จʯ͕ྑ͍݁Ռ   → આ໌จ΋෼ྨੑೳ޲্ʹد༩ 23 େΧςΰϦͷ෼ྨ݁Ռͱߟ࡯ ᶃ ᶃ ᶄ ᶄ ˞֤ࢦඪʹ͓͍ͯҰ൪ੑೳ͕ߴ͍஋Λଠࣈͱͨ͠

Slide 24

Slide 24 text

• ᶃ PrecisionҎ֎ɿ   BERT+ fi ne-tuning͕ྑ͍݁Ռ • ᶄ Precisionɿ   TF-IDF, SVM͕ྑ͍݁Ռ • େΧςΰϦͷ෼ྨ݁ՌΑΓ΋શମ తʹੑೳ͕௿͍   → Ϋϥε਺͕ଟ͍ͨΊɺશମ͔Β খΧςΰϦΛ෼ྨ͢Δ͜ͱ͸ࠔ೉ Ͱ͸ͳ͍͔ 24 খΧςΰϦͷ෼ྨ݁Ռͱߟ࡯ ᶃ ᶃ ᶄ ᶄ ˞֤ࢦඪʹ͓͍ͯҰ൪ੑೳ͕ߴ͍஋Λଠࣈͱͨ͠

Slide 25

Slide 25 text

• BERT+ fi ne-tuningϞσϧͰͷʮλΠ τϧ+આ໌จʯͷ෼ྨޡΓ݁Ռ • ಉ͡େΧςΰϦ಺ͷখΧςΰϦΛਖ਼ ͘͠෼ྨͰ͖ͳ͔ͬͨ • ࣮ࡍͷ࡞඼ςΩετΛ໨ࢹͰ֬ೝ 25 খΧςΰϦͷ෼ྨޡΓͷ݁Ռ ຌྫ খΧςΰϦ ʢେΧςΰϦʣ

Slide 26

Slide 26 text

• ਖ਼ղͷΧςΰϦࣗମ͕ਖ਼͘͠ͳ͍৔߹΋͋Γ͏Δ • ྫ͑͹ɺϔΞΞΫηαϦʔͷΧςΰϦ͕ઌʹ࡞੒͞ΕɺͦͷޙɺϔΞΰϜͷΧςΰϦ ͕࡞੒͞Εͨɺͱ͍͏ܦҢ͕͋ΔʢଞͷΧςΰϦͰ΋ಉ༷ͷมԽ͕ߟ͑ΒΕΔʣ • ݱঢ়ͷσʔλʹجͮ͘ΧςΰϦߏ଄ͷมԽΛݕग़Ͱ͖ΔՄೳੑ͕͋Δ 26 খΧςΰϦͷ෼ྨʹ͓͚Δ෼ྨޡΓͷߟ࡯ᶃ • ໨ࢹͰ֬ೝ   ϔΞΞΫηαϦʔɿ7݅ɺϔΞΰϜɿ64݅   ϔΞΞΫηαϦʔɿ0݅ɺόϨολɾϔΞΫϦοϓ ɿ47݅   όοάɿ12݅ɺτʔτόοάɿ20݅

Slide 27

Slide 27 text

27 খΧςΰϦͷ෼ྨʹ͓͚Δ෼ྨޡΓͷߟ࡯ᶄ • ʮΠϠϦϯάɺϐΞεʯͲͪΒʹ΋౰ͯ͸·Δ࡞඼͕ଟ͍   → Ұҙʹ෼ྨ͢Δඞཁ͕͋Δ͔Ͳ͏͔ɺݕ౼͕ඞཁ • ҰํɺϐΞεͷΈɾΠϠϦϯάͷΈɺͷ࡞඼͸ਖ਼͘͠෼ྨ͍ͨ͠ • ໨ࢹͰ֬ೝ   ΠϠϦϯάɿ2݅ɺϐΞεɿ13݅ɺ྆ํɿ55݅   ϐΞεɿ2݅ɺΠϠϦϯάɿ10݅ɺ྆ํɿ22݅ɺ   ɹͲͪΒͱ΋ݴ͑ͳ͍ɿ5݅

Slide 28

Slide 28 text

28 খΧςΰϦͷ෼ྨੑೳ޲্ʹ޲͚ͨࠓޙͷऔΓ૊Έ • େΧςΰϦΛ෼ྨޙʹɺখΧςΰϦΛ෼ྨ͢ΔϞσϧΛ࡞Δ • ϝϦοτɹɿখΧςΰϦͷ෼ྨੑೳͷ޲্͕ݟࠐΊΔ • σϝϦοτɿେΧςΰϦ୯ҐͰͷϞσϧߏங͕ඞཁ ɾɾ STEP1:େΧςΰϦΛ෼ྨ ˙ࠓޙͷऔΓ૊Έ ˙ݱঢ় STEP2:֤େΧςΰϦ಺͔ΒখΧςΰϦΛ෼ྨ ௚઀ɺখΧςΰϦΛ෼ྨ͍ͯ͠Δ ΞΫηαϦʔ ϑΝογϣϯ ϚεΫ ͓΋ͪΌ ΞΫηαϦʔ ࢦྠɾϦϯά ϐΞε ɾɾɾ ɾɾɾ খ খ খ খ খ େ

Slide 29

Slide 29 text

5. ·ͱΊͱࠓޙͷ՝୊

Slide 30

Slide 30 text

• ʮϋϯυϝΠυ࡞඼Λѻ͏ECαΠτʯͷ՝୊ΛؚΉλεΫͷ಺ɺ  ࡞඼ͷΧςΰϦ෼ྨͷλεΫʹ͓͍ͯ෼ྨੑೳΛൺֱͨ͠ • ݁Ռͱͯ͠ɺBERT+ fi ne-tuning͕PrecisionҎ֎ͷධՁࢦඪͰྑ͍݁Ռ   → ࡞඼ͷଟ༷ੑΛҰఆఔ౓͸ଊ͑Δ͜ͱ͕Ͱ͖ͨͷͰ͸ͳ͍͔   30 ·ͱΊ

Slide 31

Slide 31 text

• BERT௥ՃֶशʹΑΔίʔύεʹಛԽͨ͠஌ࣝͷڧԽ • খΧςΰϦͷ෼ྨʹରͯ͠͸ɺ2ஈ֊ʢେΧςΰϦˠখΧςΰϦʣͰ෼ྨ • ධՁํ๏ͷݕ౼ • ਖ਼ղͷΧςΰϦ͕ਖ਼͘͠ͳ͍ՄೳੑΛ౿·͑ͨධՁͷݕ౼ • ʮΠϠϦϯάorϐΞεʯͷΑ͏ʹɺҰҙʹ෼ྨ͢Δ͔Ͳ͏͔ͷݕ౼ • fi ne-tuningʹ༻͍ΔσʔλྔΛ૿΍͢ 31 ࠓޙͷ՝୊

Slide 32

Slide 32 text

No content