Neural Network Models for Paraphrase Identification, Semantic Textual Similarity, Natural Language Inference, and Question Answering

/FVSBM/FUXPSL.PEFMTGPS 1* 454 /-* BOE2" 8VXFJ -BOBOE8FJ9V *O1SPDPG$0-*/( QQ #FTU1BQFS"XBSE
1SFTFOUFECZ5PNPZVLJ,BKJXBSB

4FOUFODF1BJS.PEFMJOH จͱจͷؒͷҙຯతͳؔ܎ΛٻΊΔͭͷλεΫ • 1* 1BSBQISBTF*EFOUJGJDBUJPO ݴ͍׵͑ೝࣝ • 454 4FNBOUJD5FYUVBM4JNJMBSJUZ จͷྨࣅ౓ਪఆ
• /-* /BUVSBM-BOHVBHF*OGFSFODF ؚҙؔ܎ೝࣝ • 2" 2VFTUJPO"OTXFSJOH ࣭໰Ԡ౴ 4FOUFODF 4FOUFODF 4FOUFODF1BJS.PEFM -BCFM

.PUJWBUJPOBOE3FTFBSDI2VFTUJPOT • ੲ͸λεΫݻ༗ͷख๏Λ։ൃ͢Δ͔͠ͳ͔ͬͨ • ࠓ͸ڞ௨ͷϞσϧͰ1*΋454΋/-*΋2"΋શ෦ಉ͡Α͏ʹղ͚Δ • ͔͠͠ɺ֤छ405"ख๏͸dݸͷσʔλͰ͔͠ධՁ͍ͯ͠ͳ͍ Ø ͦͷϞσϧ͸ଞͷλεΫ΍σʔληοτͰ΋༗༻Ͱ͔͢ʁ Ø
γεςϜઃܭ΍ύϥϝλ࠷దԽ͕ͲΕ͚ͩੑೳΛվળ͠·͔͢ʁ 4/-* ./-* 2VPSB 63- 1*5 454 8JLJ2" 5SFD2" *OGFS4FOU 44& %FD"UU &4*.@TFR &4*.@USFF &4*.@TFR USFF 18*.

(FOFSBM'SBNFXPSLGPS4FOUFODF1BJS.PEFMJOH 4FOUFODF&ODPEJOHCBTFE 8PSE*OUFSBDUJPOCBTFE

(FOFSBM'SBNFXPSLGPS4FOUFODF1BJS.PEFMJOH *OQVU&NCFEEJOH-BZFS • ֶशࡁΈͷ୯ޠ෼ࢄදݱʢXPSEWFD(MP7FʣΛ࢖͏ͷ͕Ұൠత • 4/4σʔλʹରͯ͠͸αϒϫʔυ୯Ґͷ෼ࢄදݱ͕༗ޮ 4FOUFODF&ODPEJOHCBTFE 8PSE*OUFSBDUJPOCBTFE

(FOFSBM'SBNFXPSLGPS4FOUFODF1BJS.PEFMJOH $POUFYU&ODPEJOH-BZFS • 3W//ͱ͔$//ͱ͔-45.ͱ͔Ͱ୯ޠ͔Βจͷ෼ࢄදݱΛ࡞Δ • 4FOUFODF&ODPEJOHCBTFEϞσϧͷ৔߹͸ɺ͜ΕͰऴΘΓ 4FOUFODF&ODPEJOHCBTFE 8PSE*OUFSBDUJPOCBTFE

(FOFSBM'SBNFXPSLGPS4FOUFODF1BJS.PEFMJOH *OUFSBDUJPOBOE"UUFOUJPO-BZFS • ֤࣌ࠁͷӅΕ૚Λநग़ͯ͠ɺ୯ޠΞϥΠϝϯτ෩ͷ৘ใΛಘΔ 4FOUFODF&ODPEJOHCBTFE 8PSE*OUFSBDUJPOCBTFE

4FOUFODF&ODPEJOHCBTFE.PEFMT *OGFS4FOU #J-45..BYQPPMJOH/FUXPSL 44& 4IPSUDVU4UBDLFE 4FOUFODF&ODPEFS.PEFM

8PSE*OUFSBDUJPOCBTFE.PEFMT • ͜͜ʹςΩετΛೖྗ 18*. 1BJSXJTF8PSE *OUFSBDUJPO.PEFM %FD"UU %FDPNQPTBCMF"UUFOUJPO.PEFM &4*.
&OIBODFE4FRVFOUJBM*OGFSFODF.PEFM

%BUBTFU 4J[F &YBNQMF -BCFM 4/-* 5SBJO %FW 5FTU
5XPNFOPOCJDZDMFTDPNQFUJOHJOBSBDF .FO BSFSJEJOHCJDZDMFTPOUIFTUSFFU FOUBJMNFOU OFVUSBM DPOUSBEJDU .VMUJ/-* 5SBJO %FW 5FTU 5IF0ME0OF BMXBZTDPNGPSUFE$B`EBBO FYDFQUUPEBZ $B`EBBO LOFXUIF0ME0OFWFSZXFMM FOUBJMNFOU OFVUSBM DPOUSBEJDU 2VPSB 5SBJO %FW 5FTU 8IBU TIPVME*EPUPBWPJETMFFQJOHJODMBTT )PXEP*OPUTMFFQJOBCPSJOHDMBTT QBSBQISBTF OPOQBSBQISBTF 5XJUUFS63- 5SBJO %FW 5FTU -FUUFSXBSOFE8FMMT'BSHPPGlXJEFTQSFBEzGSBVE JO -FUUFST TVHHFTU8FMMT'BSHPTDBOEBMTUBSUFEFBSMJFS QBSBQISBTF OPOQBSBQISBTF 1*5 5SBJO %FW 5FTU &[FLJFM "OTBI XUIF%TIBEFT1PQQFEPVUMFOT &[FLJFM"OTBI XBTXFBSJOHMFOTMFTT%HMBTTFT QBSBQISBTF OPOQBSBQISBTF 454 5SBJO %FW 5FTU 5IFOQFSIBQTXFDPVMEIBWFBWPJEFEBDBUBTUSPQIF 5IFOXFNJHIUIBWFCFFOBCMFUPBWPJEBEJTBTUFS TDPSF < > 8JLJ2" 5SBJO %FW 5FTU )PXNVDIJTUBCMFTQPPO PGXBUFS *O"VTUSBMJBPOFUBCMFTQPPO NFBTVSFNFOUVOJU JT N- USVF GBMTF 5SFD2" 5SBJO %FW 5FTU 8IPXBT-JODPMO`T4FDSFUBSZPG4UBUF 8JMMJBN 4FXBSE USVF GBMTF λεΫछྨͷσʔληοτʢؚҙؔ܎ೝࣝɺݴ͍׵͑ೝࣝɺจͷྨࣅ౓ਪఆɺ࣭໰Ԡ౴ʣ

࣮ݧ݁Ռʢ੨ɿ࠷ߴɺ੺ɿ࠷௿ʣ جຊతʹΦϦδφϧͷใࠂΑΓ΋ dϙΠϯτ௿͍ Ø ݸʑͷσʔληοτ͝ͱʹ޿ൣғͳνϡʔχϯά͸͍ͯ͠ͳ͍ Ø ϥϯμϜγʔυΛዞҙతʹબΜͰ͍ͳ͍ Ø શͯͷϞσϧΛ1Z5PSDIͰ࣮૷ͨ͠
4/-* ./-* 2VPSB 63- 1*5 454 8JLJ2" 5SFD2" *OGFS4FOU 44& %FD"UU &4*.@TFR &4*.@USFF &4*.@TFR USFF 18*. ˞͋ͱͷϖʔδͰղઆ͢ΔͷͰɺ΅ʔͬͱݟͯ͘ΕͨΒ͍͍Ͱ͢

ଟ͘ͷλεΫͰ 8PSE*OUFSBDUJPOCBTFE.PEFM͕ڧ͍ ୯ޠؒͷؔ܎Λཅʹߟྀͨ͠ํ͕ྑ͍
4/-* ./-* 2VPSB 63- 1*5 454 8JLJ2" 5SFD2" 4FOUFODF&ODPEJOHCBTFE.PEFM 8PSE*OUFSBDUJPO.PEFM

-45.ʹؔ͢Δߟ࡯ %FD"UU 8PSE*OUFSBDUJPOCBTFE.PEFMXJUIPVU-45. &4*. %FD"UU XJUI #J 5SFF -45.&ODPEFST
4/-* ./-* 2VPSB 63- 1*5 454 8JLJ2" 5SFD2" %FD"UU &4*.@TFR &4*.@USFF &4*.@TFR USFF

-45.ʹΑΔจ຺৘ใͷΤϯίʔυ͕ॏཁ %FD"UU 8PSE*OUFSBDUJPOCBTFE.PEFMXJUIPVU-45. &4*. %FD"UU XJUI #J 5SFF -45.&ODPEFST

5SFF-45.ΑΓ΋ #J-45.ͷํ͕ৗʹڧ͍ %FD"UU 8PSE*OUFSBDUJPOCBTFE.PEFMXJUIPVU-45. &4*. %FD"UU XJUI #J 5SFF -45.&ODPEFST

5SFF-45.͸ 5XJUUFSͷσʔλͰߩݙ͢Δʢ63-ɺ1*5ʣ ྫ͑͹ lFWFSXPOEFSFE zͱ͍͏ϑϨʔζΛແࢹͰ͖ΔΑ͏ʹͳΔ FWFSXPOEFSFE XIZZPVSSFDPSEFEWPJDFTPVOETXFJSEUP ZPV BOEXIZEPPVSSFDPSEFEWPJDFTTPVOETPXFJSEUPVT
4/-* ./-* 2VPSB 63- 1*5 454 8JLJ2" 5SFD2" %FD"UU &4*.@TFR &4*.@USFF &4*.@TFR USFF Ξϊςʔλ͕࡞ͬͨଞͷσʔλͱ͸ҧ͍ɺ ͜ΕΒͷ5XJUUFSσʔλ͸ݱ࣮ͷςΩετ

܇࿅࣌ؒʢ4/-*ͷສจରʣ -45.ൈ͖ͷ%FD"UU͕࠷଎ɺ૚#J-45.ͷ*OGFS4FOU͕࣍ʹ଎͍
*OGFS4FOU 44& %FD"UU &4*.@TFR &4*.@USFF &4*.@TFR USFF 18*. <࣌ؒ>

୯ޠҰக཰͕௿͍΄Ͳղ͘ͷ͕೉͍͠
*OGFS4FOU 44& %FD"UU &4*.@TFR 18*. จؒͷ୯ޠҰக཰ d

จ௕͕௕͍΄Ͳղ͘ͷ͕೉͍͠
*OGFS4FOU 44& %FD"UU &4*.@TFR 18*. จ௕ UPLFOT dUPLFOT UPLFOT

ݴ͍׵͑ೝࣝʹ͓͚ΔసҠֶशʢ2VPSBສจରɺ1*5ສจରʣ
*OGFS4FOU 44& %FD"UU &4*.@TFR 18*. 2VPSBͰ܇࿅ˠ1*5ͰධՁ 1*5Ͱ܇࿅ˠ1*5ͰධՁ *OGFS4FOU 44& %FD"UU &4*.@TFR 18*. QBSBNFUFST . . , . . ύϥϝλ͕ଟ͍ͷͰେྔͷσʔλ͕ඞཁ PSυϝΠϯʹґଘ͠ͳ͍ҰൠతͳϞσϧ

IUUQTHJUIVCDPNMBOXVXFJ41.@UPPMLJU 4/-* ./-* 2VPSB 63- 1*5 454 8JLJ2" 5SFD2" *OGFS4FOU
44& %FD"UU &4*. 18*. 4FOUFODF&ODPEJOHCBTFE 8PSE*OUFSBDUJPOCBTFE

Neural Network Models for Paraphrase Identifica...

Neural Network Models for Paraphrase Identification, Semantic Textual Similarity, Natural Language Inference, and Question Answering

onizuka laboratory

More Decks by onizuka laboratory

Other Decks in Research

Featured

Transcript

/FVSBM/FUXPSL.PEFMTGPS 1* 454 /-* BOE2" 8VXFJ -BOBOE8FJ9V O1SPDPG$0-/( QQ #FTU1BQFS"XBSE

4FOUFODF1BJS.PEFMJOH จͱจͷؒͷҙຯతͳؔ܎ΛٻΊΔͭͷλεΫ • 1* 1BSBQISBTF*EFOUJGJDBUJPO ݴ͍׵͑ೝࣝ • 454 4FNBOUJD5FYUVBM4JNJMBSJUZ จͷྨࣅ౓ਪఆ

.PUJWBUJPOBOE3FTFBSDI2VFTUJPOT • ੲ͸λεΫݻ༗ͷख๏Λ։ൃ͢Δ͔͠ͳ͔ͬͨ • ࠓ͸ڞ௨ͷϞσϧͰ1΋454΋/-΋2"΋શ෦ಉ͡Α͏ʹղ͚Δ • ͔͠͠ɺ֤छ405"ख๏͸dݸͷσʔλͰ͔͠ධՁ͍ͯ͠ͳ͍ Ø ͦͷϞσϧ͸ଞͷλεΫ΍σʔληοτͰ΋༗༻Ͱ͔͢ʁ Ø

(FOFSBM'SBNFXPSLGPS4FOUFODF1BJS.PEFMJOH 4FOUFODF&ODPEJOHCBTFE 8PSE*OUFSBDUJPOCBTFE

(FOFSBM'SBNFXPSLGPS4FOUFODF1BJS.PEFMJOH OQVU&NCFEEJOH-BZFS • ֶशࡁΈͷ୯ޠ෼ࢄදݱʢXPSEWFD(MP7FʣΛ࢖͏ͷ͕Ұൠత • 4/4σʔλʹରͯ͠͸αϒϫʔυ୯Ґͷ෼ࢄදݱ͕༗ޮ 4FOUFODF&ODPEJOHCBTFE 8PSEOUFSBDUJPOCBTFE

(FOFSBM'SBNFXPSLGPS4FOUFODF1BJS.PEFMJOH $POUFYU&ODPEJOH-BZFS • 3W//ͱ͔$//ͱ͔-45.ͱ͔Ͱ୯ޠ͔Βจͷ෼ࢄදݱΛ࡞Δ • 4FOUFODF&ODPEJOHCBTFEϞσϧͷ৔߹͸ɺ͜ΕͰऴΘΓ 4FOUFODF&ODPEJOHCBTFE 8PSE*OUFSBDUJPOCBTFE

(FOFSBM'SBNFXPSLGPS4FOUFODF1BJS.PEFMJOH OUFSBDUJPOBOE"UUFOUJPO-BZFS • ֤࣌ࠁͷӅΕ૚Λநग़ͯ͠ɺ୯ޠΞϥΠϝϯτ෩ͷ৘ใΛಘΔ 4FOUFODF&ODPEJOHCBTFE 8PSEOUFSBDUJPOCBTFE

4FOUFODF&ODPEJOHCBTFE.PEFMT *OGFS4FOU #J-45..BYQPPMJOH/FUXPSL 44& 4IPSUDVU4UBDLFE 4FOUFODF&ODPEFS.PEFM

8PSEOUFSBDUJPOCBTFE.PEFMT • ͜͜ʹςΩετΛೖྗ 18. 1BJSXJTF8PSE OUFSBDUJPO.PEFM %FD"UU %FDPNQPTBCMF"UUFOUJPO.PEFM &4.

%BUBTFU 4J[F &YBNQMF -BCFM 4/-* 5SBJO %FW 5FTU

࣮ݧ݁Ռʢ੨ɿ࠷ߴɺ੺ɿ࠷௿ʣ جຊతʹΦϦδφϧͷใࠂΑΓ΋ dϙΠϯτ௿͍ Ø ݸʑͷσʔληοτ͝ͱʹ޿ൣғͳνϡʔχϯά͸͍ͯ͠ͳ͍ Ø ϥϯμϜγʔυΛዞҙతʹબΜͰ͍ͳ͍ Ø શͯͷϞσϧΛ1Z5PSDIͰ࣮૷ͨ͠

ଟ͘ͷλεΫͰ 8PSE*OUFSBDUJPOCBTFE.PEFM͕ڧ͍ ୯ޠؒͷؔ܎Λཅʹߟྀͨ͠ํ͕ྑ͍

-45.ʹؔ͢Δߟ࡯ %FD"UU 8PSEOUFSBDUJPOCBTFE.PEFMXJUIPVU-45. &4. %FD"UU XJUI #J 5SFF -45.&ODPEFST

-45.ʹΑΔจ຺৘ใͷΤϯίʔυ͕ॏཁ %FD"UU 8PSEOUFSBDUJPOCBTFE.PEFMXJUIPVU-45. &4. %FD"UU XJUI #J 5SFF -45.&ODPEFST

5SFF-45.ΑΓ΋ #J-45.ͷํ͕ৗʹڧ͍ %FD"UU 8PSEOUFSBDUJPOCBTFE.PEFMXJUIPVU-45. &4. %FD"UU XJUI #J 5SFF -45.&ODPEFST

5SFF-45.͸ 5XJUUFSͷσʔλͰߩݙ͢Δʢ63-ɺ1*5ʣ ྫ͑͹ lFWFSXPOEFSFE zͱ͍͏ϑϨʔζΛແࢹͰ͖ΔΑ͏ʹͳΔ FWFSXPOEFSFE XIZZPVSSFDPSEFEWPJDFTPVOETXFJSEUP ZPV BOEXIZEPPVSSFDPSEFEWPJDFTTPVOETPXFJSEUPVT

܇࿅࣌ؒʢ4/-ͷສจରʣ -45.ൈ͖ͷ%FD"UU͕࠷଎ɺ૚#J-45.ͷOGFS4FOU͕࣍ʹ଎͍

୯ޠҰக཰͕௿͍΄Ͳղ͘ͷ͕೉͍͠

จ௕͕௕͍΄Ͳղ͘ͷ͕೉͍͠

ݴ͍׵͑ೝࣝʹ͓͚ΔసҠֶशʢ2VPSBສจରɺ1*5ສจରʣ

IUUQTHJUIVCDPNMBOXVXFJ41.@UPPMLJU 4/-* ./-* 2VPSB 63- 15 454 8JLJ2" 5SFD2" OGFS4FOU