Share
Embed
論文紹介: Corpora Generation for Grammatical Error Correction
- 首都大 小町研 春の最先端論文紹介 2019
- Corpora Generation for Grammatical Error Correction. Lichtarge et al., NAACL2019 を紹介
- paper URL: https://www.aclweb.org/anthology/N19-1333.pdf
Satoru Katsumata
December 10, 2023
Transcript
18 slides
-
֓ཁ w จ๏ޡΓగਖ਼ޡΓจˠగਖ਼จͷ༁λεΫͱͯ͠ ѻΘΕΔ͜ͱ͕ଟ͍ ˠ༻Ͱ͖Δֶशऀؔ࿈ͷର༁ίʔύεʹݶΓ͕͋Δ w 8JLJQFEJBσʔλΛͱʹֶशσʔλΛ૿͢ 8JLJQFEJBฤूཤྺΛ༻͢Δ ಛʹɺ࠷ݶͷϑΟϧλϦϯάΛࢪͯ͠ େྔͷɺ࣮ࡍͷޡΓͰߏ͞Ε͍ͯΔ͕Ԛ͍σʔλΛ֫ಘ͢Δ
3PVOEUSJQ5SBOTMBUJPOʢ355࠶༁ʣΛ༻͢Δ UHU8JLJQFEJBநग़จTSDUHUΛ࠶༁ͨ͠ͷ ༁༝དྷͰ͋Δ͕ɺ8JLJQFEJB΄ͲԚ͘ͳ͍σʔλΛ֫ಘ͢Δ w QSFUSBJO࡞ͨ͠σʔλpOFUVOF-BOH ˠ$P/--ͱ+'-&(ͰධՁɺ1#4.5ͱ/.5ͷϋΠϒϦουϞσϧͱ Ͳ͍ͬ͜ͷੑೳ !2 -
8JLJQFEJBฤूཤྺ༝དྷͷ σʔλ࡞ w ൈ͖ग़͢લͷॲཧ .CҎ্ͷϖʔδল͘ ϖʔδશͯͷฤूΛ͏ͷͰͳ͘ɺରεέʔϧͰநग़͢Δ OϖʔδͷUPUBMSFWJTJPO w ൈ͖ग़ͨ͠ޙͷॲཧˠDPSSVQUJPOBOEpMUSBUJPO DPSSVQUJPO
TSDଆʹDIBSͷ֬ͰTQFMMJOHFSSPSΛೖΕΔ ʢআɺૠೖɺஔɺྡೖΕସ͑ʣ pMUSBUJPO ࠷େจXPSEQJFDF TSDUHUͷ߹΄ͱΜͲʢʣࣺͯΔ ˠ࠷ऴతʹSFWJTJPOEBUBTFUͷ͕ಉҰจର w ͜͜ͷ࣮UFOTPSUFOTPSͷEBUB@HFOFSBUPSTʹೖͬͯΔ log1.5 (n) !3 -
8JLJQFEJBฤूཤྺ༝དྷͷ σʔλ࡞ w ൈ͖ग़͢લͷॲཧ .CҎ্ͷϖʔδল͘ ϖʔδશͯͷฤूΛ͏ͷͰͳ͘ɺରεέʔϧͰநग़͢Δ OϖʔδͷUPUBMSFWJTJPO w ൈ͖ग़ͨ͠ޙͷॲཧˠDPSSVQUJPOBOEpMUSBUJPO DPSSVQUJPO
TSDଆʹDIBSͷ֬ͰTQFMMJOHFSSPSΛೖΕΔ ʢআɺૠೖɺஔɺྡೖΕସ͑ʣ pMUSBUJPO ࠷େจXPSEQJFDF TSDUHUͷ߹΄ͱΜͲʢʣࣺͯΔ ˠ࠷ऴతʹSFWJTJPOEBUBTFUͷ͕ಉҰจର log1.5 (n) !4 -
࠶༁༝དྷͷσʔλ࡞ w 8JLJQFEJBSFWJTJPOEBUBͰࣺͯͨTSDUHUͷσʔλʹରͯ͠ɺ ҎԼͷΑ͏ʹ3PVOEUSJQUSBOTMBUJPOΛߦ͍ٖࣅTSDΛ࡞͢Δ USBOTMBUJPONPEFM(/.5 8VFUBM w ੜͨ͠ޙͷॲཧˠDPSSVQUJPO
TSDUHUͳΔࣄྫ͕΄΅ͳ͍ˠશମͷʹͳΔΑ͏ʹ૿͢ ͷ֬ͰTQFMMJOHFSSPSʢআɺૠೖɺྡసஔʣΛಋೖ SFWJTJPOEBUBʹଘࡏ͢Δగਖ਼લޙͷΈ߹ΘͤΛөͤ͞Δ ˠSFWJTJPOEBUBͱಉ͡αΠζʹͳΔΑ͏ʹͨ͠ lang ∈ {French, German, Japanese, Russian} UHUTFOU FOXJLJ USBOTMBUFETFOU MBOHXJLJ TSDTFOU FOXJLJ USBOTMBUF FOMBOH USBOTMBUF MBOHFO !5 -
࠶༁༝དྷͷσʔλ࡞ w 8JLJQFEJBSFWJTJPOEBUBͰࣺͯͨTSDUHUͷσʔλʹରͯ͠ɺ ҎԼͷΑ͏ʹ3PVOEUSJQUSBOTMBUJPOΛߦ͍ٖࣅTSDΛ࡞͢Δ USBOTMBUJPONPEFM(/.5 8VFUBM w ੜͨ͠ޙͷॲཧˠDPSSVQUJPO
TSDUHUͳΔࣄྫ͕΄΅ͳ͍ˠશମͷʹͳΔΑ͏ʹ૿͢ ͷ֬ͰTQFMMJOHFSSPSʢআɺૠೖɺྡసஔʣΛಋೖ SFWJTJPOEBUBʹଘࡏ͢Δగਖ਼લޙͷΈ߹ΘͤΛөͤ͞Δ ˠSFWJTJPOEBUBͱಉ͡αΠζʹͳΔΑ͏ʹͨ͠ lang ∈ {French, German, Japanese, Russian} UHUTFOU FOXJLJ USBOTMBUFETFOU MBOHXJLJ TSDTFOU FOXJLJ USBOTMBUF FOMBOH USBOTMBUF MBOHFO 8JLJSFWJTJPO͔Β୯ޠҎͷFEJUΛݟ͚ͭΔ ରԠ͢ΔPSJHJOBMͱSFWJTFͷΈ߹ΘͤΛΧϯτɺ ҎԼͷΑ͏ʹPSJHJOBMQISBTFͷ֬ΛٻΊΔ ͜ͷ֬Λ༻͍ͯTSDଆʹޡΓʢPSJHJOBMQISBTFʣΛೖΕ͍ͯ͘ P(original|revised) = C(original, revised) C(revised) !6 -
*UFSBUJWF%FDPEJOHͷ w ෳճEFDPEF͢Δ͜ͱͰ ΑΓྑ͍ग़ྗΛಘΑ͏ͱ͢Δ w ᮢΛར༻ͯ͠ గਖ਼ͳ͠PS͋ΓΛൺֱ͢Δ w $PTUϞσϧͷग़ྗείΞ ˠখ͍͞΄͏͕ྑ͍
−logP(hyp|src) ˠJUFSBUJWFͰվળ͢Δ߹ɺվળ͠ͳ͍߹͕ଘࡏ͢Δ !9 -
࣮ݧઃఆ w Ϟσϧ5SBOTGPSNFS 5FOTPS5FOTPSͷUSBOTGPSNFS@DMFBO@CJH@UQV PQUJNJ[FS"EBGBDUPSMBZFS IFBE FNCFEEJN GFFEGPSXBSE CFBN
w XPSEQJFDF༻ w ֶश8JLJQFEJBSFWJTJPOPS355ˠ-BOHEBUB QSFUSBJOFQPDI CBUDITJ[F XPSEQJFDF MFBSOJOHSBUFpSTU TUFQˠͦͷ͋ͱˠ pOFUVOFFQPDIMFBSOJOHSBUFº? ʢͨͩ͠ผͰͱ͓͍ͬͯͨ-BOHʁΛEFWͱͯ͠ऩଋఆʣ w ධՁσʔλ$P/--BOE+'-&(ʢ+'-&(ͷΈTQFMMDIFDL༗ʣ w JUFSBUJWFEFDPEJOH UISFTIPME+'-&(EFWͰܾఆ͢Δ $P/--$P/--USBJOJOHTFU /6$-& ͔Β$P/--EFWTFUͱಉ͡గਖ਼จɺ ඇగਖ਼จͷׂ߹ʹͳΔΑ͏ʹTBNQMFͨ͠σʔλͰܾΊΔʢ$P/--USBJOJOHTVCTFUʣ JUFSBUJPO$P/--USBJOJOHTVCTFUͰܾఆɺ+'-&(ಉ͡Λ͍·Θͯ͠Δʁ w ͦͷ΄͔ TQFMMDIFDLFS+'-&(ͷΈ lrstep = lrstep−1 step !10 -
࣮ݧ݁Ռ 8JLJQFEJB3FWJTJPOͷ߹ w %FGBVMUEPXOTBNQMF͕.BYFEJUಛʹറͬͯͳ͍ .BYFEJUЋ࠷େFEJUEJTUBODFЋͰpMUFSJOH %XOTBNQMFЌEPXOTBNQMJOHSBUF͕ "MM্هछΛVOJPOͨ͠ͷͰֶश log1.5 (n) logβ
(n) -BOHPOMZQSFUSBJOpOFUVOF ˠQSFUSBJOͨ͠΄͏͕ྑ͍ !11 -
ଞͷγεςϜͱͷൺֱʙ #FTUTJOHMFNPEFM8JLJQFEJBSFWJTJPO.BYFEJU #FTU&OTFNCMFSFWJTJPO 355NPEFM QVCMJDEBUBͷΈ༻͍ͯ͠ΔϞσϧؒͰͷൺֱ $IPMMBNQBUUBOE/H $//FODEFD
+VOD[ZT%PXNVOUFUBM 5SBOTGPSNFSXJUIMPXSFTPVSDFUJQT (SVOELJFXJD[BOE+VOD[ZT%PXNVOU CJ(36FODEFDXJUIBUUFOUJPOBOE1#4.5 !14 -
উखʹ ଞͷQSFUSBJOϞσϧͱൺֱʙ w /""$-࠾จ*NQSPWJOH(SBNNBUJDBM&SSPS$PSSFDUJPOWJB1SF5SBJOJOHB $PQZ"VHNFOUFE"SDIJUFDUVSFXJUI6OMBCFMFE%BUB;IBPFUBM w ͜ΕࣗಈཁͰྲྀߦͬͯΔ1PJOUFS(FOFSBUPSΛ(&$ʹͬͨͰɺNPEFMͷQSFUSBJO ʹ0OF#JMMJPO8PSE#FODINBSLʢ.TFOUTʣΛجʹٖͨ͠ࣅσʔλΛ༻͍ͯ͠Δ w ۩ମతʹ0OF#JMMJPOͷPSJHJOBMΛUHUʹɺ
PSJHJOBMʹରͯ͠আɺૠೖɺஔɺೖΕସ͑ૢ࡞ΛదʹߦͬͨͷΛTSDͱͯ͠ EFOPJTJOHBVUPFODPEFSͷؾ࣋ͪͰQSFUSBJO͍ͯ͠Δ w հ͍ͯ͠Δจͱͷҧ͍NPEFMΛ͠ɺֶशσʔλ͕গྔʹͳͬͨ !15 ;IBPFUBMͷQSFUSBJOSFTVMU 3FWJTJPOQSFUSBJOͷ݁Ռ -
ࢀߟจݙ w *NQSPWJOH(SBNNBUJDBM&SSPS$PSSFDUJPOWJB1SF5SBJOJOHB$PQZ"VHNFOUFE"SDIJUFDUVSF XJUI6OMBCFMFE%BUB;IBPFUBM /""$- w (PPHMFT/FVSBM.BDIJOF5SBOTMBUJPO4ZTUFN#SJEHJOHUIF(BQCFUXFFO)VNBOBOE .BDIJOF5SBOTMBUJPO8VFUBM BS9JW w
5IF8JL&E&SSPS$PSQVT"$PSQVTPG$PSSFDUJWF8JLJQFEJB&EJUTBOE*UT"QQMJDBUJPOUP (SBNNBUJDBM&SSPS$PSSFDUJPO(SVOELJFXJD[BOE+VOD[ZT%PXNVOU *OUFSOBUJPOBM $POGFSFODFPO/BUVSBM-BOHVBHF1SPDFTTJOH w ".VMUJMBZFS$POWPMVUJPOBM&ODPEFS%FDPEFS/FVSBM/FUXPSLGPS(SBNNBUJDBM&SSPS $PSSFDUJPO$IPMMBNQBUUBOE/H """* w "QQSPBDIJOH/FVSBM(SBNNBUJDBM&SSPS$PSSFDUJPOBTB-PX3FTPVSDF.BDIJOF 5SBOTMBUJPO5BTL+VOD[ZT%PXNVOUFUBM /""$- w /FBS)VNBO-FWFM1FSGPSNBODFJO(SBNNBUJDBM&SSPS$PSSFDUJPOXJUI)ZCSJE.BDIJOF 5SBOTMBUJPO(SVOELJFXJD[BOE+VOD[ZT%PXNVOU /""$- !18