【輪講資料】Zero-shot Cross-lingual Semantic Parsing

研究室内の輪講で使った資料です。

Transcript

1. Zero-shot Cross-lingual Semantic Parsing Tom Sherborne, Mirella Lapata ACL 2022
2. ֓ཁ 2 ✓ θϩγϣοτଟݴޠҙຯղੳϞσϧɺZX-parseͷఏҊ • ݴޠؒજࡏදݱͷΞϥΠϝϯτʹண໨͠సҠֶशͷޡࠩΛ࠷খԽ • ෳ਺ͷଛࣦؔ਺Λಋೖ͢ΔϚϧνλεΫֶशʹΑΓҟͳΔݴޠͷ જࡏදݱ͕ྨࣅ͢ΔΑ͏ʹ • ର৅ݴޠͷϖΞσʔλΛඞཁͱͤͣɺӳޠϖΞͱର৅ݴޠͷࣗવ จͷΈར༻ ✓ θϩγϣοτҙຯղੳλεΫʹ͓͍ͯෳ਺ͷݴޠͰߴ͍ੑೳ
3. બΜͩཧ༝ • ݴޠԣஅతͳݚڀʹڵຯ͕͋ΔͨΊ • ಛʹݴޠԣஅతͳજࡏۭؒΛ࡞Δ࿩Ͱ໘നͦ͏ • ࣗ෼ͷݚڀΛؚΉ͞·͟·ͳసҠֶशλεΫͰར༻Ͱ͖ͦ͏ 3
4. ҙຯղੳ (Semantic Parsing) • ࣗવݴޠͷൃ࿩Λ࿦ཧܗࣜ(Logical Form)ʹม׵ • ͞·͟·ͳλεΫͰॏཁͳΠϯϑϥ • ࣭໰Ԡ౴ɺର࿩γεςϜɺػց΁ͷࢦࣔ… 4 -JTU
   fl JHIUT
   GSPN
   4BO
   'SBODJTDP
   UP
   1JUUTCVSHI
   
   4&-&$5
   %*45*/$5
   fl JHIU@ fl JHIU@JE
   '30.
   ʜ
   
5. • ෳ਺ͷݴޠ͔ΒͳΔࣗવจΛಉ͡࿦ཧܗࣜʹม׵[1] ଟݴޠҙຯղੳ (Cross-lingual Semantic Parsing) 5 <>
   .VMUJMJOHVBM
   4FNBOUJD
   1BSTJOH
   
   1BSTJOH
   .VMUJQMF
   -BOHVBHFT
   JOUP
   4FNBOUJD
   3FQSFTFOUBUJPOT 4&-&$5
   %*45*/$5
   fl JHIU@ fl JHIU@JE
   '30.
   ʜ
   -JTU
   fl JHIUT
   GSPN
   4BO
   'SBODJTDP
   UP
   1JUUTCVSHI
   αϯϑϥϯγείൃϐοπ όʔάߦ͖ͷϑϥΠτΛ 
 ϦετΞοϓ͍ͯͩ͘͠͞ɻ
   ʹ
6. ઌߦݚڀ • ӳޠͷࣗવจ-࿦ཧܗࣜͷฒྻσʔλΛ຋༁͠ɺ֤ର৅ݴޠͷࣗવ จ-࿦ཧܗࣜϖΞσʔλΛར༻ • ػց຋༁Λ༻͍Δ৔߹[2] • ػց຋༁͸׬ᘳͰ͸ͳ͍ • ಛʹ௿Ϧιʔεݴޠʹ͓͍ͯ
 ߴ඼࣭ͳػց຋༁͸೉͍͠ • ਓख຋༁Λ༻͍Δ৔߹[3] • ߴίετ 6 <>
   #PPUTUSBQQJOH
   B
   $SPTTMJOHVBM
   4FNBOUJD
   1BSTFS
   <>
   /FVSBM
   "SDIJUFDUVSFT
   GPS
   .VMUJMJOHVBM
   4FNBOUJD
   1BSTJOH ࿦ཧܗࣜ ӳޠ ର৅ݴޠ ຋༁
7. • ର৅ݴޠͷฒྻσʔλΛ࢖Θͳ͍
 θϩγϣοτ໰୊ͱ͢Δ ➡ ӳޠͷࣗવจ-࿦ཧܗࣜϖΞσʔλͱ
 ର৅ݴޠͷࣗવจͷΈར༻ ✓ Ϟσϧߏ଄ • ӳޠ͔Βಘͨજࡏදݱ͔Β
 ࿦ཧܗࣜΛੜ੒͢Δσίʔμ • ݴޠԣஅతͳʢӳޠͱྨࣅͨ͠ʣ
 જࡏදݱΛ֫ಘ͢ΔΤϯίʔμ ఏҊख๏ 7 αϯϑϥϯγείൃϐοπ όʔάߦ͖ͷϑϥΠτΛ 
 ϦετΞοϓ͍ͯͩ͘͠͞ɻ 4&-&$5
   4&-&$5
   4&-&$5
   8)&3&
   
   '30.
   4&-&$5
    Τϯίʔμ σίʔμ ❌ 4&-&$5
   %*45*/$5
   fl JHIU@ fl JHIU@JE
   '30.
   ʜ -JTU
   fl JHIUT
   GSPN
   4BO
   'SBODJTDP
   UP
   1JUUTCVSHI
   ⭕ Τϯίʔμ σίʔμ ˛
   ΞϥΠϝϯτΛߦΘͳ͍৔߹ɺݴޠ͝ͱʹ ҟͳΔજࡏදݱͱͳΓग़ྗ͕ҟͳΔ જࡏදݱ
8. ϚϧνλεΫֶश • ̍ͭͷΤϯίʔμʹର͠ෳ਺ͷ໨తؔ਺Λ༻͍ɺಉ࣌ʹ࠷దԽ[4] • ଟݴޠλεΫʹ͓͍ͯɺʮιʔεݴޠ (ӳޠͳͲ) Ͱͷ໨ඪλεΫʴ ιʔεݴޠͱର৅ݴޠͷΞϥΠϝϯτʯΛಉ࣌ʹ࠷దԽ͢Δݚڀ ͕ଘࡏ • Ի੠ݴޠཧղɺςΩετ؆ུԽɺґଘੑߏจղੳɺػց຋༁ 8 <>
   .VMUJUBTL
   4FRVFODF
   UP
   4FRVFODF
   -FBSOJOH
9. ఏҊϞσϧɿZX-Parse • ϚϧνλεΫSeq2seqϞσϧ ✓ ໨తͷλεΫʹՃ͑ิॿతͳλεΫΛಋೖ • DLF ɿ࿦ཧܗࣜͷੜ੒ • DNL ɿࣗવݴޠͷੜ੒ • LPɿݴޠ༧ଌ 9 Τϯίʔμ z DLF LP ࣗવจ DNL ֶश(EN) ਪ࿦ જࡏදݱ Transformer x 6૚ (mBARTͷΤϯίʔμ) Transformer x 6૚ ֶश(ଞݴޠ) ࿦ཧܗࣜͷੜ੒͸
 ӳޠͰͷΈֶश
10. DLF ɿ Generating Logical Forms 10 Τϯίʔμ z DLF LP List fl ights from San Francisco to Pittsburgh? DNL SELECT DISTINCT fl ight_1. fl ight_id FROM … • ӳޠͷࣗવจ͔Βಉ͡ҙຯͷ࿦ཧܗࣜΛੜ੒͢Δ ➡࿦ཧܗࣜੜ੒ೳྗΛ਎ʹ͚ͭΔ ֶश࣌ ਪ࿦࣌ જࡏදݱ
11. • ର৅ݴޠͷࣗવจʹϊΠζΛՃ͑ɺ࠶ߏங΋͘͠͸ӳޠ΁຋༁ ➡֤ݴޠͷࣗવจʹΤϯίʔμʔΛదԠͤ͞Δ • ݴޠݻ༗ͷಛੑΛ਎ʹ෇͚Δ DNL ɿ Generating Natural Language 11 Τϯίʔμ z DLF LP αϯϑϥϯγείൃϐοπ όʔάߦ͖ͷϑϥΠτΛ
 ϦετΞοϓ͍ͯͩ͘͠͞ɻ DNL Н ϊΠζ Н αϯϑϥϯγείൃϐοπ όʔάߦ͖ͷϑϥΠτΛ
 ϦετΞοϓ͍ͯͩ͘͠͞ɻ List fl ights from San Francisco to Pittsburgh? ֶश࣌ ਪ࿦࣌ જࡏදݱ
12. LPɿ Language Prediction 12 Τϯίʔμ z DLF LP List fl ights from San Francisco to Pittsburgh? DNL English ֶश࣌ ਪ࿦࣌ • ೖྗจͷݴޠΛ෼ྨثͰ༧ଌ͢Δ ➡ΤϯίʔμʹݴޠΛ۠ผͤ͞Δ જࡏදݱ
13. LPɿ Language Prediction 13 Τϯίʔμ z DLF LP List fl ights from San Francisco to Pittsburgh? DNL English ֶश࣌ ਪ࿦࣌ • ೖྗจͷݴޠΛ෼ྨثͰ༧ଌ͢Δ ➡ΤϯίʔμʹݴޠΛ۠ผͤ͞Δ ✦ ٯ఻೻࣌ʹޯ഑Λ൓సͤ͞Δ ➡ ݴޠΛ۠ผͤ͞ͳ͍ • ݴޠʹͱΒΘΕͳ͍දݱ જࡏදݱ
14. • ಉ࣌ʹ̏ͭͷ໨తؔ਺Λ࠷దԽ • ࿦ཧܗࣜͷੜ੒ʴݴޠ͝ͱͷಛ௃ΛֶशʴݴޠΛ۠ผ͠ͳ͍ • ର৅ݴޠͷϖΞίʔύεΛඞཁͱ͠ͳ͍θϩγϣοτҙຯղੳ ZX-Parse 14 Τϯίʔμ z DLF LP DNL ٯ఻೻࣌ ࿦ཧܗࣜͷੜ੒ ࣗવݴޠͷੜ੒ ʢ࠶ߏ੒/຋༁ʣ ݴޠ༧ଌ − ∂LLP ∂θ ∂LNL ∂θ ∂LLF ∂θ જࡏදݱ
15. σʔληοτ ✦ ҙຯղੳσʔληοτ • ATIS • ཱྀߦυϝΠϯͷଟݴޠࣗવจͱ࿦ཧܗࣜͷϖΞίʔύε • ӳޠ, ϑϥϯεޠ, ϙϧτΨϧޠ, εϖΠϯޠ, υΠπޠ, தࠃޠ, ώϯσΟʔޠ, τϧίޠ • Overnight • ̔υϝΠϯͷӳޠࣗવจͱ࿦ཧܗࣜͷϖΞίʔύε • ਪ࿦࣌ͷΈ຋༁Ͱ࡞੒͞ΕͨதࠃޠͱυΠπޠσʔλΛར༻ ✦ ࣗવݴޠσʔληοτ • MKQA • ࣭໰จͷର༁ίʔύε • ӳޠ, ϑϥϯεޠ, ϙϧτΨϧޠ, εϖΠϯޠ, υΠπޠ, தࠃޠ • ParaCrawl • Webର༁ίʔύε 15
16. σʔληοτ ✦ ҙຯղੳσʔληοτ • ATIS • ཱྀߦυϝΠϯͷଟݴޠࣗવจͱ࿦ཧܗࣜͷϖΞίʔύε • ӳޠ, ϑϥϯεޠ, ϙϧτΨϧޠ, εϖΠϯޠ, υΠπޠ, தࠃޠ, ώϯσΟʔޠ, τϧίޠ • Overnight • ̔υϝΠϯͷӳޠࣗવจͱ࿦ཧܗࣜͷϖΞίʔύε • ਪ࿦࣌ͷΈ຋༁Ͱ࡞੒͞ΕͨதࠃޠͱυΠπޠσʔλΛར༻ ✦ ࣗવݴޠσʔληοτ • MKQA • ࣭໰จͷର༁ίʔύε • ӳޠ, ϑϥϯεޠ, ϙϧτΨϧޠ, εϖΠϯޠ, υΠπޠ, தࠃޠ • ParaCrawl • Webର༁ίʔύε 16
17. ຋༁ϕʔεϥΠϯ ✦ “࠷খݶͷ౒ྗ”ϕʔεϥΠϯ • ػց຋༁Λ༻͍ͯ࡞੒ͨ͠ϕʔεϥΠϯ • Translate-Test • ςετηοτΛӳޠʹ຋༁ • Translate-Train • ֶशηοτΛର৅ݴޠʹ຋༁ ✦ “࠷େݶͷ౒ྗ”ϕʔεϥΠϯ • Monolingual Training • ֶशηοτΛਓखͰର৅ݴޠʹ຋༁ 17
18. ࣮ݧ݁Ռɿ຋༁ϕʔεϥΠϯͱͷൺֱ • ӳޠҎ֎ͷݴޠͷθϩγϣοτҙຯղੳλεΫͰSOTA 18 • Monolingual Training: ֶशηοτΛਓखͰର৅ݴޠʹ຋༁ • Translate-Train: ֶशηοτΛର৅ݴޠʹ຋༁ • Translate-Test: ςετηοτΛӳޠʹ຋༁ • ATIS: ࣭໰จͱ࿦ཧܗࣜͷϖΞσʔλ • Overnight: ෳ਺υϝΠϯͷจͱ࿦ཧܗࣜͷϖΞσʔλ ※ώϯσΟʔޠ(HI)ɺτϧίޠ(TR)͸ର༁ίʔύε͕
 ଘࡏ͠ͳ͍ͨΊิॿ໨తͰͷֶशʹؚ·Ε͍ͯͳ͍
19. ࣮ݧ݁Ռɿ࠷খݶͷ౒ྗϕʔεϥΠϯͱͷൺֱ • ิॿ໨తͰͷֶशΛߦͳͬͨશͯͷݴޠ(HI,TRҎ֎)ʹ͓͍ͯੑೳ޲্ • Overnight͸ATIS΄Ͳੑೳ޲্͍ͯ͠ͳ͍ • υϝΠϯ͕ଟ༷ͳͨΊɺ՝୊͕ෳ߹తʹͳ͍ͬͯΔ • ӳޠʹ͍ۙݴޠͰ͸Translate-Train͸Translate-TestΛԼճΔ͕ɺԕ͍ݴޠ Ͱ͸ੑೳ͕௿Լ͢Δ • ػց຋༁ͷํ޲ͷӨڹʁ 19
20. ࣮ݧ݁ՌɿαϒλεΫͷӨڹ 20 • LFͷΈͷ৔߹ɺTranslate-TestΑΓ΋ੑೳ͕௿͍ • NLͱLPͲͪΒ͔௥Ճ͢Δ࣌ɺNLΛ௥Ճͨ͠ํ͕ੑೳ͕ߴ͍ • ֤ݴޠͷࣗવݴޠจʹదԠ͢Δํ͕େࣄ
21. ࣮ݧ݁ՌɿαϒλεΫͷӨڹ 21 • LFͷΈͷ৔߹ɺTranslate-TestΑΓ΋ੑೳ͕௿͍ • NLͱLPͲͪΒ͔௥Ճ͢Δ࣌ɺNLΛ௥Ճͨ͠ํ͕ੑೳ͕ߴ͍ • ֤ݴޠͷࣗવจʹదԠ͢Δ͜ͱ͕େࣄ
22. ࣮ݧ݁ՌɿαϒλεΫͷӨڹ • αϒλεΫͰֶशʹؚ·Εͳ͍ݴޠ(HI,TR)Ͱ΋ੑೳ޲্ • ௚઀ֶश͍ͯ͠ͳͯ͘΋ݴޠؒͷજࡏදݱ͕վળ͞Ε͍ͯΔ 22
23. ࣮ݧ݁ՌɿαϒλεΫͷӨڹ • αϒλεΫͰֶशʹؚ·Εͳ͍ݴޠ(HI,TR)Ͱ΋ੑೳ޲্ • ௚઀ֶश͍ͯ͠ͳͯ͘΋ݴޠؒͷજࡏදݱ͕վળ͞Ε͍ͯΔ 23 ˝
    ຒΊࠐΈͷՄࢹԽ
24. ࣮ݧ݁ՌɿαϒλεΫͷӨڹ • ୯ݴޠͷΈར༻͢Δ৔߹(τ = 0.0)ΑΓ΋຋༁΋ߦ͏৔߹(τ = 0.5) ͷੑೳ͕ߴ͍ • ద౰ͳจΛར༻͢Δ৔߹(ParaCrawl)ΑΓ΋ਪ࿦࣌ͷೖྗͱಉ࣭͡ ໰จΛֶशʹར༻͢Δ৔߹(MKQA)ͷੑೳ͕ߴ͍ 24
25. ·ͱΊ ✓ θϩγϣοτଟݴޠҙຯղੳϞσϧɺZX-parseͷఏҊ • જࡏදݱͷΞϥΠϝϯτʹண໨͠సҠֶशͷޡࠩΛ࠷খԽ • ෳ਺ͷଛࣦؔ਺Λಋೖ͢ΔϚϧνλεΫֶशʹΑΓҟͳΔݴޠͷ જࡏදݱ͕ྨࣅ͢ΔΑ͏ʹ • ର৅ݴޠͷϖΞσʔλΛඞཁͱͤͣɺӳޠϖΞͱର৅ݴޠͷࣗવ จͷΈར༻ ✓ θϩγϣοτҙຯղੳλεΫʹ͓͍ͯෳ਺ͷݴޠͰߴ͍ੑೳ 25