Making Neural QA as Simple as Possible but not Simpler

Transcript

1. Making Neural QA as Simple as Possible but not Simpler ୩ޱହ࢙
2. ໨࣍  QAλεΫͷ֓ཁͱ՝୊
    ఏҊख๏
    ࣮ݧ
    ·ͱΊ !2
3. QAλεΫͷ֓ཁͱ՝୊
4. ֓ཁ ՝୊
   ༷ʑͳ2"λεΫͰఏҊ͞Ε͍ͯΔϞσϧ͸ߏ଄͕ෳࡶ͗ͯ͢ɺຊ࣭తʹ Կ͕ॏཁͳͷ͔Θ͔Βͳ͍
   ԿΛ͢Δ͔
   γϯϓϧͳϕʔεϞσϧΛఏҊ͠ɺॏཁͳϞδϡʔϧ୳Δɻ !4
5. QAλεΫ ֓ཁʛͲͷ2"λεΫͰ΋جຊతͳྲྀΕ͸ಉ͡
    ࣭໰จΛॲཧ
    ॴ༩ͷෳ਺จॻʗ୯จॻΛॲཧʛ༩͑ΒΕͳ͍͜ͱ΋͋Δ
    ͱͷ݁ՌΛߟྀ͠ɺղ౴Λநग़ʗੜ੒
   ຊݚڀͰ͸ɺ୯จॻͰղ౴Λநग़͢Δ2"λεΫ
   σʔληοτ͸SQuADͱNewsQAΛ࢖༻ !5
6. طଘͷQAϞσϧͷ՝୊ ݱঢ়
   ࣭໰จͱจষͷ৘ใΛͲ͏
   ѻ͏͔ Interaction Layer ʹ ஫ྗ
   ໰୊఺
   ϕʔεϞσϧ͕ͳ͍ͨΊɺ
   Ϟδϡʔϧͷॏཁੑ͕ෆ໌
   !6
7. طଘͷQAϞσϧͷ՝୊ ݱঢ়
   ࣭໰จͱจষͷ৘ใΛͲ͏
   ѻ͏͔ Interaction Layer ʹ ஫ྗ
   ໰୊఺
   ϕʔεϞσϧ͕ͳ͍ͨΊɺ
   Ϟδϡʔϧͷॏཁੑ͕ෆ໌
   Interaction Layer ࣭໰จΛߟྀͨ͠୯จॻͷදݱ Λ֫ಘ͢Δ͜ͱ͕໨త !7
8. 2. ఏҊख๏
9. ఏҊϞσϧͷ֓ཁ ࣭໰จͱ୯จॻΛೖྗ͠ɺղ౴ͷൣғΛग़ྗ ࣭໰จʛೖྗ
   Where do water droplets collide with ice crystals to form precipitation? ୯จॻʛೖྗ
   …Precipitation forms as smaller droplets coalesce via collision with other rain drops or ice crystals within20 a21 cloud22… ղ౴ͷൣғʛग़ྗ
   TUBSUʗ
   FOEʗ
   ղ౴ʛϞσϧͷग़ྗͰ͸ͳ͍
   within20 a21 cloud22 !9
10. ֤Ϟδϡʔϧͷ໾ׂ ୯จॻ
    #J-45.ͰFODPEF ղ౴ͷൣғ
    wTUBSUͱFOEΛ༧ଌ
    wֶश࣌͸ਖ਼ղͷTUBSUΛ༩͑Δ teacher forcing ͭͷϞδϡʔϧ͕ϕʔε ࣭໰จ
    #J-45.ͰFODPEF !10
11. γϯϓϧͳख๏Ͱղ͚ΔͷͰ͸? !11
12. γϯϓϧͳख๏Ͱղ͚ΔͷͰ͸? ࣌ؒΛදٙ͢໰ࢺ ࣌ؒΛද͍ͯ͠Δޠͱͯ͠ɺ
    ͭͷީิʹߜΓࠐΊΔ !12
13. γϯϓϧͳख๏Ͱղ͚ΔͷͰ͸? ࣭໰จʹग़ݱ͢Δ ୯ޠͱಉ͡୯ޠ A
     !13
14. γϯϓϧͳख๏Ͱղ͚ΔͷͰ͸? ࣭໰จʹग़ݱ͢Δ ୯ޠͱಉ͡୯ޠ A
     ࣭໰จதͷ୯ޠͰॴ༩ͷจॻதʹ
    ग़ݱ͢Δ୯ޠ͸ॏཁ !14
15. ఏҊख๏ʛContext Matching ࣭໰จதͷ୯ޠͷ୯จॻதͰͷग़ݱΛߟྀ wiqb j = I(∃i : x j = q i ) sim i,j = v wiq (x j ⊙ q i ), v wiq ∈ ℝn wiqw j = ∑ i softmax(sim i,⋅ ) j binary weighted ୯ޠ෼ࢄදݱʹfeatureΛ
    concat͢Δ͚ͩ !15
16. Context Matchingʛbinary wiqb j = I(∃i : x j = q i ) ࣭໰จதͷ୯ޠ ୯จॻதͷ୯ޠ ୯จॻ …Precipitation forms as smaller droplets coalesce via collision with other rain drops or ice crystals within a cloud… ࣭໰จ Where do water droplets collide with ice crystals to form precipitation? wiqb j = 0 1 smaller droplets coalesce 0 0 via 0 collision with 0 other rain 0 drops 0 0 1 or ice crystals 1 1 !16
17. sim i,j = v wiq (x j ⊙ q i ), v wiq ∈ ℝn wiqw j = ∑ i softmax(sim i,⋅ ) j Context Matchingʛweighted ୯จॻதͷ
    ୯ޠ෼ࢄදݱ ࣭໰จதͷ
    ୯ޠ෼ࢄදݱ ྨࣅ౓
    ୯จॻதͷ୯ޠͱ࣭໰จதͷ୯ޠͷ
    ྨࣅ౓Λ෼ࢄදݱʹج͍ͮͯܭࢉ softmax ࣭໰จதͷ͋Δ୯ޠqi ͔Β ୯จॻதͷ͋Δશ୯ޠxɾʹର͢Δ attentionείΞΛܭࢉ !17
18. wiqw j = ∑ i softmax(sim i,⋅ ) j Context Matchingʛweighted water droplets collide with ice crystals sm aller droplets coalesce via collision w ith other rain drops or ice crystals ୯จॻ sim ߦํ޲Ͱ softmax ྻํ޲Ͱsum ࣭໰จ softmax ࣭໰จதͷ͋Δ୯ޠqi ͔Β ୯จॻதͷ͋Δશ୯ޠxɾʹର͢Δ attentionείΞΛܭࢉ !18
19. wiqw j = ∑ i softmax(sim i,⋅ ) j Context Matchingʛweighted water droplets collide with ice crystals sm aller droplets coalesce via collision w ith other rain drops or ice crystals ୯จॻ sim ߦํ޲Ͱ softmax ྻํ޲Ͱsum ࣭໰จ softmax ࣭໰จதͷ͋Δ୯ޠqi ͔Β ୯จॻதͷ͋Δશ୯ޠxɾʹର͢Δ attentionείΞΛܭࢉ !19
20. ϞσϧৄࡉʛEncoder ෼ࢄදݱʹHighway NetworkΛద༻ޙBiLSTM Highway Network BiLSTM & Projection Layer forwardͱbackwardͷग़ྗΛ Ͳ͏૊Έ߹ΘͤΔ͔ !20
21. ϞσϧৄࡉʛAnswer Layer ࣭໰จΛϕΫτϧʹू໿ ղ౴ͷ։࢝ҐஔʗऴྃҐஔΛ༧ଌ ։࢝Ґஔ ऴྃҐஔ ։࢝Ґஔͷ ӅΕ૚Λߟྀ !21
22. 3. ࣮ݧ
23. ࣮ݧ݁ՌʛSQuAD ఏҊख๏ΛೖΕΔͱ ϙΠϯτ !23
24. ࣮ݧ݁ՌʛSQuAD ैདྷ๏Λ௥ՃͰߋʹϒʔετ
    DIBSFNC
    JOUSBDUJPO
    MBZFS
    CFBNTFBSDI !24
25. ਫ਼౓ൺֱʛSQuAD ΧελϚΠζͨ͠ఏҊ๏Ͱ
    ଞख๏Λѹ౗ !25
26. ·ͱΊ ఏҊख๏ʛ୯จॻͷ୯ޠͷ࣭໰จதͰͷ࢖ΘΕํΛߟྀ
    w binaryʛ࣭໰จதͰͷ୯ޠͷग़ݱΛʗͰදݱ
    w weightedʛ෼ࢄදݱʹΑΓ୯ޠͷྨࣅ౓Λܭࢉ
    ࣮ݧ݁Ռ
    w ఏҊख๏Λ௥Ճ͢Δ͚ͩͰɺ ϙΠϯτਫ਼౓޲্
    w ఏҊϞσϧʹैདྷ๏Λ௥Ճ͢ΔͱैདྷϞσϧΛѹ౗͢Δਫ਼౓ !27