Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Unsupervised Abstractive Summarization Based on Tree-Structured Topic Guidance and Rate-Distortion Theory

Unsupervised Abstractive Summarization Based on Tree-Structured Topic Guidance and Rate-Distortion Theory

NLPコロキウムで使用したスライド
https://nlp-colloquium-jp.github.io/schedule/2022-03-23_masaru-isonuma/

Masaru Isonuma

March 24, 2022
Tweet

More Decks by Masaru Isonuma

Other Decks in Technology

Transcript

  1. Unsupervised Abstractive Summarization
    Based on Tree-Structured Topic Guidance & Rate-Distortion Theory
    Masaru Isonuma

    View Slide

  2. ࣗݾ঺հ
    2
    • үপ େ/Masaru Isonuma
    • ܦྺ
    – 2015೥3݄ ౦ژେֶ޻ֶ෦ ଔۀʢେ࿨ɾṦํݚڀࣨʣ
    – 2017೥3݄ ౦ژେֶେֶӃ޻ֶܥݚڀՊम࢜՝ఔ मྃʢࡔాɾ৿ݚڀࣨʣ
    ʢίϯαϧςΟϯάاۀۈ຿Λܦͯʣ
    – 2021೥9݄ ౦ژେֶେֶӃ޻ֶܥݚڀՊത࢜՝ఔ मྃʢࡔాɾ৿ݚڀࣨʣ
    – ݱࡏ ౦ژେֶେֶӃ޻ֶܥݚڀՊ ࡔాɾ৿ݚڀࣨ ಛ೚ݚڀһ
    ܦྺ
    ݚڀ಺༰
    • ڭࢣͳ͠/ऑڭࢣ෇͖ཁ໿
    – จॻ෼ྨͱͷϚϧνλεΫֶशʹΑΔॏཁจநग़ (EMNLP’17)
    – ஊ࿩ߏ଄Λଊ͑ͨڭࢣͳ͠ϔουϥΠϯੜ੒ (ACL’19)
    – τϐοΫߏ଄ʹجͮ͘ڭࢣͳ͠ཁ໿ੜ੒ (TACL’21)
    • τϐοΫϞσϧ
    – ໦ߏ଄χϡʔϥϧτϐοΫϞσϧ (ACL’20)
    ίϯαϧ͔Βڭࢣͳ͠ཁ໿ݚڀʹࢸͬͨܦҢʢʁʣ
    ͕هࡌ͞Ε͍ͯΔnoteهࣄ
    https://note.com/jst_kisokenkyu/n/n90d06ec74985

    View Slide

  3. • τϐοΫ໦ߏ଄ʹجͮ͘ڭࢣͳ͠ཁ໿ੜ੒
    w/ J. Mori, D. Bollegala, I. Sakata (TACL’21)
    ຊ೔ͷྲྀΕ
    3
    લ൒ɿ͜Ε·Ͱͷݚڀ ޙ൒ɿݱࡏͷݚڀ಺༰
    • Ϩʔτ࿪Έཧ࿦ʹجͮ͘ڭࢣͳ͠ཁ໿ੜ੒
    overall
    service
    food
    atmo-
    sphere
    location
    place
    taste price
    The food here is fantastic, easily the best
    sub sandwiches in the Arizona area.
    The shop is local and family run, so I
    definitely choose it over a lot of the large
    national chains that are all around town.
    The staff are extremely friendly and will
    always go above and beyond in creating
    a delicious sandwich for you.
    You will not be let down by the great
    food that they make here!
    ҰൠԽ
    Έ͔Μ͕޷͖
    ΓΜ͕͝޷͖
    ੺͍ڕ͕޷͖
    ੨͍ڕ͕޷͖
    Ռ෺͕޷͖
    ڕ͕޷͖
    จষ ཁ໿
    1bit
    2bit

    View Slide

  4. ࡢࠓɺࣗಈจॻཁ໿͸ඈ༂తʹਐԽ͍ͯ͠Δ
    matthew kenney, 34, said he smoked flakka
    before he went streaking . was arrested on
    saturday after run through fort lauderdale,
    florida . drug is made from same version of
    stimulant used to produce bath salts . it causes
    euphoria, hallucinations, psychosis and
    superhuman strength . kenney has prior
    arrests and was hospitalized for a psychiatric
    evaluation .
    ৽ฉهࣄ
    matthew kenney, 34, told police he smoked
    flakka before he streaked through traffic in
    fort lauderdale while only wearing a pair of
    sneakers . he said he was escaping imaginary
    killers who he believed stole his clothes and
    wanted to murder him . kenney has previous
    arrests for disorderly conduct, making a riot
    and possession of a controlled substance .
    ࣗಈཁ໿ (PEGASUS; Zhang et al., 2020) ਓ͕࡞੒ͨ͠ࢀরཁ໿
    • ਺ສ݅୯Ґͷࢀরཁ໿ͷύλʔϯΛֶश͢Δڭࢣ͋Γख๏Ͱඈ༂తʹੑೳ͕޲্
    • ௚ۙͰ͸৽ฉهࣄͳͲͷσʔληοτͰਓͱಉ౳ͷੑೳΛୡ੒ (PEGASUS; Zhang et al., 2020)

    2022/4/14 4
    ………………………………………………
    ………………………………………………
    ………………………………………………
    ………………………………………………
    ………………………………………………
    ………………………………………………
    ………………………………………………
    ………………………………………………
    ………………………………………………
    ………………………………………………
    ………………………………………………
    ………………………………………………
    ………………………………………………
    ………………………………………………
    ………………………………………………
    …………………………………

    View Slide

  5. ͔͠͠ࢀরཁ໿͕গͳ͍จॻʹ͸ద༻͕ࠔ೉
    The food here is fantastic, easily the best sub
    sandwiches in the Arizona area. The shop is
    local and family run, so I definitely choose it
    over a lot of the large national chains that are
    all around town. The staff are extremely
    friendly and will always go above and beyond
    in creating a delicious sandwich for you. You
    will not be let down by the great food that
    they make here!
    ………………………………………………
    ………………………………………………
    ………………………………………………
    ………………………………………………
    ………………………………………………
    ………………………………………………
    ………………………………………………
    ………………………………………………
    ………………………………………………
    ………………………………………………
    ………………………………………………
    ………………………………………………
    ………………………………………………
    ………………………………………………
    ………………………………………………
    …………………………………
    McGurkee's is a family run restaurant that has
    been serving up Italian sandwiches for over
    30 years .
    • ྫ͑͹Web্ͷϨϏϡʔ΍SNS౤ߘͳͲͷҙݟจॻ͸τϐοΫ͕ଟ༷Ͱɺࢀরཁ໿ͷ༻ҙ͕ࠔ೉
    • ͜͏ͨ͠จॻʹ͸ɺڭࢣ͋Γख๏͸͏·͘ػೳ͠ͳ͍

    2022/4/14 5
    YelpϨϏϡʔ ࣗಈཁ໿ (PEGASUS; Zhang et al., 2020) ਓ͕࡞੒ͨ͠ࢀরཁ໿

    View Slide

  6. τϐοΫ໦ߏ଄Λख͕͔Γʹ͢Δ͜ͱͰࢀরཁ໿Λෆཁʹ͢Δ
    6
    The food here is fantastic, easily the best
    sub sandwiches in the Arizona area.
    The shop is local and family run, so I
    definitely choose it over a lot of the large
    national chains that are all around town.
    The staff are extremely friendly and will
    always go above and beyond in creating
    a delicious sandwich for you.
    You will not be let down by the great
    food that they make here!
    The sesame bread is amazing. The sandwich was huge but I
    ended up eating the whole thing. Very very good! Delicious.
    Cheap. Friendly. Quick (if you want it to be!). What more
    could you ask for? The atmosphere at McGurkee's is
    reminiscent of an old town go-to with the friendliest staff in the
    valley. The sandwiches are inexpensive and are, in my opinion,
    the best Italian subs in AZ. Get the Sicilian if you want to know
    what a real sub tastes like. Awesome sandwiches best bread,
    had the Pastrami Sandwich and it was a great experience with
    the homemade bread, pastrami meat and the condiments
    included, would not change a thing. Did I mention the bread? It
    is toasted and has a great taste and firm consistency. The
    restaurant is an older quaint building which is nice and fun.
    better One of the best sandwiches in town. There is nothing not
    to like about this place, great sandwiches and a friendly staff of
    people working behind the counter. Forget about the chains and
    give this place a try you willl be glad you did. It's been a while
    since I've been in this place, and I figured it was time for a
    good sammie. I've had a few of their sandwiches in the past, but
    today I got the Athenian chicken sandwich. The onions and feta
    cheese adds a nice flavor to it. I would recommend this place if
    you are in the area or for someone who wants to venture out
    from their usual sandwich joint.
    • τϐοΫ໦ߏ଄Λਪఆ
    • τϐοΫຖʹཁ໿จΛੜ੒
    YelpϨϏϡʔ ཁ໿
    ఏҊख๏
    overall
    service
    food
    atmo-
    sphere
    location
    place
    taste price

    View Slide

  7. จॻதͷจͷજࡏ෼෍Λࠞ߹Ψ΢ε෼෍Ͱදݱ
    જࡏ෼෍ʢࠞ߹Ψ΢ε෼෍ʣ
    ໦ߏ଄τϐοΫϞσϧͰ
    τϐοΫൺ཰Λਪఆ͠ɺ
    ରԠؔ܎ΛֶशʢVAEʣ
    0.1
    0.5 0.4
    Food Service
    Overall
    The food is delicious and
    the service is top notch.
    2021/7/7 7
    จॻதͷจ
    food service
    overall

    View Slide

  8. ࠞ߹Ψ΢ε෼෍Λ෼ղ͠ɺτϐοΫຖͷจͷ෼෍ʹ෼ղ
    food service
    overall
    I love this restaurant.
    The sandwich is amazing. Wonderful service and staffs.
    ෼ղ
    small
    large
    small
    જࡏ෼෍ʢࠞ߹Ψ΢ε෼෍ʣ
    જࡏ෼෍ʢ୯ๆΨ΢ε෼෍ʣ
    ਌ͷ෼ࢄ͕ࢠΑΓ΋େ͖͘ͳΔ
    Α͏ʹֶश͢Δ͜ͱͰɺཻ౓ͷ
    ҟͳΔཁ໿จΛੜ੒
    0.1
    0.5 0.4
    Food Service
    Overall
    The food is delicious and
    the service is top notch.
    2021/7/7 8
    food service
    overall
    จॻதͷจ
    ཁ໿จ
    ฏۉΛσίʔυ
    ໦ߏ଄τϐοΫϞσϧͰ
    τϐοΫൺ཰Λਪఆ͠ɺ
    ରԠؔ܎ΛֶशʢVAEʣ

    View Slide

  9. • طଘͷڭࢣͳ͠ੜ੒ܕཁ໿΍ࣄલֶशཁ໿ϞσϧʢPEGASUSʣΛ্ճΔ/ڝ߹͢ΔੑೳΛ֬ೝ
    – ਓखධՁͰ͸ಛʹཁ໿ͷcoherencyɺinformativenessʹ͍ͭͯߴධՁ
    • ಛʹPEGASUSʹରͯ͠͸ڭࢣͳ͠ઃఆͰ͸ఏҊ๏ͷੑೳ͕ஶ্͘͠ճΓɺ
    ڭࢣ͋Γઃఆʹ͓͍ͯ΋ఏҊ๏͕ڝ߹͢Δ/্ճΔ͜ͱΛ֬ೝʢग़൛ޙͷ௥Ճ࣮ݧʣ
    ଟจॻཁ໿ʹͯੑೳΛݕূ
    ڭࢣͳ͠நग़ܕ
    ڭࢣͳ͠ੜ੒ܕ
    ఏҊ๏
    2021/7/7 9
    ڭࢣ͋Γੜ੒ܕ

    View Slide

  10. જࡏ෼෍ͷ෼ࢄ͕খ͘͞ͳΔͱੜ੒͞ΕΔจ͕ৄࡉʹͳΔ
    2021/7/7 10
    1. A great thing to go to the place.
    11. Friendly staff and I appreciated the food.
    12. There is a lot of a great meal.
    112. Food was great, and the service was great.
    111. The staff is very friendly and knowledgable.
    121. The meal was great, but the food was fantastic.
    122. The was delicious and the fish tacos were great.
    τϐοΫจͷજࡏ෼෍ʢPCAʣ ੜ੒͞ΕͨτϐοΫจ
    ਂ͘ͳΔ΄Ͳ
    ෼ࢄ͸খ͍͞
    ਂ͘ͳΔ΄Ͳ
    ੜ੒จ͸ৄࡉ

    View Slide

  11. • ͜Ε·Ͱ༷ʑͳڭࢣͳ͠ཁ໿Ϟσϧ͕ఏҊ
    – จϕΫτϧͷΫϥελॏ৺ɾத৺ੑ
    (Radev et al., 2000; Erkan & Radev, EMNLP’04)
    – จॻϕΫτϧͷฏۉ
    (Chu & Liu, ICML’19; Bražinskas, ACL’20)
    – ֤τϐοΫ΁ͷ෼ղ
    (Isonuma et al., TACL’21) ͳͲͳͲ…
    • ͲΕ΋΋ͬͱ΋Β͍͕͠ɺԿ͕ຊ౰ʹඞཁͳͷ
    ͔/ຊ࣭తʹಉ֓͡೦ͳͷ͔Θ͔Βͳ͍
    → ʢڭࢣͳ͠ʣจষཁ໿ͷഎܠཧ࿦͕΄͍͠
    ͜Ε·Ͱͷڭࢣͳ͠ཁ໿ݚڀʢࣗ෼ͷݚڀؚΉʣͷ൓লʢʁʣ
    11
    ཧֶతͳ໘ ޻ֶతͳ໘
    • ڭࢣͳ͠ཁ໿ݚڀͷ๣ΒͰɺࣄલֶशϞσϧ
    ʹΑΔڭࢣ͋Γཁ໿͕ඈ༂తʹਐԽ
    – BART (Lewis et al., ACL’20) ,
    – PEGASUS (Zhang et al., 2020), etc.
    • ҟͳΔΞʔΩςΫνϟͷཁ໿Ϟσϧʹ
    ࣄલֶशࡁΈϞσϧΛ૊ΈࠐΉͷ͸݁ߏ೉͍͠
    (Liu et al., EMNLP’19)
    → ৽͍͠ڭࢣͳ͠ཁ໿ϞσϧͰ͸ͳ͘ɺ
    ৽͍͠ڭࢣͳֶ͠शํ๏͕΄͍͠

    View Slide

  12. • จষཁ໿Λ৘ใཧ࿦ͷݴ༿Ͱද͢ͱɺ
    ඇՄٯσʔλѹॖͱΈͳͤΔ
    • ͕ͨͬͯ͠ඇՄٯσʔλѹॖͷഎܠཧ࿦Ͱ͋Δ
    Ϩʔτ࿪Έཧ࿦ (Shannon, 1959; Berger, 1971) ͸ɺ
    จষཁ໿ͷഎܠཧ࿦ʹ΋ͳΔͷͰ͸ͳ͍͔
    ൓লΛ౿·͑ͨݱࡏͷݚڀ֓ཁ
    12
    ཧֶతͳ໘ ޻ֶతͳ໘
    • Ϩʔτ࿪Έཧ࿦ʹجͮ͘จষཁ໿ͷ৽͍͠໨త
    ؔ਺ʢֶशํ๏ʣΛఏҊ
    • จষཁ໿Ϟσϧ𝑝(𝑦|𝑥)͸ԿͰ΋Α͘ɺ
    ࣄલֶशϞσϧΛ࢖͏͜ͱ΋Մೳ
    • ϋΠύʔύϥϝʔλ΍࿪Έؔ਺ʢޙड़ʣͰ
    ཁ໿ͷ௕͞΍಺༰Λ੍ޚՄೳ
    Έ͔Μ͕޷͖
    ΓΜ͕͝޷͖
    ੺͍ڕ͕޷͖
    ੨͍ڕ͕޷͖
    Ռ෺͕޷͖
    ڕ͕޷͖
    จষ ཁ໿
    1bit
    2bit
    จॻ𝑥 ཁ໿𝑦
    จষཁ໿
    Ϟσϧ
    𝑝(𝑦|𝑥)
    𝑝(𝑦|𝑥) = argmin
    !(#|%)
    𝐿[𝑝 𝑦 𝑥 ]

    View Slide

  13. • ೖྗʢѹॖ͍ͨ͠৘ใʣΛ𝑋ɺग़ྗʢ𝑋Λѹॖͨ͠৘ใʣΛ𝑌ͱ͢Δ
    𝑋͸Ұ༷෼෍𝑝(𝑥)ʹج͖ͮɺ𝑌͸𝑝(𝑦|𝑥)ʹج͖ͮఆ·ΔͱԾఆ͢Δ
    Ϩʔτ࿪Έཧ࿦ͱ͸
    13
    0
    1
    2
    3
    0
    1
    2
    3
    0
    1
    2
    3
    0.5
    2.5
    𝑋 𝑌
    𝑋 𝑌
    0
    1
    2
    3
    0.5
    2.5
    𝑋 𝑌
    𝑝(𝑦|𝑥)
    (c)
    (b)
    (a)

    View Slide

  14. • ೖྗʢѹॖ͍ͨ͠৘ใʣΛ𝑋ɺग़ྗʢ𝑋Λѹॖͨ͠৘ใʣΛ𝑌ͱ͢Δ
    𝑋͸Ұ༷෼෍𝑝(𝑥)ʹج͖ͮɺ𝑌͸𝑝(𝑦|𝑥)ʹج͖ͮఆ·ΔͱԾఆ͢Δ
    • Ϩʔτ࿪Έཧ࿦ͱ͸ɺѹॖޙͷ৘ใྔʢϨʔτʣΛҰఆʹͨ͠΋ͱͰɺ
    Ͳ͜·Ͱѹॖલޙͷޡࠩʢ࿪ΈʣΛখ͘͞Ͱ͖Δ͔ʹ͍ͭͯ࿦ͨ͡ɺඇՄٯσʔλѹॖͷجૅཧ࿦
    Ϩʔτ࿪Έཧ࿦ͱ͸
    14
    0
    1
    2
    3
    0
    1
    2
    3
    0
    1
    2
    3
    0.5
    2.5
    𝑋 𝑌
    𝑋 𝑌
    2bit
    1bit
    (c)
    0
    1
    2
    3
    0.5
    2.5
    𝑋 𝑌
    Ϩʔτ:
    1bit
    (b)
    (a)
    1bit
    খ ແ
    =
    >
    Ϩʔτ
    ࿪Έ
    2bit
    1bit

    <
    >
    𝑝(𝑦|𝑥)

    View Slide

  15. • Ϩʔτ࿪Έཧ࿦Ͱ͸Ϩʔτͱ࿪ΈΛҎԼͰఆࣜԽ
    – Ϩʔτɿ૬ޓ৘ใྔ 𝐼 𝑋; 𝑌 = 𝐻 𝑋 − 𝐻 𝑋 𝑌 ʢ𝑌Λ஌Δ͜ͱͰݮΔ𝑋ͷෆ͔֬͞ʣ
    – ࿪Έɿޡࠩͷظ଴஋ Ε[𝑑 𝑥, 𝑦 ] = ∑!,#
    𝑝 𝑥 𝑝 𝑦|𝑥 𝑑 𝑥, 𝑦 ʢ𝑑 𝑥, 𝑦 ͸೚ҙͷ࿪Έؔ਺ʣ
    Ϩʔτͱ࿪ΈͷఆࣜԽ
    15
    𝐼 𝑋; 𝑌 = 1
    Ε[𝑑 𝑥, 𝑦 ] = 0.25 Ε[𝑑 𝑥, 𝑦 ] = 0
    =
    >
    Ϩʔτ
    ࿪Έ
    ʢe.g., 2৐ޡࠩʣ
    𝐼 𝑋; 𝑌 = 2
    𝐼 𝑋; 𝑌 = 1
    Ε[𝑑 𝑥, 𝑦 ] = 4.25
    <
    >
    0
    1
    2
    3
    0
    1
    2
    3
    0
    1
    2
    3
    0.5
    2.5
    𝑋 𝑌
    𝑋 𝑌
    2bit
    1bit
    0
    1
    2
    3
    0.5
    2.5
    𝑋 𝑌
    Ϩʔτ:
    1bit
    𝑝(𝑦|𝑥)
    (c)
    (b)
    (a)

    View Slide

  16. • Ϩʔτͱ࿪Έͷ૒ํΛ࠷খԽ͢Δ࠷దͳѹॖϞσϧ𝑝(𝑦|𝑥)͸ɺԼهࣜΛ࠷খԽ͢Δ͜ͱͰಘΒΕΔ
    𝐿 𝑝 𝑦 𝑥 = 𝐼 𝑋; 𝑌 + 𝛽Ε[𝑑 𝑥, 𝑦 ]
    Ϩʔτ࿪Έཧ࿦ʹجͮ͘σʔλѹॖͷఆࣜԽ
    16
    0
    1
    2
    3
    0
    1
    2
    3
    0
    1
    2
    3
    0.5
    2.5
    𝑋 𝑌
    𝑋 𝑌
    0
    1
    2
    3
    0.5
    2.5
    𝑋 𝑌
    𝑝(𝑦|𝑥)
    𝛽 = େ
    𝛽 = খ
    ࠷దԽޙ
    ࠷దԽલ
    𝐼 𝑋; 𝑌 = 1
    Ε[𝑑 𝑥, 𝑦 ] = 0.25 Ε[𝑑 𝑥, 𝑦 ] = 0
    =
    >
    Ϩʔτ
    ࿪Έ
    ʢe.g., 2৐ޡࠩʣ
    𝐼 𝑋; 𝑌 = 2
    𝐼 𝑋; 𝑌 = 1
    Ε[𝑑 𝑥, 𝑦 ] = 4.25
    <
    >

    View Slide

  17. • ಉ༷ʹɺจॻΛ𝑥ɺཁ໿Λ𝑦ͱ͠ɺ𝑝(𝑦|𝑥)Λཁ໿Ϟσϧʢe.g., Τϯίʔμσίʔμʣͱͨ͠ͱ͖ɺ
    ཁ໿λεΫ΋·ͨ𝐿 𝑝 𝑦 𝑥 Λ࠷খԽ͢Δ໰୊ͱͯ͠ଊ͑ΒΕΔͷͰ͸ͳ͍͔
    𝐿 𝑝 𝑦 𝑥 = 𝐼 𝑋; 𝑌 + 𝛽Ε[𝑑 𝑥, 𝑦 ]
    • 𝛽Ͱཁ໿ͷ௕͞΍୯ޠͷछྨ਺ΛίϯτϩʔϧͰ͖ͦ͏
    Ϩʔτ࿪Έཧ࿦ʹجͮ͘จষཁ໿ͷఆࣜԽ
    17
    Έ͔Μ͕޷͖
    ΓΜ͕͝޷͖
    ੺͍ڕ͕޷͖
    ੨͍ڕ͕޷͖
    Έ͔Μ͕޷͖
    ΓΜ͕͝޷͖
    ੺͍ڕ͕޷͖
    ੨͍ڕ͕޷͖
    𝑋 𝑌
    𝛽 = େ
    𝛽 = খ
    ࠷దԽޙ
    ࠷దԽલ
    Έ͔Μ͕޷͖
    ΓΜ͕͝޷͖
    ੺͍ڕ͕޷͖
    ੨͍ڕ͕޷͖
    Ռ෺͕޷͖
    ڕ͕޷͖
    𝑋 𝑌
    Έ͔Μ͕޷͖
    ΓΜ͕͝޷͖
    ੺͍ڕ͕޷͖
    ੨͍ڕ͕޷͖
    Ռ෺͕޷͖
    ڕ͕޷͖
    𝑋 𝑌
    ཁ໿͕୹͍or୯ޠͷछྨ͕গͳ͍ ཁ໿͕௕͍or୯ޠͷछྨ͕ଟ͍

    View Slide

  18. • ڭࢣͳ͠ཁ໿͸ɺจ/จॻϕΫτϧͷฏۉ΍த৺ੑΛ༻͍Δ΋ͷ͕ଟ͍ (Radev et al., 2000; Erkan & Radev,
    EMNLP’04; Chu & Liu, ICML’19; Bražinskas, ACL’20; Isonuma et al., TACL’21, etc.)
    • ͜ΕΒͷख๏͸͍ͣΕ΋Ϩʔτͷ্ݶΛݻఆ͠ͳ͕ΒԿΒ͔ͷ࿪ΈΛ࠷খԽ͍ͯ͠ΔͱҰൠԽͰ͖Δ
    طଘͷڭࢣͳ͠ཁ໿ख๏ͱͷ઀఺
    18
    ੜ੒ܕྫ
    (Isonuma et
    al., 2021)
    நग़ܕྫ
    (Radev et
    al., 2000)
    Ϩʔτ࿪Έཧ࿦తݟํ
    طଘख๏
    GMMͷ֤Ψ΢ε෼෍ͷฏۉ
    ˔͔Βཁ໿จΛσίʔυ
    จϕΫτϧ จ𝑥 ཁ໿จ𝑦
    จϕΫτϧ
    จ𝑥 ཁ໿จ𝑦
    จϕΫτϧͷ֤Ϋϥελฏۉ
    ʹ࠷΋͍ۙཁ໿จ˔Λநग़
    ϋʔυͳ𝑝(𝑦|𝑥)ͷ΋ͱͰɺϨʔτ
    ͷ্ݶΛΫϥελ਺ʹݻఆ͠ɺ
    ࿪Έʢೋ৐ޡࠩʣΛ࠷খԽ
    ιϑτͳ𝑝(𝑦|𝑥)ͷ΋ͱͰɺϨʔτ
    ͷ্ݶΛΨ΢ε෼෍ͷ਺ʹݻఆ͠ɺ
    ࿪Έʢೋ৐ޡࠩʣΛ࠷খԽ

    View Slide

  19. • ڭࢣ͋Γཁ໿ֶशͷաఔͰɺϨʔτ𝐼 𝑋; 𝑌 ͱ࿪ΈΕ[𝑑 𝑥, 𝑦 ]͕Ͳ͏มԽ͢Δ͔ݟͯΈΔ
    – ࣄલֶशࡁBART (Lewis et al., ACL’20) Λ༻͍ͯɺ CNN-DailyMailσʔληοτͰཁ໿ੜ੒λεΫΛֶश
    – ೖྗͷ୯ޠ෼෍𝒑!
    ͱग़ྗͷ୯ޠ෼෍𝒑#
    ؒͷKL৘ใྔΛ࿪Έؔ਺ͱͯ͠࢖༻ (Peyrard, ACL’19)
    Ε[𝑑 𝑥, 𝑦 ] = ;
    !,#
    𝑝 𝑥 𝑝 𝑦|𝑥 D$%[𝒑#||𝒑!]
    • Ծઆɿ
    – BARTͰ͸ೖྗจΛͦͷ··ग़ྗ͢ΔΑ͏ʹࣄલֶशΛߦ͏ͨΊɺཁ໿ֶशલ͸ࠨͷঢ়ଶʹ͍ۙ
    – ཁ໿ֶश͕ਐΉͱɺग़ྗ௕͕୹͘ͳΔor୯ޠͷछྨ͕গͳ͘ͳΔ͜ͱͰɺӈͷঢ়ଶʹۙͮ͘
    ॳظతͳ࣮ݧ
    19
    Έ͔Μ͕޷͖
    ΓΜ͕͝޷͖
    ੺͍ڕ͕޷͖
    ੨͍ڕ͕޷͖
    Έ͔Μ͕޷͖
    ΓΜ͕͝޷͖
    ੺͍ڕ͕޷͖
    ੨͍ڕ͕޷͖
    𝑋 𝑌
    Έ͔Μ͕޷͖
    ΓΜ͕͝޷͖
    ੺͍ڕ͕޷͖
    ੨͍ڕ͕޷͖
    𝑋 𝑌
    Ռ෺͕޷͖
    ڕ͕޷͖
    ཁ໿ֶशલ Ϩʔτɿେ ࿪Έɿখ ཁ໿ֶशޙ Ϩʔτɿখ ࿪Έɿେ

    View Slide

  20. • ֓ͶԾઆ௨Γͷ݁Ռ͕ಘΒΕͨ
    – ʢࠨਤʣֶश͕ਐΉͱϨʔτ͕খ͘͞ͳΓɺ࿪Έ͕େ͖͘ͳΔ
    – ʢӈਤʣϨʔτͷ૿ݮʹԠͯ͡ɺཁ໿௕͕૿ݮ͢Δ
    ࣮ݧ݁Ռ
    20
    Ϩʔτ ࿪Έ
    ֶशεςοϓ਺
    Ϩʔτʢࠨ࣠ʣ
    ࿪Έʢӈ࣠ʣ
    Ϩʔτ ཁ໿௕
    Ϩʔτʢࠨ࣠ʣ
    ཁ໿௕ʢӈ࣠ʣ
    ֶशεςοϓ਺

    View Slide

  21. • ֶश͕ਐΉʹͭΕɺϨʔτͱ࿪Έͷ߹ܭ 𝐿 𝑝 𝑦 𝑥 = 𝐼 𝑋; 𝑌 + 𝛽Ε[𝑑 𝑥, 𝑦 ]͕খ͘͞ͳΔ
    • ࿪Έؔ਺͸ݕ౼ͷ༨஍͋Γʢ୯ޠ෼෍Ͱ͸ͳ͘ɺ୯ޠຒΊࠐΈΛ༻͍ͨ΋ͷͳͲʣ
    ࣮ݧ݁Ռ
    21
    Ϩʔτ𝐼 𝑋; 𝑌
    ࿪ΈΕ[𝑑 𝑥, 𝑦 ]
    100εςοϓ໨
    200εςοϓ໨

    ࠨԼํ޲
    ʹਐΉ

    View Slide

  22. • ༷ʑͳڭࢣͳ͠ཁ໿ख๏͕ɺϨʔτͱ࿪Έͷ࠷খԽʹؼண͞ΕΔ͜ͱΛࣔͨ͠
    – τϐοΫจੜ੒ʹΑΔڭࢣͳ͠ཁ໿΋ͦͷҰͭ
    • ࣮ࡍʹɺڭࢣ͋Γཁ໿ֶशͷաఔͰϨʔτͱ࿪Έͷ߹ܭ͕খ͘͞ͳΔ͜ͱΛ֬ೝͨ͠
    – ࠓޙ͸ٯʹɺϨʔτͱ࿪ΈΛ࠷খԽ͢Δ͜ͱͰɺจষཁ໿λεΫΛղ͚Δ͔ݕূ
    – ߋʹɺϨʔτ࿪Έཧ࿦Λ֦ுͨ͠࿮૊ΈͰ͋Δinformation bottleneck๏ʹΑΓɺ
    aspect-based summarizationͱ͍ͬͨɺཁ໿಺༰Λ੍ޚ͢Δඞཁͷ͋Δจষཁ໿λεΫΛղ͚ͳ͍͔ݕূ
    • ߋʹɺΩϟϓγϣϯੜ੒ͳͲɺจষҎ֎ͷσʔλΛจষʹʮѹॖʯ͢ΔλεΫΛ
    ͜ͷ࿮૊ΈͰఆࣜԽͰ͖ͳ͍͔௅ઓ͍ͨ͠ʢໝ૝ʣ
    ·ͱΊͱࠓޙͷల๬
    22

    View Slide