Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
抽出的文書要約における hetero graph の応用 Heterogeneous Grap...
Search
Sponsored
·
SiteGround - Reliable hosting with speed, security, and support you can count on.
→
uchi_k
September 06, 2020
Programming
0
1.2k
抽出的文書要約における hetero graph の応用 Heterogeneous Graph Neural Networks for Extractive Document Summarization
ACL 2020 に採択された Heterogeneous Graph Neural Networks for Extractive Document Summarization を読んでいます。
uchi_k
September 06, 2020
Tweet
Share
More Decks by uchi_k
See All by uchi_k
ACL2020 Category Survey: Sentiment Analysis
uchi_k
2
3.4k
前処理が単語埋め込みに与える影響 A Comprehensive Analysis of Preprocessing for Word Representation Learning in Affective Tasks
uchi_k
2
1.1k
Graph Neural Networks のビジネス応用可能性 heterogeneous graph と論文再現性について
uchi_k
1
3.4k
ACL精神医療論文まとめ 8min LT
uchi_k
0
1.4k
【論文紹介】医用画像への転移学習の有効性について Transfusion: Understanding Transfer Learning for Medical Imaging
uchi_k
4
3.7k
Graph: A Survey of Graph Neural Networks, Embedding, Tasks and Applications
uchi_k
1
1.2k
Other Decks in Programming
See All in Programming
JPUG勉強会 OSSデータベースの内部構造を理解しよう
oga5
2
230
AHC061解説
shun_pi
0
320
Claude Code の Skill で複雑な既存仕様をすっきり整理しよう
yuichirokato
1
290
CSC307 Lecture 13
javiergs
PRO
0
310
AIに仕事を丸投げしたら、本当に楽になれるのか
dip_tech
PRO
0
180
Head of Engineeringが現場で回した生産性向上施策 2025→2026
gessy0129
0
210
今更考える「単一責任原則」 / Thinking about the Single Responsibility Principle
tooppoo
3
1.4k
Geminiの機能を調べ尽くしてみた
naruyoshimi
0
200
2026年は Rust 置き換えが流行る! / 20260220-niigata-5min-tech
girigiribauer
0
220
ご飯食べながらエージェントが開発できる。そう、Agentic Engineeringならね。
yokomachi
1
280
atmaCup #23でAIコーディングを活用した話
ml_bear
4
740
AIとペアプロして処理時間を97%削減した話 #pyconshizu
kashewnuts
1
200
Featured
See All Featured
BBQ
matthewcrist
89
10k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
47
8k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
55
3.3k
10 Git Anti Patterns You Should be Aware of
lemiorhan
PRO
659
61k
Fireside Chat
paigeccino
42
3.8k
Paper Plane (Part 1)
katiecoart
PRO
0
5.1k
Designing for Timeless Needs
cassininazir
0
150
Have SEOs Ruined the Internet? - User Awareness of SEO in 2025
akashhashmi
0
280
Exploring anti-patterns in Rails
aemeredith
2
280
svc-hook: hooking system calls on ARM64 by binary rewriting
retrage
2
140
Data-driven link building: lessons from a $708K investment (BrightonSEO talk)
szymonslowik
1
950
Stewardship and Sustainability of Urban and Community Forests
pwiseman
0
130
Transcript
Heterogeneous Graph Neural Networks for Extractive Document Summarization
ڮ ݎࢤ uchi_k @__uchi_k__ About me yuni, inc. ද nlpaper.challenge
ӡӦ Freelance Machine Learning ɹɹɹɹɹEngineer / Researcher former ژେใӃ, ະ౿16 FreakOut Machine Learning Engineer
nlpaper.challenge ࣗવݴޠॲཧͷΛ͍Ζ͍Ζ͢ΔࣾձਓɾֶੜɾݚڀऀͷίϛϡχςΟ ʢϘϥϯςΟΞத৺ͰӡӦʣ "$-ͷશཏΛࢦͯ͠ɺ"$-ެࣜʹ͋Δʹै͍ɺͷ Λઃఆͯ͠ɺͦΕͧΕͷνʔϜʹ͔ΕͯαʔϕΠ ຊఔͷจΛಡΈɺٞ-5ձͳͲΛ͍ͯ͠·ͨ͠
ACL2020 ੜܥɺάϥϑܥͷจ͕͔ͳΓ૿͑ͨҹ #&35 3P#&35BͷࣄલֶशݴޠϞσϧʹؔ͢Δݴٴ͕΄΅ඞͣ͋Δ ࠶ݱੑͷࢹ࣮ͷԠ༻͔Βɺࢦඪͷݟ͕͠ਐΜͩ ϕετϖʔύʔɺ/-1λεΫͷςετέʔεΈ͍ͨͳͷΛఆ ٛͯ͠௨աΛݟΑ͏Έ͍ͨͳΛ͍ͯͨ͠Γ ,OPXMFEHFHSBQIʹճؼͯ͠ɺάϥϑ্Ͱͷԋࢉάϥϑߏɺֶ शΛߦ͏Α͏ͳ͕૿Ճ Ҏ্ɺࢲݟͰͨ͠
)FUFSPHFOFPVT(SBQI/FVSBM/FUXPSLT GPS&YUSBDUJWF%PDVNFOU4VNNBSJ[BUJPO #abstract จॻཁͰɺηϯςϯεؒͷؔੑͷϞσϧԽ͕ ඇৗʹॏཁɻैདྷɺ3//ϕʔεͷख๏ͰܥྻͰ ϞσϧԽ͍ͯͨ͠ %BORJOH8BOH 4IBOHIBJ,FZ-BCPSBUPSZPG*OUFMMJHFOU*OGPSNBUJPO1SPDFTTJOH 'VEBO6OJWFSTJUZ FUBM
"$- நग़తจॻཁͰηϯςϯεؒͷؔੑΛදݱ͢ΔͨΊʹ IFUFSPHFOFPVTHSBQIΛಋೖ͠ɺ4P5"Λୡ֦ுੑͳͲʹ͍ͭͯݕূͨ͠ɻ จॻͷҙຯߏܥྻΑΓάϥϑߏͷํ͕దͯ͠ ͍Δ͜ͱ͕࠷ۙͷݚڀͰΘ͔͖͍ͬͯͯΔ͕ɺྑ͍ άϥϑߏ·ͩఏҊ͞Ε͍ͯͳ͔ͬͨ ୯ޠϊʔυͱจϊʔυΛ࣋ͭIFUFSPͳHSBQIߏ ΛఏҊ͠ɺ୯จॻɾଟจॻཁͦΕͧΕͰ 4P5"Λୡɻ֦ுੑʹ͍ͭͯٞͨ͠
#abstract #extractive document summarization ݩͷจॻ͔Βؔ࿈͢ΔจॻΛऔΓग़ͯ͠ɺཁ ͱͯ͠࠶ߏ͢ΔλεΫ நग़తจॻཁ ୯ޠΛܦ༝ͨ͠จͷؔੑΛදݱ͢ΔIFUFSPHSBQIΛఆٛ υΩϡϝϯτͷ֤ηϯςϯεΛ#JEJSFDUJPOBM-45.ͰϕΫτϧԽɻ͜Ε ʹΑͬͯηϯςϯεͷҙຯΛଊ͑ͨϕΫτϧ͕࡞ΒΕΔʢXPSEMBZFSʣ
நग़ܕͱɺදݱΛநԽͯ͠θϩ͔ΒཁจΛ ࡞ΔੜܕɺͦΕΒͷࠞ߹ͷύλʔϯ͕͋Δ ͞Βʹ͜ͷϕΫτϧಉ࢜ͷؔੑΛ#JEJSFDUJPOBM-45.Ͱֶश͢Δ ʢTFOUFODFMBZFSʣ ηϯςϯεΛநग़͢Δ֬Λग़ྗ 4VNNB3V//FS ॳظͷݚڀ
)FUFSPHFOFPVT(SBQI ࣮ੈքͷάϥϑIFUFSPHFOFPVTͳͷ͕ଟ͍ ࣮ੈքͷάϥϑɺҟͳΔಛۭؒͷ༷ʑͳλΠϓͷϊʔυɾΤοδͰ ߏ͞Ε͍ͯΔ #abstract #heterogeneous graph
#model overview ηϯςϯεͷΈΛϊʔυͱͯ͠άϥϑΛߏங͢ ΔͷͰͳ͘ɺηϯςϯεΛͭͳ͙հͷΑ ͏ͳϊʔυΛՃ 1SPQPTFE(SBQI ୯ޠΛܦ༝ͨ͠จͷؔੑΛදݱ͢ΔIFUFSPHSBQIΛఆٛ จใͰ୯ޠϊʔυΛߋ৽Ͱ͖Δ ଞͷϊʔυλ ΠϓΛՃ͢ΔͳͲͷ֦ுੑ͕͋ΔɺͳͲͷར
͜ͷจͰɺ࠷খҙຯ୯ҐΛ୯ޠʹ͍ͯ͠ Δɻྫ͑ɺΑΓநԽͯ͠୯ޠͷҙຯ֓೦ ΛϊʔυλΠϓͱ͢Δ͜ͱ໘നͦ͏ HSBQIJOJUJBMJ[Fˠ("5Ͱߋ৽ˠηϯςϯε ಛ͔ΒཁจʹՃ͢Δ͔൱͔ͷྨΛ ղ͘ɺͱ͍͏खॱ
#model overview #learning step HSBQIJOJUJBMJ[FSͰɺจʹΧʔωϧαΠζͷҟ ͳΔ$//Λద༻ͯ͠OHSBNಛΛநग़ʢہ ॴಛʣɺ࣍ʹ#J-45.ͰηϯςϯεϨϕϧͷ ಛΛநग़ʢେҬಛʣ 1SPQPTFE(SBQI ֶशखॱͱNPEFMPWFSWJFX
୯ޠϊʔυͱจϊʔυͷؔੑʹؔ͢Δใͱ ͯ͠ɺUGJEGΛΤοδಛͰ༻͢Δ άϥϑಛ(SBQI"UUFOUJPO/FUXPSLͰ ߋ৽
#model overview #graph attention network ࣗͱपғʹͦΕͧΕॏΈΛ͔͚ͨϕΫτϧ͔ΒBUUFOUJPOΛܭࢉ ͠ɺपลϊʔυ͔ΒͷBHHSFHBUJPOʹར༻ (SBQI"UUFOUJPO/FUXPSL άϥϑ্ͰͷBUUFOUJPOΛఆٛ "UUFOUJPO
ྡϊʔυ "UUFOUJPOΛܭࢉ͢Δؔ "UUFOUJPOΛߟྀͨ͠ BHHSFHBUJPO άϥϑूͷڑؔΛɺάϥϑߏʹґଘ͠ͳ͍BUUFOUJPOͱͯ͠ ఆֶٛ͠शϕʔεͰٻΊΔɺΈ͍ͨͳ ϊʔυಛ
#dataset #train test split %BUBTFU ୯จॻཁͰͭɺෳจॻཁͰͭͷσʔληοτͰ࣮ݧ • ୯จॻཁͰ࠷͘ར༻͞Ε͍ͯΔϕϯνϚʔΫσʔληοτ • USBJO
WBMJE UFTUσʔλͦΕͧΕ $//%BJMZ.BJM2"σʔλ • /FX:PSL5JNFT"OOPUBUFE$PSQVT 4BOEIBVT ͔Βऩू͞Εͨ୯จॻཁ σʔληοτ • USBJO WBMJE UFTUσʔλͦΕͧΕ ݅ /:5 .VMUJ/FXT • ෳจॻཁσʔληοτ • ͦΕͧΕʙͷจॻʹର͠ɺਓ͕ؒॻ͍ͨཁ͕͋Δ • USBJO WBMJE UFTUσʔλͦΕͧΕ
#experiment #setting #hyper-parameter #preprocessing 4FUUJOH)ZQFSQBSBNFUFST લॲཧ άϥϑ ࣮ݧ ετοϓϫʔυ۟ಡͷআڈ ೖྗจॻͷ࠷େΛจʹ
ઃఆ UGJEGԼҐΛআڈ ޠኮΛʹ੍ݶ ࣍ݩͷ(MP7FͰຒΊࠐΈ จϕΫτϧαΠζͰॳظԽ Τοδಛྔ ࣍ݩͰॳظԽ IFBE όοναΠζ ֶशF "EBN FQPDIͰMPTT ͕Լ͕Βͳ͍߹FBSMZTUPQQJOH ୯จॻཁͰ্Ґจ ෳจॻཁͰ্ҐจΛબ
#methods #extractor • &YU#J-45. ◦$// #J-45. ◦จॻΛจͷܥྻͱΈͳ͠จؔΛֶश͢Δ • &YU5SBOTGPSNFS ◦5SBOTGPSNFS
USBOTGPSNFS ◦શจͷϖΞϫΠζ૬ޓ࡞༻Λֶश ◦จϨϕϧͷશ࿈݁άϥϑͱΈͳͤΔ • )4( )FUFS4VN(SBQI ◦ఏҊख๏ɻจ୯ޠจͷؔੑΛάϥϑͰϞσϧԽ ◦)4(ͰϊʔυྨʹΑͬͯཁจΛબ͠ɺ͞ΒʹUSJHSBN CMPDLJOHʹΑͬͯUSJHSBN͕ࣅ͍ͯΔจΛআ֎͠ੑΛ͑ͨόʔ δϣϯ࣮ݧ .FUIPET
#result #CNN/DailyMail 3FTVMUʢ୯จॻཁɿ$//%BJMZ.BJMʣ $//%BJMZ.BJMͰͷ୯จॻཁͷ݁Ռɻطଘख๏ͯ͢Λ্ճΔείΞ͕ಘΒΕͨɻ -&"%͕ϕʔεϥΠϯɺ 03"$-&͕VQQFSCPVOE MBCFM QSFWJPVTTUVEZ QSPQPTFENFUIPE จ຺όϯσΟουͱͯ͠ఆٛ
ͨ͠)&3ʹؔͯ͠ಛʹϙϦ γʔ͋Γͳ࣮͠ݧ͠ɺ͍ͣΕ উͪ ʢ#&35Λ͍ͬͯͳ͍ʣશͯͷطଘख๏ΑΓߴ͍είΞ͕ಘΒΕͨ 306(& -ͰධՁɻͦΕ ͧΕHSBN HSBN Ұக͢Δ ࠷ܥྻͷྨࣅͷείΞ
#result #CNN/DailyMail 3FTVMUʢ୯จॻཁɿ$//%BJMZ.BJMʣ จܥྻશଓάϥϑΛར༻ͨ͠ख๏ͱൺΔ͜ͱͰɺ IFUFSPHSBQIߏͷ༗༻ੑ͕ࣔ͞Εͨɻ &YUNFUIPE QSPQPTFENFUIPE จܥྻɺશଓάϥϑΛͬ ͨ&YU#J-45. &YU
5SBOTGPSNFSΑΓߴ͍είΞ IFUFSPHSBQIΛ͏͜ͱͰɺ ηϯςϯεؒͷෆཁͳ݁߹ΛޮՌ తʹআڈͰ͖͍ͯΔ
#result #NYT50 3FTVMUʢ୯จॻཁɿ/:5ʣ /:5Ͱͷ୯จॻཁͷ࣮ݧ݁Ռɻ$//%BJMZ.BJMͱجຊతʹಉ͕͡ݟΒΕͨɻ جຊతʹ$//%BJMZ.BJM ͱಉ͡ͰɺఏҊख๏͕طଘ ख๏Λ্ճ͍ͬͯΔ QSPQPTFENFUIPE USJHSBNCMPDLJOH͋Γ όʔδϣϯ͕ҐͰͳ͍
ͷͳͥɾɾɾʁ ˠ$//%BJMZ.BJMͰॏෳͷ গͳ͍Օॻ͖Λ࿈݁͢Δܗࣜ ͕ͩɺ/:5ͰΩʔϑ Ϩʔζ͕ෳճొ͢ΔͳͲॏ ෳ͕͋ΔɻͳͷͰɺUSJHSBN CMPDLJOHͰ/:5Ͱε ίΞΛग़ͮ͠Β͍ͷͰ
#ablation #CNN/DailyMail ୯ޠϑΟϧλϦϯάͷআͰ 3 3-είΞݮগ 3 είΞ૿Ճ "CMBUJPO $//%BJMZ.BJMͰBCMBUJPO͠ϞδϡʔϧͷߩݙΛௐͨɻ ୯ޠϑΟϧλϦϯάʹΑΓɺಛʹॏཁͳ୯ޠϊʔυʹϑΥʔΧεͰ͖Δར
͕CJHSBNใΛࣦ͏σϝϦοτΛ্ճ͍ͬͯΔͷͰͳ͍͔ ("5ؒͷSFTJEVBM DPOOFDUJPOΛআ͢Δ͜ͱͰ είΞ͕େ͖͘ݮগ ("5ͷSFTJEVBMDPOOFDUJPOɺIFUFSPHSBQIʹ͓͚ΔผλΠϓͷ ϊʔυ͔ΒͷूͰཧతʹॏཁͳͷͰ୯ͳΔ݁߹Ͱஔ͖͑Ͱ͖ͳ͍
#result #multidocument )4( )%4(ڞʹطଘख๏Λ্ճ ΔείΞ͕ಘΒΕ͍ͯͯɺಛʹ )%4(ͰείΞ্ঢ͕େ͖͍ 3FTVMUʢଟจॻཁʣ ଟจॻཁͰจॻϊʔυΛՃͨ͠ఏҊख๏Ͱݕূ จॻϊʔυͷՃ͕ଟจॻཁʹ ޮՌతͰ͋Δ͜ͱ͕ࣔࠦ
USJHSBNCMPDLJOH͕ޮ͍͍ͯͳ͍ ͷɺ͓ͦΒ͖ͬ͘͞ͱಉ͡ཧ༝ ఏҊख๏Ͱ୯ʹϊʔυλΠϓΛՃ͢Δ͚ͩͰผλεΫʹԠ༻Ͱ͖͓ͯ Γɺൃలੑ͕ߴ͍ QSPQPTFENFUIPE
#qualitative analysis #degree ୯ޠϊʔυͷ͕ߴ͍ͱɺͦͷ୯ޠ ͷग़ݱ͕ଟ͍ͱ͍͏͜ͱʹͳΓจॻ ͷΛʢଟগʣද͢ 2VBMJUBUJWF"OBMZTJT ୯ޠϊʔυͷ͕༩͑ΔӨڹΛௐࠪ ୯ޠϊʔυ͕͋Δ͜ͱͰɺจใͷूͱେҬදݱͷ͕ߦΘΕ͍ͯΔՄ ೳੑ͕ࣔࠦ͞ΕΔ
୯ޠͷͱ306(&͕ൺྫ ˠੑͷߴ͍จॻ΄Ͳཁ͠қ͍ ͕ߴ͍ͱෳͷจͷใΛू͢ Δ͜ͱ͕Ͱ͖ɺϞσϧͷԸܙΛΑΓڧ ͘ड͚Δ͜ͱ͕Ͱ͖Δͱߟ͑ΒΕΔ
#qualitative analysis #source จॻ͕૿Ճ͢Δ͜ͱͰɺϕʔεϥΠϯ ্ঢ͢Δ͕ఏҊख๏ͰԼ͠ จͰฒͿ 2VBMJUBUJWF"OBMZTJT ଟจॻཁͰɺจॻͷͷӨڹΛௐࠪ จॻͷ૿ՃͰ)&5&346.(3"1)ͱ)&5&3%0$46.(3"1)ͷੑ
ೳ͕֦ࠩେจॻͱจॻͷ͕ؔෳࡶʹͳΔ΄Ͳɺจॻϊʔυͷར͕Α Γେ͖͘ͳΔ 'JSTUɺΧόϨοδΛ֬อͰ͖Δ จষΛ֤จॻ͔Βڧ੍తʹநग़Ͱ͖Δ จॻͷ૿Ճʹ͍ɺશจͷओࢫΛΧ όʔͰ͖ΔݶΒΕͨͷจΛநग़͢Δ ͜ͱ͕ࠔʹͳ͍ͬͯͨ͘Ί
#key points ·ͱΊ IFUFSPHSBQIΛ͏͜ͱͰɺจॻཁʹpOFHSBJOFEͳҙຯ୯Ґ Λಋೖ͢Δ͜ͱ͕Ͱ͖ɺจɾจষؒͷؔੑͷϞσϦϯάͷ༗ޮੑ ͕͔֬ΊΒΕͨ ख๏ͷ֦ுੑߴ͘ɺ୯จॻཁ͔ΒϊʔυλΠϓͷՃͷΈͰଟจ ॻཁʹରԠՄೳ IFUFSPHSBQIʹಛԽͨ͠ख๏ʢϝλύεΛͬͨαϒάϥϑͷఆ ٛɺIFUFSPHSBQIʹର͢ΔBUUFOUJPOʣΛࢼ͢ͱ໘ന͍͔
ࠓޙ#&35ࣄલֶशϞσϧΛ͍Ζ͍Ζݕ౼͍ͨ͠ͱͷ͜ͱ චऀܰ͘৮Ε͍͕ͯͨɺ୯ޠϊʔυʹͨΔ෦͕ҙຯϊʔυ·Ͱ நԽ͞ΕͨΓͨ͠Βख๏ͷ༏Ґੑ͕ΑΓ׆͔͞ΕΔͱࢥ͏ɻͦ͏Ͱ ͳͯ͘ɺϊʔυλΠϓͷՃ͍Ζ͍Ζࢼͤͦ͏