Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
抽出的文書要約における hetero graph の応用 Heterogeneous Grap...
Search
uchi_k
September 06, 2020
Programming
0
1.2k
抽出的文書要約における hetero graph の応用 Heterogeneous Graph Neural Networks for Extractive Document Summarization
ACL 2020 に採択された Heterogeneous Graph Neural Networks for Extractive Document Summarization を読んでいます。
uchi_k
September 06, 2020
Tweet
Share
More Decks by uchi_k
See All by uchi_k
ACL2020 Category Survey: Sentiment Analysis
uchi_k
2
3.3k
前処理が単語埋め込みに与える影響 A Comprehensive Analysis of Preprocessing for Word Representation Learning in Affective Tasks
uchi_k
2
1.1k
Graph Neural Networks のビジネス応用可能性 heterogeneous graph と論文再現性について
uchi_k
1
3.3k
ACL精神医療論文まとめ 8min LT
uchi_k
0
1.3k
【論文紹介】医用画像への転移学習の有効性について Transfusion: Understanding Transfer Learning for Medical Imaging
uchi_k
4
3.6k
Graph: A Survey of Graph Neural Networks, Embedding, Tasks and Applications
uchi_k
1
1.2k
Other Decks in Programming
See All in Programming
The state patternの実践 個人開発で培ったpractice集
miyanokomiya
0
150
A Gopher's Guide to Vibe Coding
danicat
0
170
パスタの技術
yusukebe
1
400
ライブ配信サービスの インフラのジレンマ -マルチクラウドに至ったワケ-
mirrativ
2
260
Rancher と Terraform
fufuhu
0
100
decksh - a little language for decks
ajstarks
4
21k
Laravel Boost 超入門
fire_arlo
1
120
CSC305 Summer Lecture 06
javiergs
PRO
0
100
LLMOpsのパフォーマンスを支える技術と現場で実践した改善
po3rin
8
980
コーディングは技術者(エンジニア)の嗜みでして / Learning the System Development Mindset from Rock Lady
mackey0225
2
570
Portapad紹介プレゼンテーション
gotoumakakeru
1
130
サイトを作ったらNFCタグキーホルダーを爆速で作れ!
yuukis
0
490
Featured
See All Featured
Building Better People: How to give real-time feedback that sticks.
wjessup
367
19k
Optimizing for Happiness
mojombo
379
70k
For a Future-Friendly Web
brad_frost
179
9.9k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
31
2.5k
Code Review Best Practice
trishagee
70
19k
Producing Creativity
orderedlist
PRO
347
40k
GitHub's CSS Performance
jonrohan
1031
460k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
49
3k
The World Runs on Bad Software
bkeepers
PRO
70
11k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
507
140k
Automating Front-end Workflow
addyosmani
1370
200k
How to train your dragon (web standard)
notwaldorf
96
6.2k
Transcript
Heterogeneous Graph Neural Networks for Extractive Document Summarization
ڮ ݎࢤ uchi_k @__uchi_k__ About me yuni, inc. ද nlpaper.challenge
ӡӦ Freelance Machine Learning ɹɹɹɹɹEngineer / Researcher former ژେใӃ, ະ౿16 FreakOut Machine Learning Engineer
nlpaper.challenge ࣗવݴޠॲཧͷΛ͍Ζ͍Ζ͢ΔࣾձਓɾֶੜɾݚڀऀͷίϛϡχςΟ ʢϘϥϯςΟΞத৺ͰӡӦʣ "$-ͷશཏΛࢦͯ͠ɺ"$-ެࣜʹ͋Δʹै͍ɺͷ Λઃఆͯ͠ɺͦΕͧΕͷνʔϜʹ͔ΕͯαʔϕΠ ຊఔͷจΛಡΈɺٞ-5ձͳͲΛ͍ͯ͠·ͨ͠
ACL2020 ੜܥɺάϥϑܥͷจ͕͔ͳΓ૿͑ͨҹ #&35 3P#&35BͷࣄલֶशݴޠϞσϧʹؔ͢Δݴٴ͕΄΅ඞͣ͋Δ ࠶ݱੑͷࢹ࣮ͷԠ༻͔Βɺࢦඪͷݟ͕͠ਐΜͩ ϕετϖʔύʔɺ/-1λεΫͷςετέʔεΈ͍ͨͳͷΛఆ ٛͯ͠௨աΛݟΑ͏Έ͍ͨͳΛ͍ͯͨ͠Γ ,OPXMFEHFHSBQIʹճؼͯ͠ɺάϥϑ্Ͱͷԋࢉάϥϑߏɺֶ शΛߦ͏Α͏ͳ͕૿Ճ Ҏ্ɺࢲݟͰͨ͠
)FUFSPHFOFPVT(SBQI/FVSBM/FUXPSLT GPS&YUSBDUJWF%PDVNFOU4VNNBSJ[BUJPO #abstract จॻཁͰɺηϯςϯεؒͷؔੑͷϞσϧԽ͕ ඇৗʹॏཁɻैདྷɺ3//ϕʔεͷख๏ͰܥྻͰ ϞσϧԽ͍ͯͨ͠ %BORJOH8BOH 4IBOHIBJ,FZ-BCPSBUPSZPG*OUFMMJHFOU*OGPSNBUJPO1SPDFTTJOH 'VEBO6OJWFSTJUZ FUBM
"$- நग़తจॻཁͰηϯςϯεؒͷؔੑΛදݱ͢ΔͨΊʹ IFUFSPHFOFPVTHSBQIΛಋೖ͠ɺ4P5"Λୡ֦ுੑͳͲʹ͍ͭͯݕূͨ͠ɻ จॻͷҙຯߏܥྻΑΓάϥϑߏͷํ͕దͯ͠ ͍Δ͜ͱ͕࠷ۙͷݚڀͰΘ͔͖͍ͬͯͯΔ͕ɺྑ͍ άϥϑߏ·ͩఏҊ͞Ε͍ͯͳ͔ͬͨ ୯ޠϊʔυͱจϊʔυΛ࣋ͭIFUFSPͳHSBQIߏ ΛఏҊ͠ɺ୯จॻɾଟจॻཁͦΕͧΕͰ 4P5"Λୡɻ֦ுੑʹ͍ͭͯٞͨ͠
#abstract #extractive document summarization ݩͷจॻ͔Βؔ࿈͢ΔจॻΛऔΓग़ͯ͠ɺཁ ͱͯ͠࠶ߏ͢ΔλεΫ நग़తจॻཁ ୯ޠΛܦ༝ͨ͠จͷؔੑΛදݱ͢ΔIFUFSPHSBQIΛఆٛ υΩϡϝϯτͷ֤ηϯςϯεΛ#JEJSFDUJPOBM-45.ͰϕΫτϧԽɻ͜Ε ʹΑͬͯηϯςϯεͷҙຯΛଊ͑ͨϕΫτϧ͕࡞ΒΕΔʢXPSEMBZFSʣ
நग़ܕͱɺදݱΛநԽͯ͠θϩ͔ΒཁจΛ ࡞ΔੜܕɺͦΕΒͷࠞ߹ͷύλʔϯ͕͋Δ ͞Βʹ͜ͷϕΫτϧಉ࢜ͷؔੑΛ#JEJSFDUJPOBM-45.Ͱֶश͢Δ ʢTFOUFODFMBZFSʣ ηϯςϯεΛநग़͢Δ֬Λग़ྗ 4VNNB3V//FS ॳظͷݚڀ
)FUFSPHFOFPVT(SBQI ࣮ੈքͷάϥϑIFUFSPHFOFPVTͳͷ͕ଟ͍ ࣮ੈքͷάϥϑɺҟͳΔಛۭؒͷ༷ʑͳλΠϓͷϊʔυɾΤοδͰ ߏ͞Ε͍ͯΔ #abstract #heterogeneous graph
#model overview ηϯςϯεͷΈΛϊʔυͱͯ͠άϥϑΛߏங͢ ΔͷͰͳ͘ɺηϯςϯεΛͭͳ͙հͷΑ ͏ͳϊʔυΛՃ 1SPQPTFE(SBQI ୯ޠΛܦ༝ͨ͠จͷؔੑΛදݱ͢ΔIFUFSPHSBQIΛఆٛ จใͰ୯ޠϊʔυΛߋ৽Ͱ͖Δ ଞͷϊʔυλ ΠϓΛՃ͢ΔͳͲͷ֦ுੑ͕͋ΔɺͳͲͷར
͜ͷจͰɺ࠷খҙຯ୯ҐΛ୯ޠʹ͍ͯ͠ Δɻྫ͑ɺΑΓநԽͯ͠୯ޠͷҙຯ֓೦ ΛϊʔυλΠϓͱ͢Δ͜ͱ໘നͦ͏ HSBQIJOJUJBMJ[Fˠ("5Ͱߋ৽ˠηϯςϯε ಛ͔ΒཁจʹՃ͢Δ͔൱͔ͷྨΛ ղ͘ɺͱ͍͏खॱ
#model overview #learning step HSBQIJOJUJBMJ[FSͰɺจʹΧʔωϧαΠζͷҟ ͳΔ$//Λద༻ͯ͠OHSBNಛΛநग़ʢہ ॴಛʣɺ࣍ʹ#J-45.ͰηϯςϯεϨϕϧͷ ಛΛநग़ʢେҬಛʣ 1SPQPTFE(SBQI ֶशखॱͱNPEFMPWFSWJFX
୯ޠϊʔυͱจϊʔυͷؔੑʹؔ͢Δใͱ ͯ͠ɺUGJEGΛΤοδಛͰ༻͢Δ άϥϑಛ(SBQI"UUFOUJPO/FUXPSLͰ ߋ৽
#model overview #graph attention network ࣗͱपғʹͦΕͧΕॏΈΛ͔͚ͨϕΫτϧ͔ΒBUUFOUJPOΛܭࢉ ͠ɺपลϊʔυ͔ΒͷBHHSFHBUJPOʹར༻ (SBQI"UUFOUJPO/FUXPSL άϥϑ্ͰͷBUUFOUJPOΛఆٛ "UUFOUJPO
ྡϊʔυ "UUFOUJPOΛܭࢉ͢Δؔ "UUFOUJPOΛߟྀͨ͠ BHHSFHBUJPO άϥϑूͷڑؔΛɺάϥϑߏʹґଘ͠ͳ͍BUUFOUJPOͱͯ͠ ఆֶٛ͠शϕʔεͰٻΊΔɺΈ͍ͨͳ ϊʔυಛ
#dataset #train test split %BUBTFU ୯จॻཁͰͭɺෳจॻཁͰͭͷσʔληοτͰ࣮ݧ • ୯จॻཁͰ࠷͘ར༻͞Ε͍ͯΔϕϯνϚʔΫσʔληοτ • USBJO
WBMJE UFTUσʔλͦΕͧΕ $//%BJMZ.BJM2"σʔλ • /FX:PSL5JNFT"OOPUBUFE$PSQVT 4BOEIBVT ͔Βऩू͞Εͨ୯จॻཁ σʔληοτ • USBJO WBMJE UFTUσʔλͦΕͧΕ ݅ /:5 .VMUJ/FXT • ෳจॻཁσʔληοτ • ͦΕͧΕʙͷจॻʹର͠ɺਓ͕ؒॻ͍ͨཁ͕͋Δ • USBJO WBMJE UFTUσʔλͦΕͧΕ
#experiment #setting #hyper-parameter #preprocessing 4FUUJOH)ZQFSQBSBNFUFST લॲཧ άϥϑ ࣮ݧ ετοϓϫʔυ۟ಡͷআڈ ೖྗจॻͷ࠷େΛจʹ
ઃఆ UGJEGԼҐΛআڈ ޠኮΛʹ੍ݶ ࣍ݩͷ(MP7FͰຒΊࠐΈ จϕΫτϧαΠζͰॳظԽ Τοδಛྔ ࣍ݩͰॳظԽ IFBE όοναΠζ ֶशF "EBN FQPDIͰMPTT ͕Լ͕Βͳ͍߹FBSMZTUPQQJOH ୯จॻཁͰ্Ґจ ෳจॻཁͰ্ҐจΛબ
#methods #extractor • &YU#J-45. ◦$// #J-45. ◦จॻΛจͷܥྻͱΈͳ͠จؔΛֶश͢Δ • &YU5SBOTGPSNFS ◦5SBOTGPSNFS
USBOTGPSNFS ◦શจͷϖΞϫΠζ૬ޓ࡞༻Λֶश ◦จϨϕϧͷશ࿈݁άϥϑͱΈͳͤΔ • )4( )FUFS4VN(SBQI ◦ఏҊख๏ɻจ୯ޠจͷؔੑΛάϥϑͰϞσϧԽ ◦)4(ͰϊʔυྨʹΑͬͯཁจΛબ͠ɺ͞ΒʹUSJHSBN CMPDLJOHʹΑͬͯUSJHSBN͕ࣅ͍ͯΔจΛআ֎͠ੑΛ͑ͨόʔ δϣϯ࣮ݧ .FUIPET
#result #CNN/DailyMail 3FTVMUʢ୯จॻཁɿ$//%BJMZ.BJMʣ $//%BJMZ.BJMͰͷ୯จॻཁͷ݁Ռɻطଘख๏ͯ͢Λ্ճΔείΞ͕ಘΒΕͨɻ -&"%͕ϕʔεϥΠϯɺ 03"$-&͕VQQFSCPVOE MBCFM QSFWJPVTTUVEZ QSPQPTFENFUIPE จ຺όϯσΟουͱͯ͠ఆٛ
ͨ͠)&3ʹؔͯ͠ಛʹϙϦ γʔ͋Γͳ࣮͠ݧ͠ɺ͍ͣΕ উͪ ʢ#&35Λ͍ͬͯͳ͍ʣશͯͷطଘख๏ΑΓߴ͍είΞ͕ಘΒΕͨ 306(& -ͰධՁɻͦΕ ͧΕHSBN HSBN Ұக͢Δ ࠷ܥྻͷྨࣅͷείΞ
#result #CNN/DailyMail 3FTVMUʢ୯จॻཁɿ$//%BJMZ.BJMʣ จܥྻશଓάϥϑΛར༻ͨ͠ख๏ͱൺΔ͜ͱͰɺ IFUFSPHSBQIߏͷ༗༻ੑ͕ࣔ͞Εͨɻ &YUNFUIPE QSPQPTFENFUIPE จܥྻɺશଓάϥϑΛͬ ͨ&YU#J-45. &YU
5SBOTGPSNFSΑΓߴ͍είΞ IFUFSPHSBQIΛ͏͜ͱͰɺ ηϯςϯεؒͷෆཁͳ݁߹ΛޮՌ తʹআڈͰ͖͍ͯΔ
#result #NYT50 3FTVMUʢ୯จॻཁɿ/:5ʣ /:5Ͱͷ୯จॻཁͷ࣮ݧ݁Ռɻ$//%BJMZ.BJMͱجຊతʹಉ͕͡ݟΒΕͨɻ جຊతʹ$//%BJMZ.BJM ͱಉ͡ͰɺఏҊख๏͕طଘ ख๏Λ্ճ͍ͬͯΔ QSPQPTFENFUIPE USJHSBNCMPDLJOH͋Γ όʔδϣϯ͕ҐͰͳ͍
ͷͳͥɾɾɾʁ ˠ$//%BJMZ.BJMͰॏෳͷ গͳ͍Օॻ͖Λ࿈݁͢Δܗࣜ ͕ͩɺ/:5ͰΩʔϑ Ϩʔζ͕ෳճొ͢ΔͳͲॏ ෳ͕͋ΔɻͳͷͰɺUSJHSBN CMPDLJOHͰ/:5Ͱε ίΞΛग़ͮ͠Β͍ͷͰ
#ablation #CNN/DailyMail ୯ޠϑΟϧλϦϯάͷআͰ 3 3-είΞݮগ 3 είΞ૿Ճ "CMBUJPO $//%BJMZ.BJMͰBCMBUJPO͠ϞδϡʔϧͷߩݙΛௐͨɻ ୯ޠϑΟϧλϦϯάʹΑΓɺಛʹॏཁͳ୯ޠϊʔυʹϑΥʔΧεͰ͖Δར
͕CJHSBNใΛࣦ͏σϝϦοτΛ্ճ͍ͬͯΔͷͰͳ͍͔ ("5ؒͷSFTJEVBM DPOOFDUJPOΛআ͢Δ͜ͱͰ είΞ͕େ͖͘ݮগ ("5ͷSFTJEVBMDPOOFDUJPOɺIFUFSPHSBQIʹ͓͚ΔผλΠϓͷ ϊʔυ͔ΒͷूͰཧతʹॏཁͳͷͰ୯ͳΔ݁߹Ͱஔ͖͑Ͱ͖ͳ͍
#result #multidocument )4( )%4(ڞʹطଘख๏Λ্ճ ΔείΞ͕ಘΒΕ͍ͯͯɺಛʹ )%4(ͰείΞ্ঢ͕େ͖͍ 3FTVMUʢଟจॻཁʣ ଟจॻཁͰจॻϊʔυΛՃͨ͠ఏҊख๏Ͱݕূ จॻϊʔυͷՃ͕ଟจॻཁʹ ޮՌతͰ͋Δ͜ͱ͕ࣔࠦ
USJHSBNCMPDLJOH͕ޮ͍͍ͯͳ͍ ͷɺ͓ͦΒ͖ͬ͘͞ͱಉ͡ཧ༝ ఏҊख๏Ͱ୯ʹϊʔυλΠϓΛՃ͢Δ͚ͩͰผλεΫʹԠ༻Ͱ͖͓ͯ Γɺൃలੑ͕ߴ͍ QSPQPTFENFUIPE
#qualitative analysis #degree ୯ޠϊʔυͷ͕ߴ͍ͱɺͦͷ୯ޠ ͷग़ݱ͕ଟ͍ͱ͍͏͜ͱʹͳΓจॻ ͷΛʢଟগʣද͢ 2VBMJUBUJWF"OBMZTJT ୯ޠϊʔυͷ͕༩͑ΔӨڹΛௐࠪ ୯ޠϊʔυ͕͋Δ͜ͱͰɺจใͷूͱେҬදݱͷ͕ߦΘΕ͍ͯΔՄ ೳੑ͕ࣔࠦ͞ΕΔ
୯ޠͷͱ306(&͕ൺྫ ˠੑͷߴ͍จॻ΄Ͳཁ͠қ͍ ͕ߴ͍ͱෳͷจͷใΛू͢ Δ͜ͱ͕Ͱ͖ɺϞσϧͷԸܙΛΑΓڧ ͘ड͚Δ͜ͱ͕Ͱ͖Δͱߟ͑ΒΕΔ
#qualitative analysis #source จॻ͕૿Ճ͢Δ͜ͱͰɺϕʔεϥΠϯ ্ঢ͢Δ͕ఏҊख๏ͰԼ͠ จͰฒͿ 2VBMJUBUJWF"OBMZTJT ଟจॻཁͰɺจॻͷͷӨڹΛௐࠪ จॻͷ૿ՃͰ)&5&346.(3"1)ͱ)&5&3%0$46.(3"1)ͷੑ
ೳ͕֦ࠩେจॻͱจॻͷ͕ؔෳࡶʹͳΔ΄Ͳɺจॻϊʔυͷར͕Α Γେ͖͘ͳΔ 'JSTUɺΧόϨοδΛ֬อͰ͖Δ จষΛ֤จॻ͔Βڧ੍తʹநग़Ͱ͖Δ จॻͷ૿Ճʹ͍ɺશจͷओࢫΛΧ όʔͰ͖ΔݶΒΕͨͷจΛநग़͢Δ ͜ͱ͕ࠔʹͳ͍ͬͯͨ͘Ί
#key points ·ͱΊ IFUFSPHSBQIΛ͏͜ͱͰɺจॻཁʹpOFHSBJOFEͳҙຯ୯Ґ Λಋೖ͢Δ͜ͱ͕Ͱ͖ɺจɾจষؒͷؔੑͷϞσϦϯάͷ༗ޮੑ ͕͔֬ΊΒΕͨ ख๏ͷ֦ுੑߴ͘ɺ୯จॻཁ͔ΒϊʔυλΠϓͷՃͷΈͰଟจ ॻཁʹରԠՄೳ IFUFSPHSBQIʹಛԽͨ͠ख๏ʢϝλύεΛͬͨαϒάϥϑͷఆ ٛɺIFUFSPHSBQIʹର͢ΔBUUFOUJPOʣΛࢼ͢ͱ໘ന͍͔
ࠓޙ#&35ࣄલֶशϞσϧΛ͍Ζ͍Ζݕ౼͍ͨ͠ͱͷ͜ͱ චऀܰ͘৮Ε͍͕ͯͨɺ୯ޠϊʔυʹͨΔ෦͕ҙຯϊʔυ·Ͱ நԽ͞ΕͨΓͨ͠Βख๏ͷ༏Ґੑ͕ΑΓ׆͔͞ΕΔͱࢥ͏ɻͦ͏Ͱ ͳͯ͘ɺϊʔυλΠϓͷՃ͍Ζ͍Ζࢼͤͦ͏