Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Dynamic Multi-Level Multi-Task Learning for Sentence Simplification
Search
onizuka laboratory
October 17, 2018
Research
0
50
Dynamic Multi-Level Multi-Task Learning for Sentence Simplification
弊研究室で行なったCOLING2018読み会の発表資料です。
onizuka laboratory
October 17, 2018
Tweet
Share
More Decks by onizuka laboratory
See All by onizuka laboratory
Phrase-Based & Neural Unsupervised Machine Translation
onilab
0
110
Tell-and-Answer: Towards Explainable Visual Question Answering using Attributes and Captions
onilab
0
64
Card-660: A Reliable Evaluation Framework for Rare Word Representation Models
onilab
0
31
A Word-Complexity Lexicon and A Neural Readability Ranking Model for Lexical Simplification
onilab
0
98
Integrating Transformer and Paraphrase Rules for Sentence Simplification
onilab
0
56
An Auto-Encoder Matching Model for Learning Utterance-Level Semantic Dependency in Dialogue Generation
onilab
0
48
Generating More Interesting Responses in Neural Conversation Models with Distributional Constraints
onilab
0
98
Modeling Multi-turn Conversation with Deep Utterance Aggregation
onilab
0
91
Learning Semantic Sentence Embeddings using Pair-wise Discriminator
onilab
0
110
Other Decks in Research
See All in Research
3D Human Mesh Estimationについていくつかまとめてみた / Survey about 3D Human Mesh Estimation
nttcom
0
200
MLtraq: Track your AI experiments at hyperspeed
micheda
1
110
方策の長期性能に対する効率的なオフライン評価・学習 (Long-term Off-Policy Evaluation and Learning)
usaito
PRO
2
180
Evolutionary Optimization ofModel Merging Recipes (2024/04/17, NLPコロキウム)
iwiwi
9
2.9k
Introduction of NII S. Koyama's Lab (AY2024)
skoyamalab
0
110
FMP L3 Year 1 Project Proposal
haiinya
0
150
第14回対話システムシンポジウム EMNLP 2023 参加報告
atsumoto
0
150
Source Code Diff Revolution (JetBrains Open Reading Club)
tsantalis
0
260
クロスモーダル表現学習の研究動向: 音声関連を中心として
ryomasumura
3
590
Webスケールデータセットに対する実用的なポイズニング手法 / Poisoning Web-Scale Training Datasets is Practical
nttcom
0
120
ゼロからわかるリザバーコンピューティング
kurotaky
1
290
Combating Misinformation in the age of LLMs
teacherpeterpan
0
130
Featured
See All Featured
How to Ace a Technical Interview
jacobian
272
22k
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
panda_program
60
14k
Designing for Performance
lara
601
67k
Scaling GitHub
holman
457
140k
Mobile First: as difficult as doing things right
swwweet
216
8.6k
KATA
mclloyd
15
12k
Building Better People: How to give real-time feedback that sticks.
wjessup
355
18k
The Invisible Customer
myddelton
114
12k
Atom: Resistance is Futile
akmur
259
25k
Statistics for Hackers
jakevdp
789
220k
Design by the Numbers
sachag
274
18k
The MySQL Ecosystem @ GitHub 2015
samlambert
243
12k
Transcript
$ )BO(VP 3BNBLBOUI 1BTVOVSV .PIJU #BOTBM %ZOBNJD.VMUJ-FWFM.VMUJ5BTL -FBSOJOHGPS4FOUFODF 4JNQMJGJDBUJPO
#ݪ େو $0-*/(ಡΈձʢʣ
࣍ l ֓ཁ l Ϟσϧ l ධՁํ๏ l ݁Ռ l
"CMBUJPOT"OBMZTJT
֓ཁ จͷฏқԽ l ׂɺআɺݴ͍͑ͳͲͰɺՄಡੑΛ্͛Δ l ༗ޮͳೖྗจཧతʹؚΊΔ͖ʢؚҙʣ #BTFMJOF l ϙΠϯλίϐʔϝΧχζϜͷTFRTFR
֓ཁ ఏҊख๏ l ϚϧνλεΫɾϚϧνϨϕϧ֊ ◦ ิॿλεΫʢؚҙɾݴ͍͑ʣ͕ɺҟͳΔϨϕϧͷ֊ l λεΫͷΓସ͑ํΛಈతʹֶश ◦ ଟόϯσΟοτϕʔεͷ܇࿅ख๏
ධՁɾੳ l ࣗಈධՁʢ4"3*ɺ',(-ʣͱखಈධՁͰ༏Εͨ
࣍ l ֓ཁ l Ϟσϧ l ධՁํ๏ l ݁Ռ l
"CMBUJPOT"OBMZTJT Ø Baseline Ø Ø Ø Ø Ø
Ϟσϧɿ#BTFMJOF ϙΠϯλίϐʔจฏқԽϞσϧ l ҎԼͷΛඋ͑ͨTFRTFRϞσϧ ◦ Ξςϯγϣϯʢ#BIEBOBV FUBM ʣ ◦ ϙΠϯλίϐʔϝΧχζϜʢ4FFFUBM
ʣ
Ϟσϧɿ#BTFMJOF Ξςϯγϣϯ 4FFFUBM l(FU5P5IF1PJOU4VNNBSJ[BUJPOXJUI1PJOUFS(FOFSBUPS/FUXPSLTz BS9JWF
Ϟσϧɿ#BTFMJOF ϙΠϯλίϐʔϝΧχζϜ 4FFFUBM l(FU5P5IF1PJOU4VNNBSJ[BUJPOXJUI1PJOUFS(FOFSBUPS/FUXPSLTz BS9JWF
ϞσϧɿิॿλεΫ̍ ؚҙੜλεΫ l ฏқจೖྗʹै͏͖ l ҙຯతʢߴʣ l ؚҙੜϞσϧ܇࿅༻σʔληοτ ◦ 4/-*
#PXNBOFUBM ◦ .VMUJ/-* 8JMMJBNTFUBM
ϞσϧɿิॿλεΫ̎ ݴ͍͑ੜλεΫ l ߏจޠኮͰɺฒସ͑ॻ͖͑ l ޠኮ౷ޠʢʣ l ݴ͍͑ੜλεΫ༻܇࿅σʔληοτ ◦ 1BSB/.5
8JFUJOH BOE(JNQFM B
ϞσϧɿϚϧνλεΫϞσϧ
ϞσϧɿϚϧνλεΫֶश TFRTFRͰ l Τϯίʔμσίʔμͷɺύϥϝʔλڞ༗ ఏҊख๏ l ڞ༗ͰɺύϥϝʔλͷҰ෦Λඇެ։ɺ Γʢؔ࿈දݱʣΛڞ༗ l ڞ༗ύϥϝʔλΛଛࣦؔͷϖφϧςΟ߲Ͱɺ
͋Δڑࢦඪʹ͚ۙͮΔ
ϞσϧɿϚϧνϨϕϧڞ༗ TFRTFRͷ֤֊ͰҟͳΔػೳΛͭ l #FMJOLPW FUBM l ʢೖྗଆʣʹ୯ޠߏΛֶशʢݴ͍͑ʣ l ߴʹҙຯʹযΛ߹ΘͤΔʢؚҙʣ
Ϟσϧɿιϑτڞ༗ ϋʔυڞ༗ l ڞ༗͢ΔύϥϝʔλΛ݁ͼ͚ͭΔ ιϑτڞ༗ l ҟͳΔλεΫ͕ɺύϥϝʔλۭؒͷͲͷ෦Λڞ༗͢Δ ͔બͰ͖Δ l ଛࣦؔ
! " = −log () * +; " + . "/ − 0/ ◦ " ओλεΫʢฏқԽʣͷશͯͷύϥϝʔλ ◦ "/ 0/ ओλεΫͱิॿλεΫͷڞ༗ύϥϝʔλͷαϒηοτ
ϞσϧɿϚϧνλεΫ܇࿅ طଘख๏ l ʢ੩తͳʣࠞ߹ൺ !"" : !$% : !&& ◦
ओλεΫʢฏқԽʣɿؚҙɿݴ͍͑ ఏҊख๏ l ੩తͰͳ͘ಈతʹ܇࿅͍ͨ͠
Ϟσϧɿಈతࠞ߹ൺֶश ଟόϯσΟοτ l XJUI#PMU[NBOOFYQMPSBUJPO ,BFMCMJOH FUBM l XJUIࢦҠಈฏۉߋ৽ϧʔϧ
Ϟσϧɿಈతࠞ߹ൺֶश ଟόϯσΟοτ
Ϟσϧɿಈతࠞ߹ൺֶश ଟόϯσΟοτ l ̏λεΫͷઃఆͷू߹ !" , ⋯ , !% l
֤ &' ͰɺΛબ͠ɺใु ()* ΛಘΔ ◦ ใुɺओλεΫͷෛͷ WBMJEBUJPOMPTT l όϯσΟοτίϯτϩʔϥͷํ l ֤ &' Ͱͷ֤ + ͷߦಈਪఆ
࣍ l ֓ཁ l Ϟσϧ l ධՁํ๏ l ݁Ռ l
"CMBUJPOT"OBMZTJT
ධՁํ๏ɿσʔληοτ 5SBJO 7BMJE 5FTU /FXTFMB 4NBMM8JLJ
-BSHF8JLJ 4/-* .VMUJ/-* ؚҙੜ༻ ʢ߹Θͤͯʣ 8JFUJOH(JNQFM ݴ͍͑༻ . ʢ߹Θͤͯʣ,
ධՁํ๏ɿධՁࢦඪ ࣗಈධՁ l 4"3* l ',(- ◦ ͍จΛධՁ͕ͪ͠ʢ4IBSEMPX ʣ l
#-&6 ◦ ฏқԽͰؔ࿈ੑ͕͍ʢ;IVFUBM ଞʣ ◦ มߋ͠ͳ͍อकతͳγεςϜ͕ධՁ͞Ε͕ͪʢಉ্ʣ
ධՁํ๏ɿධՁࢦඪ खಈධՁ l ྲྀெੑʢ'MVFODZʣ ◦ JTUIFPVUQVUHSBNNBUJDBMBOEXFMMGPSNFE l ଥੑʢ"EFRVBDZʣ ◦ UPXIBUFYUFOUJTUIFNFBOJOHFYQSFTTFEJOUIFPSJHJOBM
TFOUFODFQSFTFSWFEJOUIFPVUQVU l ฏқੑʢ4JNQMJDJUZʣ ◦ JTUIFPVUQVUTJNQMFSUIBOUIFPSJHJOBMTFOUFODF
࣍ l ֓ཁ l Ϟσϧ l ධՁํ๏ l ݁Ռ l
"CMBUJPOT"OBMZTJT
݁Ռ l #BTFMJOFWTطଘख๏ ◦ /FXTFMB Ͱ ',(-༏উɺ4"3*࣍ ◦ 8JLJͰಉ
݁Ռ l #BTFMJOF &OU1BSWT#BTFMJOFطଘख๏ ◦ /FXTFMB ͷ ',(-ͱ 4"3*Ͱ༏উ l
#BTFMJOF &OU 1BSWT#BTFMJOF &OU1BS ◦ /FXTFMB 8JLJͰ ',(- 4"3*͍͍ͩͨ༏উ
݁Ռ l ʢิॿλεΫͷʣࠞ߹ൺʢಈతWT੩తʣ ◦ /FXTFMB ͱ 8JLJ4NBMM Ͱ 4"3* ༏উ
݁Ռ ਓखධՁ l XBZWTଞ ◦ ฏқੑʢ4JNQMJDJUZʣͰ༏উ ◦ ଥੑʢ"EFRVBDZʣ %3&44-4 ʹෛ͚ͨˠฏқԽॏࢹ
◦ ฏۉείΞɺ%3&44-4ΑΓྑ͍
݁Ռ ೖग़ྗͷҰக l XBZ͍ʢ (SPVOEUSVUIʹ͍ۙʣ l %3&44-4 ߴ͍ ◦ ଥੑʢ"EFRVBDZʣ͕ߴ͔ͬͨͷᰐ͚Δ
࣍ l ֓ཁ l Ϟσϧ l ධՁํ๏ l ݁Ռ l
"CMBUJPOT"OBMZTJT
"CMBUJPOT"OBMZTJT ଞͷڞ༗ख๏ʁ l ʴߴ͕༏উ
"CMBUJPOT"OBMZTJT ڞ༗ʢιϑτ WTϋʔυʣʁ l ιϑτ͕ 4"3*ڧ͍
"CMBUJPOT"OBMZTJT 4"3*ͷৄࡉ l XBZ༏উ
"CMBUJPOT"OBMZTJT ̎ͭͷଟόϯσΟοτख๏ l ͭɿ࠷ޙΛهͯࠞ͠߹ൺΛݻఆ͢Δ l 4"3*ʢ̍ͭʣWT ʢ̎ͭʣ
"CMBUJPOT"OBMZTJT ̎ͭͷଟόϯσΟοτख๏
"CMBUJPOT"OBMZTJT ϚϧνλεΫֶश WTσʔλ֦ு l ຊʹɺิॿλεΫͷzػೳz͕ੑೳ্ͤͨ͞ʁ ◦ ิॿλεΫ༻ͷlσʔλՃzͷӨڹͰͳ͍ʁ l ิॿλεΫ༻σʔλʢ4/-* .VMUJ/-*
ͱ 1BSB/.5ʣΛɺݸผʹ FNCFEEJOHͯ͠ ओλεΫϞσϧʹΈࠐΜͩ l ఏҊख๏͕େ෯ʹ༏Ε͍ͯͨ
"CMBUJPOT"OBMZTJT ྫ