snlp8-2016-09-12

第8回最先端NLP勉強会 発表資料
See also http://penzant.net/files/snlp8-2016-09-12.pdf

Transcript

1. Ronbun reading: A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task Danqi Chen and Jason Bolton and Christopher D. Manning Saku Sugawara (U Tokyo, Aizawa-lab) ୈ 8 ճ࠷ઌ୺ NLP ษڧձ September 12, 2016
2. εϥΠυ͸ SpeakerDeck ͕ݟ͑ͮΒ͍৔߹͸ http://penzant.net/files/snlp8-2016-09-12.pdf Λ͝Β Μ͍ͩ͘͞ 2 / 26
3. Ronbun A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task Danqi Chen and Jason Bolton and Christopher D. Manning https://arxiv.org/pdf/1606.02858v2.pdf ACL2016 ఏग़൛͔Βਫ਼౓͕޲্͍ͯ͠ΔʢACL ൃද͸ͦͷ ಺༰ʣ http://cs.stanford.edu/people/danqi/ bib/paper/slide https://github.com/danqi/rc-cnn-dailymail only README??? Figs are quoted from the original paper or the slides 3 / 26
4. Abstract CNN/Daily Mail ಡղλεΫ (Hermann+ 2015) ͷͨΊͷϞσϧ ͱͦͷ෼ੳ ΄΅ݶքͷείΞ͕ग़ͨʢͱओு͍ͯ͠Δʣ 4 / 26
5. Background: Reading Comprehension ՝୊จͱͦΕʹؔΘΔ໰͍ΛಡΈɺԿΒ͔ͷ৘ใΛฦ͢ ݀ຒΊɺ ՝୊จ → બ୒ࢶ or ൈ͖ग़͠ ͳͲͷܗࣜ 5 / 26
6. Background: Reading Comprehension Figure: طଘλεΫྫ (Rajpurkar+ 2016) 6 / 26
7. Background: Big Data vs. Realistic ਓखͰσʔλΛ࡞Ζ͏ͱ͢ΔͱͲ͏ͯ͠΋খ͘͞ͳΔ MCTest (Richardson+ 2013) [web]: 660*4 questions ProcessBank (Berant+ 2014) [web]: 585 questions ࣗಈతʹ࡞Δͱͨ͘͞Μσʔλ͕Ͱ͖Δ΋ͷͷɺ࣭͕ո͍͠ CNN/Daily Mail (Hermann+ 2015) SQuAD (Rajpurkar+ 2016) [web] ਓखͰൺֱత࣭ͷྑ͍σʔλΛͨ͘͞Μ࡞ͬͨྫʁ LAMBADA (Paperno+ 2016) [web] ·ͩͪΌΜͱಡΜͰͳ͍Ͱ͕ͨ͢ͿΜ͓͢͢Ί 7 / 26
8. CNN/Daily Mail Dataset (DeepMind QA Dataset) Paper: Teaching Machines to Read and Comprehend http://arxiv.org/pdf/1506.03340v3.pdf (NIPS 2015) Site: http://cs.nyu.edu/~kcho/DMQA/ 8 / 26
9. CNN/Daily Mail Dataset (DeepMind QA Dataset) CNN ΍ Daily Mail ͷهࣄλΠτϧ΍ݟग़͕֘͠౰Օॴͷཁ໿ ʹͳ͍ͬͯΔ λΠτϧ΍ݟग़͠ͷ entity ෦෼Λ݀ʹͯͦ͠Ε͕Կ͔Λ౴͑͞ ͤΔλεΫ هࣄ͸ͨ͘͞Μ͋ΔͷͰͨ͘͞Μ࡞ΕΔ (context, query, answer) Ͱ 1 ୯Ґ هࣄ಺༰ʹ౴͕͑ग़ͯ͜ͳ͍Α͏ͳ΋ͷ͸࡞Βͳ͍ 9 / 26
10. CNN/Daily Mail Dataset: Example 10 / 26
11. จ຺จʹ͓͚Δ correct answer ͷස౓ 11 / 26
12. σʔληοτͷߏ੒ ͨ͘͞Μ 12 / 26
13. ݩ࿦จͰఏҊ͞Ε͍ͯΔ Neural Network Models 13 / 26
14. ͜͜·Ͱલ࠲ ຊݚڀͷߩݙ Ϟσϧɿ؆୯ͳ΍ͭͰ͍͍ͩͨྑ͍είΞ͕ग़ͨ 1. Entity-Centric Classifier (ൺֱɾ෼ੳ༻ʁ) 2. End-to-end Neural Network (state of the art) ෼ੳɿ͍͍ͩͨྑ͍෼ੳΛͯ͠ϊΠζΛআ্͍ͨݶΛ༩͑ͨ ϥϯμϜʹબΜͩ 100 ໰Λ෼ྨ Τϥʔͳ͍͠ෆ໌ྎͳ΋ͷ͕ 25 ໰͋ͬͨ → 25% ͸ແҙຯʁ 14 / 26
15. Entity-Centric Classifier ީิͱͳΔ entities ʹ͍ͭͯҎԼΛಛ௃ʹͨ͠ vector f Λߏ੒ ౴͑ͷ entity ͷॱҐ͕ߴ͘ͳΔΑ͏ʹ weight vector θ Λֶश θ⊤fp,q (a) > θ⊤fp,q (e), ∀e ∈ E\{a} p: passage, q: question, e: entity, a: answer, E: entities Algorithm: LambdaMart 15 / 26
16. End-to-end Neural Network 16 / 26
17. End-to-end Neural Network 17 / 26
18. End-to-end Neural Network 1. Encoding Bi-directional RNN + GRU ACL ൛Ͱ͸ LSTM ͕ͩͬͨվగ൛ (v2) Ͱ͸ RNN+GRU ʹมߋ 1. hR i = RNN(hR i−1 , w(pi)), hL i = RNN(hL i+1 , w(pi)) 2. pi = concat(hR i , hL i ) 2. Attention 1. αi = softmaxi q⊤Wspi 2. o = ∑ i αipi α: prob. distribution (=attention), q: question embedding, pi: contextual embedding for pi (i-th word in the passage), Ws: weight matrix used for a bilinear term (it frexibly computes a similarity between q and pi), o: output vector 3. Prediction a = argmaxa∈p W⊤ a o 18 / 26
19. Diffs from previous model (Hermann+ 2015) 1. bilinear term using Ws , instead of a tanh layer similarity between q and pi ͷදݱͷ࢓ํΛม͑ͨ ॊೈੑ্͕͕ͬͨʁ 2. o: output vector Λ࠷ऴతͳ༧ଌʹ࢖͏աఔͰ༨ܭͳܭࢉΛڬ ·ͳ͍Α͏ʹͨ͠ ݩͷϞσϧͰ͸มͳϨΠϠʔΛ͍Ζ͍Ζט·͍ͤͯͨ 3. prediction ର৅ͷ vocaburaly Λ entity ͚ͩʹͨ͠ ݩͷϞσϧ͸ग़ݱ͢Δ͢΂ͯͷޠΛީิʹ͍ͯͨ͠ 19 / 26
20. Result 20 / 26
21. Analysis - classifier model - ablation 21 / 26
22. Analysis - sampled questions 22 / 26
23. Analysis - sampled questions 23 / 26
24. Analysis - result 24 / 26
25. Analysis - accuracies for each category 25 / 26
26. ·ͱΊ γϯϓϧͳϞσϧ͕ྑ͔ͬͨ semantic matching Λֶश͢Δʹ͸ neural model ͕΍ͬͺΓྑ ͍ʢ୯ͳΔ classifier ͱൺ΂Δͱʣ CNN/Daily Mail ͸΄ͱΜͲ಄ଧͪ: σʔλʹϊΠζ͕ଟ͍ɺࣗ ಈͰ࡞Εͨͷ͸ྑ͍͚Ͳ࣭΋େࣄʢຊ౰ʹಡղతͳਪ࿦ΛଌΔ ͷʹ໾ཱͭͷ͔ʁʣ ͜͏͍͏σʔληοτΛ൱ఆ͢Δඞཁ͸ͳ͘ɺΑΓ realistic ͳ σʔληοτͷͨΊͷֶशσʔλͱͯ͠׆͔ͤΔ͸ͣ ͍Ζ͍Ζσʔληοτ͕૿͑ͯΔ͠ reading comprehension ྲྀ ߦͯ͠·͢Ͷ ײ૝: ࣮σʔλݟͯஸೡʹ෼ੳ͢Δͷ͕େࣄ 26 / 26