表現学習時代の生成語彙論ことはじめ @PFIセミナー

Transcript

1. ¿Ó½Ǝ»: —ÐŇǖĦ!5;&D
   ĉPreferred Infrastructure Ù¦Ʋǁ (@unnonouno)
   2014/10/16 PFIVZX^
   
2. ƝLJŘ
   Ù¦Ʋǁ (@unnonouno) !  ŪåŇĭ¸GdV\fuSh€XGÁnj½Ǝ !  ĘƬĭ¸E/H05GKA' !  “‚ú߸ýŵġIBMôLjŎPFI 8
   
3. mrep5c}P1KA%-
   !  řĹƑƚ8›%3ÖĨåŇ4íƾ4A' !  pip install mrep 4S€\f}
   9
   
4. Ÿ§:Ú
   !  word2vecP†ù8ƄŇ:¿Ó½ƎŹK:;7%P%A %-
   :
   
5. ¿Ó½Ǝ:ÚPň3Ĉ0-!5
   ;
   ƣƵ.5ăJM3LƄ Ň:+&)PrWf} Į :†8ƅ%ñDJM L:4;7
   
6. —ÐŇǖĦ Generative Lexicon (GL) <
   
7. ®:£Ć
   !  —ÐŇǖĦ813:ƴƄ7šʼn !  ¿Ó½Ǝ5Êů:Ð5:¨Ē
   =
   18EA%3Œ:Ú OJ7,
   
8. The Generative Lexicon [Pustejovsky95]
   !  James PustejovskyĀdž !  ÄƸA4;ƟDA)Q4% - !  TimeML:ĀdžŽ4EĐL !  „Å; ¿Ó813:ÚP %Iƙ03-:4<0K %-
   >
   ÊÌ5¥%
   http://www.cs.brandeis.edu/~jamesp/
   
9. „Ň:„7J·¦é—:„ [·¦05]
   !  ·¦ǍƆ—ÐŇǖÊů Ħ !  ®ƟQ4LǕƥ† ; _ ;
   ?
   
10. Æ8ħă8%-´ē
    !  ·¦ǍƆ—ÐŇǖÊůĦ[·¦05]:Û12 ƽ !  James Pustejovsky, Introduction to Generative Lexicon. [Pustejovsky05] !  ě„ƲëGenerative Lexicon:NjÇ [ě„ 06]
    76
    
11. ɋ:Êů813ă3BI
    !  ‰č8ɋ5å03EĬî5%3:ɋ5æŲ 5%3:ɋL 77
    ƛŸ:8‹0-
    ŒP8ƍ-
    ¥ƺ48‹0-
    
12. gR:Êů813ă3BI
    !  wj5%3:gR5‹Ò'L”Ï5%3:gR:Ê ůL !  é:ɋ:ŗ5™&I7!5ĸ!037 
    78
    Ps€V4Ǒ0-
    Jš03«#
13. !  ă3BL5ƄŇ;1:ƌ7LĎćP â03L!58Ñ2 !  ċOMœ8I03ÊůģO03L !  !:Êů:ģµ8;ęª:ĨƏÎ:I7E: ĐL !  %Eƌ7LåŇ4Eǃ-ÓłL
    79
    
14. —ÐŇǖĦ:ăœ
    !  1:Ňǖ8ęª:ļŋPâ03L !  đǓ8Ī&3+:ļŋJƦ§ƌ7L Êů—Ð#M3L5ăL
    7:
    ɋ
    Ĭî:ɋ
    æŲ:ɋ
    $_
    
15. ŅÊ ™ŸƌĺŇ5êĺŇ;ĔĖ#ML
    7;
    !  êĺŇ;Êů¢8ŒJ:¨ĒPâ03LÔ çÇ !  -A-A™&Ÿ870-Ň;™ŸƌĺŇ5% 3ĔĖ#ML
    
16. Ù¦;7*!ÚÝ8ŰůPâ03L
    Ī¹‡ƄŇGƣŇ~r}:ÊůPA Ʊ7Ë݉ŏê5ă3L
    7<
    %( ¿þņ6™&Êů
    %( ¿þ™&.6ņÊů
    
17. Ňǖ:ĝŃļŋWU|Rļŋ5E:P ăL
    ŇPń­'LƠÎGł:ğ 1.  ļÐWU|RConstitutive !  ƋēG£Ć76Œ4ˆÒ3LPā' 2.  řĚWU|RFormal !  +MŒ7:‡ì:NjōPā' 3.  ²¢WU|RTelic !  ÁĊG²¢Pā' 4.  ƱWU|RAgentive !  +MP—Bˆ'¾ÿGÄƉPā'
    7=
    
18. Œ‚ģ+.
    7>
    
19. ŗ ò:WU|Rļŋ
    7?
    1. ļÐ `Syǀ k€g}
    2. řĚ ŀKîÁ nj
    3. ²¢ ŀLĄĩ 'L
    4. Ʊ æB©3L ÿL
    @1AKN`KBC0 L4GM*!J&DIBQS[^T
    
20. Œ4!%-úßĠÇ7:
    !  ò;Ŋ¾'L5ƿǏ:Ƥø!!4; ²¢WU|RĐL !  +:-D…;«:đ:Êů.5øDŽ4 L 86
    ŔŌ:ò;ş
    ŔŌ:ò;\ngş
    
21. Ƈő ƿǏ:ƤøPįŁ'L5Y€f87L
    !  Ɯö;ä8LE:5ƿǏ:ƤøPįŁ%3L !  ƶ8ärbg8703-Jƭǎ:^| o;­J8ÀŪ 1Ƨ:åŇĭ¸5:PGK-9
    87
    ƭǎ ä4Dž3-Q.I Ɯö4 lhtS  60/ ä4 Ɯö4  ƭǎ .Jä:†8LƜö:†.I 
    \^[]U5@CLASSICAOP
    
22. 7Q4WU|RļŋPăL:
    !  ê:ƄŇ:Êů8¨%3é8•-I7ƿǏ:• øūĢ'L !  +:-D-Eáơ#M-:I7¿Ó4Êů ª0-K+0K7đĖ:Êů87L!5ĐL !  !:ƿǏ:•øJ³ï:đ†4:Êů—Ð# ML:4;7
    88
    
23. begin8;į×:Êů—Ð#ML
    !  +E+Ebegin:58;‹ƒGł4; !  beginį×:ÊůPâ03L5ĈIKđǓ8Ī &3Êů—Ð#ML5ăL?Ū
    89
    begin the meeting
    begin a dance
    begin the book
    begin the movie
    
24. ęÐco-composition
    1.  begin:žN8;łÒL;(.book;ł4 ;7 , 2.  bookPł5%3øDŽ%I`SqÍÈ
    3.  WU|RJúßPÂ03L ‰¶:—Ðq^\P 5Ž= 8:
    begin
    the book
    
    
25. cf. ƿǏ:ŝģŶ
    1.  double:žN8;double:ŞÒL;(.10; double4;7 , 2.  10Pdouble:Ş5%3øDŽ%I
    3.  Vx\f:œ¯:úßPÂ03L ‰¶:—Ðq^\P13*",5Ž= 8;
    double x =
    10
    
    
26. ħă ³;—ÐŇǖĦ8;ĴK31:ļŋL
    ĝŃļŋ;—ÐŇǖĦ:41ļŋ:†:11 !  Ƴļŋ !  łļŋ !  ĝŃļŋ !  Ňǖżƕļŋ ®;ėŷ%A'I
    8<
    
27. !!A4:A5D
    !  ĥŇǖ8;+:ċOMœ8¨'LúßPâ0 3LWU|Rļŋ !  ŹŴ:ƄŇ5śćªK:øDŽ478 !:WU|R:úßPċ03¥%ÊůP—Ð 'LęÐ
    8=
    
28. ŪåŇĭ¸:†4—ÐŇǖĦ;6MJċOM 3-
    !  åŇĭ¸½ƒ:«430÷ !  ACL:«4300÷
    8>
    ÊÌ5Ĥ7
    
29. /H05!M;‚ģ
    !  11:Ňǖ4!MJ:úß
    8?
    5 . $./LHE , @A M5BAM +3
    45. $.#/R%BFSY]W2M( $
30. WU|Rļŋ;‚ĨũdV\fJ¾Ʈõ%-
    !  Web:íƾóČJ…’4ÿÐ%-m`€ Pܹ%3ǐˆ'Lœ¯ [Cimiano+07] !  ęĸǔ°5ðǒG¾ǒ:ƠÎPĝƯ8%3{ €V€X½ƎPƂ¹'Lœ¯ [ĻŖ+12]
    96
    5;ċ:EƴƄ4;7#+
    
31. Ĥ%ƖJ•7%3BL
    !  ƄŇ6:I7ƃ:ƄŇ5‰Ƽ8ċOML 5!58›'LüŭP±ųµ%3L !  6:I7ƄŇ5ċOML8I03ƄŇ: ÊůEģOL 97
    
32. !:Ú6!4
    Ÿ§: Statistical Semantics:Ú8ǃ3L
    98
    
33. !!4‰°—ÐŇǖĦJŮM3 Êů:Ð:ÚP%A'
    99
    
34. Statistical Semantics:#J
    !  ŹŴƄŇ:ˆÓǔ°76:IJè¢7ÎŃJ+:Ň: ÊůPăL !  ‹ƚ‘øGhz{}ibf76:’¯¹JML
    9:
    Statistical Semantics is the study of "how the statistical patterns of human word usage can be used to figure out what people mean, at least to a level sufficient for information access”(ACL wiki)
    
35. ƄŇ:rWf}¿Ó
    !  Ÿ§:Ú;ƄŚ:ƄŇ:ÊůP¿Ó%- !  ®§;ŹŴƄŇ5:¨Ē4ÊůģOLÚ7: 4/(J7Lo~]:Êů67 L: 9;
    
36. —ÐŇǖĦ5Statistical Semantics:¨Ē
    !  —ÐŇǖĦ !  ĝ¡:đǓ8LƄŇ:ˆÓ:ÊůPă3L !  5+:I7ģµPĸ!'-D:Ňǖ:ļ ŋPă3L
    !  Statistical Semantics !  ŹŴƄŇ:‘Ɯ8I03‰Ɣ¢7+:ƄŇ:Êů Pă3L !  ƀ±¢7ƄŇ:ˆÓ8›'LÊů8åſ%37 ®J'LI 9<
    
37. Principle of compositionality (Frege1892]
    … the meaning of a complex expression is determined by the meanings of its constituent expressions and the rules used to combine them (wikipedia )         
    9=
    
38. ·Š#Q:\{SgA5A03L
    9>
    https://speakerdeck.com/mamoruk/a-tensor-based-factorization-model-of-semantic-compositionality
    
39. ¿ÓrWf}:Ð;ĵ%ĕIKƻĕķ [Mitchell+08]
    !  ĵ%ĕGƻĕ76rWf}:Ðœ¯Pılj !  ÇƑ"5:ƻĕMultiplicative¬0/F0-
    9?
    
40. Statistical Semantics5:¨Ē
    !  Statistical SemanticsŸ§:£Ć;6/J 5å5#-(:rWf}PƮõ'LŎŠ !  Compositionality:ŎŠ;+:rWf}: :ŎŠ ĞœP™8øœØEL%¿Ó½Ǝ:ˆÓ 4ƁK‡KP•)3L
    :6
    
41. Êů:ÐP‹ƚ4¿Ó'L (MV-RNN) [Socher +12]
    !  ĥƄŇ8;rWf}5‹ƚ›Ī'L !  ŵƹ#ML5ƪ:‹ƚP¼’:rWf}8ƻ3 #J8žĶř7ģŶ f PŦż¢8‹ !  RootA4ƗKľ'5đ:¿ÓrWf}õJML :7
    
42. GGd€_}. [Cruys+13]
    !  ÆŇ¾ǒ²¢Ň:31Pd€_}Pċ03æ) 3L:.5ăL
    :8
    
43. Dynamic Convolutional Neural Network [Kalchbrenner+14]
    !  CNNPċ03ƄŇ:A5AKPæB‡ 3 !  ļđÞ4;7Żǃ:ŜČõJML
    :9
    
44. Ƈő CNN5compositionality
    !  Ŀ~Sy4;ŧD3ijÏ¢ƄƷ7úß !  ˜Ă:~Sy87L8¶M3+MJæBO#K IKƣƵ7Êů:LúßPûŭ%3L
    ::
    Layer1
    Layer3
    Layer5
    [Zelier+14]OP'
    
45. åŇĭ¸;ĸÔņ
    :;
    
    4
    nW^}
    Tb[
    ¤‘
    ǂ
    źř
    ŸƑ
    ƄŇ
    –Ú
    æB‡ L
    ‘ø'L
    ¿Ó
    ƄŇ
    o~]
    đ
    M-
    M-
    
46. !:œŐ8—ÐŇǖĦ:ü•Pď)7
    !  —ÐŇǖĦ;CompositionalityP…’4űƘ% 3±ų2L-D8Ľ¸#M-¸Ħ5•M7  !  šƕ½Ǝ76:m{aSvZofĸ+7 ®!+ĜŨ:ü•GŹŴ‘¦P•ą'> :<
    
47. Ãà:Ú
    :=
    ƣƵ.5ăJM3LƄ Ň:+&)PrWf} Į :†8ƅ%ñDJM L:4;7
    
48. ęÐP¿ÓrWf}85K!C [Tsubaki+13]
    !  ƪƪ8ŕŢPſ@%3¥%rWf}PÿL !  4-rWf}JÊů:Ð‹OML
    :>
    run companyM
    Mrun
    companyM JMrun
    company LOQ
    runM JMcompany
    Mcompany
    run LOQ
    run companyM)"
    '48
    
49. ƄŇ:Ð8›'L1:ŁÔ
    !  —ÐŇǖĦ !  åŇÓłPűƘ%3Awe}µ%I5'L īB !  Ť:׸we} !  űƘ#M-åŇe`P‡’ń­'LI7we }:ŬƘ !  ¿Ó½Ǝųšƕ½Ǝų !  ļÐ5¿Ó:½ƎP™8‹5%3L !MJ:RSeR;ƞţ8¨Ē%3L
    :?
    
50. A5D
    !  —ÐŇǖĦ5; !  Ňǖ;8›'LÀƊº7üŭJđǓ8Ī&3Ê ů—Ð#ML5ăL !  Ň:Êů:Ð8¨'LŎŠ !  ƄŇ:¿ÓrWf}PÐ%3ƣ¿Ó:ÊůP ÿL !  ¿Ó½Ǝpv8ŀ03®İ‹03L åŇ½:ü•5åŇĭ¸:¥%’¯:㏠E05ÕC ;6
    
51. "ƓƢK5"$A%-
    ;7
    
52. ħăđƫ (1/3)
    !  [Pustejovsky95] James Pustejovsky. The Generative Lexicon. MIT Press. !  [·¦05] ·¦ǍƆ. (4+$. N%ˆư !  [Pustejovsky05] James Pustejovsky. Introduction to Generative Lexicon. Foundations of SemanticsƩĺ´ē. http://www.cs.brandeis.edu/~jamesp/classes/LING130/ !  [ě„06] ě„Ʋë. Generative Lexicon2. Ɛ:ğ†NJ̓:ŏÌƈ ;8
    
53. ħăđƫ(2/3)
    !  [Cimiano+07] Philipp Cimiano, Johanna Wenderoth. Automatic Acquisition of Ranked Qualia Structures from the Web. ACL2007 !  [ĻŖ+12] ĻŖ˜ƨ, ·Šť, ě„Ʋë. (4$
    (!'&)  . 0. NLP2012.
    !  [Mitchell+08] Jeff Mitchell, Mirella Lapata. Vector-based Models of Semantic Composition. ACL2008. !  [Socher+12] Richard Socher, Brody Huval, Christopher D. Manning, Andrew Y. Ng. Semantic Compositionality through Recursive Matrix-Vector Spaces. EMNLP2012. ;9
    
54. ħăđƫ(3/3)
    !  [Cruys+13] Tim Van de Cruys, Thierry Poibeau, Anna Korhonen. A Tensor-based Factorization Model of Semantic Compositionality. NAACL2013. !  [Kalchbrenner+14] Nal Kalchbrenner, Edward Grefenstette, Phil Blunsom. A Convolutional Neural Network for Modelling Sentences. ACL2014. !  [Zelier+14] Matthew D. Zeiler, Rob Fergus. Visualizing and Understanding Convolutional Networks. ECCV2014. !  [Tsubaki+13] Masashi Tsubaki, Kevin Duh, Masashi Shimbo, Yuji Matsumoto. Modeling and Learning Semantic Co-Compositionality through Prototype Projections and Neural Networks. EMNLP2013 ;: