Multimodal Grounding for Language Processing

弊研究室で行なったCOLING2018読み会の発表資料です。

Transcript

1. 
    Multimodal Grounding for Language Processing 20181017 NomotoEriko 1
2. ### š # &0 NLP  +$ 65#5- šContents: Ø"! Ø # &0 '8 Ø # 3/*2 Ø  # NLP Multimodal Grounding for Language Processing 2 %7. 4(
   )35- ,1
3. 
    Multimodal Grounding for Language Processing 3
4. HF>T NLP šA>O… "/(# '*!.@dQaBX g6O š_QR>O… KIEWLS 8e^?2JO  Conceptual grounding: ]`QZ3fDZb7P14N: U ↑ S<),%+/$,M=0 ]` [c  4 !.&- YZC;hVZC;h\ZC;h9ZC;G
   DZ56M=
5. 
   Multimodal Grounding for Language Processing 5
6. # 3 -
   ($ '+ . ,! &"  3 - š,)* (Cross-modal transfer) š,'+ (Cross-modal interpretation) š%# (Joint multimodal processing) 6 #-
7. @>? (Cross-modal transfer) š
   E*D8F &E*A):F -+ 1! 92  C š"G Ø07E;=→4# '45F ØB,(.E→B6F ØE< /#→F 7 $3%C
8. GAF (Cross-modal interpretation) š$)=(
   
   /,>3D+ G@6 . K &)B"I
   L?*B"142;<M š!N Ø9#80LHE  5:M Ø LJ&C-5:M 8 %7'K
9. ;$ 3 (Joint multimodal processing) š=- %/ F  *' #27 : $(>
    šI Ø+*CDG6B<* "4H Ø@A7E&)8G5? 6B 3H 9  3 !F 09.1,
10. 
      Multimodal Grounding for Language Processing 10
11. ,(+ 3 0 š,( 
     $ * š 3  .- Ø%#,( '" (Concept representations) Ø !'" (Projection) Ø&/),( '" (Compositional representations) 11 ,(+
12. .)50/( (Concept representations) š.)50 7&;.)9 #, '!( Ø<:.)50 6,:1 +50 '!( Ø
     2 8 *"3 %- 12 50$4
13. ,*:21) (Concept representations) š(&!+.F Ø"6BA7C%#/D Ø↑ “cat” ;<5 “panther”  >@5 “dog”   : E 8$ 
    :
    Ø04 = 3
    ?=- 13 :2'9
14. 2394 (Projection) š86>:4 D 5. &(" 86 +$)
     A š&("@0- (Mapping)  /?>:<@ (Joint representation space)  7;86>:4   B*'!#1=  ;C 14 %) &,)>:1= ←Mapping ↓ Joint learning
15. ./40 (Projection) š!"=)( (Mapping) Ø!"+!" )( !: #$% → #$' -9  Ø87>13
    @6(*?  '&  %$2<,:  maximize sim(#$% , #$' ), minimize sim(#$% , ./0123$' ) 15 #!%#;5-9
16. -/70 (Projection) š&@?8;B (Joint representation space) Ø%#$ :9C2+35A( ü) ?8'6<E1. F üB 5A =*E1.
    F š4D 2 B  !,>    16  " ?8,>
17. ;NB KA @5 šOE*KA6L/- E*  I-6L  šCD=P U$#  <
    8>7:  Ø( E* RFS3.90&TJ  Ø," MG? E*'U 94 Q2 )+% 17 !KA1H
18. 
        NLP Multimodal Grounding for Language Processing 18
19. NLP #&$'&,> š957.A=<4 ØD(6' (simLex-999 etc…) 1  7.A=@2 B* Ø/0E #&$'&7.A=%!$'&- š+)?95 8C?95 :; Ø#&$'&7.A=<4" Ø#&$'&7.A=8C?953
       19 #&$'& NLP
20. NLP  !%"'% +1 š3,47 Ø0) 3, 6/.( Ø0)-# $& ' 2)-,5 ' (imSitu) 
    
    *8
     20  !%"'% NLP imSitu*- (http://imsitu.org/ )
21. +. š)&$ Ø "
    >< š!,% #&$ Ø!,% #'(% "
    >< š- % Ø* &$ 
    / 21  NLP
22. 
     Multimodal Grounding for Language Processing 22
23.  NLP NLP 
     " š# š! š NLP 23