夏のトップカンファレンス論文読み会 / InnovationMeetup20170918csn_cvpr2k17

夏のトップカンファレンス論文読み会（2017/09/18）での発表資料です。

- connpass: https://abeja-innovation-meetup.connpass.com/event/63466/
- 著者実装: https://github.com/andreasveit/conditional-similarity-networks
- 散布図: https://github.com/crcrpar/conditional_similarity_networks_pytorch

Transcript

1. © 2017 VASILY,Inc. Նͷ τοϓΧϯϑΝϨϯε ࿦จಡΈձ Conditional Similarity Networks (CVPR2017) 2017/09/18 Mon.
2. © 2017 VASILY,Inc. ໨࣍ ▸ ໨తʹ߹Θͤͯɺಛ௃ྔϕΫτϧͷۭؒΛ෼ׂ͢Δ
   ▸ ओு
   ▸ ैདྷͷྨࣅ౓ܭࢉʹ͍ͭͯ
   ▸ $POEJUJPOBM
   4JNJMBSJUZ
   /FUXPSLT
   ▸ ࣮ݧ݁Ռ
   ▸ ௥ࢼ
   ▸ ײ૝
   ▸ 3FGFSFODFT 
3. © 2017 VASILY,Inc. ಛ௃ྔϕΫτϧͷۭؒΛ෼ׂ͢Δ 
4. © 2017 VASILY,Inc. ࿦จͷओு ▸ ʢඇʣྨࣅ౓ͱͯ͠ಛ௃ྔϕΫτϧಉ࢜ͷڑ཭͕࢖͑Δ͕ɺ
 ୯Ұͷج४ʹΑΔྨࣅ౓͔͠ܭࢉͰ͖ͳ͍
   ▸ ྨࣅ౓ͷج४͸ෳ਺͋Δͷ͕Ұൠత
   ▸ ʮҰຕͷը૾ʹෳ਺ͷଐੑ BUUS ʯͰ͋Δ͜ͱ͕ଟ͍
   ▸ FH
   ۺͷଐੑɿछྨɺඥͷ༗ແɺώʔϧͷߴ͞ɺʜ
   ▸ ಛ௃ྔϕΫτϧʹϚεΫΛద༻͠ɺ
 ಛ௃ྔϕΫτϧͷۭؒΛ෼ׂ͢Δ͜ͱͰෳ਺ͷྨࣅ౓ܭࢉΛՄೳʹ͢Δ 
5. © 2017 VASILY,Inc. දهʹؔͯ͠ ▸ ಛ௃ྔϕΫτϧɿ$//ͷग़ྗͳͲ
   ▸ ಛ௃ྔɿಛ௃ྔϕΫτϧͷཁૉ
   ▸ ৚݅ɺҙຯɺλεΫɿྨࣅ౓ܭࢉͷج४ 
6. © 2017 VASILY,Inc. ྨࣅ౓ܭࢉͷطଘख๏(1/5) ▸ $//ͷதؒ૚ͷग़ྗΛಛ௃ྔϕΫτϧͱͯ͠ɺಛ௃ྔؒͷڑ཭Λྨࣅ౓ͱͯ͠ѻ͑Δ
   ▸ ❌
   ෳ਺ͷྨࣅ౓ΛҰͭͷۭؒͰ͸ѻ͑ͳ͍  CNN feature input feature space
7. © 2017 VASILY,Inc. ྨࣅ౓ܭࢉͷطଘख๏(2/5) ▸ *NBHF/FU౳Ͱࣄલֶशͨ͠Ϟσϧͷશ݁߹૚ͷग़ྗͰ΋ྨࣅ౓ͷܭࢉࣗମ͸Ͱ͖Δ͕ɺ
 λεΫʹదͨ͠ಛ௃ྔϕΫτϧΛநग़͢ΔΑ͏ʹֶशΛ͢Δख๏͕͋Δ
   
   4JBNFTF
   /FUXPSLT
   
   5SJQMFU
   -PTT 
8. © 2017 VASILY,Inc. ྨࣅ౓ܭࢉͷطଘख๏(3/5) ▸ 4JBNFTF
   /FUXPSLT<
   B><
   C>
   ▸ ࣅ͍ͯΔը૾ʹ͸ͳΔ΂͘ڑ཭͕ۙ͘ͳΔΑ͏ʹֶश͢Δ
   ▸ ࣅ͍ͯΔը૾ͷϖΞͰֶश͢Δ  Image from γϞηϥ ΤυΨʔɹSiameseωοτϫʔΫϞσϧΛ༻͍ͨը૾ಛ௃ྔநग़
9. © 2017 VASILY,Inc. ྨࣅ౓ܭࢉͷطଘख๏(4/5) ▸ 5SJQMFU
   MPTT<
   B><
   C>
   ▸ ࣅ͍ͯΔը૾͸ۙ͘ɺࣅ͍ͯͳ͍ը૾͸ԕ͘ͳΔΑ͏ʹֶश͢Δ
   ▸ ʡʢΫΤϦը૾ɺࣅ͍ͯΔը૾ɺࣅͯͳ͍ը૾ʣʡͷτϦϓϨοτͰֶश
   ▸ 4JBNFTF
   /FUΑΓॊೈ  query positive negative train d_p d_n L_triplet = max{0, d_n - d_p + h}, h: margin
10. © 2017 VASILY,Inc. ྨࣅ౓ܭࢉͷطଘख๏(5/5) ▸ ʮࣅ͍ͯΔʯ͸ը૾ͷଐੑ͝ͱʹܭࢉͰ͖Δͱخ͍͠
    ▸ ͭͷج४ʹ͖ͭɺͭͷϞσϧΛ༻ҙ͢Δͷ͸ޮ཰͕ѱ͍  ・ ・ ・ model for notion1 model for notion2 ・ ・ ・ ・ ・ ・
11. © 2017 VASILY,Inc. ▸ ͭͷϞσϧͰෳ਺ͷྨࣅܭࢉΛ࣮ݱ͢ΔͨΊʹϚεΫΛಋೖ
    ▸ ಛ௃ྔϕΫτϧʹϚεΫΛద༻ͯ͠ಛ௃ྔϕΫτϧͷۭؒΛ෼ղ͢Δɻ Conditional Similarity Networks (1/3)  特徴量空間 マスク 色空間 形空間
12. © 2017 VASILY,Inc. Conditional Similarity Networks (2/3) - mask  ݻఆϚεΫ
    $4/
    
    pYFE
    EJTKPJOU
    NBTLT
    " ಛ௃ྔϕΫτϧΛ౳෼͢Δ
    # ڞ༗͞ΕΔಛ௃ྔ͸ͳ͍
     ֶशϚεΫ
    $4/
    MFBSOFE
    NBTLT
    " ֓೦ຖʹ༻͍Δಛ௃ྔͱରԠ͢ΔॏΈΛֶश͢Δ
    # ڞ༗͞ΕΔಛ௃ྔ͕͋Δ
    $ ॏΈ͸ਖ਼ن෼෍ͰॳظԽ  ᶃ ᶄ
13. © 2017 VASILY,Inc. ▸ ϚεΫ͸ຒΊࠐΈߦྻͰ࣮૷Ͱ͖Δɻ
    ▸ ߦྻ͸ʢ৚݅਺ʣʷʢಛ௃ྔϕΫτϧͷ࣍ݩʣ Conditional Similarity Networks (3/3)  Similirarity Network (ResNet-18) ・・・ mask input feature condition specific feature condition specify
14. © 2017 VASILY,Inc. ఆࣜԽ ▸ $POEJUJPOBM
    5SJQMFU
    -PTT
    
    ▸ ਖ਼ଇԽ߲
    ▸ ಛ௃ྔநग़ʹରͯ͠
    ▸ ͓ؾ࣋ͪɿ௒ٿ໘্ʹຒΊࠐΈ͍ͨ
    ▸ ϚεΫʹରͯ͠
    
    ▸ ͓ؾ࣋ͪɿεύʔεʹ͍ͨ͠
    ▸ શମɿ  c: ৚݅ɺΧςΰϦ m_c: cʹରԠ͢ΔϚεΫ h: Ϛʔδϯ
15. © 2017 VASILY,Inc. ࣮ݧ   U4/&ʹΑΔಛ௃ྔϕΫτϧͷՄࢹԽ
     ֶशͨ͠ϚεΫͷಛ௃
     ਫ਼౓ൺֱ
     σʔληοταΠζͷӨڹ
     5SJQMFU
    -PTTΛ༻ֶ͍ͨश͕ಛ௃ྔϕΫτϧʹ༩͑ΔӨڹ
16. © 2017 VASILY,Inc. Ϟσϧ ▸ ஶऀ࣮૷ɿIUUQTHJUIVCDPNBOESFBTWFJUDPOEJUJPOBMTJNJMBSJUZOFUXPSLT
    ▸ *NBHF/FUֶशࡁΈ3FT/FU<>ΛϕʔεʹɺҎԼͷϞσϧΛൺֱ
    4UBOEBSE
    5SJQMFU
    /FUXPSL
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    DPOEJUJPOΛ༻͍ͣʹτϦϓϨοτΛֶश
    4FU
    PG
    4QFDJBMJ[FE
    5SJQMFU
    /FUXPSLT
    DPOEJUJPO͝ͱʹϞσϧΛֶश
    $4/
    pYFE
    KPJOU
    NBTLT
    $4/
    MFBSOFE
    NBTLT 
17. © 2017 VASILY,Inc. σʔληοτ ▸ 65
    ;BQQPT,
    ▸ ۺͷը૾ ຕ
    ▸ ࣍ͷͭͷج४ͰτϦϓϨοτΛ࡞੒
     छྨɿϒʔπɺαϯμϧɺFUD
     ੑผɿஉੑɺঁੑɺঁͷࢠɺஉͷࢠ
     ώʔϧͷߴ͞ɿʙΠϯν
     ด͡ํɿͻ΋ɺόοΫϧɺεϦοϙϯɺFUD
    ▸ ϒϥϯυ৘ใʢ෼ྨλεΫʹ࣮ݧʣ  image from http://vision.cs.utexas.edu/projects/finegrained/utzap50k/
18. © 2017 VASILY,Inc. t-SNEʹΑΔಛ௃ྔϕΫτϧͷՄࢹԽ ▸ ʢ্ʣɿʮۺͷకΊํʯʹؔ͢Δۭؒ
    ▸ ʢԼʣɿʮۺͷछྨʯʹؔ͢Δۭؒ
    ▸ ৭΍ܗͷӨڹ͕΄΅ͳ͍
    ▸ શ͘֎ݟ͕ࣅ͍ͯͳͯ͘΋৚݅ʹैͬͯ
 ຒΊࠐ·Ε͍ͯΔ 
19. © 2017 VASILY,Inc. ֶशͨ͠ϚεΫͷಛ௃ ▸ ֶशϚεΫ͕ʮૄʯ
    ▸ ৚݅͝ͱʹڞ௨͢Δಛ௃ྔ͕ଘࡏ͢Δ  ԣɿಛ௃ྔϕΫτϧͷ
 ɹɹΠϯσοΫε
    ॎɿॏΈ
20. © 2017 VASILY,Inc. ਫ਼౓ൺֱ ▸ ςετσʔλͷτϦϓϨοτʹର͢Δਫ਼౓
    ▸ τϦϓϨοτ
    Y
    Z
    [ ʹରͯ͠ɺZͱ[ͷͲͪΒ͕Yʹࣅ͍ͯΔ͔ 
21. © 2017 VASILY,Inc. σʔληοταΠζͷӨڹ ▸ ʙ ɿݻఆϚεΫɾ৚݅͝ͱʹϞσϧ͕ਖ਼֬
    ▸  ʙɿֶशϚεΫ͕༗ར  ԣɿσʔλ਺
    ॎɿޡΓ཰
22. © 2017 VASILY,Inc. Triplet LossΛ༻ֶ͍ͨश͕ಛ௃ྔʹ༩͑ΔӨڹ ▸ సҠֶशͰͷਫ਼౓Ͱൺֱ
    ▸ ॏΈݻఆͷ$// 3FT/FU ʹશ݁߹૚Λ௥Ճ͠ϒϥϯυ༧ଌλεΫΛֶश
    ▸ ैདྷͷτϦϓϨοτֶश͕ྼ͍ͬͯΔ 
23. © 2017 VASILY,Inc. ࣮ݧ·ͱΊ ▸ $4/ʹΑΓ৚݅ʹదͨ͠ಛ௃ྔϕΫτϧ͕ܭࢉͰ͖ΔΑ͏ʹͳ͍ͬͯΔ
    ▸ $4/
    MFBSOFE
    NBTLͷֶश͸ΑΓଟ͘ͷσʔλ͕ඞཁʹͳΔ
    ▸ ҟͳΔج४Ͱ΋Ұ෦ͷಛ௃ྔ͸ڞ௨͍ͯͯ͠ɺਫ਼౓ʹߩݙ͍ͯ͠Δ
    ▸ 5SJQMFU
    MPTTʹΑΓ$//͕ٻΊΔಛ௃ྔ͕UBTL
    TQFDJpDʹͳ͍ͬͯΔ 
24. © 2017 VASILY,Inc. ௥ࢼ ▸ ஶऀ࣮૷͕ެ։͞Ε͍ͯΔͷͰखݩͰಈ͔͠·ͨ͠ɻ
    ▸ ࿦จͱͷҧ͍
    ▸ όοναΠζɿˠʢϝϞϦ͕଍Γͳ͔ͬͨͨΊʣ
    ▸ 
    USJQMFU
     
    ▸ ਫ਼౓ʢ࿦จͰ͸ʣ
    ▸ 
    WBMJEBUJPO
    EBUB
    FQPDI
    
    ▸ 
    UFTU
    EBUB 
25. © 2017 VASILY,Inc. ϚεΫ ▸ ΄΅ݻఆϚεΫ
     ۺͷछྨ
 ϒʔπɺεϦοϙϯͳͲ
     ۺͷకΊํ
     ର৅ͱͳΔੑผ
     ώʔϧͷߴ͞ 
26. © 2017 VASILY,Inc. ৚݅͝ͱͷಛ௃ྔ ▸ ҎԼͷσʔλͷࢄ෍ਤΛ࡞੒͠·ͨ͠ɻ
    ▸ ֶशσʔλʹؚ·Ε͍ͯͳ͍σʔλ
    ▸ WBMJEBUJPO
    EBUBTFU
    ▸ 63-
    IUUQTHJUIVCDPNDSDSQBSDPOEJUJPOBM@TJNJMBSJUZ@OFUXPSLT@QZUPSDIUSFF NBTUFSWJTVBMJ[BUJPO 
27. © 2017 VASILY,Inc. ௥ࢼͷ݁Ռʹ͍ͭͯ ▸ ਫ਼౓΍ϚεΫͷॏΈ͸֤৚݅ͷσʔλ਺͕,ͱগ͠গͳ͔ͬͨͷ͔΋͠Εͳ͍
    ▸ ࿦จͰ͸,Ҏ্ͰֶशϚεΫ͕༏੎ʹͳ͍ͬͯͨͷͰɺ
 ݻఆϚεΫ΍৚݅͝ͱʹϞσϧΛֶशͯ͠ൺֱ͢Δඞཁ͕͋ΔʢGVUVSF
    XPSLʣ 
28. © 2017 VASILY,Inc. ৚݅ɿώʔϧͷߴ͞ ▸ WBMJEBUJPO
    EBUBTFUͷը૾Ͱ࣮ߦ
    ▸ ৭΍ܗͳͲ৚݅Ҏ֎ͷଐੑ͔Βͷ
 Өڹ͸ऑ͍ɻ 
29. © 2017 VASILY,Inc. ײ૝ ▸ Α͋͘Γͦ͏͕ͩෳࡶͳ՝୊ΛɺγϯϓϧͳΞϓϩʔνͰղܾ͍ͯ͠Δɻ
    ▸ ࣮૷ͱͯ͠͸&NCFE
    NBUSJYͱཁૉੵΛ௥Ճ͚ͨͩ͠ɻ
    ▸ ࣮ݧ͕ஸೡɻ
    ▸ ײ֮తʹ͸Θ͔͍͕ͬͯͨɺτϦϓϨοτϩεʹΑͬͯ$//͕நग़͢Δಛ௃ྔ͕
 ࿪ΊΒΕΔͷΛ࣮ݧతʹ͍ࣔͯ͠Δɻ
    ▸ ൣғΛߜΕ͹ɺඞཁͳಛ௃ྔϕΫτϧͷ࣍ݩ͸ͦ͜·Ͱ͍Βͳ͍ͷ͔΋͠Εͳ͍ɻ
    ▸ ࿦จͰ͸ʙ࣍ݩఔ౓ʹݟ͑Δɻ 
30. © 2017 VASILY,Inc. References  4JBNFTF
    /FUXPSLT
    B 4JBNFTF
    /FVSBM
    /FUXPSLT
    GPS
    0OFTIPU
    *NBHF
    3FDPHOJUJPO
    IUUQTXXXDTVUSPOUPDBdHLPDIpMFTNTDUIFTJTQEG
    C %JTDSJNJOBUJWF
    -FBSOJOH
    PG
    %FFQ
    $POWPMVUJPOBM
    'FBUVSF
    1PJOU
    %FTDSJQUPST
    IUUQIJDTXBTFEBBDKQdFTJNPQVCMJDBUJPOT 4JNP4FSSB*$$7QEG
     5SJQMFU
    MPTT
    B -FBSOJOH
    'JOFHSBJOFE
    *NBHF
    4JNJMBSJUZ
    XJUI
    %FFQ
    3BOLJOH
    IUUQTBSYJWPSHBCT
    C *O
    %FGFOTF
    PG
    UIF
    5SJQMFU
    -PTT
    GPS
    1FSTPO
    3F*EFOUJpDBUJPO
    IUUQTBSYJWPSHBCT
     *-473$
    DIBNQJPO
    NPEFMT
    B 4QFFEBDDVSBDZ
    USBEFP⒎T
    GPS
    NPEFSO
    DPOWPMVUJPOBM
    PCKFDU
    EFUFDUPST