Upgrade to Pro — share decks privately, control downloads, hide ads and more …

夏のトップカンファレンス論文読み会 / InnovationMeetup20170918csn_cvpr2k17

38fbb481037166a3e7facba1020b7a7c?s=47 Masaki Kozuki
September 14, 2017
1.2k

夏のトップカンファレンス論文読み会 / InnovationMeetup20170918csn_cvpr2k17

夏のトップカンファレンス論文読み会(2017/09/18)での発表資料です。

- connpass: https://abeja-innovation-meetup.connpass.com/event/63466/
- 著者実装: https://github.com/andreasveit/conditional-similarity-networks
- 散布図: https://github.com/crcrpar/conditional_similarity_networks_pytorch

38fbb481037166a3e7facba1020b7a7c?s=128

Masaki Kozuki

September 14, 2017
Tweet

Transcript

  1. © 2017 VASILY,Inc. Նͷ τοϓΧϯϑΝϨϯε ࿦จಡΈձ Conditional Similarity Networks (CVPR2017)

    2017/09/18 Mon.
  2. © 2017 VASILY,Inc. ໨࣍ ▸ ໨తʹ߹Θͤͯɺಛ௃ྔϕΫτϧͷۭؒΛ෼ׂ͢Δ ▸ ओு ▸ ैདྷͷྨࣅ౓ܭࢉʹ͍ͭͯ

    ▸ $POEJUJPOBM4JNJMBSJUZ/FUXPSLT ▸ ࣮ݧ݁Ռ ▸ ௥ࢼ ▸ ײ૝ ▸ 3FGFSFODFT 
  3. © 2017 VASILY,Inc. ಛ௃ྔϕΫτϧͷۭؒΛ෼ׂ͢Δ 

  4. © 2017 VASILY,Inc. ࿦จͷओு ▸ ʢඇʣྨࣅ౓ͱͯ͠ಛ௃ྔϕΫτϧಉ࢜ͷڑ཭͕࢖͑Δ͕ɺ
 ୯Ұͷج४ʹΑΔྨࣅ౓͔͠ܭࢉͰ͖ͳ͍ ▸ ྨࣅ౓ͷج४͸ෳ਺͋Δͷ͕Ұൠత ▸

    ʮҰຕͷը૾ʹෳ਺ͷଐੑ BUUS ʯͰ͋Δ͜ͱ͕ଟ͍ ▸ FH ۺͷଐੑɿछྨɺඥͷ༗ແɺώʔϧͷߴ͞ɺʜ ▸ ಛ௃ྔϕΫτϧʹϚεΫΛద༻͠ɺ
 ಛ௃ྔϕΫτϧͷۭؒΛ෼ׂ͢Δ͜ͱͰෳ਺ͷྨࣅ౓ܭࢉΛՄೳʹ͢Δ 
  5. © 2017 VASILY,Inc. දهʹؔͯ͠ ▸ ಛ௃ྔϕΫτϧɿ$//ͷग़ྗͳͲ ▸ ಛ௃ྔɿಛ௃ྔϕΫτϧͷཁૉ ▸ ৚݅ɺҙຯɺλεΫɿྨࣅ౓ܭࢉͷج४

    
  6. © 2017 VASILY,Inc. ྨࣅ౓ܭࢉͷطଘख๏(1/5) ▸ $//ͷதؒ૚ͷग़ྗΛಛ௃ྔϕΫτϧͱͯ͠ɺಛ௃ྔؒͷڑ཭Λྨࣅ౓ͱͯ͠ѻ͑Δ ▸ ❌ෳ਺ͷྨࣅ౓ΛҰͭͷۭؒͰ͸ѻ͑ͳ͍  CNN

    feature input feature space
  7. © 2017 VASILY,Inc. ྨࣅ౓ܭࢉͷطଘख๏(2/5) ▸ *NBHF/FU౳Ͱࣄલֶशͨ͠Ϟσϧͷશ݁߹૚ͷग़ྗͰ΋ྨࣅ౓ͷܭࢉࣗମ͸Ͱ͖Δ͕ɺ
 λεΫʹదͨ͠ಛ௃ྔϕΫτϧΛநग़͢ΔΑ͏ʹֶशΛ͢Δख๏͕͋Δ 4JBNFTF/FUXPSLT 5SJQMFU-PTT 

  8. © 2017 VASILY,Inc. ྨࣅ౓ܭࢉͷطଘख๏(3/5) ▸ 4JBNFTF/FUXPSLT<B><C> ▸ ࣅ͍ͯΔը૾ʹ͸ͳΔ΂͘ڑ཭͕ۙ͘ͳΔΑ͏ʹֶश͢Δ ▸ ࣅ͍ͯΔը૾ͷϖΞͰֶश͢Δ

     Image from γϞηϥ ΤυΨʔɹSiameseωοτϫʔΫϞσϧΛ༻͍ͨը૾ಛ௃ྔநग़
  9. © 2017 VASILY,Inc. ྨࣅ౓ܭࢉͷطଘख๏(4/5) ▸ 5SJQMFUMPTT<B><C> ▸ ࣅ͍ͯΔը૾͸ۙ͘ɺࣅ͍ͯͳ͍ը૾͸ԕ͘ͳΔΑ͏ʹֶश͢Δ ▸ ʡʢΫΤϦը૾ɺࣅ͍ͯΔը૾ɺࣅͯͳ͍ը૾ʣʡͷτϦϓϨοτͰֶश

    ▸ 4JBNFTF/FUΑΓॊೈ  query positive negative train d_p d_n L_triplet = max{0, d_n - d_p + h}, h: margin
  10. © 2017 VASILY,Inc. ྨࣅ౓ܭࢉͷطଘख๏(5/5) ▸ ʮࣅ͍ͯΔʯ͸ը૾ͷଐੑ͝ͱʹܭࢉͰ͖Δͱخ͍͠ ▸ ͭͷج४ʹ͖ͭɺͭͷϞσϧΛ༻ҙ͢Δͷ͸ޮ཰͕ѱ͍  ・

    ・ ・ model for notion1 model for notion2 ・ ・ ・ ・ ・ ・
  11. © 2017 VASILY,Inc. ▸ ͭͷϞσϧͰෳ਺ͷྨࣅܭࢉΛ࣮ݱ͢ΔͨΊʹϚεΫΛಋೖ ▸ ಛ௃ྔϕΫτϧʹϚεΫΛద༻ͯ͠ಛ௃ྔϕΫτϧͷۭؒΛ෼ղ͢Δɻ Conditional Similarity Networks

    (1/3)  特徴量空間 マスク 色空間 形空間
  12. © 2017 VASILY,Inc. Conditional Similarity Networks (2/3) - mask 

    ݻఆϚεΫ $4/pYFEEJTKPJOUNBTLT  " ಛ௃ྔϕΫτϧΛ౳෼͢Δ # ڞ༗͞ΕΔಛ௃ྔ͸ͳ͍  ֶशϚεΫ $4/MFBSOFENBTLT  " ֓೦ຖʹ༻͍Δಛ௃ྔͱରԠ͢ΔॏΈΛֶश͢Δ # ڞ༗͞ΕΔಛ௃ྔ͕͋Δ $ ॏΈ͸ਖ਼ن෼෍ͰॳظԽ  ᶃ ᶄ
  13. © 2017 VASILY,Inc. ▸ ϚεΫ͸ຒΊࠐΈߦྻͰ࣮૷Ͱ͖Δɻ ▸ ߦྻ͸ʢ৚݅਺ʣʷʢಛ௃ྔϕΫτϧͷ࣍ݩʣ Conditional Similarity Networks

    (3/3)  Similirarity Network (ResNet-18) ・・・ mask input feature condition specific feature condition specify
  14. © 2017 VASILY,Inc. ఆࣜԽ ▸ $POEJUJPOBM5SJQMFU-PTT ▸ ਖ਼ଇԽ߲ ▸ ಛ௃ྔநग़ʹରͯ͠

    ▸ ͓ؾ࣋ͪɿ௒ٿ໘্ʹຒΊࠐΈ͍ͨ ▸ ϚεΫʹରͯ͠ ▸ ͓ؾ࣋ͪɿεύʔεʹ͍ͨ͠ ▸ શମɿ  c: ৚݅ɺΧςΰϦ m_c: cʹରԠ͢ΔϚεΫ h: Ϛʔδϯ
  15. © 2017 VASILY,Inc. ࣮ݧ   U4/&ʹΑΔಛ௃ྔϕΫτϧͷՄࢹԽ  ֶशͨ͠ϚεΫͷಛ௃ 

    ਫ਼౓ൺֱ  σʔληοταΠζͷӨڹ  5SJQMFU-PTTΛ༻ֶ͍ͨश͕ಛ௃ྔϕΫτϧʹ༩͑ΔӨڹ
  16. © 2017 VASILY,Inc. Ϟσϧ ▸ ஶऀ࣮૷ɿIUUQTHJUIVCDPNBOESFBTWFJUDPOEJUJPOBMTJNJMBSJUZOFUXPSLT ▸ *NBHF/FUֶशࡁΈ3FT/FU<>ΛϕʔεʹɺҎԼͷϞσϧΛൺֱ 4UBOEBSE5SJQMFU/FUXPSLDPOEJUJPOΛ༻͍ͣʹτϦϓϨοτΛֶश 4FUPG4QFDJBMJ[FE5SJQMFU/FUXPSLTDPOEJUJPO͝ͱʹϞσϧΛֶश

    $4/pYFEKPJOUNBTLT $4/MFBSOFENBTLT 
  17. © 2017 VASILY,Inc. σʔληοτ ▸ 65;BQQPT, ▸ ۺͷը૾ ຕ ▸

    ࣍ͷͭͷج४ͰτϦϓϨοτΛ࡞੒  छྨɿϒʔπɺαϯμϧɺFUD  ੑผɿஉੑɺঁੑɺঁͷࢠɺஉͷࢠ  ώʔϧͷߴ͞ɿʙΠϯν  ด͡ํɿͻ΋ɺόοΫϧɺεϦοϙϯɺFUD ▸ ϒϥϯυ৘ใʢ෼ྨλεΫʹ࣮ݧʣ  image from http://vision.cs.utexas.edu/projects/finegrained/utzap50k/
  18. © 2017 VASILY,Inc. t-SNEʹΑΔಛ௃ྔϕΫτϧͷՄࢹԽ ▸ ʢ্ʣɿʮۺͷకΊํʯʹؔ͢Δۭؒ ▸ ʢԼʣɿʮۺͷछྨʯʹؔ͢Δۭؒ ▸ ৭΍ܗͷӨڹ͕΄΅ͳ͍

    ▸ શ͘֎ݟ͕ࣅ͍ͯͳͯ͘΋৚݅ʹैͬͯ
 ຒΊࠐ·Ε͍ͯΔ 
  19. © 2017 VASILY,Inc. ֶशͨ͠ϚεΫͷಛ௃ ▸ ֶशϚεΫ͕ʮૄʯ ▸ ৚݅͝ͱʹڞ௨͢Δಛ௃ྔ͕ଘࡏ͢Δ  ԣɿಛ௃ྔϕΫτϧͷ


    ɹɹΠϯσοΫε ॎɿॏΈ
  20. © 2017 VASILY,Inc. ਫ਼౓ൺֱ ▸ ςετσʔλͷτϦϓϨοτʹର͢Δਫ਼౓ ▸ τϦϓϨοτ Y Z

    [ ʹରͯ͠ɺZͱ[ͷͲͪΒ͕Yʹࣅ͍ͯΔ͔ 
  21. © 2017 VASILY,Inc. σʔληοταΠζͷӨڹ ▸ ʙ ɿݻఆϚεΫɾ৚݅͝ͱʹϞσϧ͕ਖ਼֬ ▸  ʙɿֶशϚεΫ͕༗ར

     ԣɿσʔλ਺ ॎɿޡΓ཰
  22. © 2017 VASILY,Inc. Triplet LossΛ༻ֶ͍ͨश͕ಛ௃ྔʹ༩͑ΔӨڹ ▸ సҠֶशͰͷਫ਼౓Ͱൺֱ ▸ ॏΈݻఆͷ$// 3FT/FU

    ʹશ݁߹૚Λ௥Ճ͠ϒϥϯυ༧ଌλεΫΛֶश ▸ ैདྷͷτϦϓϨοτֶश͕ྼ͍ͬͯΔ 
  23. © 2017 VASILY,Inc. ࣮ݧ·ͱΊ ▸ $4/ʹΑΓ৚݅ʹదͨ͠ಛ௃ྔϕΫτϧ͕ܭࢉͰ͖ΔΑ͏ʹͳ͍ͬͯΔ ▸ $4/MFBSOFENBTLͷֶश͸ΑΓଟ͘ͷσʔλ͕ඞཁʹͳΔ ▸ ҟͳΔج४Ͱ΋Ұ෦ͷಛ௃ྔ͸ڞ௨͍ͯͯ͠ɺਫ਼౓ʹߩݙ͍ͯ͠Δ

    ▸ 5SJQMFUMPTTʹΑΓ$//͕ٻΊΔಛ௃ྔ͕UBTLTQFDJpDʹͳ͍ͬͯΔ 
  24. © 2017 VASILY,Inc. ௥ࢼ ▸ ஶऀ࣮૷͕ެ։͞Ε͍ͯΔͷͰखݩͰಈ͔͠·ͨ͠ɻ ▸ ࿦จͱͷҧ͍ ▸ όοναΠζɿˠʢϝϞϦ͕଍Γͳ͔ͬͨͨΊʣ

    ▸ USJQMFU  ▸ ਫ਼౓ʢ࿦จͰ͸ʣ ▸  WBMJEBUJPOEBUB FQPDI  ▸  UFTUEBUB 
  25. © 2017 VASILY,Inc. ϚεΫ ▸ ΄΅ݻఆϚεΫ  ۺͷछྨ
 ϒʔπɺεϦοϙϯͳͲ 

    ۺͷకΊํ  ର৅ͱͳΔੑผ  ώʔϧͷߴ͞ 
  26. © 2017 VASILY,Inc. ৚݅͝ͱͷಛ௃ྔ ▸ ҎԼͷσʔλͷࢄ෍ਤΛ࡞੒͠·ͨ͠ɻ ▸ ֶशσʔλʹؚ·Ε͍ͯͳ͍σʔλ ▸ WBMJEBUJPOEBUBTFU

    ▸ 63-IUUQTHJUIVCDPNDSDSQBSDPOEJUJPOBM@TJNJMBSJUZ@OFUXPSLT@QZUPSDIUSFF NBTUFSWJTVBMJ[BUJPO 
  27. © 2017 VASILY,Inc. ௥ࢼͷ݁Ռʹ͍ͭͯ ▸ ਫ਼౓΍ϚεΫͷॏΈ͸֤৚݅ͷσʔλ਺͕,ͱগ͠গͳ͔ͬͨͷ͔΋͠Εͳ͍ ▸ ࿦จͰ͸,Ҏ্ͰֶशϚεΫ͕༏੎ʹͳ͍ͬͯͨͷͰɺ
 ݻఆϚεΫ΍৚݅͝ͱʹϞσϧΛֶशͯ͠ൺֱ͢Δඞཁ͕͋ΔʢGVUVSFXPSLʣ 

  28. © 2017 VASILY,Inc. ৚݅ɿώʔϧͷߴ͞ ▸ WBMJEBUJPOEBUBTFUͷը૾Ͱ࣮ߦ ▸ ৭΍ܗͳͲ৚݅Ҏ֎ͷଐੑ͔Βͷ
 Өڹ͸ऑ͍ɻ 

  29. © 2017 VASILY,Inc. ײ૝ ▸ Α͋͘Γͦ͏͕ͩෳࡶͳ՝୊ΛɺγϯϓϧͳΞϓϩʔνͰղܾ͍ͯ͠Δɻ ▸ ࣮૷ͱͯ͠͸&NCFENBUSJYͱཁૉੵΛ௥Ճ͚ͨͩ͠ɻ ▸ ࣮ݧ͕ஸೡɻ

    ▸ ײ֮తʹ͸Θ͔͍͕ͬͯͨɺτϦϓϨοτϩεʹΑͬͯ$//͕நग़͢Δಛ௃ྔ͕
 ࿪ΊΒΕΔͷΛ࣮ݧతʹ͍ࣔͯ͠Δɻ ▸ ൣғΛߜΕ͹ɺඞཁͳಛ௃ྔϕΫτϧͷ࣍ݩ͸ͦ͜·Ͱ͍Βͳ͍ͷ͔΋͠Εͳ͍ɻ ▸ ࿦จͰ͸ʙ࣍ݩఔ౓ʹݟ͑Δɻ 
  30. © 2017 VASILY,Inc. References  4JBNFTF/FUXPSLT B 4JBNFTF/FVSBM/FUXPSLTGPS0OFTIPU*NBHF3FDPHOJUJPOIUUQTXXXDTVUSPOUPDBdHLPDIpMFTNTDUIFTJTQEG C %JTDSJNJOBUJWF-FBSOJOHPG%FFQ$POWPMVUJPOBM'FBUVSF1PJOU%FTDSJQUPSTIUUQIJDTXBTFEBBDKQdFTJNPQVCMJDBUJPOT

    4JNP4FSSB*$$7QEG  5SJQMFUMPTT B -FBSOJOH'JOFHSBJOFE*NBHF4JNJMBSJUZXJUI%FFQ3BOLJOHIUUQTBSYJWPSHBCT C *O%FGFOTFPGUIF5SJQMFU-PTTGPS1FSTPO3F*EFOUJpDBUJPOIUUQTBSYJWPSHBCT  *-473$DIBNQJPONPEFMT B 4QFFEBDDVSBDZUSBEFP⒎TGPSNPEFSODPOWPMVUJPOBMPCKFDUEFUFDUPST