Upgrade to Pro — share decks privately, control downloads, hide ads and more …

ファッションアイテム検索における深層学習の活用/Fashion Item Retrieval using Deep Learning

tn1031
June 25, 2017

ファッションアイテム検索における深層学習の活用/Fashion Item Retrieval using Deep Learning

tn1031

June 25, 2017
Tweet

More Decks by tn1031

Other Decks in Technology

Transcript

  1. © 2017 VASILY,Inc. ࣗݾ঺հ ‣ தଜ୓ຏ!UO ‣ σʔλαΠΤϯςΟετ ‣ ུྺɿ

      ΞΫηϯνϡΞ  ݱࡏ 7"4*-: ‣ ۀ຿ɿ  ػցֶशɾը૾ೝࣝ  ΞϧΰϦζϜͷ։ൃ  αʔϏε΁ͷద༻ 
  2. © 2017 VASILY,Inc. ໊ࣾגࣜձࣾ7"4*-: ϰΝγϦʔ 7"4*-: *OD ઃཱ೥݄ ॴࡏ஍౦ژ౎඼઒۠੢ޒ൓ాδχΞεϏϧ' ैۀһ໊

    ࢿຊۚԯ
 ୅දऔక໾ۚࢁ༟थ
 औక໾ࠓଜխ޾ઍ༿େี
 ओཁגओ άϩʔϏεϕϯνϟʔΩϟϐλϧ ҏ౻஧ςΫϊϩδʔϕϯνϟʔζ (.0ϕϯνϟʔύʔτφʔζ ,%%*גࣜձࣾ גࣜձࣾߨஊࣾ
  3. © 2017 VASILY,Inc. Google Play ϕετΞϓϦ J04ΞϓϦ ɾ"QQ4UPSF#&45 ɾ&TTFOUJBMೝఆ "OESPJEΞϓϦ

    ɾ೥(PPHMF1MBZϕετΞϓϦ ɾ೥(PPHMF1MBZϕετΞϓϦ ɾ೥(PPHMF1MBZϕετΞϓϦ
 ɹɹϕετΠϊϕʔςΟϒΞϓϦେ৆ ɾ(PPHMF͔Βτοϓσϕϩούʔೝఆ ೥࿈ଓϕετΞϓϦड৆͸ ੈքͰ7"4*-:͚ͩ
  4. © 2017 VASILY,Inc. ໨࣍  ϑΝογϣϯΞΠςϜݕࡧͱը૾ೝࣝ  ΞΠςϜݕࡧʹ͓͚Δը૾ೝࣝͷՁ஋  ϑΝογϣϯ෼໺Ͱѻ͏ը૾

     ϑΝογϣϯΞΠςϜͷݕࡧͱਂ૚ֶश  ਂ૚ֶशͷ׆༻ࣄྫ  ࠓޙͷऔΓ૊Έ ༧ఆ 
  5. © 2017 VASILY,Inc. ΞΠςϜݕࡧʹ͓͚Δը૾ೝࣝͷՁ஋ ͜ͷͭͷΞΠςϜͷ۠ผʹد༩͢Δ৘ใ͸Կ͔ ʻ.VOJDIʼϒϥ΢εʢ./5ʣ =  ϒϥϯυɿϛϡχοΫʦ.6/*$)ʧ ΧςΰϦɿϒϥ΢ε

    ৭ɿ྘ʢάϦʔϯʣ ΞΠςϜઆ໌ɿ Ϟʔυͳண͜ͳ͠Λ׎͑ΔϘϦϡʔϜεϦʔϒͷϒϥ΢εͰ͢ɻ କΛંΓฦͯ͠௚ઢతͳΧοτΛڧௐ͢Δ͜ͱͰɺ०ͷσΟςΠ ϧ΋ͻͱຯҧ͏ݟ͑ํʹɻγϯϓϧͳ਎ࠒͱͷόϥϯε΋ྑ޷Ͱ ͢ɻγϫʹͳΓʹ͍͘৳ॖૉࡐΛ࢖͓ͬͯΓɺσΠϦʔʹؾܰʹ औΓೖΕΒΕΔͷ͕ϙΠϯτɻ͖ͬ͢Γͱ։͍ͨ7ωοΫ͕େਓ ͷঁੑΒ͠͞Λԋग़͠·͢ɻ <8&#ݶఆ঎඼><΢Υογϟϒϧ>ϑϨΞεϦʔϒϒϥ΢ε =  ϒϥϯυɿΦΫτʔϒϧʦ0$50#3&ʧ ΧςΰϦɿϒϥ΢ε ৭ɿ྘ʢάϦʔϯʣ ΞΠςϜઆ໌ɿ τϨϯυΛҙࣝͨ͠ϑϨΞεϦʔϒτοϓεɻ͖ͬ͢Γͱൈ͚ײ ͷ͋ΔωοΫϥΠϯ΍όοΫγϣοτ͕ঁੑΒ͍͠งғؾͮ͘Γ ͷϙΠϯτɻ্඼͞΋͓͠ΌΕײ΋ซͤ࣋ͬͨҰ඼͸ɺͲΜͳੈ ୅ͷํ͔Β΋޷ධͰ͢ɻΦϑΟε΋σʔτ΋͓೚ͤͷσβΠϯͰ ͢ɻ ςΩετͰൺֱͨ͠৔߹ 
  6. © 2017 VASILY,Inc. ΞΠςϜݕࡧʹ͓͚Δը૾ೝࣝͷՁ஋ ͜ͷͭͷΞΠςϜͷ۠ผʹد༩͢Δ৘ใ͸Կ͔ ʻ.VOJDIʼϒϥ΢εʢ./5ʣ =  ϒϥϯυɿϛϡχοΫʦ.6/*$)ʧ ΧςΰϦɿϒϥ΢ε

    ৭ɿ྘ʢάϦʔϯʣ ΞΠςϜઆ໌ɿ Ϟʔυͳண͜ͳ͠Λ׎͑ΔϘϦϡʔϜεϦʔϒͷϒϥ΢εͰ͢ɻ କΛંΓฦͯ͠௚ઢతͳΧοτΛڧௐ͢Δ͜ͱͰɺ०ͷσΟςΠ ϧ΋ͻͱຯҧ͏ݟ͑ํʹɻγϯϓϧͳ਎ࠒͱͷόϥϯε΋ྑ޷Ͱ ͢ɻγϫʹͳΓʹ͍͘৳ॖૉࡐΛ࢖͓ͬͯΓɺσΠϦʔʹؾܰʹ औΓೖΕΒΕΔͷ͕ϙΠϯτɻ͖ͬ͢Γͱ։͍ͨ7ωοΫ͕େਓ ͷঁੑΒ͠͞Λԋग़͠·͢ɻ <8&#ݶఆ঎඼><΢Υογϟϒϧ>ϑϨΞεϦʔϒϒϥ΢ε =  ϒϥϯυɿΦΫτʔϒϧʦ0$50#3&ʧ ΧςΰϦɿϒϥ΢ε ৭ɿ྘ʢάϦʔϯʣ ΞΠςϜઆ໌ɿ τϨϯυΛҙࣝͨ͠ϑϨΞεϦʔϒτοϓεɻ͖ͬ͢Γͱൈ͚ײ ͷ͋ΔωοΫϥΠϯ΍όοΫγϣοτ͕ঁੑΒ͍͠งғؾͮ͘Γ ͷϙΠϯτɻ্඼͞΋͓͠ΌΕײ΋ซͤ࣋ͬͨҰ඼͸ɺͲΜͳੈ ୅ͷํ͔Β΋޷ධͰ͢ɻΦϑΟε΋σʔτ΋͓೚ͤͷσβΠϯͰ ͢ɻ ςΩετͰൺֱͨ͠৔߹ କʹؔ͢Δઆ໌ ϑϨΞεϦʔϒ WTϘϦϡʔϜεϦʔϒʁ ۗʹؔ͢Δઆ໌ ൈ͚ײͷ͋ΔωοΫϥΠϯͱ͸ʁ 
  7. © 2017 VASILY,Inc. ΞΠςϜݕࡧʹ͓͚Δը૾ೝࣝͷՁ஋ ΞΠςϜͷྨࣅ౓ΛධՁ͢Δ্Ͱը૾͸͔ܽͤͳ͍৘ใͰ͋Δ ‣ ΞΠςϜͷڞ௨఺΍ࠩ෼͕೺ѲͰ͖Δ  ςΩετʹൺ΂ؚͯ·ΕΔ৘ใͷྔ͕ଟ͍ ‣

    ΞΠςϜͷը૾͸ίϯςΩετΛؚΜͰ͍Δ  ը૾ΛݟΔ͚ͩͰར༻γʔϯ΍ίʔσΟωʔτ͕૝૾Ͱ͖Δࣄ͕͋Δ ‣ ϑΝογϣϯ༻ޠͷࣙॻ΍Φϯτϩδʔ͕ඞཁͳ͍  ϑΝογϣϯ͸ྲྀߦʹහײͳ෼໺ͷͨΊɺࣙॻͷϝϯςͷ޻਺Λ࡟ݮ Ͱ͖Δ͜ͱͷϝϦοτ͸ඇৗʹେ͖͍ ෳࡶͳ৘ใΛநग़Մೳͳը૾ೝٕࣝज़͸ϑΝογϣϯͱ૬ੑ͕ྑ͍ 
  8. © 2017 VASILY,Inc. ϑΝογϣϯ෼໺Ͱѻ͏ը૾ ঎඼ը૾ͱεφοϓը૾ͱ͍͏छྨͷυϝΠϯ͕ଘࡏ͠ɺ༻్͕ҟͳΔ ঎඼ը૾ εφοϓը૾ ‣ ஔ͖ࡱΓ΍ϚωΩϯͷը૾͕ ଟ͍

    ‣ ਖ਼໘͔Βݟͨ࣌ͷσβΠϯ͕ Θ͔Γ΍͍͢ ‣ ண༻Πϝʔδ͕༙͖ʹ͍͘ ‣ Ϟσϧ͕ண༻ͨ͠ը૾ ‣ ண༻࣌ͷҹ৅΍߹Θͤͷɹ ࢀߟʹͳΔ ‣ ϙʔζ΍എܠʹ΋޻෉Λɹ ڽΒ͍ͯ͠Δ ѻ͏ը૾ʹԠͯ͡ΞϓϩʔνΛม͑Δඞཁ͕͋Δ 
  9. © 2017 VASILY,Inc. ϑΝογϣϯΞΠςϜͷݕࡧͱਂ૚ֶश ‣ ༷ʑͳυϝΠϯͷը૾Λಉ͡ΞʔΩςΫνϟͰѻ͑Δ  ΞϓϦέʔγϣϯશମΛγϯϓϧʹอͯΔ ‣ ಛ௃நग़ΛࣗಈԽͰ͖Δ

     ϑΝογϣϯಛ༗ͷෳࡶͳίϯςΩετͷநग़͕௿ίετͰ࣮ݱͰ͖Δ ‣ 4JEFJOGPSNBUJPOͷར༻͕؆୯  ΧςΰϦ΍ଐੑͷ৚݅෇͖Ϟσϧ͕؆୯ʹఆٛͰ͖Δ ‣ Ϟσϧͷֶशʹॆ෼ͳྔͷσʔλ͕ଘࡏ͢Δ  ϑΝογϣϯΞΠςϜ͸೔ʑ૿͑ଓ͚αϯϓϧ਺ͷෆ଍ʹࠔΔ͜ͱ͸ͳ͍  ϑΝογϣϯ෼໺ʹ͓͚Δը૾ೝࣝͱਂ૚ֶश͸਌࿨ੑ͕ߴ͍
  10. © 2017 VASILY,Inc. ໨࣍  ϑΝογϣϯΞΠςϜݕࡧͱը૾ೝࣝ  ਂ૚ֶशͷ׆༻ࣄྫ  *UFN*UFNݕࡧ

     4OBQ*UFNݕࡧ  *UFN4OBQݕࡧ  ײ֮తͳΞΠςϜݕࡧ  ࠓޙͷऔΓ૊Έ ༧ఆ 
  11. © 2017 VASILY,Inc. ਂ૚ֶशͷ׆༻ࣄྫ  ݕࡧΫΤϦͱݕࡧର৅ʹԠ࣮ͯ͡ݱํ๏͕ҟͳΔ *UFN*UFNݕࡧ ݕࡧର৅ ࣮ݱํ๏ ঎඼ը૾

    ঎඼ը૾ ଐੑ ঎඼ը૾ εφοϓը૾ ঎඼ը૾ ঎඼ը૾ εφοϓը૾ ঎඼ը૾ "VUPFODPEFS 44%  4JNJMBSJUZMFBSOJOH 7"& ("/ 4OBQ*UFNݕࡧ *UFN4OBQݕࡧ ײ֮తͳɹɹɹ ΞΠςϜݕࡧ ݕࡧΫΤϦ
  12. © 2017 VASILY,Inc. *UFN*UFNݕࡧ "VUPFODPEFSͰநग़ͨ͠ಛ௃ʹରͯۙ͠๣୳ࡧΛ࣮ߦ͢Δ  &ODPEFS %FDPEFS ಛ௃ྔ ݕ

    ࡧ ֶ श &ODPEFS ‣ ೖྗը૾Λ෮ݩ͢ΔΑ͏ʹɹ ֶश ‣ தؒ૚ʹ͸෮ݩʹඞཁͳ৘ใ ͕อଘ͞ΕΔ ‣ &ODPEFSΛ௨ͯ͠ը૾Λಛ ௃ྔʹม׵͢Δ ‣ ಛ௃্ۭؒͰۙ๣୳ࡧΛ࣮ߦ ͯ͠ྨࣅΞΠςϜΛݕࡧ͢Δ
  13. © 2017 VASILY,Inc. 4OBQ*UFNݕࡧ 44%ͰΞΠςϜʹ֘౰͢ΔྖҬΛݕग़ɺநग़ͨ͠ಛ௃ྔΛ࢖ͬͯྨࣅ౓Λֶश  44% tops pants bags

    Anchor Positive Negative CNN CNN CNN ֶ श ݕ ग़ ‣ 44%Λ࢖ͬͯεφοϓը૾Λ ΧςΰϦͰύʔε͢Δ ‣ ύʔεͨͦ͠ΕͧΕͷྖҬΛ ݕࡧΫΤϦͱΈͳ͢ ‣ ҟͳΔυϝΠϯͷը૾Λڞ௨ ͷωοτϫʔΫͰຒΊࠐΉ ‣ εφοϓը૾༝དྷͷಛ௃ྔͷ ·ΘΓʹͦͷ঎඼ը૾͕ɹɹ Ϛοϐϯά͞ΕΔΑ͏ʹֶश ͢Δ
  14. © 2017 VASILY,Inc. ײ֮తͳΞΠςϜݕࡧ ঎඼ը૾ʹଐੑΛ෇Ճͯ͠໨తͷΞΠςϜΛݕࡧ͢Δ  ը૾ ଐੑʢϕΫτϧʣ Red Flare

    Maxi ︙ ‣ ঎඼ը૾ʹଐੑΛՃࢉݮࢉ͠ͳ͕Β୳͢ ‣ ଐੑͷԋࢉ͸ΠϯλϥΫςΟϒʹߦ͑Δ ‣ ݕࡧΫΤϦͷݴޠԽ͕೉͍͠ͱ͖ʹ༗ޮ ֓ཁ ‣ 7"& ‣ ("/ ‣ ۙࣅۙ๣୳ࡧ ख๏
  15. © 2017 VASILY,Inc. ײ֮తͳΞΠςϜݕࡧ 7"&ͱ("/Λ૊Έ߹ΘͤͨϞσϧΛར༻ͯ͠ಛ௃ۭؒΛுΔ  Encoder Decoder Discriminator ը૾

    ଐੑ ը૾ ը૾ɹɹɹ ಛ௃ྔ ༧ଌʢຊ෺/ِ෺ʣ ༧ଌʢଐੑʣ ը૾ʢ෮ݩʣ ‣ ଐੑʹΑΓ৚݅෇͚ͨ7"&Ͱಛ௃Λநग़͢Δ ‣ %JTDSJNJOBUPS͸ೖྗը૾ͷਅِ༧ଌʹՃ͑ͯଐੑ΋༧ଌ͢Δ ‣ ͋Δଐੑʹ஫໨ͨ࣌͠ɺଐੑͷڧ͞ʹԠͯ͡نଇਖ਼͘͠ αϯϓϧ͕഑ஔ͞Ε͍ͯΔঢ়ଶΛ໨ࢦ͢ ‣ ଐੑͷڧ͞ΛมԽͤͯ͞ݕࡧΛ࣮ݱ͢Δ
  16. © 2017 VASILY,Inc. ໨࣍  ϑΝογϣϯΞΠςϜݕࡧͱը૾ೝࣝ  ਂ૚ֶशͷ׆༻ࣄྫ  ࠓޙͷऔΓ૊Έ

    ༧ఆ   ײ֮తͳΞΠςϜݕࡧͷߴ౓Խ  ίʔσΟωʔτͷࣗಈੜ੒ 
  17. © 2017 VASILY,Inc. ײ֮తͳΞΠςϜݕࡧͷߴ౓Խ  5SJQMFUMPTTʹΑΔྨࣅ౓ֶश͸খ͞ͳվળΛੵΈ্͛Δඞཁ͕͋Δ ਂ͍ωοτϫʔΫ ઙ͍ωοτϫʔΫ ‣ ਂ͍Ϟσϧ͸৭ͳͲͷ୯७ͳߏ଄͕ফࣦ͢Δ͜ͱ͕͋Δ

    ‣ ઙ͍ύεΛ௥Ճ͢Δ͜ͱͰ୯७ͳߏ଄ͷফࣦΛ๷ࢭ ઙ͍ύεͷ௥Ճ "ODIPS1PTJUJWF/FHBUJWFͷҐஔؔ܎ʹԠͨ͡ਖ਼ଇԽ Embedding Embedding margin Anchor Positive Negative Embedding margin ظ଴͢Δ ঢ়ଶ ๬·͘͠ ͳ͍ঢ়ଶ ֶश ‣ 1PTJUJWF/FHBUJWFͷҐஔʹରͯ͠"ODIPS͕ԕ͗͢Δ ‣ ఺͕ద੾ͳҐஔʹϚοϐϯά͞ΕΔΑ͏ʹേଇΛ՝͢
  18. © 2017 VASILY,Inc. ίʔσΟωʔτͷࣗಈੜ੒ ‣ τϐοΫϞσϧʹΑΔΞϓϩʔν ‣ 5PQTͱ#PUUPNTͦΕͧΕͷτϐοΫΛԾఆ͠ɺ୯ҰͷϞσϧͰಉ࣌ʹѻ͏  5PQTY#PUUPNTͷ૊Έ߹Θͤ͸ઌߦݚڀ͕͋Δ͕ɺ఺Ҏ্͸੒ޭྫ͕গͳ͍

    'BTIJPO$PPSEJOBUFT3FDPNNFOEFS4ZTUFNVTJOH1IPUPHSBQITGSPN'BTIJPO.BHB[JOFT ‣ ঎඼ը૾͔Β$//Λ༻͍ͯಛ௃நग़ ‣ 4JBNFTFOFUXPSLͰڞىؔ܎Λֶश -FBSOJOH7JTVBM$MPUIJOH4UZMFXJUI)FUFSPHFOFPVT%ZBEJD$PPDDVSSFODFT
  19. © 2017 VASILY,Inc. ίʔσΟωʔτͷࣗಈੜ੒  ίʔσΟωʔτΛֶश͠Α͏ͱͳΔͱ՝୊͕ଟ͍ ‣ ੜ੒͞ΕͨίʔσΟωʔτͷධՁ͕೉͍͠  ίʔσΟωʔτΛఆྔతʹධՁ͢Δ͜ͱ͸Ͱ͖ͳ͍

     قઅ΍ྲྀߦͷӨڹΛड͚Δ ‣ ΞΠςϜͷ૊Έ߹Θ͔ͤΒͳΔσʔλ͸ඇৗʹ4QBSTF  ϑΝογϣϯΞΠςϜ͸ଟछଟ༷ͳͷͰؔ܎σʔλͱΈͳ͢ͱඇৗʹ4QBSTFʹͳΔ  ը૾ಛ௃ྔΛར༻ͯ͠%FOTFʹอͭ ‣ ίϯςΩετΛͲͷ༷ʹѻ͏͔  ίʔσΟωʔτʹ͸ελΠϧͷ֓೦͕ଘࡏ͢Δ  τϐοΫϞσϧ΍.FNPSZ/FUXPSLͷΑ͏ʹந৅౓ͷߴ͍৘ใΛѻ͑ΔΞʔΩςΫνϟ͕ඞཁʹͳΔͱࢥΘΕΔ
  20. © 2017 VASILY,Inc. ·ͱΊ  ϑΝογϣϯΞΠςϜݕࡧͱը૾ೝࣝ  ϑΝογϣϯ෼໺Ͱը૾ೝࣝ΍ਂ૚ֶशΛ࢖͏ҙٛʹ͍ͭͯ  ѻ͏ը૾σʔλͷυϝΠϯʹ͍ͭͯ

     ׆༻ࣄྫ  ୯७ͳྨࣅΞΠςϜݕࡧ͸"VUPFODPEFS  ΫϩευϝΠϯͳݕࡧ͸44% 4JNJMBSJUZMFBSOJOH  ײ֮తͳΞΠςϜݕࡧ͸ੜ੒Ϟσϧ͔ΒͷΞϓϩʔν  ࠓޙͷऔΓ૊Έ  ײ֮తͳΞΠςϜݕࡧͷߴ౓Խ  ίʔσΟωʔτͷੜ੒ͱ՝୊ 
  21. © 2017 VASILY,Inc. ࠓޙͷ׆ಈ  .*36Ͱൃද͠·͢ ‣ ೔࣌  

    ‣ ձ৔  ޿ౡࠃࡍձٞ৔ ‣ λΠτϧ  ʮτϦϓϨοτଛࣦؔ਺ͷॏΈ෇͚ʹΑΔֶशͷ ޮ཰Խʯ 14 IUUQDWJNJQTKPSKQ.*36
  22. © 2017 VASILY,Inc. ࢀߟ  17JODFOU )-BSPDIFMMF *-BKPJF :#FOHJP 1".BO[BHPM4UBDLFEEFOPUJOHBVUPFODPEFST-FBSOJOHVTFGVMSFQSFTFOUBUJPOTJOBEFFQ

    OFUXPSLXJUIBMPDBMEFOPUJOHDSJUFSJPO*O1SPD*$.-   8-JV %"OHVFMPW %&SIBO $4[FHFEZ 43FFE44%4JOHMF4IPU.VMUJ#PY%FUFDUPSBS9JW   +8BOH :4POH 5-FVOH $3PTFOCFSH +8BOH +1IJMCJO #$IFO :8V-FBSOJOH'JOF(SBJOFE*NBHF4JNJMBSJUZXJUI%FFQ3BOLJOH *O1SPD$713   "#--BSTFO 4,4”OEFSCZ 08JOUIFS"VUPFODPEJOHCFZPOEQJYFMTVTJOHBMFBSOFETJNJMBSJUZNFUSJDBS9JW   ";IBJ %,JTMZVL :+JOH .'FOH &5[FOH +%POBIVF :-%V 5%BSSFMM7JTVBMEJTDPWFSZBUQJOUFSFTU*O1SPD888   #;IBP +'FOH 98V 4:BO.FNPSZ"VHNFOUFE"UUSJCVUF.BOJQVMBUJPO/FUXPSLTGPS*OUFSBDUJWF'BTIJPO4FBSDI*O1SPD$713    5*XBUB 48BUBOBCF )4BXBEB'BTIJPO$PPSEJOBUFT3FDPNNFOEFS4ZTUFN6TJOH1IPUPHSBQITGSPN'BTIJPO.BHB[JOFT*O1SPD *+$"*   "7FJU #,PWBDT 4#FMM +.D"VMFZ ,#BMB 4#FMPOHJF-FBSOJOH7JTVBM$MPUIJOH4UZMFXJUI)FUFSPHFOFPVT%ZBEJD$PPDDVSSFODFT *O1SPD*$$7