$30 off During Our Annual Pro Sale. View Details »

ファッションアイテムの類似画像検索を実装してみました/Fashion Tech Meetup #2 LT

tn1031
March 22, 2016

ファッションアイテムの類似画像検索を実装してみました/Fashion Tech Meetup #2 LT

2016/03/22
Fashion Tech Meetup #2

tn1031

March 22, 2016
Tweet

More Decks by tn1031

Other Decks in Technology

Transcript

  1. ϑΝογϣϯΞΠςϜͷ ྨࣅը૾ݕࡧΛ࣮૷ͯ͠Έ·ͨ͠ 2016/03/22 FASHION TECH MEETUP #2 Presented by @tn1031,

    VASILY Inc.
  2. 0. ࣗݾ঺հ ࣗݾ঺հ ▸ தଜ ୓ຏ / @tn1031 ▸ σʔλαΠΤϯςΟετ

    ▸ SIer(2೥൒) -> VASILY(3िؒ) ▸ ػցֶशΛઐ߈ ▸ SHIROBAKO͸ਓੜ 2 @tn1031 ਓ޻஌ೳ͸ɹɹɹɹɹ झຯͰᅂΉఔ౓ SHIROBAKOͷଚ͍ը૾
  3. 1. औΓ૊Έͷഎܠ ྨࣅը૾ݕࡧ͕͋Δͱྑ͍৔໘ ʮཉ͍͠ΞΠςϜ͸͋Δ͚Ͳɺߴͯ͘ख͕ग़ͳ͍ɻʯ ʮଥڠͯ͠ങͬͨޙʹɺࣗ෼͕ങͬͨ΋ͷΑΓ΋ྑ͍΋ͷ͕ݟ͔ͭΔɻʯ 3 ྨࣅը૾ݕࡧ͕͋Ε͹ ʮࣅͨΞΠςϜΛ୳͠·ΘΔख͕ؒল͚Δʂʯ ʮଥڠͤͣʹ೰Ή͜ͱ͕Ͱ͖Δʂʯ

  4. 2. ը૾ݕࡧʹ͍ͭͯ ը૾ݕࡧʹ͸ओʹ̎छྨ͋Γ·͢ ςΩετϕʔεͷݕࡧ ▸ Image meta search ▸ ը૾ʹ෇ਵ͢Δϝλσʔλ΍ɹ

    ςΩετΛར༻ͨ͠ݕࡧ 4 ը૾ϕʔεͷݕࡧ ▸ Content-based image retrieval (CBIR) ▸ ςΩετ৘ใΛ࢖Θͣɺը૾ͷಛ௃ (৭ɺܗঢ়ͳͲ)Λར༻ͨ͠ݕࡧ ը૾σʔλ ը૾σʔλ ςΩετσʔλ ࢖͑Δ৘ใ͸ɹ ը૾σʔλ͚ͩ
  5. 2. ը૾ݕࡧʹ͍ͭͯ ը૾ݕࡧʹ͸ओʹ̎छྨ͋Γ·͢ ςΩετϕʔεͷݕࡧ ▸ Image meta search ▸ ը૾ʹ෇ਵ͢Δϝλσʔλ΍ɹ

    ςΩετΛར༻ͨ͠ݕࡧ 5 ը૾ϕʔεͷݕࡧ ▸ Content-based image retrieval (CBIR) ▸ ςΩετ৘ใΛ࢖Θͣɺը૾ͷಛ௃ (৭ɺܗঢ়ͳͲ)Λར༻ͨ͠ݕࡧ ը૾σʔλ ը૾σʔλ ࢖͑Δ৘ใ͸ɹ ը૾σʔλ͚ͩ ςΩετσʔλ ࠓճ͸ͪ͜Βʹ௅ઓ
  6. 2. ը૾ݕࡧʹ͍ͭͯ ը૾ݕࡧ͸ѹॖͱڑ཭ܭࢉͰ͢ ը૾ݕࡧͷجຊతͳߟ͑ํ ▸ ͳΔ΂͘௿࣍ͷۭؒʹѹॖ͠ɺѹॖͨ͠ϕΫτϧͷڑ཭ʹج͍ͮͯྨࣅ౓Λఆٛ͢Δ ▸ ࣅ͍ͯΔը૾ಉ࢜ͷڑ཭͕ۙ͘ɺࣅ͍ͯͳ͍ը૾ͱͷڑ཭͕ԕ͘ͳΔΑ͏ʹѹॖ͢Δ 6 ಛ௃ྔۭؒ

    f(x) ѹॖ ͍ۙ(ࣅ͍ͯΔ) ԕ͍(ࣅ͍ͯͳ͍) ը૾σʔλ ॎԣ480pixelͷ৔߹ɺ࣍ݩ਺͸ 480x480x3 = 691200 dim ը૾ಛ௃ྔ ը૾σʔλΛදݱ͢Δ௿࣍ͷϕΫτϧ ը૾Λѹॖ(=ಛ௃நग़)͢Δؔ਺Λ Ͳͷ༷ʹઃܭ͢Δ͔͕େࣄ
  7. 3. ྨࣅը૾ݕࡧ CBIRΛࢼͯ͠Έ·ͨ͠ 7 3௨Γͷํ๏Ͱ࣮૷ 1. Color histogram + Histogram

    of oriented gradients (HOG) - ίϯϐϡʔλϏδϣϯ෼໺ͷ఻౷తͳಛ௃நग़ํ๏ 2. Convolutional Neural Network (CNN) based model - σΟʔϓϥʔχϯά(ࣝผϞσϧ)ʹΑΔಛ௃நग़ 3. Deep Convolutional Generative Adversarial Networks (DCGAN) - σΟʔϓϥʔχϯά(ੜ੒Ϟσϧ)ʹΑΔಛ௃நग़
  8. 3. ྨࣅը૾ݕࡧ > 3.1. COLOR HISTOGRAM + HOG 1. COLOR

    HISTOGRAM + HOG ▸ ը૾ͷHSV஋ΛώετάϥϜԽ ▸ ը૾ͷً౓ޯ഑ΛώετάϥϜԽ ▸ 2छྨͷώετάϥϜΛ݁߹ͯ͠ը૾ͷಛ௃ྔͱ͢Δ 8 HSV஋நग़ άϨʔɹɹ εέʔϧ ৭৘ใώετάϥϜ ޯ഑৘ใώετάϥϜ ը૾ಛ௃ྔ ޯ഑நग़
  9. 3. ྨࣅը૾ݕࡧ > 3.1. COLOR HISTOGRAM + HOG 1. COLOR

    HISTOGRAM + HOG 9 ←ΫΤϦը૾ ݕࡧ݁Ռ ↓ ←ΫΤϦը૾ ݕࡧ݁Ռ ↓
  10. 3. ྨࣅը૾ݕࡧ > 3.2. CNN BASED MODEL 2. CNN BASED

    MODEL ▸ CNNΛimage netͰֶशͤ͞Δ ▸ ֶशࡁΈCNNʹΞΠςϜը૾ͱΧςΰϦϥϕϧΛ౤ೖͯ͠࠶ֶशͤ͞Δ ▸ શ݁߹૚ͷग़ྗΛը૾ಛ௃ྔͱ͢Δ 10 CNN શ݁߹૚ 4096ϊʔυ જࡏ૚ 64ϊʔυ ग़ྗ૚ 7ϊʔυ ΧςΰϦɹ ༧ଌ ը૾ಛ௃ྔ ݕࡧ࣌ͷڑ཭ܭࢉʹ࢖༻ ը૾ͷϋογϡ஋ ݕࡧର৅ͷߜࠐʹ࢖༻ ̍̍̌ɾɾ̍̌
  11. 3. ྨࣅը૾ݕࡧ > 3.2. CNN BASED MODEL 2. CNN BASED

    MODEL 11 ←ΫΤϦը૾ ݕࡧ݁Ռ ↓ ←ΫΤϦը૾ ݕࡧ݁Ռ ↓
  12. 3. ྨࣅը૾ݕࡧ > 3.3. DCGAN 3. DCGAN ▸ DCGANͰGeneratorͱDiscriminatorͷֶशΛߦ͏ ▸

    ֶशࡁΈGeneratorΛ༻͍ͯVectorizerͷֶशΛߦ͏ ▸ ֶशࡁΈVectorizerΛ༻͍ͯը૾Λ100࣍ݩͷϕΫτϧʹม׵͢Δ 12 DCGAN DISCRIPTOR GENERATOR TRAINED DISCRIPTOR TRAINED GENERATOR TRAINED GENERATOR VECTORIZER 100࣍ݩ ϕΫτϧ(ཚ਺) ը૾ੜ੒(ِ෺) TRAINEDɹ VECTORIZER ΞΠςϜը૾ 100࣍ݩ ϕΫτϧ 100࣍ݩ ϕΫτϧ ↓ ը૾ಛ௃ྔ Ϟσϧֶश ಛ௃நग़
  13. 3. ྨࣅը૾ݕࡧ > 3.3. DCGAN 3. DCGAN 13 DCGAN DISCRIPTOR

    GENERATOR TRAINED DISCRIPTOR TRAINED GENERATOR TRAINED GENERATOR VECTORIZER 100࣍ݩ ϕΫτϧ(ཚ਺) ը૾ੜ੒(ِ෺) TRAINEDɹ VECTORIZER ΞΠςϜը૾ 100࣍ݩ ϕΫτϧ 100࣍ݩ ϕΫτϧ ↓ ը૾ಛ௃ྔ Ϟσϧֶश ಛ௃நग़ ฐࣾςοΫϒϩάͰ΋·ͱΊ͍ͯ·͢ http://tech.vasily.jp/entry/fashion-deep-learning
  14. 3. ྨࣅը૾ݕࡧ > 3.3. DCGAN 3. DCGAN 14 ←ΫΤϦը૾ ݕࡧ݁Ռ

    ↓ ←ΫΤϦը૾ ݕࡧ݁Ռ ↓
  15. 3. ྨࣅը૾ݕࡧ > 3.4. ֤छ๏ͷൺֱ ࢖ͬͯΈͨײ૝ 15 COLOR HISTOGRAM +

    HOG CNN BASED MODEL DCGAN ख๏ ϝϦοτ σϝϦοτ ݕࡧ݁Ռͷ੍ޚ͕؆୯ લॲཧ͕େม ѹॖ཰͕ѱ͍ લॲཧָ͕ ϋογϡΛར༻ͨ͠ݕࡧ ඞཁͳ৘ใֶ͕शͷաఔͰ མͪΔ͜ͱ͕͋Δ લॲཧָ͕ ѹॖ཰͕ྑ͍ ݕࡧ݁Ռͷ੍ޚ͕ҋ
  16. 4. ·ͱΊͱࠓޙͷ՝୊ ·ͱΊ ▸ ྨࣅը૾ݕࡧػೳΛ࣮૷ͨ͠ - ݁Ռʹख๏ͷݸੑ͕ݟΕͯ໘ന͍ 16 ࠓޙͷ՝୊ ▸

    ݕࡧ଎౓޲্ - ॠ࣌ʹݕࡧ݁Ռ͕ฦͬͯ͜ͳ͍ͱ࢖͑ͳ͍ ▸ αʔϏεΛݟਾ͑ͨվળ - Ϣʔβ͕ຊ౰ʹݟ͍ͨ৘ใɺཉ͍͠ػೳ͸Կ͔
  17. ͝ਗ਼ௌ ͋Γ͕ͱ͏͍͟͝·ͨ͠ We are hiring !! ڵຯͷ͋ΔํͷೖࣾΛ͓଴͓ͪͯ͠Γ·͢ʂʂ

  18. ςΩετ ࢀߟ ▸ HoG - http://www.vision.cs.chubu.ac.jp/joint_hog/pdf/HOG +Boosting_LN.pdf ▸ CNN based

    model - http://www.iis.sinica.edu.tw/papers/song/18378-F.pdf ▸ DCGAN - http://arxiv.org/abs/1511.06434 - http://tech.vasily.jp/entry/fashion-deep-learning 18