ファッションアイテムの類似画像検索を実装してみました/Fashion Tech Meetup #2 LT

2016/03/22
Fashion Tech Meetup #2

Transcript

1. ϑΝογϣϯΞΠςϜͷ ྨࣅը૾ݕࡧΛ࣮૷ͯ͠Έ·ͨ͠ 2016/03/22 FASHION TECH MEETUP #2 Presented by @tn1031, VASILY Inc.
2. 0. ࣗݾ঺հ ࣗݾ঺հ ▸ தଜ ୓ຏ / @tn1031 ▸ σʔλαΠΤϯςΟετ ▸ SIer(2೥൒) -> VASILY(3िؒ) ▸ ػցֶशΛઐ߈ ▸ SHIROBAKO͸ਓੜ 2 @tn1031 ਓ޻஌ೳ͸ɹɹɹɹɹ झຯͰᅂΉఔ౓ SHIROBAKOͷଚ͍ը૾
3. 1. औΓ૊Έͷഎܠ ྨࣅը૾ݕࡧ͕͋Δͱྑ͍৔໘ ʮཉ͍͠ΞΠςϜ͸͋Δ͚Ͳɺߴͯ͘ख͕ग़ͳ͍ɻʯ ʮଥڠͯ͠ങͬͨޙʹɺࣗ෼͕ങͬͨ΋ͷΑΓ΋ྑ͍΋ͷ͕ݟ͔ͭΔɻʯ 3 ྨࣅը૾ݕࡧ͕͋Ε͹ ʮࣅͨΞΠςϜΛ୳͠·ΘΔख͕ؒল͚Δʂʯ ʮଥڠͤͣʹ೰Ή͜ͱ͕Ͱ͖Δʂʯ
4. 2. ը૾ݕࡧʹ͍ͭͯ ը૾ݕࡧʹ͸ओʹ̎छྨ͋Γ·͢ ςΩετϕʔεͷݕࡧ ▸ Image meta search ▸ ը૾ʹ෇ਵ͢Δϝλσʔλ΍ɹ ςΩετΛར༻ͨ͠ݕࡧ 4 ը૾ϕʔεͷݕࡧ ▸ Content-based image retrieval (CBIR) ▸ ςΩετ৘ใΛ࢖Θͣɺը૾ͷಛ௃ (৭ɺܗঢ়ͳͲ)Λར༻ͨ͠ݕࡧ ը૾σʔλ ը૾σʔλ ςΩετσʔλ ࢖͑Δ৘ใ͸ɹ ը૾σʔλ͚ͩ
5. 2. ը૾ݕࡧʹ͍ͭͯ ը૾ݕࡧʹ͸ओʹ̎छྨ͋Γ·͢ ςΩετϕʔεͷݕࡧ ▸ Image meta search ▸ ը૾ʹ෇ਵ͢Δϝλσʔλ΍ɹ ςΩετΛར༻ͨ͠ݕࡧ 5 ը૾ϕʔεͷݕࡧ ▸ Content-based image retrieval (CBIR) ▸ ςΩετ৘ใΛ࢖Θͣɺը૾ͷಛ௃ (৭ɺܗঢ়ͳͲ)Λར༻ͨ͠ݕࡧ ը૾σʔλ ը૾σʔλ ࢖͑Δ৘ใ͸ɹ ը૾σʔλ͚ͩ ςΩετσʔλ ࠓճ͸ͪ͜Βʹ௅ઓ
6. 2. ը૾ݕࡧʹ͍ͭͯ ը૾ݕࡧ͸ѹॖͱڑ཭ܭࢉͰ͢ ը૾ݕࡧͷجຊతͳߟ͑ํ ▸ ͳΔ΂͘௿࣍ͷۭؒʹѹॖ͠ɺѹॖͨ͠ϕΫτϧͷڑ཭ʹج͍ͮͯྨࣅ౓Λఆٛ͢Δ ▸ ࣅ͍ͯΔը૾ಉ࢜ͷڑ཭͕ۙ͘ɺࣅ͍ͯͳ͍ը૾ͱͷڑ཭͕ԕ͘ͳΔΑ͏ʹѹॖ͢Δ 6 ಛ௃ྔۭؒ f(x) ѹॖ ͍ۙ(ࣅ͍ͯΔ) ԕ͍(ࣅ͍ͯͳ͍) ը૾σʔλ ॎԣ480pixelͷ৔߹ɺ࣍ݩ਺͸ 480x480x3 = 691200 dim ը૾ಛ௃ྔ ը૾σʔλΛදݱ͢Δ௿࣍ͷϕΫτϧ ը૾Λѹॖ(=ಛ௃நग़)͢Δؔ਺Λ Ͳͷ༷ʹઃܭ͢Δ͔͕େࣄ
7. 3. ྨࣅը૾ݕࡧ CBIRΛࢼͯ͠Έ·ͨ͠ 7 3௨Γͷํ๏Ͱ࣮૷ 1. Color histogram + Histogram of oriented gradients (HOG) - ίϯϐϡʔλϏδϣϯ෼໺ͷ఻౷తͳಛ௃நग़ํ๏ 2. Convolutional Neural Network (CNN) based model - σΟʔϓϥʔχϯά(ࣝผϞσϧ)ʹΑΔಛ௃நग़ 3. Deep Convolutional Generative Adversarial Networks (DCGAN) - σΟʔϓϥʔχϯά(ੜ੒Ϟσϧ)ʹΑΔಛ௃நग़
8. 3. ྨࣅը૾ݕࡧ > 3.1. COLOR HISTOGRAM + HOG 1. COLOR HISTOGRAM + HOG ▸ ը૾ͷHSV஋ΛώετάϥϜԽ ▸ ը૾ͷً౓ޯ഑ΛώετάϥϜԽ ▸ 2छྨͷώετάϥϜΛ݁߹ͯ͠ը૾ͷಛ௃ྔͱ͢Δ 8 HSV஋நग़ άϨʔɹɹ εέʔϧ ৭৘ใώετάϥϜ ޯ഑৘ใώετάϥϜ ը૾ಛ௃ྔ ޯ഑நग़
9. 3. ྨࣅը૾ݕࡧ > 3.1. COLOR HISTOGRAM + HOG 1. COLOR HISTOGRAM + HOG 9 ←ΫΤϦը૾ ݕࡧ݁Ռ ↓ ←ΫΤϦը૾ ݕࡧ݁Ռ ↓
10. 3. ྨࣅը૾ݕࡧ > 3.2. CNN BASED MODEL 2. CNN BASED MODEL ▸ CNNΛimage netͰֶशͤ͞Δ ▸ ֶशࡁΈCNNʹΞΠςϜը૾ͱΧςΰϦϥϕϧΛ౤ೖͯ͠࠶ֶशͤ͞Δ ▸ શ݁߹૚ͷग़ྗΛը૾ಛ௃ྔͱ͢Δ 10 CNN શ݁߹૚ 4096ϊʔυ જࡏ૚ 64ϊʔυ ग़ྗ૚ 7ϊʔυ ΧςΰϦɹ ༧ଌ ը૾ಛ௃ྔ ݕࡧ࣌ͷڑ཭ܭࢉʹ࢖༻ ը૾ͷϋογϡ஋ ݕࡧର৅ͷߜࠐʹ࢖༻ ̍̍̌ɾɾ̍̌
11. 3. ྨࣅը૾ݕࡧ > 3.2. CNN BASED MODEL 2. CNN BASED MODEL 11 ←ΫΤϦը૾ ݕࡧ݁Ռ ↓ ←ΫΤϦը૾ ݕࡧ݁Ռ ↓
12. 3. ྨࣅը૾ݕࡧ > 3.3. DCGAN 3. DCGAN ▸ DCGANͰGeneratorͱDiscriminatorͷֶशΛߦ͏ ▸ ֶशࡁΈGeneratorΛ༻͍ͯVectorizerͷֶशΛߦ͏ ▸ ֶशࡁΈVectorizerΛ༻͍ͯը૾Λ100࣍ݩͷϕΫτϧʹม׵͢Δ 12 DCGAN DISCRIPTOR GENERATOR TRAINED DISCRIPTOR TRAINED GENERATOR TRAINED GENERATOR VECTORIZER 100࣍ݩ ϕΫτϧ(ཚ਺) ը૾ੜ੒(ِ෺) TRAINEDɹ VECTORIZER ΞΠςϜը૾ 100࣍ݩ ϕΫτϧ 100࣍ݩ ϕΫτϧ ↓ ը૾ಛ௃ྔ Ϟσϧֶश ಛ௃நग़
13. 3. ྨࣅը૾ݕࡧ > 3.3. DCGAN 3. DCGAN 13 DCGAN DISCRIPTOR GENERATOR TRAINED DISCRIPTOR TRAINED GENERATOR TRAINED GENERATOR VECTORIZER 100࣍ݩ ϕΫτϧ(ཚ਺) ը૾ੜ੒(ِ෺) TRAINEDɹ VECTORIZER ΞΠςϜը૾ 100࣍ݩ ϕΫτϧ 100࣍ݩ ϕΫτϧ ↓ ը૾ಛ௃ྔ Ϟσϧֶश ಛ௃நग़ ฐࣾςοΫϒϩάͰ΋·ͱΊ͍ͯ·͢ http://tech.vasily.jp/entry/fashion-deep-learning
14. 3. ྨࣅը૾ݕࡧ > 3.3. DCGAN 3. DCGAN 14 ←ΫΤϦը૾ ݕࡧ݁Ռ ↓ ←ΫΤϦը૾ ݕࡧ݁Ռ ↓
15. 3. ྨࣅը૾ݕࡧ > 3.4. ֤छ๏ͷൺֱ ࢖ͬͯΈͨײ૝ 15 COLOR HISTOGRAM + HOG CNN BASED MODEL DCGAN ख๏ ϝϦοτ σϝϦοτ ݕࡧ݁Ռͷ੍ޚ͕؆୯ લॲཧ͕େม ѹॖ཰͕ѱ͍ લॲཧָ͕ ϋογϡΛར༻ͨ͠ݕࡧ ඞཁͳ৘ใֶ͕शͷաఔͰ མͪΔ͜ͱ͕͋Δ લॲཧָ͕ ѹॖ཰͕ྑ͍ ݕࡧ݁Ռͷ੍ޚ͕ҋ
16. 4. ·ͱΊͱࠓޙͷ՝୊ ·ͱΊ ▸ ྨࣅը૾ݕࡧػೳΛ࣮૷ͨ͠ - ݁Ռʹख๏ͷݸੑ͕ݟΕͯ໘ന͍ 16 ࠓޙͷ՝୊ ▸ ݕࡧ଎౓޲্ - ॠ࣌ʹݕࡧ݁Ռ͕ฦͬͯ͜ͳ͍ͱ࢖͑ͳ͍ ▸ αʔϏεΛݟਾ͑ͨվળ - Ϣʔβ͕ຊ౰ʹݟ͍ͨ৘ใɺཉ͍͠ػೳ͸Կ͔
17. ͝ਗ਼ௌ ͋Γ͕ͱ͏͍͟͝·ͨ͠ We are hiring !! ڵຯͷ͋ΔํͷೖࣾΛ͓଴͓ͪͯ͠Γ·͢ʂʂ
18. ςΩετ ࢀߟ ▸ HoG - http://www.vision.cs.chubu.ac.jp/joint_hog/pdf/HOG +Boosting_LN.pdf ▸ CNN based model - http://www.iis.sinica.edu.tw/papers/song/18378-F.pdf ▸ DCGAN - http://arxiv.org/abs/1511.06434 - http://tech.vasily.jp/entry/fashion-deep-learning 18