ϑΝογϣϯΞΠςϜͷ ྨࣅը૾ݕࡧΛ࣮૷ͯ͠Έ·ͨ͠ 2016/03/22 FASHION TECH MEETUP #2 Presented by @tn1031, VASILY Inc.

0. ࣗݾ঺հ ࣗݾ঺հ ▸ தଜ ୓ຏ / @tn1031 ▸ σʔλαΠΤϯςΟετ ▸ SIer(2೥൒) -> VASILY(3िؒ) ▸ ػցֶशΛઐ߈ ▸ SHIROBAKO͸ਓੜ 2 @tn1031 ਓ޻஌ೳ͸ɹɹɹɹɹ झຯͰᅂΉఔ౓ SHIROBAKOͷଚ͍ը૾

1. औΓ૊Έͷഎܠ ྨࣅը૾ݕࡧ͕͋Δͱྑ͍৔໘ ʮཉ͍͠ΞΠςϜ͸͋Δ͚Ͳɺߴͯ͘ख͕ग़ͳ͍ɻʯ ʮଥڠͯ͠ങͬͨޙʹɺࣗ෼͕ങͬͨ΋ͷΑΓ΋ྑ͍΋ͷ͕ݟ͔ͭΔɻʯ 3 ྨࣅը૾ݕࡧ͕͋Ε͹ ʮࣅͨΞΠςϜΛ୳͠·ΘΔख͕ؒল͚Δʂʯ ʮଥڠͤͣʹ೰Ή͜ͱ͕Ͱ͖Δʂʯ

2. ը૾ݕࡧʹ͍ͭͯ ը૾ݕࡧʹ͸ओʹ̎छྨ͋Γ·͢ ςΩετϕʔεͷݕࡧ ▸ Image meta search ▸ ը૾ʹ෇ਵ͢Δϝλσʔλ΍ɹ ςΩετΛར༻ͨ͠ݕࡧ 4 ը૾ϕʔεͷݕࡧ ▸ Content-based image retrieval (CBIR) ▸ ςΩετ৘ใΛ࢖Θͣɺը૾ͷಛ௃ (৭ɺܗঢ়ͳͲ)Λར༻ͨ͠ݕࡧ ը૾σʔλ ը૾σʔλ ςΩετσʔλ ࢖͑Δ৘ใ͸ɹ ը૾σʔλ͚ͩ

2. ը૾ݕࡧʹ͍ͭͯ ը૾ݕࡧʹ͸ओʹ̎छྨ͋Γ·͢ ςΩετϕʔεͷݕࡧ ▸ Image meta search ▸ ը૾ʹ෇ਵ͢Δϝλσʔλ΍ɹ ςΩετΛར༻ͨ͠ݕࡧ 5 ը૾ϕʔεͷݕࡧ ▸ Content-based image retrieval (CBIR) ▸ ςΩετ৘ใΛ࢖Θͣɺը૾ͷಛ௃ (৭ɺܗঢ়ͳͲ)Λར༻ͨ͠ݕࡧ ը૾σʔλ ը૾σʔλ ࢖͑Δ৘ใ͸ɹ ը૾σʔλ͚ͩ ςΩετσʔλ ࠓճ͸ͪ͜Βʹ௅ઓ

2. ը૾ݕࡧʹ͍ͭͯ ը૾ݕࡧ͸ѹॖͱڑ཭ܭࢉͰ͢ ը૾ݕࡧͷجຊతͳߟ͑ํ ▸ ͳΔ΂͘௿࣍ͷۭؒʹѹॖ͠ɺѹॖͨ͠ϕΫτϧͷڑ཭ʹج͍ͮͯྨࣅ౓Λఆٛ͢Δ ▸ ࣅ͍ͯΔը૾ಉ࢜ͷڑ཭͕ۙ͘ɺࣅ͍ͯͳ͍ը૾ͱͷڑ཭͕ԕ͘ͳΔΑ͏ʹѹॖ͢Δ 6 ಛ௃ྔۭؒ f(x) ѹॖ ͍ۙ(ࣅ͍ͯΔ) ԕ͍(ࣅ͍ͯͳ͍) ը૾σʔλ ॎԣ480pixelͷ৔߹ɺ࣍ݩ਺͸ 480x480x3 = 691200 dim ը૾ಛ௃ྔ ը૾σʔλΛදݱ͢Δ௿࣍ͷϕΫτϧ ը૾Λѹॖ(=ಛ௃நग़)͢Δؔ਺Λ Ͳͷ༷ʹઃܭ͢Δ͔͕େࣄ

3. ྨࣅը૾ݕࡧ CBIRΛࢼͯ͠Έ·ͨ͠ 7 3௨Γͷํ๏Ͱ࣮૷ 1. Color histogram + Histogram of oriented gradients (HOG) - ίϯϐϡʔλϏδϣϯ෼໺ͷ఻౷తͳಛ௃நग़ํ๏ 2. Convolutional Neural Network (CNN) based model - σΟʔϓϥʔχϯά(ࣝผϞσϧ)ʹΑΔಛ௃நग़ 3. Deep Convolutional Generative Adversarial Networks (DCGAN) - σΟʔϓϥʔχϯά(ੜ੒Ϟσϧ)ʹΑΔಛ௃நग़

3. ྨࣅը૾ݕࡧ > 3.1. COLOR HISTOGRAM + HOG 1. COLOR HISTOGRAM + HOG ▸ ը૾ͷHSV஋ΛώετάϥϜԽ ▸ ը૾ͷً౓ޯ഑ΛώετάϥϜԽ ▸ 2छྨͷώετάϥϜΛ݁߹ͯ͠ը૾ͷಛ௃ྔͱ͢Δ 8 HSV஋நग़ άϨʔɹɹ εέʔϧ ৭৘ใώετάϥϜ ޯ഑৘ใώετάϥϜ ը૾ಛ௃ྔ ޯ഑நग़

3. ྨࣅը૾ݕࡧ > 3.1. COLOR HISTOGRAM + HOG 1. COLOR HISTOGRAM + HOG 9 ←ΫΤϦը૾ ݕࡧ݁Ռ ↓ ←ΫΤϦը૾ ݕࡧ݁Ռ ↓

3. ྨࣅը૾ݕࡧ > 3.2. CNN BASED MODEL 2. CNN BASED MODEL ▸ CNNΛimage netͰֶशͤ͞Δ ▸ ֶशࡁΈCNNʹΞΠςϜը૾ͱΧςΰϦϥϕϧΛ౤ೖͯ͠࠶ֶशͤ͞Δ ▸ શ݁߹૚ͷग़ྗΛը૾ಛ௃ྔͱ͢Δ 10 CNN શ݁߹૚ 4096ϊʔυ જࡏ૚ 64ϊʔυ ग़ྗ૚ 7ϊʔυ ΧςΰϦɹ ༧ଌ ը૾ಛ௃ྔ ݕࡧ࣌ͷڑ཭ܭࢉʹ࢖༻ ը૾ͷϋογϡ஋ ݕࡧର৅ͷߜࠐʹ࢖༻ ̍̍̌ɾɾ̍̌

3. ྨࣅը૾ݕࡧ > 3.2. CNN BASED MODEL 2. CNN BASED MODEL 11 ←ΫΤϦը૾ ݕࡧ݁Ռ ↓ ←ΫΤϦը૾ ݕࡧ݁Ռ ↓

3. ྨࣅը૾ݕࡧ > 3.3. DCGAN 3. DCGAN ▸ DCGANͰGeneratorͱDiscriminatorͷֶशΛߦ͏ ▸ ֶशࡁΈGeneratorΛ༻͍ͯVectorizerͷֶशΛߦ͏ ▸ ֶशࡁΈVectorizerΛ༻͍ͯը૾Λ100࣍ݩͷϕΫτϧʹม׵͢Δ 12 DCGAN DISCRIPTOR GENERATOR TRAINED DISCRIPTOR TRAINED GENERATOR TRAINED GENERATOR VECTORIZER 100࣍ݩ ϕΫτϧ(ཚ਺) ը૾ੜ੒(ِ෺) TRAINEDɹ VECTORIZER ΞΠςϜը૾ 100࣍ݩ ϕΫτϧ 100࣍ݩ ϕΫτϧ ↓ ը૾ಛ௃ྔ Ϟσϧֶश ಛ௃நग़

3. ྨࣅը૾ݕࡧ > 3.3. DCGAN 3. DCGAN 13 DCGAN DISCRIPTOR GENERATOR TRAINED DISCRIPTOR TRAINED GENERATOR TRAINED GENERATOR VECTORIZER 100࣍ݩ ϕΫτϧ(ཚ਺) ը૾ੜ੒(ِ෺) TRAINEDɹ VECTORIZER ΞΠςϜը૾ 100࣍ݩ ϕΫτϧ 100࣍ݩ ϕΫτϧ ↓ ը૾ಛ௃ྔ Ϟσϧֶश ಛ௃நग़ ฐࣾςοΫϒϩάͰ΋·ͱΊ͍ͯ·͢ http://tech.vasily.jp/entry/fashion-deep-learning

3. ྨࣅը૾ݕࡧ > 3.3. DCGAN 3. DCGAN 14 ←ΫΤϦը૾ ݕࡧ݁Ռ ↓ ←ΫΤϦը૾ ݕࡧ݁Ռ ↓

3. ྨࣅը૾ݕࡧ > 3.4. ֤छ๏ͷൺֱ ࢖ͬͯΈͨײ૝ 15 COLOR HISTOGRAM + HOG CNN BASED MODEL DCGAN ख๏ ϝϦοτ σϝϦοτ ݕࡧ݁Ռͷ੍ޚ͕؆୯ લॲཧ͕େม ѹॖ཰͕ѱ͍ લॲཧָ͕ ϋογϡΛར༻ͨ͠ݕࡧ ඞཁͳ৘ใֶ͕शͷաఔͰ མͪΔ͜ͱ͕͋Δ લॲཧָ͕ ѹॖ཰͕ྑ͍ ݕࡧ݁Ռͷ੍ޚ͕ҋ

4. ·ͱΊͱࠓޙͷ՝୊ ·ͱΊ ▸ ྨࣅը૾ݕࡧػೳΛ࣮૷ͨ͠ - ݁Ռʹख๏ͷݸੑ͕ݟΕͯ໘ന͍ 16 ࠓޙͷ՝୊ ▸ ݕࡧ଎౓޲্ - ॠ࣌ʹݕࡧ݁Ռ͕ฦͬͯ͜ͳ͍ͱ࢖͑ͳ͍ ▸ αʔϏεΛݟਾ͑ͨվળ - Ϣʔβ͕ຊ౰ʹݟ͍ͨ৘ใɺཉ͍͠ػೳ͸Կ͔

͝ਗ਼ௌ ͋Γ͕ͱ͏͍͟͝·ͨ͠ We are hiring !! ڵຯͷ͋ΔํͷೖࣾΛ͓଴͓ͪͯ͠Γ·͢ʂʂ

ςΩετ ࢀߟ ▸ HoG - http://www.vision.cs.chubu.ac.jp/joint_hog/pdf/HOG +Boosting_LN.pdf ▸ CNN based model - http://www.iis.sinica.edu.tw/papers/song/18378-F.pdf ▸ DCGAN - http://arxiv.org/abs/1511.06434 - http://tech.vasily.jp/entry/fashion-deep-learning 18