Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Fashion Tech x Machine Learning/twm_fashion_ml

tn1031
April 16, 2016
5.5k

Fashion Tech x Machine Learning/twm_fashion_ml

tn1031

April 16, 2016
Tweet

More Decks by tn1031

Transcript

  1. ࣗݾ঺հ ▸ தଜ ୓ຏ / @tn1031 ▸ σʔλαΠΤϯςΟετ ▸ SIer(2೥൒)

    -> VASILY ▸ ػցֶशΛઐ߈ ▸ ϊϯύϥϝτϦοΫϕΠζ ▸ ม෼ϕΠζ ▸ ϊϯύϥຊָ͕͠Έ ▸ TokyoWebMiningͷొஃ͸2ճ໨ ▸ લճ͸ɺը૾ೝࣝʹ͍ͭͯൃද 3 @tn1031 ਓ޻஌ೳ͸ɹɹɹɹɹ झຯͰᅂΉఔ౓
  2. Fashion x Technologyʁ ‣ ϑΝογϣϯۀք͸ɺ ‣ Ϗδωεͷن໛͕େ͖͍ ‣ ੜ׆ʹ͔ܽͤͳ͍࢈ۀ ‣

    IT͕ਁಁ͍ͯ͠ͳ͍ 11 ϑΝογϣϯۀքʹɹɹɹɹɹ ITͷ೾͕དྷͯΔΒ͍͠
  3. Fashioning Data: A 2015 Update Data Innovations from the Fashion

    Industry 12 http://www.oreilly.com/data/free/fashioning-data.csp Έ͍ͨͳ͜ͱ͕ॻ͍ͯ͋Δຊ
  4. Fashion x Technologyͷࣄྫ ‣ ը૾Λѻ͏αʔϏε͕ଟ͍ ‣ ෰Λങ͏্Ͱ͔ܽͤͳ͍ཁૉ ‣ ݕࡧ΍ύʔιφϥΠζͷधཁ͕͋Δ ‣

    େྔͷΞΠςϜ͔Β໨తͷΞΠςϜΛ୳͢ ‣ ݸਓͷझ޲Λ൓өͨ͠ϥϯΩϯά 17 ϑΝογϣϯͱػցֶशͷ ਌࿨ੑ͸େ͖͍
  5. FashionΛର৅ʹͨ͠ݚڀͰѻΘΕΔ໰୊ 19 ը૾ܥͷख๏͸ϑΝογϣϯσʔλͱͷ਌࿨ੑ͕ߴ͍ɻҎԼͷ3߲໨͕ओྲྀͳҹ৅ɻ ΞΠςϜݕग़ ύʔε ྖҬ෼ׂ ϑΝογϣϯ ΞΠςϜݕࡧ Ϩίϝϯυ ςʔϚ

    λεΫ ϑΝογϣϯྖҬ΁ͷద༻ ը૾͔ΒΞΠςϜʹ֘౰͢ΔՕॴΛಛఆ͠ɺ ύʔε΍ΧςΰϦͷ൑ผΛߦ͏ ѻ͏ը૾͕ݶఆ͞ΕΔҝɺ ࣄલ஌ࣝΛ׆༻Ͱ͖Δ ΫΤϦʹҰகɾྨࣅ͢Δ ΞΠςϜΛఏࣔ͢Δ ΤοδΛ༻͍ͨΞΠςϜͷ۠ผ͕ࠔ೉ͳҝɺ ৭΍ฑ΍ૉࡐͷ৘ใΛ൓өͰ͖ΔϞσϧ͕ ඞཁ ߦಈσʔλ΍ΞΠςϜؒྨࣅ౓Λར༻ͯ͠ɺ Ϣʔβ΁ͷΞΠςϜఏࣔΛ࠷దԽ͢Δ ߦಈσʔλͷଞʹΞΠςϜͷը૾ಛ௃ྔ΍ ྲྀߦΛϞσϧʹ૊ΈࠐΉ͜ͱ͕Ͱ͖Δ
  6. Fashion Apparel Detection: The Role of Deep Convolutional Neural Network

    and Pose-dependent Priors (WACV 2016) ‣ ϑΝογϣϯΞΠςϜͷݕग़ ‣ ண͍ͯΔ΋ͷ͚ͩͰͳ͘ɺ਎ʹ෇͚Δ΋ͷ΋ର৅ɹɹɹɹɹɹɹɹɹ (hat, glasses, bag, pants, shoes and so on.) ‣ ֶशʹDeep Convolutional Neural NetworkΛར༻ ‣ ֤ΞΠςϜ͕ग़ݱ͠΍͍͢ҐஔΛࣄલ஌ࣝͱֶͯ͠शʹ׆༻͢Δ࢓૊Έ ΛఏҊ 21 ֓ཁ Fashion Apparel Detection: The Role of Deep Convolutional Neural Network and Pose-dependent Priors http://arxiv.org/abs/1411.5319
  7. Fashion Apparel Detection: The Role of Deep Convolutional Neural Network

    and Pose-dependent Priors (WACV 2016) 22 Contribution ‣ domain-specific priorsͳࣄલ৘ใΛ׆༻ֶͨ͠शํ๏ ΛఏҊͨ͠ ‣ ϋϯυόοά͸खट΍खͷۙ͘ʹ͋Γɺۺ͸଍ͷۙ͘ʹग़ݱ͠ қ͍ ‣ ΞΠςϜͷେ͖͞͸͍ࣸͬͯΔਓͷେ͖͞ʹൺྫ͢Δ ‣ ϙʔζݕग़ΛߦͬͯΞΠςϜͷݕग़ਫ਼౓ΛߴΊͨ ‣ ΞΠςϜީิྖҬͷ࠲ඪͱϙʔζݕग़ͷग़ྗΛjoint͍ͤͯ͞Δ
  8. Fashion Apparel Detection: The Role of Deep Convolutional Neural Network

    and Pose-dependent Priors (WACV 2016) 23 Overview of the proposed algorithm ‣ ΞΠςϜީิྖҬ͔Βͷಛ௃நग़͸CNNͰߦ͏ ‣ ͦΕͧΕͷΫϥεͷग़ݱ֬཰Λ1-vs-rest SVMsͰݟੵ΋Δ ‣ ϙʔζਪఆʹجͮ͘Ґஔ৘ใʹΑͬͯ֬཰Λิਖ਼͢Δ ‣ Non-Maximum suppressionͰ੔͑ͯ࠷ऴతͳग़ྗͱ͢Δ
  9. Fashion Apparel Detection: The Role of Deep Convolutional Neural Network

    and Pose-dependent Priors (WACV 2016) 24 Model Object Proposal ‣ ը૾͔ΒΞΠςϜྖҬͷީิΛऔಘ͢Δ ‣ ͦΕͧΕͷީิྖҬʹ͍ͭͯΞΠςϜͷ༗ແΛ൑ผ ثͰॲཧͯ͠false positiveΛݮΒ͢ 0CKFDU1SPQPTBM *NBHF'FBUVSF&YUSBDUJPO 47.USBJOJOH 1SPCBCJMJTUJDGPSNVMBUJPO "QQFBSBODFCBTFE1PTUFSJPS (FPNFUSJD1SJPST Image Feature Extraction ‣ ֶशࡁΈCaffeNetΛ༻͍Δ ‣ ग़ྗ૚ͷखલͷग़ྗ4096࣍ݩͷϕΫτϧΛಛ௃ྔͱ ͢Δ
  10. Fashion Apparel Detection: The Role of Deep Convolutional Neural Network

    and Pose-dependent Priors (WACV 2016) 25 Model Appearance-based Posterior ‣ SVMͷύϥϝʔλΛར༻ͯ͠posteriorΛܭࢉ͢Δ ‣ ɹ ‣ λ͕൑ผਫ਼౓ʹޮ͍ͯ͘ΔͷͰɺ࠷ऴతͳ݁Ռʹجͮ ͖λΛνϡʔχϯά͢Δ 0CKFDU1SPQPTBM *NBHF'FBUVSF&YUSBDUJPO 47.USBJOJOH 1SPCBCJMJTUJDGPSNVMBUJPO "QQFBSBODFCBTFE1PTUFSJPS (FPNFUSJD1SJPST SVM training ‣ ΞΠςϜͷΫϥε(bag,hat,...)ͦΕͧΕʹ͍ͭͯઢܗ SVMͰ2஋൑ผΛߦ͏
  11. Fashion Apparel Detection: The Role of Deep Convolutional Neural Network

    and Pose-dependent Priors (WACV 2016) 26 Model 0CKFDU1SPQPTBM *NBHF'FBUVSF&YUSBDUJPO 47.USBJOJOH 1SPCBCJMJTUJDGPSNVMBUJPO "QQFBSBODFCBTFE1PTUFSJPS (FPNFUSJD1SJPST Geometric Priors ‣ Ґஔ৘ใʹؔ͢ΔpriorΛ2छྨఆٛ͢Δ ‣ ۣܗͷप௕ʹ͍ͭͯɺ ‣ ۣܗͷΞεϖΫτൺʹ͍ͭͯɺ ‣ ۣܗͷத৺࠲ඪʹ͍ͭͯɺ Probabilistic formulation ‣ Appearance-based PosteriorͱGeometric Priors͔ ΒPosteriorΛܭࢉ͢Δ
  12. Fashion Apparel Detection: The Role of Deep Convolutional Neural Network

    and Pose-dependent Priors (WACV 2016) 27 Result
  13. Fashionista: A Fashion-aware Graphical System for Exploring Visually Similar Items

    (WWW 2016) ‣ ϑΝογϣϯΞΠςϜͷݕࡧγεςϜ ‣ ࢹ֮తʹྨࣅ͍ͯ͠ΔΞΠςϜͷݕࡧγεςϜΛ։ൃͨ͠ ‣ graphical interfaceʹͩ͜ΘΓ͕͋Δ ‣ ྲྀߦΓΛ౿·্͑ͨͰࣅ͍ͯΔΞΠςϜΛఏࣔ͢Δ ‣ ֶशʹ༻͍Δ৘ใ ‣ ը૾ಛ௃ྔ ‣ AmazonͷߪങσʔλͱλΠϜελϯϓ 29 ֓ཁ Fashionista: A Fashion-aware Graphical System for Exploring Visually Similar Items http://cseweb.ucsd.edu/~chunbinlin/papers/www16demo.pdf
  14. Fashionista: A Fashion-aware Graphical System for Exploring Visually Similar Items(WWW

    2016) 30 Contribution ‣ ࢹ֮తʹྨࣅ͍ͯ͠ΔΞΠςϜΛ୳ͤΔgraphical interface ”Fashionista”Λ։ൃͨ͠ ‣ ࢹ֮తͳ৘ใ(ը૾ಛ௃ྔ)Λྨࣅ౓ܭࢉʹར༻͢Δ ‣ ”Fashionista”͸τϨϯυΛՄࢹԽͯ͠ࠓྲྀߦ͍ͬͯΔΞΠςϜΛ୳͠ ΍͍ͯ͘͢͠Δ ‣ Amazon͔Β11೥ʹΘͨͬͯɺ0.6 million items, 1.8 million users, and 3.2 million user-item
  15. Fashionista: A Fashion-aware Graphical System for Exploring Visually Similar Items(WWW

    2016) 31 A screenshot of Fashionista http://132.239.95.211:8080/demowww/index.jsp
  16. Fashionista: A Fashion-aware Graphical System for Exploring Visually Similar Items(WWW

    2016) 33 Fashion Learner 5SBJOJOH%BUB ‣ Amazonͷߪങσʔλɹɹ (user, item, timestamp) ‣ pretrained CNN͔Βɹɹɹɹ ը૾ಛ௃ྔΛநग़ 'BTIJPO-FBSOFS ,OPXMFEHF ‣ ΞΠςϜಉ͕࢜ࣅ͍ͯΔʗࣅͯ ͍ͳ͍ͷ৘ใ ‣ tŖSNEͰѹॖͯ͠ฏ໘ʹɹ Ϛοϐϯά ‣ ΞΠςϜͷ͓͠ΌΕ౓ͷਪҠ ‣ Matrix Factorizationϕʔεͷख ๏ͰɺߦಈσʔλʹՃ͑ͯը૾ ಛ௃ྔΛར༻͢Δ ‣ ҎԼͷ࣌ܥྻ৘ใΛѻ͏ ‣ ΞΠςϜͷັྗʹ͍ͭͯͷ࣌ ؒਪҠ ‣ ࢹ֮త৘ใʹର͢ΔϢʔβ ͷ൓ԠͷਪҠ ‣ Ϣʔβຖͷᅂ޷ͷมԽ
  17. “Hi, Magic Closet, Tell Me What toWear!”(ACM MM 2012) ‣

    occasionʹର͢ΔϨίϝϯσʔγϣϯ ‣ Wedding΍σʔτͱ͍ͬͨoccasionΛೖྗ͢Δͱɺࣗ਎ ͷϑΥτΞϧόϜ΍ΦϯϥΠϯγϣοϓ͔Βద੾ͳ෰૷Λ ୳ͯ͠ఏҊͯ͘͠ΕΔ ‣ Magic Closet͕ղ͘໰୊͸2छྨ ‣ wear properly(ଐੑ)ɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹ ʮόϯέοτʹ͸ύϯπεʔπΑΓυϨε͕ɹద͍ͯ͠Δʯ ‣ wear aesthetically(ඒ͠͞)ɹɹɹɹɹɹɹɹɹɹɹɹɹɹ ʮ੺͍Tγϟπʹ͸྘ΑΓനͷύϯπ͕ద͍ͯ͠Δʯ 35 ֓ཁ “Hi, Magic Closet, Tell Me What to Wear!" http://www.lv-nus.org/papers/2012/magic_closet-MM12.pdf
  18. “Hi, Magic Closet, Tell Me What toWear!”(ACM MM 2012) 37

    Magic Closetͷ࢓૊Έ TDFOBSJPɿPDDBTJPOΛࢦఆ ʮ*BNGPSBDPOGFSFODF QMFBTF SFDPNNFOETPNFUIJOHGSPNNZ QIPUPBMCVN ʯ TDFOBSJPɿPDDBTJPOͱ্൒਎ͷ෰Λࢦఆ ʮ*BNHPJOHPOBEBUF QMFBTFpOE TPNFUIJOHUPNBUDIXJUIUIJT5 TIJSU ʯ
  19. “Hi, Magic Closet, Tell Me What toWear!”(ACM MM 2012) 38

    Magic Closetͷ࢓૊Έ ز͔ͭͷ؍఺͔ΒϨίϝϯυΛߦ͏ B "UUSJCVUFWT0DDBTJPO.PEFM ‣ ෰ͷଐੑͱPDDBTJPOͷ૬ੑ C "UUSJCVUFWT"UUSJCVUF.PEFM ‣ ෰ͷଐੑಉ࢜ͷ૬ੑ ্൒਎ͱԼ൒ ਎  D $MPUIJOH"UUSJCVUF&TUJNBUJPO ‣ ෰ͷଐੑͷਪఆ
  20. “Hi, Magic Closet, Tell Me What toWear!”(ACM MM 2012) 39

    Magic Closetͷ࢓૊Έ ݁Ռͷग़ྗ PDDBTJPOΛࢦఆͨ͠৔߹ ྫɿDPOGFSFODF  ‣ ద੾ͳ෰ ͷ૊Έ߹Θͤ ΛఏҊ PDDBTJPOͱ্൒਎ͷ෰Λࢦఆͨ͠৔߹ ྫɿ EBUF 5TIJSU  ‣ ࢦఆͨ͠෰͓ΑͼPDDBTJPOͱ૬ੑͷྑ͍ ෰ΛఏҊ
  21. “Hi, Magic Closet, Tell Me What toWear!”(ACM MM 2012) ‣

    ҎԼͷ2ͭͷ໰୊ʹ࠷ॳʹऔΓ૊Μͩ ‣ ϑΥτΞϧόϜ͔Βoccasionʹదͨ͠෰૷Λ୳ͯ͠ਪન͢Δ ‣ ΫΤϦͷΞΠςϜ΍occasionʹ૬ੑͷྑ͍ΞΠςϜΛECαΠτ͔Β୳͢ ‣ What-to-Wear (WoW)σʔληοτΛߏஙͨ͠ ‣ 24,417ຕɺ౰࣌࠷େڃΒ͍͠ ‣ occasionͱ෰ͷ૬ੑΛಉ࣌ʹֶश͢Δlatent SVMΛఏҊͨ͠ 40 Contribution
  22. “Hi, Magic Closet, Tell Me What toWear!”(ACM MM 2012) ‣

    ֶशσʔλ ‣ ‣ :্൒਎/Լ൒਎ͷը૾ಛ௃ྔ ʗ :্൒਎ͷଐੑ(৭ͱ͔)ʗ :Լ൒਎ͷଐੑ ʗ :occasionΧςΰϦ 41 Model ( x, au, al, o ) ‣ Recommendation function x al au o
  23. “Hi, Magic Closet, Tell Me What toWear!”(ACM MM 2012) 42

    Model 'FBUVSFWT0DDBTJPO ‣ ը૾ಛ௃ྔͱoccasion ‣ ઢܗϞσϧ 'FBUVSFWT"UUSJCVUF ‣ ը૾ಛ௃ྔͱ෰ͷଐੑ ‣ ઢܗϞσϧ "UUSJCVUFWT"UUSJCVUF ‣ ෰ͷଐੑͷ૊Έ߹Θͤ ͷ૬ੑ ‣ ্൒਎ͱԼ൒਎ͷϖΞ ʹ͍ͭͯܭࢉ͢Δ ‣ ্Լͷ෰ͷ૬ੑʹ͍ͭ ֶͯश͢Δ "UUSJCVUFWT0DDBTJPO ‣ ෰ͷଐੑͱoccasion ‣ ઢܗϞσϧ ‣ occasionʹదͨ͠෰૷ Λֶश͢Δ
  24. “Hi, Magic Closet, Tell Me What toWear!”(ACM MM 2012) ‣

    ӡ༻࣌ ‣ occasionʹదͨ͠෰ͷϨίϝϯυ ‣ Լ൒਎givenͰ্൒਎ͷ෰ͷϨίϝϯυ ‣ ͜͜Ͱɺ 43 Model x ⇤ = arg maxx 2 X t fw(x, o) x ⇤ u = arg maxxu 2 X t u fw([xu; xl], o) fw(x, o) = max au,al w T (x, au, al, o)
  25. “Hi, Magic Closet, Tell Me What toWear!”(ACM MM 2012) 44

    Result ΞΠςϜͷଐੑ ಉ࢜ͷ૬ੑ ଐੑͱoccasionͷ૬ੑ
  26. ࢀߟ ‣ http://cs229.stanford.edu/proj2009/McDanielsWorsley.pdf ‣ http://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/Simo- Serra_Neuroaesthetics_in_Fashion_2015_CVPR_paper.pdf ‣ http://labs.ebay.com/sites/default/files/13_stylefinder_cvpr_ebayresearchlabs.pdf ‣ http://vision.is.tohoku.ac.jp/~kyamagu/papers/yamaguchi_cvpr2012.pdf

    ‣ http://arxiv.org/abs/1411.5319 ‣ http://labs.ebay.com/sites/default/files/14_fashionrecommender_kdd_ebayresearchlabs.pdf ‣ http://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/ Chen_Deep_Domain_Adaptation_2015_CVPR_paper.pdf ‣ http://arxiv.org/abs/1603.07063 ‣ http://arxiv.org/abs/1506.04757 ‣ http://cseweb.ucsd.edu/~chunbinlin/papers/www16demo.pdf ‣ http://arxiv.org/pdf/1505.07922.pdf ‣ http://www.iis.sinica.edu.tw/papers/song/18378-F.pdf ‣ http://arxiv.org/abs/1602.01585 ‣ http://www.lv-nus.org/papers/2012/magic_closet-MM12.pdf ‣ http://arxiv.org/abs/1405.4013 49