Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Search fashion items by colors

shoby
March 22, 2016

Search fashion items by colors

shoby

March 22, 2016
Tweet

More Decks by shoby

Other Decks in Technology

Transcript

  1. Search fashion items by colors Fablic, inc. Shota Izaka ⛄

  2. Shota Izaka • Fablic, inc. • Developing Fril • iOS

    / Ruby on Rails / Image Processing • Search by colors / Apple Watch App / Swift migration  !TIPCZTIPCZ
  3. What is Fril? • Fashion market place app • Similar

    to Poshmark • Buy and sell fashion items • Fril mediates trading 
  4. Outline 

  5. Outline • What is color search? • Why we implemented

    it? • Implementation • Create color palette • Color detection by image processing • Result 
  6. What is color search? 

  7. 

  8. What is color search? • Narrow down items by colors

    • e.g. One-piece dress & white • e.g. Coat & brown • Detect colors automatically • Sellers input nothing • Color detection from item images 
  9. Why we implemented it? 

  10. Why we implemented it? • Most users choosing clothing colors

    • Questionnaire • Interview • Search word logs • To specialize in fashion • Fashion specialized search features 
  11. Implementation #1 Create color palette 

  12. Use RGB values  255 * 255 * 255 =

    16,581,375 colors
  13. Create color palette  12 colors ☺

  14. Why 12 colors? 

  15. Why 12 colors? • From human color recognition research[1] •

    All languages have basic 11 color terms • 11 colors are universal • Red, yellow, green, blue, orange, purple, pink, brown, gray, black, white • From search queries including color names • Listed queries in descending order • Merged into basic 11 colors (e.g camel is brown) • Leaved beige because it can't merge  [1] Berlin B.and Kay P, “Basic Color Terms:Their Universality and Evolution.”
  16. Implementation #2 Color detection by image processing 

  17. Color detection by image processing • Color correction by white

    balancing • Remove background • Detect a mode color • Choose nearest color palette 
  18. Color correction by white balancing 

  19. Color correction by white balancing • Correct colors of images

    taken under fluorescent lamp • Typical fluorescent lights make images greenish • Fashion images usually taken inside a room • Correct colors with Retinex[2] • A method modeled human visual features • Do not affect photoshopped images  [2] E. Land and J. McCann, “Lightness and Retinex theory,”
  20. require 'RMagick' image = ImageList.new('image.jpg') red_extrema = image.channel_extrema(Magick::RedChannel) green_extrema =

    image.channel_extrema(Magick::GreenChannel) blue_extrema = image.channel_extrema(Magick::BlueChannel) red_max = red_extrema.last.to_f green_max = green_extrema.last.to_f blue_max = blue_extrema.last.to_f white_balanced_image = image white_balanced_image = white_balanced_image.convolve_channel(1, [green_max / red_max], Magick::RedChannel) white_balanced_image = white_balanced_image.convolve_channel(1, [green_max / blue_max], Magick::BlueChannel)  Retinex using RMagick
  21.  Before After green white

  22. Remove background 

  23. Remove background • Eliminate the effect of background colors •

    Make background mask to remove • Make background mask • Detect edges • Flood magenta from corners • Extract magenta regions as background mask 
  24.  Original Detect edges Flood magenta Background mask

  25. Detect a mode color 

  26. Detect a mode color • Detect a mode color from

    the central portion • Fashion items usually appear in center of photo • Central portion is less affected by background colors • Detect a mode color • Crop the central portion • Reduce colors into 8 • Composite background mask • Make histogram and detect a mode 
  27.  Original Crop center ˍ Reduce colors Composite mask Make

    histogram & Detect mode #A3886B
  28. Choose nearest color palette 

  29. Choose nearest color palette • RGB to 12 colors •

    255*255*255ʢ16,581,375ʣ → 12 • Use color name survey[3] • 196,608 RGB values are named • We converted to HSL and used the range of each color names • Choose nearest color palette • Conver RGB to HSL • Choose nearest palette considering HSL range  IUUQCMPHYLDEDPNDPMPSTVSWFZSFTVMUT
  30. Me: ”Succeeded!” 

  31. A internal user: “It’s strange” 

  32. General colors != Fashion colors 

  33. Color recognition is different by fashion knowledge 

  34.  8IBUDPMPSJTUIJT HSFFO #81D4CB

  35.  5J⒎BOZlCMVFz

  36. Tuned up with showing to internal uses 

  37.  Completed

  38. Result 

  39. Accuracy of color detection 

  40. Accuracy of color detection • Checked 30 items visually in

    major categories • Average is 48.36% • Well appeared colors are well • Black: 90.67% • Red: 79.33% • Pink: 72.67% • White: 56.67% • Less appeared colors affected by false detection of other colors 
  41. Item found proportion in search result 

  42. Item found ratio in search result • We defined item

    found proportion • Percentage of users who opened a item in search result • Item found • Keyword search: about 69% • Color search: about 75% • Color search become easier to find items than keyword search 
  43. Summary 

  44. Summary • We implemented color search • Researched user buying

    behavior and implemented fashion specialized search • Color palette were created from human recognition research and search queries • Correcting color, removing background, cropping center and detect nearest color palette • Color search become easier to find items than keyword search 
  45. Thanks  ⛄