Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Search fashion items by colors

shoby
March 22, 2016

Search fashion items by colors

shoby

March 22, 2016

Transcript

45 slides
  1. Shota Izaka • Fablic, inc. • Developing Fril • iOS

    / Ruby on Rails / Image Processing • Search by colors / Apple Watch App / Swift migration  !TIPCZTIPCZ
  2. What is Fril? • Fashion market place app • Similar

    to Poshmark • Buy and sell fashion items • Fril mediates trading 
  3. Outline • What is color search? • Why we implemented

    it? • Implementation • Create color palette • Color detection by image processing • Result 
  4. 

  5. What is color search? • Narrow down items by colors

    • e.g. One-piece dress & white • e.g. Coat & brown • Detect colors automatically • Sellers input nothing • Color detection from item images 
  6. Why we implemented it? • Most users choosing clothing colors

    • Questionnaire • Interview • Search word logs • To specialize in fashion • Fashion specialized search features 
  7. Why 12 colors? • From human color recognition research[1] •

    All languages have basic 11 color terms • 11 colors are universal • Red, yellow, green, blue, orange, purple, pink, brown, gray, black, white • From search queries including color names • Listed queries in descending order • Merged into basic 11 colors (e.g camel is brown) • Leaved beige because it can't merge  [1] Berlin B.and Kay P, “Basic Color Terms:Their Universality and Evolution.”
  8. Color detection by image processing • Color correction by white

    balancing • Remove background • Detect a mode color • Choose nearest color palette 
  9. Color correction by white balancing • Correct colors of images

    taken under fluorescent lamp • Typical fluorescent lights make images greenish • Fashion images usually taken inside a room • Correct colors with Retinex[2] • A method modeled human visual features • Do not affect photoshopped images  [2] E. Land and J. McCann, “Lightness and Retinex theory,”
  10. require 'RMagick' image = ImageList.new('image.jpg') red_extrema = image.channel_extrema(Magick::RedChannel) green_extrema =

    image.channel_extrema(Magick::GreenChannel) blue_extrema = image.channel_extrema(Magick::BlueChannel) red_max = red_extrema.last.to_f green_max = green_extrema.last.to_f blue_max = blue_extrema.last.to_f white_balanced_image = image white_balanced_image = white_balanced_image.convolve_channel(1, [green_max / red_max], Magick::RedChannel) white_balanced_image = white_balanced_image.convolve_channel(1, [green_max / blue_max], Magick::BlueChannel)  Retinex using RMagick
  11. Remove background • Eliminate the effect of background colors •

    Make background mask to remove • Make background mask • Detect edges • Flood magenta from corners • Extract magenta regions as background mask 
  12. Detect a mode color • Detect a mode color from

    the central portion • Fashion items usually appear in center of photo • Central portion is less affected by background colors • Detect a mode color • Crop the central portion • Reduce colors into 8 • Composite background mask • Make histogram and detect a mode 
  13. Choose nearest color palette • RGB to 12 colors •

    255*255*255ʢ16,581,375ʣ → 12 • Use color name survey[3] • 196,608 RGB values are named • We converted to HSL and used the range of each color names • Choose nearest color palette • Conver RGB to HSL • Choose nearest palette considering HSL range  IUUQCMPHYLDEDPNDPMPSTVSWFZSFTVMUT
  14. Accuracy of color detection • Checked 30 items visually in

    major categories • Average is 48.36% • Well appeared colors are well • Black: 90.67% • Red: 79.33% • Pink: 72.67% • White: 56.67% • Less appeared colors affected by false detection of other colors 
  15. Item found ratio in search result • We defined item

    found proportion • Percentage of users who opened a item in search result • Item found • Keyword search: about 69% • Color search: about 75% • Color search become easier to find items than keyword search 
  16. Summary • We implemented color search • Researched user buying

    behavior and implemented fashion specialized search • Color palette were created from human recognition research and search queries • Correcting color, removing background, cropping center and detect nearest color palette • Color search become easier to find items than keyword search 

More Decks by shoby

Other Decks in Technology