Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Food Image Object Detection and Classification

C1595f6a99fc51c0fb8e04b54863dbeb?s=47 Leszek Rybicki
February 16, 2017

Food Image Object Detection and Classification

Part 1: Detection

C1595f6a99fc51c0fb8e04b54863dbeb?s=128

Leszek Rybicki

February 16, 2017
Tweet

Transcript

  1. Food Image Object Detection and Classification Challenges and Solutions

  2. Part 1: Detection

  3. 自己紹介 • リビツキ レシェック • ポーランド出身 • 2016~ クックパッド • github:

    lunardog
  4. Warning! This presentation contains images that may cause severe drooling

    and stomach grumbling. @cookpad
  5. History 歴史

  6. ImageNet KWWSLPDJHQHWRUJ

  7. ImageNet Large Scale Visual Recognition Competition KWWSZZZLPDJHQHWRUJFKDOOHQJHV/695&

  8. ILSVRC 2010 task Classification )RUHDFKLPDJHDOJRULWKPV ZLOOSURGXFHDOLVWRIDWPRVW REMHFWFDWHJRULHVLQWKH GHVFHQGLQJRUGHURI FRQILGHQFH KWWSZZZLPDJHQHWRUJFKDOOHQJHV/695&

  9. ILSVRC 2011 tasks 1. Classification 2. *Classification with localization *tester

    task
  10. KWWSFVQVWDQIRUGHGXV\OODEXVKWPO Classification + Localization

  11. ILSVRC 2012 tasks 1. Classification 2. Classification with localization 3.

    Fine-grained classification
  12. Fine-grained classification KWWSZZZLPDJHQHWRUJFKDOOHQJHV/695&

  13. AlexNet ,PDJHQHWFODVVLILFDWLRQZLWKGHHSFRQYROXWLRQDOQHXUDOQHWZRUNV $.UL]KHYVN\,6XWVNHYHU*(+LQWRQ$GYDQFHVLQQHXUDOLQIRUPDWLRQ SURFHVVLQJV\VWHPV

  14. ILSVRC 2013 tasks 1. Detection 2. Classification 3. Classification with

    localization
  15. ILSVRC 2014 tasks 1. Detection 2. Classification 3. Classification with

    localization
  16. Object Detection KWWSFVQVWDQIRUGHGXV\OODEXVKWPO

  17. Deep Learning KWWSVGHYEORJVQYLGLDFRP

  18. ILSVRC 2015 tasks 1. Object detection 2. Object localization 3.

    *Object detection from video 4. *Scene classification
  19. ILSVRC 2016 tasks 1. Object localization 2. Object detection 3.

    Object detection from video 4. Scene classification 5. Scene parsing
  20. Cookpad 2016

  21. 画像データセット 1997年~ レシピ数:国内約260万 + 国外 + つくれぽ + 手順写真 17言語、60カ国

    ※数字は2017年02月時点のものです
  22. 画像解析の研究関心 • これは料理ですか? • どの料理ですか? • 料理はどこですか? • 。。。 Part

    2
  23. Where is the food? 料理はどこですか?

  24. ゴール )LQGIRRGLQWKHLPDJHGUDZ DERXQGLQJER[DURXQGWKH IRRGLWHPLQFOXGLQJWKH GLVKLIYLVLEOH

  25. ,IWKHUHDUHPXOWLSOHLWHPV GUDZDERXQGLQJER[ DURXQGHDFKRQH ゴール

  26. ground truth bounding box > 0.9 We count it as

    a positive detection if Intersection over Union ratio is greater than 0.9. ƴ
  27. QXPEHURIWUXHSRVLWLYHV QXPEHURIJURXQGWUXWKER[HV ƴ ƴ ƴ QXPEHURIWUXHSRVLWLYHV QXPEHURIJHQHUDWHGER[HV 再現率 (precision) (recall)

    ƴ ƴ
  28. Methods

  29. 1. Build a classifier 2. Pick Regions of Interest 3.

    Run classifier on each region 4. Remove duplicate detections IDEA
  30. Fast, Faster R-CNN  5LFKIHDWXUHKLHUDUFKLHVIRUDFFXUDWHREMHFWGHWHFWLRQDQGVHPDQWLFVHJPHQWDWLRQ 5RVV*LUVKLFN-HII'RQDKXH7UHYRU'DUUHOO-LWHQGUD0DOLN  )DVWHU5&117RZDUGV5HDO7LPH2EMHFW'HWHFWLRQZLWK5HJLRQ3URSRVDO1HWZRUNV 6KDRTLQJ5HQ.DLPLQJ+H5RVV*LUVKLFN-LDQ6XQ 

    )DVW5&11 5RVV*LUVKLFN
  31. 問題 1. Computational cost 2. Context is important 3. ...but

    context can be confusing. KDQG IRRG JUDVV IRRG KWWSSL[DED\FRP
  32. Single Shot Detector  66'6LQJOH6KRW0XOWL%R['HWHFWRU :HL/LX'UDJRPLU$QJXHORY'XPLWUX(UKDQ&KULVWLDQ6]HJHG\ 6FRWW5HHG&KHQJ<DQJ)X$OH[DQGHU&%HUJ

  33. Either The Least Or Most Employable Person Ever 7KH+XIILQJWRQ3RVW JLWKXEFRPSMUHGGLH

    SMUHGGLHFRPGDUNQHW ZZZNDJJOHFRPSMUHGGLH Joseph Redmon
  34. You Only Look Once  <RX2QO\/RRN2QFH8QLILHG 5HDO7LPH2EMHFW'HWHFWLRQ -RVHSK5HGPRQ6DQWRVK'LYYDOD5RVV *LUVKLFN$OL)DUKDGL 'HF

    <2/2%HWWHU)DVWHU 6WURQJHU -RVHSK5HGPRQ$OL)DUKDGL
  35. <RX2QO\/RRN2QFH8QLILHG5HDO7LPH2EMHFW'HWHFWLRQ -RVHSK5HGPRQ6DQWRVK'LYYDOD5RVV*LUVKLFN$OL)DUKDGL YOLO in Context

  36. None