Food Image Object Detection and Classification

Part 1: Detection

Transcript

1. Food Image Object Detection and Classification Challenges and Solutions
2. Part 1: Detection
3. 自己紹介 • リビツキ　レシェック • ポーランド出身 • 2016~ クックパッド • github: lunardog
4. Warning! This presentation contains images that may cause severe drooling and stomach grumbling. @cookpad
5. History 歴史
6. ImageNet KWWSLPDJHQHWRUJ
7. ImageNet Large Scale Visual Recognition Competition KWWSZZZLPDJHQHWRUJFKDOOHQJHV/695&
8. ILSVRC 2010 task Classification )RUHDFKLPDJHDOJRULWKPV ZLOOSURGXFHDOLVWRIDWPRVW REMHFWFDWHJRULHVLQWKH GHVFHQGLQJRUGHURI FRQILGHQFH KWWSZZZLPDJHQHWRUJFKDOOHQJHV/695&
9. ILSVRC 2011 tasks 1. Classification 2. *Classification with localization *tester task
10. KWWSFVQVWDQIRUGHGXV\OODEXVKWPO Classification + Localization
11. ILSVRC 2012 tasks 1. Classification 2. Classification with localization 3. Fine-grained classification
12. Fine-grained classification KWWSZZZLPDJHQHWRUJFKDOOHQJHV/695&
13. AlexNet ,PDJHQHWFODVVLILFDWLRQZLWKGHHSFRQYROXWLRQDOQHXUDOQHWZRUNV $.UL]KHYVN\,6XWVNHYHU*(+LQWRQ$GYDQFHVLQQHXUDOLQIRUPDWLRQ SURFHVVLQJV\VWHPV
14. ILSVRC 2013 tasks 1. Detection 2. Classification 3. Classification with localization
15. ILSVRC 2014 tasks 1. Detection 2. Classification 3. Classification with localization
16. Object Detection KWWSFVQVWDQIRUGHGXV\OODEXVKWPO
17. Deep Learning KWWSVGHYEORJVQYLGLDFRP
18. ILSVRC 2015 tasks 1. Object detection 2. Object localization 3. *Object detection from video 4. *Scene classification
19. ILSVRC 2016 tasks 1. Object localization 2. Object detection 3. Object detection from video 4. Scene classification 5. Scene parsing
20. Cookpad 2016
21. 画像データセット 1997年~ レシピ数：国内約260万 + 国外 + つくれぽ + 手順写真 17言語、60カ国 ※数字は2017年02月時点のものです
22. 画像解析の研究関心 • これは料理ですか？ • どの料理ですか？ • 料理はどこですか？ • 。。。 Part 2
23. Where is the food? 料理はどこですか？
24. ゴール )LQGIRRGLQWKHLPDJHGUDZ DERXQGLQJER[DURXQGWKH IRRGLWHPLQFOXGLQJWKH GLVKLIYLVLEOH
25. ,IWKHUHDUHPXOWLSOHLWHPV GUDZDERXQGLQJER[ DURXQGHDFKRQH ゴール
26. ground truth bounding box > 0.9 We count it as a positive detection if Intersection over Union ratio is greater than 0.9. ƴ
27. QXPEHURIWUXHSRVLWLYHV QXPEHURIJURXQGWUXWKER[HV ƴ ƴ ƴ QXPEHURIWUXHSRVLWLYHV QXPEHURIJHQHUDWHGER[HV 再現率 (precision) (recall) ƴ ƴ
28. Methods
29. 1. Build a classifier 2. Pick Regions of Interest 3. Run classifier on each region 4. Remove duplicate detections IDEA
30. Fast, Faster R-CNN  5LFKIHDWXUHKLHUDUFKLHVIRUDFFXUDWHREMHFWGHWHFWLRQDQGVHPDQWLFVHJPHQWDWLRQ 5RVV*LUVKLFN-HII'RQDKXH7UHYRU'DUUHOO-LWHQGUD0DOLN  )DVWHU5&117RZDUGV5HDO7LPH2EMHFW'HWHFWLRQZLWK5HJLRQ3URSRVDO1HWZRUNV 6KDRTLQJ5HQ.DLPLQJ+H5RVV*LUVKLFN-LDQ6XQ  )DVW5&11 5RVV*LUVKLFN
31. 問題 1. Computational cost 2. Context is important 3. ...but context can be confusing. KDQG IRRG JUDVV IRRG KWWSSL[DED\FRP
32. Single Shot Detector  66'6LQJOH6KRW0XOWL%R['HWHFWRU :HL/LX'UDJRPLU$QJXHORY'XPLWUX(UKDQ&KULVWLDQ6]HJHG\ 6FRWW5HHG&KHQJ<DQJ)X$OH[DQGHU&%HUJ
33. Either The Least Or Most Employable Person Ever 7KH+XIILQJWRQ3RVW JLWKXEFRPSMUHGGLH SMUHGGLHFRPGDUNQHW ZZZNDJJOHFRPSMUHGGLH Joseph Redmon
34. You Only Look Once  <RX2QO\/RRN2QFH8QLILHG 5HDO7LPH2EMHFW'HWHFWLRQ -RVHSK5HGPRQ6DQWRVK'LYYDOD5RVV *LUVKLFN$OL)DUKDGL 'HF <2/2%HWWHU)DVWHU 6WURQJHU -RVHSK5HGPRQ$OL)DUKDGL
35. <RX2QO\/RRN2QFH8QLILHG5HDO7LPH2EMHFW'HWHFWLRQ -RVHSK5HGPRQ6DQWRVK'LYYDOD5RVV*LUVKLFN$OL)DUKDGL YOLO in Context