Food Image Object Detection and Classification

Transcript

1. Food Image Object Detection and Classification Challenges and Solutions

2. Part 1: Detection

3. 自己紹介 • リビツキ　レシェック • ポーランド出身 • 2016~ クックパッド • github:

   lunardog

4. Warning! This presentation contains images that may cause severe drooling

   and stomach grumbling. @cookpad

5. History 歴史

6. ImageNet KWWSLPDJHQHWRUJ

7. ImageNet Large Scale Visual Recognition Competition KWWSZZZLPDJHQHWRUJFKDOOHQJHV/695&

8. ILSVRC 2010 task Classification )RUHDFKLPDJHDOJRULWKPV ZLOOSURGXFHDOLVWRIDWPRVW REMHFWFDWHJRULHVLQWKH GHVFHQGLQJRUGHURI FRQILGHQFH KWWSZZZLPDJHQHWRUJFKDOOHQJHV/695&

9. ILSVRC 2011 tasks 1. Classification 2. *Classification with localization *tester

   task

10. KWWSFVQVWDQIRUGHGXV\OODEXVKWPO Classification + Localization

11. ILSVRC 2012 tasks 1. Classification 2. Classification with localization 3.

    Fine-grained classification

12. Fine-grained classification KWWSZZZLPDJHQHWRUJFKDOOHQJHV/695&

13. AlexNet ,PDJHQHWFODVVLILFDWLRQZLWKGHHSFRQYROXWLRQDOQHXUDOQHWZRUNV $.UL]KHYVN\,6XWVNHYHU*(+LQWRQ$GYDQFHVLQQHXUDOLQIRUPDWLRQ SURFHVVLQJV\VWHPV

14. ILSVRC 2013 tasks 1. Detection 2. Classification 3. Classification with

    localization

15. ILSVRC 2014 tasks 1. Detection 2. Classification 3. Classification with

    localization

16. Object Detection KWWSFVQVWDQIRUGHGXV\OODEXVKWPO

17. Deep Learning KWWSVGHYEORJVQYLGLDFRP

18. ILSVRC 2015 tasks 1. Object detection 2. Object localization 3.

    *Object detection from video 4. *Scene classification

19. ILSVRC 2016 tasks 1. Object localization 2. Object detection 3.

    Object detection from video 4. Scene classification 5. Scene parsing

20. Cookpad 2016

21. 画像データセット 1997年~ レシピ数：国内約260万 + 国外 + つくれぽ + 手順写真 17言語、60カ国

    ※数字は2017年02月時点のものです

22. 画像解析の研究関心 • これは料理ですか？ • どの料理ですか？ • 料理はどこですか？ • 。。。 Part

    2

23. Where is the food? 料理はどこですか？

24. ゴール )LQGIRRGLQWKHLPDJHGUDZ DERXQGLQJER[DURXQGWKH IRRGLWHPLQFOXGLQJWKH GLVKLIYLVLEOH

25. ,IWKHUHDUHPXOWLSOHLWHPV GUDZDERXQGLQJER[ DURXQGHDFKRQH ゴール

26. ground truth bounding box > 0.9 We count it as

    a positive detection if Intersection over Union ratio is greater than 0.9. ƴ

27. QXPEHURIWUXHSRVLWLYHV QXPEHURIJURXQGWUXWKER[HV ƴ ƴ ƴ QXPEHURIWUXHSRVLWLYHV QXPEHURIJHQHUDWHGER[HV 再現率 (precision) (recall)

    ƴ ƴ

28. Methods

29. 1. Build a classifier 2. Pick Regions of Interest 3.

    Run classifier on each region 4. Remove duplicate detections IDEA

30. Fast, Faster R-CNN  5LFKIHDWXUHKLHUDUFKLHVIRUDFFXUDWHREMHFWGHWHFWLRQDQGVHPDQWLFVHJPHQWDWLRQ 5RVV*LUVKLFN-HII'RQDKXH7UHYRU'DUUHOO-LWHQGUD0DOLN  )DVWHU5&117RZDUGV5HDO7LPH2EMHFW'HWHFWLRQZLWK5HJLRQ3URSRVDO1HWZRUNV 6KDRTLQJ5HQ.DLPLQJ+H5RVV*LUVKLFN-LDQ6XQ 

    )DVW5&11 5RVV*LUVKLFN

31. 問題 1. Computational cost 2. Context is important 3. ...but

    context can be confusing. KDQG IRRG JUDVV IRRG KWWSSL[DED\FRP

32. Single Shot Detector  66'6LQJOH6KRW0XOWL%R['HWHFWRU :HL/LX'UDJRPLU$QJXHORY'XPLWUX(UKDQ&KULVWLDQ6]HJHG\ 6FRWW5HHG&KHQJ<DQJ)X$OH[DQGHU&%HUJ

33. Either The Least Or Most Employable Person Ever 7KH+XIILQJWRQ3RVW JLWKXEFRPSMUHGGLH

    SMUHGGLHFRPGDUNQHW ZZZNDJJOHFRPSMUHGGLH Joseph Redmon

34. You Only Look Once  <RX2QO\/RRN2QFH8QLILHG 5HDO7LPH2EMHFW'HWHFWLRQ -RVHSK5HGPRQ6DQWRVK'LYYDOD5RVV *LUVKLFN$OL)DUKDGL 'HF

    <2/2%HWWHU)DVWHU 6WURQJHU -RVHSK5HGPRQ$OL)DUKDGL

35. <RX2QO\/RRN2QFH8QLILHG5HDO7LPH2EMHFW'HWHFWLRQ -RVHSK5HGPRQ6DQWRVK'LYYDOD5RVV*LUVKLFN$OL)DUKDGL YOLO in Context

36. None