20180210_Cookpad_TechConf2018_YoheiKIKUTA

Talk at Cookpad TechConf 2018 ( https://techconf.cookpad.com/2018/ ).

Transcript

1. ٠ా ངฏ
   ݚڀ։ൃ෦
   Solve “unsolved”
   image recognition problems
   in service applications
   Cookpad Inc.
   Feb 10th, 2018
   

   View Slide

2. ࣗݾ঺հ → https://github.com/yoheikikuta/resume
   ɾ໊લɿ٠ా ངฏ @yohei_kikuta
   ɾॴଐɿݚڀ։ൃ෦
   ɾݞॻɿϦαʔνΤϯδχΞ
   ɹɹɹɹത࢜ʢཧֶʣ
   ɾઐ໳ɿը૾෼ੳ
   ɾ޷෺ɿম͖ᰤࢠɺण࢘ɺDr Pepper
   2
   

   View Slide

3. ໨࣍
   3
   ɾݚڀ։ൃ෦ͷ঺հ
   ɾ࣮ۀ຿ʹ͓͚Δը૾෼ੳͷࠔ೉
   ɾΫοΫύουͰ۩ମతʹը૾෼ੳʹऔΓ૊ΜͰ͍Δࣄྫͷ঺հ
   - ྉཧ͖Ζ͘ɿ೚ҙͷը૾ͷྉཧ/ඇྉཧ෼ྨ
   - Ϩγϐ෼ྨɿྉཧը૾ͷϨγϐΧςΰϦ෼ྨ
   - ϞόΠϧ࣮૷ɿϞόΠϧͰಈ͘ྉཧը૾෼ྨͷϞσϧߏங
   ɾ·ͱΊ
   

   View Slide

4. ݚڀ։ൃ෦ͷϝϯόʔ
   4
   ৽نٕज़Λ׆༻ͨ͠αʔϏεͷ։ൃɾվળ
   [ର৅ྖҬ]

   σʔλ࡞੒ɺը૾෼ੳɺࣗવݴޠॲཧ

   ର࿩ɺ৯จԽɺIoTσόΠεɺ։ൃج൫੔උ
   

   View Slide

5. ݚڀ։ൃ෦ͷऔΓ૊Έ
   5
   

   View Slide

6. ݚڀ։ൃ෦ͷऔΓ૊Έɿը૾෼ੳ
   6
   ྉཧ/ඇྉཧ൑ఆ http://techlife.cookpad.com/entry/2017/09/14/161756

   http://techlife.cookpad.com/entry/2017/11/08/132538
   ྉཧ/ඇྉཧ෼ྨɺϨγϐ෼ྨɺ௒ղ૾ɺϑΟϧλ࡞੒ɺͳͲ
   

   View Slide

7. ݚڀ։ൃ෦ͷऔΓ૊Έɿࣗવݴޠॲཧ
   7
   http://techlife.cookpad.com/entry/2015/09/30/170015

   http://techlife.cookpad.com/entry/2017/10/30/080102
   MYϑΥϧμͷࣗಈ੔ཧɺࡐྉදهͷਖ਼نԽɺͳͲ
   

   View Slide

8. ݚڀ։ൃ෦ͷऔΓ૊ΈɿAmazon Echo ޲͚ͷΫοΫύουεΩϧ
   http://techlife.cookpad.com/entry/2017/11/21/181206

   http://techlife.cookpad.com/entry/2017/11/22/alexa-skilldesign
   

   View Slide

9. ݚڀ։ൃ෦ͷऔΓ૊ΈɿΠϯϑϥ؀ڥͱαʔϏε΁ͷܨ͗ࠐΈ
   9
   https://youtu.be/Jw9CpQkCvpM
   

   View Slide

10. ݚڀ։ൃ෦ͷऔΓ૊Έɿ৯จԽݚڀ
    10
    https://cookpad.com/kitchen/14604664

    https://info.cookpad.com/pr/news/press_2016_1208
    

    View Slide

11. ݚڀ։ൃ෦ͷऔΓ૊Έɿֶज़ํ໘΁ͷߩݙ
    11
    ɾ֤छֶձ΁ͷ࿦จ౤ߘ΍εϙϯαʔ

    ɹ IJCAI, SIGIR, JSAI, ALNP, IPSJ, CEA, XSIG2017, …
    ɾݚڀ༻ʹσʔληοτΛఏڙ

    ɹ https://www.nii.ac.jp/dsc/idr/cookpad/cookpad.html
    ɾίϯϖςΟγϣϯ༻ʹ໰୊ઃఆͱσʔληοτΛఏڙ

    ɹ- ਓ޻஌ೳٕज़ઓུձٞ౳ओ࠵ ୈ1ճAIνϟϨϯδίϯςετ

    ɹ https://deepanalytics.jp/compe/31 20170331ऴྃ

    ɹ- JSAI Cup 2018 ਓ޻஌ೳֶձσʔλղੳίϯϖςΟγϣϯ

    ɹ https://deepanalytics.jp/compe/59 20180329క੾
    

    View Slide

12. ࠓ೔͸ը૾෼ੳͷ࿩Λ͠·͢
    

    View Slide

13. ࣮ۀ຿ʹ͓͚Δը૾෼ੳͷࠔ೉ɿͦ΋ͦ΋ղ͚͍ͯΔ໰୊Ͱ͸ʁ
    13
    ෼ྨ໰୊͸ʮղ͚ͨʯ
    0
    7.5
    15
    22.5
    30
    2010 2011 2012 2013 2014 2015 2016 2017
    2.25
    2.99
    3.57
    7.41
    11.2
    15.3
    25.8
    28.2
    Classification error [%]
    Deep Learning !!
    human ability
    

    View Slide

14. ࣮ۀ຿ʹ͓͚Δը૾෼ੳͷࠔ೉ɿͦ΋ͦ΋ղ͚͍ͯΔ໰୊Ͱ͸ʁ
    14
    ෼ྨ໰୊͸ʮղ͚ͨʯ※ཧ૝తͳঢ়گԼͰ͸
    ɾద੾ͳϥϕϧͷ෇༩

    ɹ Ұఆ౓Ҏ্ͷ࣭Ͱ֤ը૾ʹϥϕϧ͕෇༩͞Ε͍ͯΔ
    ɾద੾ͳΧςΰϦͷઃܭ

    ɹ ࢹ֮తʹ෼ྨͰ͖ΔΑ͏ͳΧςΰϦʹ෼͚ΒΕ͍ͯΔ
    ɾclosed set

    ɹ ֶशσʔλͷ෼෍ͱςετσʔλͷ෼෍͕౳͍͠
    

    View Slide

15. ࣮ۀ຿ʹ͓͚Δը૾෼ੳͷࠔ೉ɿͦ΋ͦ΋ղ͚͍ͯΔ໰୊Ͱ͸ʁ
    15
    ཧ૝ ≠ ݱ࣮
    ɾద੾ͳϥϕϧͷ෇༩ɿ˚ ͋Δఔ౓σʔλྔͰΧόʔՄೳ
    ɾద੾ͳΧςΰϦͷઃܭɿ☓ {ϥʔϝϯ, ύελ, ΧϧϘφʔϥ} ͳͲ
    ɾclosed setɿ☓ ςετσʔλ͸ଟ༷Ͱ͔ͭಈత
    ͦ΋ͦ΋࣮αʔϏεͰղ͘΂͖໰୊͸ଟ͘ͷ৔߹ ”ؒҧ͍ͬͯΔ”

    → trial & error Ͱղ͘΂͖໰୊͕Կ͔Λ໌Β͔ʹ͍ͯ͘͠ͷ͕ओ୊
    

    View Slide

16. զʑ͕ͲͷΑ͏ʹͦΕΒͷ໰୊ʹऔΓ૊ΜͰ͍Δ͔ʁ
    ɾྉཧ͖Ζ͘Ͱͷػೳ

    ɹ Ϣʔβͷ࣋ͭը૾Λྉཧ/ඇྉཧ෼ྨ
    ɾϨγϐ෼ྨͰͷػೳ

    ɹ ྉཧࣸਅΛద੾ͳϨγϐʹ෼ྨ
    ɾྉཧ/ඇྉཧ෼ྨϞσϧͷϞόΠϧ࣮૷

    ɹ ϞσϧΛϞόΠϧʹҠ২ͯ͠ϓϥΠόγʔͷ໰୊ͳͲΛղܾ
    16
    ۩ମతͳࣄྫͷ঺հ
    

    View Slide

17. ۩ମతͳࣄྫɿྉཧ͖Ζ͘Ͱͷྉཧ/ඇྉཧ෼ྨ
    ɾTechConf2017 Ͱ΋঺հ
    ɾྉཧͷࣸਅΛࣗಈతʹ෼ྨͯ͠දࣔɹ
    ɹ- CNNʹΑΔ෼ྨͰྉཧը૾Λநग़
    ɹ- ৯ࣄͷৼΓฦΓ΍ͭ͘ΕΆͷଅਐ
    ɾ20180206࣌఺Ͱ
    ɹ- Ϣʔβ਺ɿ19ສਓҎ্
    ɹ- ྦྷੵྉཧຕ਺ɿ1900ສຕҎ্
    17
    ྉཧ͖Ζ͘ͷਐԽͱݱࡏ
    https://speakerdeck.com/ayemos/real-world-machine-learning
    

    View Slide

18. ۩ମతͳࣄྫɿྉཧ͖Ζ͘Ͱͷྉཧ/ඇྉཧ෼ྨ
    18
    ػցֶशͷ؍఺͔Βॏཁͳ఺
    ɾΫΠοΫελʔτ
    ɹը૾෼ੳͷ஌ݟ͕ෆे෼ͳͱ͖͔Β CaffeNet Ͱૉૣ࣮͘૷
    ɾϞσϧͷվળͱۤखͳΧςΰϦͷߟྀ
    ɹ Inception V3 ͷ࢖༻΍ multi-class Ϟσϧͷ࢖༻
    ɾςετσʔλͷ֦ॆ
    ɹࣾһ͔ΒσʔλΛूΊ࣮ͯ؀ڥʹ͍ۙঢ়گͰݕূ
    ɾہॴੑΛऔΓࠐΉͨΊͷύονԽ
    ɹࣸਅͷҰ෦ʹྉཧ͕͍ࣸͬͯΔঢ়گʹదԠ
    http://techlife.cookpad.com/entry/2017/09/14/161756

    http://techlife.cookpad.com/entry/2017/11/08/132538
    

    View Slide

19. ۩ମతͳࣄྫɿྉཧ͖Ζ͘Ͱͷྉཧ/ඇྉཧ෼ྨ
    19
    ɾہॴੑΛऔΓࠐΉͨΊͷύονԽ
    ɹ- ෦෼తͳྉཧը૾Λर͍͍ͨʢsegmentation ·Ͱ͸͍Βͳ͍ʣ
    ɹ- ը૾Λύονʹ෼͚ͯͦΕͧΕͰ෼ྨ͢ΔϞσϧΛߏங
    

    View Slide

20. ۩ମతͳࣄྫɿྉཧࣸਅͷϨγϐΧςΰϦ෼ྨ
    ɾྉཧࣸਅΛద੾ͳϨγϐΧςΰϦʹ෼ྨ
    ɾ୯७ͳ෼ྨʹݟ࣮͑ͯ͸ඇৗʹ೉͍͠
    ɹ- open set ʹ͓͚Δ༧ଌ
    ɹ- ༧ଌର৅ͷΧςΰϦͷઃܭ
    ɹ- ྨࣅΧςΰϦͷଘࡏ
    ɾ༷ʑͳ࣮ݧΛܦͯϞσϧΛ࡞੒
    ɹ- ྨࣅΧςΰϦͷ෼ྨͱ precision ʹ஫ྗ
    20
    ྉཧ͖Ζ͘ͷͦͷઌ΁
    

    View Slide

21. ۩ମతͳࣄྫɿྉཧࣸਅͷϨγϐΧςΰϦ෼ྨ
    21
    ػցֶशͷ؍఺͔Βॏཁͳ఺
    ɾ෼ྨͷର৅ͱͳΔΧςΰϦͷઃܭ
    ɹαʔϏεͱ݉Ͷ߹͍ΛਤΓͭͭ༧ଌର৅ΧςΰϦΛબఆ
    ɾྨࣅΧςΰϦʹର͢Δ෼ྨ
    ɹ ΧςΰϦؒͷྨࣅ౓͕େ͖͘ҟͳΔͷͰఆྔతͳධՁ๏ΛߟҊ
    ɾopen set ͳ෼ྨ໰୊ʹ͓͚Δ precision ͷ֬อ
    ɹOne vs. Rest ෼ྨثΛ૊Έ߹Θͤͯ precision ΛߴΊΔΑ͏ௐ੔
    ɾධՁํ๏ͷઃܭ
    ɹΦϯϥΠϯͰ͸ϑΟʔυόοΫɺΦϑϥΠϯͰ͸σʔλ࡞੒
    ࿦จ : https://arxiv.org/abs/1802.01267
    

    View Slide

22. ۩ମతͳࣄྫɿྉཧࣸਅͷϨγϐΧςΰϦ෼ྨ
    22
    ɾΧςΰϦߏ଄ͱΧςΰϦؒྨࣅ౓ͷఆࣜԽ
    ɹ- ੜ੒Ϟσϧͷ؍఺ɺϥϕϧ෇͚ͷ֬཰ੑɺ༧ଌϥϕϧͱͷؔ܎
    ɹ- ֶशϞσϧͷ ”ޡ෼ྨ” ͔ΒΧςΰϦؒྨࣅ౓Λఆٛ
    

    View Slide

23. ۩ମతͳࣄྫɿྉཧࣸਅͷϨγϐΧςΰϦ෼ྨ
    23
    ɾ࣮ࡍͷΧςΰϦઃܭͷεςοϓ
    ɹ- ϝλσʔλ͔ΒશΧςΰϦΛநग़ʢશ෦Ͱ1,000ΧςΰϦఔ౓ʣ
    ↓
    ɹ- ࢹ֮తͰͳ͍΋ͷ΍਺͕গͳ͍΋ͷΛআ֎ʢେࡼྉཧͳͲʣ
    ↓
    ɹ- αʔϏεʹ͓͍ͯ༗༻ͦ͏ͳ΋ͷΛਓྗͰநग़ʢ͜͜ॏཁʣ
    ↓
    ɹ- ޡ෼ྨʹجͮ͘ྨࣅ౓Ͱ౷ഇ߹ʢ࠷ऴతʹ50ΧςΰϦఔ౓ʣ
    ྫʣ΍͖ͦ͹ͱϏʔϑϯΛಉ͡ΧςΰϦͱͯ͠౷߹
    

    View Slide

24. ۩ମతͳࣄྫɿྉཧࣸਅͷϨγϐΧςΰϦ෼ྨ
    24
    ɾprecision ΛߴΊΔͨΊʹ One vs. Rest ෼ྨثʹΑΔϞσϧΛߏங
    ɹ- ར఺ɿݸʑͷΧςΰϦʹ߹Θͤͨॊೈͳઃܭ͕Մೳ
    ɹ- ܽ఺ɿॱ൪΍ᮢ஋ͳͲ hand crafted ͳ෦෼΋গͳ͘ͳ͍
    feature
    extractor
    for c in {αϥμ, ύελ, …}
    0
    1
    ྉཧը૾Ͱ

    pre-train ͨ͠

    Inception V3
    1
    0
    αϥμ
    next
    next
    f2 ͸ྨࣅ౓͕ߴ͍ΧςΰϦ
    ͚ͩΛूΊֶͯशͨ͠
    One vs. Rest ෼ྨث
    

    View Slide

25. ۩ମతͳࣄྫɿը૾෼ྨϞσϧͷϞόΠϧ΁ͷҠ২
    ɾղܾ͍ͨ͠໰୊͸·ͩ·ͩ͋Δ
    ɹ- ଈ࣌ੑɿࡱͬͨࣸਅ͕Ͱ͖Δ͚ͩૣ͘൓ө͞Εͯཉ͍͠
    ɹ- ػີੑɿϢʔβͷࣸਅ͸ݟ͍ͯͳ͍͕৺ཧత఍߅͸࢒Δ
    ɹ- ֦େੑɿܭࢉࢿݯΛ؆୯ʹεέʔϧ͍ͤͨ͞
    ɹ- Ԡ༻ੑɿΞϓϦ಺Ͱ෼ྨ༷ͯ͠ʑͳαʔϏεʹԠ༻͍ͨ͠
    ɾϞόΠϧ࣮૷ͷػӡ
    ɹ - ܰྔͰߴੑೳͳϞσϧ͕֤छଘࡏ ʢSqueezeNet ΍ MobileNetʣ
    ɹ - ֤छϥΠϒϥϦͷॆ࣮ʢCore ML ΍ TensorFlow Liteʣ
    25
    ϞόΠϧ΁ͷҠߦ
    

    View Slide

26. ۩ମతͳࣄྫɿը૾෼ྨϞσϧͷϞόΠϧ΁ͷҠ২
    26
    ػցֶशͷ؍఺͔Βॏཁͳ఺
    ɾਫ਼౓Λग़དྷΔݶΓམͱͣܰ͞ྔͳϞσϧΛ࡞Δ
    ɹܰྔԽΛ໨తͱͨ͠ߏ଄΍ྔࢠԽͳͲͷཧղ
    ɾϞόΠϧଆͱͷ࿈ܞ
    ɹ iOS ΍ Android ଆͷ஌ݟ͕ෆՄܽ
    ɾ৘ใ͕গͳ͍தͰͷϓϩδΣΫτਪਐ
    ɹ ػցֶशͱϞόΠϧͷͦΕͧΕͷྖҬͰਂ͍ཧղ͕ॏཁ
    ɾϥΠϒϥϦͷόʔδϣϯґଘੑͳͲΛద੾ʹѻ͏
    ɹྫʣcoremltools ͸ 201802 ·Ͱ͸ python 2.7 ܥͰͷΈར༻Մ
    

    View Slide

27. ۩ମతͳࣄྫɿը૾෼ྨϞσϧͷϞόΠϧ΁ͷҠ২
    27
    ɾྉཧ/ඇྉཧ෼ྨϞσϧΛϞόΠϧʹҠ২
    ɹ- MobileNet ͱہॴԽͷͨΊͷύονԽΛ߹Θͤͨߏ଄
    ɹ- αʔό্ͷ࣮ݧʢը૾20,000ຕఔ౓ʣͰ 1% ఔ౓ͷਖ਼౴཰ͷࠩ
    ɹ- iOS, Android ڞʹ࣮ػͰݕূ͓ͯ͠Γಉఔ౓ͷੑೳ
    ɾBristol ΦϑΟε΁ͷग़ு࣌ʹਐΊͨϓϩδΣΫτ
    ɹ- iOS, Android ΤϯδχΞʹڠྗͯ͠΋Β͍ҰؾʹਐΜͩ
    ɹ- ࠃ಺֎ͰਐΊ͍͚ͯͦ͏ͳτϐοΫ
    

    View Slide

28. ۩ମతͳࣄྫɿը૾෼ྨϞσϧͷϞόΠϧ΁ͷҠ২
    28
    ɾAndroid (Pixel 2 at Bristol) Ͱͷ࣮ݧ݁Ռ
    Original Quantized
    Model Size 12 [MB] 3.3 [MB]
    Accuracy 0.97 0.97
    Precision 0.98 0.98
    Recall 0.96 0.96
    CPU Usage 40-60 [%] 40-60 [%]
    Memory Usage 120 [MB] 90 [MB]
    FPS 7.54 [FPS] 7.72 [FPS]
    DEMO
    

    View Slide

29. ը૾෼ੳͷίϯϖ΍ͬͯ·͢ʂ
    JSAI Cup 2018 క੾ : 20180329
    https://deepanalytics.jp/compe/59
    

    View Slide

30. ·ͱΊ
    30
    ໰୊Λఆٛ͠ɺͦΕΛਵ࣌ߋ৽ͯ͠ղ͍͍ͯ͘
    ɾ࣮ۀ຿Ͱͷը૾෼ੳͷ໰୊͸·ͩ·ͩ “ղ͚ͯͳ͍“

    ɹ ਖ਼֬ʹ͸ղ͘΂͖໰୊͕໌֬ʹఆٛͰ͖͍ͯΔ͜ͱ͕গͳ͍
    ɾࢼߦࡨޡͷ຤ʹͦΕΛݱঢ়ͷٕज़Ͱղ͚Δ໰୊ʹམͱ͠ࠐΉ

    ɹ ը૾෼ੳͷཁૉٕज़͸੒ख़͖͍ͯͯͯ͜͠Ε͕ॏཁͳϑΣʔζ

    ɹ ΫοΫύουͰ͸ྉཧ͖Ζ͘΍Ϩγϐ෼ྨʹը૾෼ੳΛಋೖ
    ɾϞόΠϧ΁ͷҠ২΍ಈըͳͲ͕ը૾෼ੳͷ࣍ͷ໘നͦ͏ͳྖҬ
    

    View Slide

31. ࠷ޙʹɿΫοΫύου͸ʢগͳ͘ͱ΋ࣗ෼ʹͱͬͯ͸ʣಇ͖΍͍͢
    31
    ݚڀ։ൃ෦Ͱಇ͘͜ͱ = ྑήʔ
    ɾྑετʔϦʔ

    ɹʮຖ೔ͷྉཧΛָ͠Έʹ͢Δʯͱ͍͏ϛογϣϯͷԼͰڠಇ
    ɾߴࣗ༝౓

    ɹ৽͍͠௅ઓʹॏ͖Λஔ͍͍ͯͯ trial & error Λਪ঑
    ɾָγεςϜ

    ɹैۀһ͕ಇ͖΍͘͢ύϑΥʔϚϯεΛग़͠΍͍͢؀ڥ
    

    View Slide

32. [એ఻] ਓ޻஌ೳֶձओ࠵ͷNIPS2018ใࠂձͰൃද͠·͢
    32
    https://www.ai-gakkai.or.jp/no74_jsai_seminar/
    

    View Slide