Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Cooking with Food Photos

Leszek Rybicki
November 02, 2017

Cooking with Food Photos

From a food/non-food classifier to multi-class semantic segmentation from partially labeled data.

Leszek Rybicki

November 02, 2017
Tweet

More Decks by Leszek Rybicki

Other Decks in Research

Transcript

  1. $PPLJOHXJUI
    'PPE1IPUPT
    5PLZP.-,JUDIFO

    -FT[FL3ZCJDLJ

    View full-size slide

  2. *OUSP
    • Leszek Rybicki
    • github: @lunardog
    • from Poland
    • 2016 Cookpad R&D
    • really bad at selfies

    View full-size slide

  3. $)"--&/(&
    *TJUGPPE

    View full-size slide

  4. http://techlife.cookpad.com/entry/2017/09/14/161756

    View full-size slide

  5. 'PPE/POGPPE
    twitter: @teenybiscuit

    View full-size slide

  6. w QFPQMFBSFOPUGPPE
    w QFUTBSFOPUGPPE
    w QPUUFEQMBOUTBSFOPUGPPE
    w FWJMBMJFOTBSFOPUGPPE
    w JGJUIBTUFYUPOJU 

    JU`TQSPCBCMZOPUGPPE
    /PUGPPE

    View full-size slide

  7. DCNN
    global pooling
    fully connected
    RGB x 240 x 240 pixels
    2048 features x 8 x 8
    one-hot class vector
    2048 features
    food plant person pet other

    View full-size slide

  8. GPPEOPOGPPEBDDVSBDZ

    QSFDJTJPOSFDBMM

    View full-size slide

  9. 0OUIFGFODF
    w GPPEXJUIPUIFSJUFNT
    w GPPEXJUIUFYU
    w GPPE CVUTNBMMJOUIF
    QIPUP
    w FEJCMFQMBOUT
    w QFPQMFXJUIGPPE

    View full-size slide

  10. 'PPEJTTPDJBM
    w 8FDPPLBOEFBUXJUIPVS
    GBNJMZBOEGSJFOET
    w 'PPEJTBUUIFDFOUFSPG
    JNQPSUBOUFWFOUT
    w 8IFOXFVTFBCJOBSZ
    DMBTTJpFSGPSGPPEQIPUPT
    BSFO`UXFMPTJOHUIF
    IVNBODPOOFDUJPO
    https://pixabay.com/

    View full-size slide

  11. 3VMF
    "QIPUPUIBU

    DBOCFDSPQQFEUP
    DPOUBJONPTUMZGPPE

    JTBQIPUPPGGPPE

    View full-size slide

  12. DCNN
    global pooling
    fully connected
    RGB x 240 x 240 pixels
    2048 features x 8 x 8
    one-hot class vector
    2048 features
    DCNN
    global pooling
    1x1 convo layer
    RGB x 240 x 240 pixels
    2048 features x 8 x 8
    one-hot class vector
    food not food not

    View full-size slide

  13. w %$//*ODFQUJPO7
    w USBJOFEPOBMMGPPEBOE
    BMMOPOGPPEQIPUPT

    USBJOJOHJTSFBMMZGBTU

    w BWFSBHFQPPMJOH
    w IFBWZESPQPVU

    w OPHMPCBMQPPMJOHMBZFS
    BGUFSUSBJOJOH
    w VTFSFTVMUJOHIFBUNBQ
    GPSDMBTTJpDBUJPO
    DCNN
    global pooling
    1x1 convo layer
    RGB x 512 x 512 pixels
    2048 features x 14 x 14
    one-hot class vector
    food not

    View full-size slide

  14. GPPEOPOGPPEBDDVSBDZ

    QSFDJTJPOSFDBMM

    View full-size slide

  15. GPPEOPOGPPEBDDVSBDZ

    QSFDJTJPOSFDBMM

    View full-size slide

  16. http://techlife.cookpad.com/entry/2017/09/14/161756
    https://storialaw.jp/blog/3420

    View full-size slide

  17. test images from https://snappygoat.com/
    food person

    View full-size slide

  18. food person
    test images from https://snappygoat.com/

    View full-size slide

  19. $)"--&/(&
    *OHSFEJFOUT

    View full-size slide

  20. $-"44*':5)*4

    View full-size slide

  21. test images from https://snappygoat.com/

    View full-size slide

  22. test images from https://snappygoat.com/

    View full-size slide