Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Calorie Estimation in a Real-World Recipe Service/iaai-20

j.harashima
February 09, 2020
12k

Calorie Estimation in a Real-World Recipe Service/iaai-20

j.harashima

February 09, 2020
Tweet

Transcript

  1. $BMPSJF&TUJNBUJPO
    JOB3FBM8PSME3FDJQF4FSWJDF
    +VO)BSBTIJNB .BLPUP)JSBNBUTV 4BUPTIJ4BOKP
    $PPLQBE*OD

    View full-size slide

  2. 4VNNBSZ
    8FIBWFJOUSPEVDFEUIFGPMMPXJOHUISFFUBTLT
    JOUP$PPLQBE
    $BMPSJFFTUJNBUJPO NBJOUBTL

    *OHSFEJFOUOPSNBMJ[BUJPO TVCUBTL

    4FSWJOHFTUJNBUJPO TVCUBTL

    8FIBWFBQQMJFENBDIJOFMFBSOJOH .-
    CBTFE
    NFUIPETGPSUIFTFUBTLTBOEEFQMPZFEUIFNJO
    PVSSFBMXPSMESFDJQFTFSWJDF

    View full-size slide

  3. $PPLQBE
    1MBUGPSNGPSQFPQMFUPTIBSF

    SFDJQFJEFBTBOEDPPLJOHUJQT
    NJMMJPOVTFSTJODPVOUSJFT

    NJMMJPOSFDJQFT IBMGBSFJO+BQBOFTF

    View full-size slide

  4. $BMPSJF&TUJNBUJPO

    &TUJNBUFUIFOVNCFSPGDBMPSJFT

    QFSTFSWJOHJOBSFDJQF
    DBMPSJF XIPMF

    BNPVOU

    View full-size slide

  5. 6TFJOHSFEJFOUTBOEBNPVOU
    TVNPGUIFDBMPSJFTJOUIFJOHSFEJFOUT
    OVNCFSPGTFSWJOHTPGUIFSFDJQF
    $BMPSJF&TUJNBUJPO

    c(r) =

    i∈Ir
    c(i) ⋅ q(i)/100
    s(r)

    View full-size slide

  6. HSBNTPGUIFJOHSFEJFOU

    JOUIFSFDJQF
    )PXEPXFPCUBJOc(i)
    γϝδ

    TIJNFKJNVTISPPN


    χϯδϯ

    DBSSPU


    ʜ ʜ

    JOHSFEJFOUEJDUJPOBSZ
    ͠Ί͡
    TIJNFKJNVTISPPN

    ͠Ί͡JTO`UUIFTBNF

    BTγϝδ
    c(r) =

    i∈Ir
    c(i) ⋅ q(i)/100
    s(r)
    *OHSFEJFOU/PSNBMJ[BUJPO

    DBMPSJFTQFSHSBNT

    PGBOJOHSFEJFOU

    View full-size slide

  7. 4FWFSBMJTTVFTNVTUCFTPMWFE
    1FPQMFDBOEFTDSJCFUIFJOHSFEJFOUTJOUIFJSSFDJQFTGSFFMZ JONBOZEJ⒎FSFOU

    XBZT
    FH ͠ΐ͏Ώ TPZTBVDF
    IBTNPSFUIBOFYQSFTTJPOTJO$PPLQBE TVDIBT
    γϣ΢Ϣ ͠ΐ͏༉ ͓ে༉ BOEὑ͠ΐ͏༉
    *OHSFEJFOU/PSNBMJ[BUJPO

    0SJHJOBMGPSN $BOPOJDBMGPSN
    0SUIPHSBQIJD
    WBSJBOUT
    ਓࢀ
    DBSSPU

    ʹΜ͡Μ
    DBSSPU

    "CCSFWJBUJPOT
    ৽ͨ·
    TQSJOHPOJPO

    ৽ͨ·Ͷ͗
    TQSJOHPOJPO

    4QFMMJOH
    NJTUBLFT
    ΞϘΨυ
    BWPDBEP

    ΞϘΧυ
    BWPDBEP

    0UIFS
    അླḕ
    QPUBUP

    ͡Ό͕͍΋
    QPUBUP

    0SJHJOBMGPSN $BOPOJDBMGPSN
    4ZNCPMT
    ὑຯḩ
    ὑNJTP

    ຯḩ
    NJTP

    1BSFOUIFTFT
    ༉ ᖱΊ༻

    PJM GPSGSZJOH


    PJM

    1SFpY
    ͓࠭౶
    TVHBS

    ࠭౶
    TVHBS

    0UIFS
    ͋Ε͹ϩʔζϚϦʔ
    SPTFNBSZ JGBOZ

    ϩʔζϚϦʔ

    SPTFNBSZ

    &YBNQMFTPGJOHSFEJFOUOPSNBMJ[BUJPO

    View full-size slide

  8. "QQMZBOFODPEFSEFDPEFSNPEFM
    5IFFODPEFSUBLFTUIFPSJHJOBMJOHSFEJFOUOBNFBTBOJOQVU

    5IFEFDPEFSHFOFSBUFTBDBOPOJDBMGPSNBTBOPVUQVU
    %PO`UVTFBUUFOUJPONFDIBOJTNTCFDBVTFUIFJOHSFEJFOUOBNFTBSFTIPSUTP

    UIFNFUIPEXPVMEOPUCFF⒎FDUJWF
    *OHSFEJFOU/PSNBMJ[BUJPO

    y1
    yJ
    y2

    canonical form of the ingredient
    decoder
    original string of an ingredient
    encoder
    x1
    x2
    xI

    softmax softmax softmax

    View full-size slide

  9. )PXEPXFPCUBJOs(r)

    OVNCFSPGTFSWJOHTPGUIFSFDJQF
    খുݸ෼
    TNBMMCPXMT

    IPXNBOZQFPQMFJT
    UIJTSFDJQFGPS
    c(r) =

    i∈Ir
    c(i) ⋅ q(i)/100
    s(r)
    4FSWJOH&TUJNBUJPO

    View full-size slide

  10. 4FSWJOH&TUJNBUJPO

    4FWFSBMJTTVFTNVTUCFTPMWFE
    1FPQMFBMTPGSFFMZFYQSFTTBOBNPVOUJOUIFJSSFDJQFT
    5JUMF "NPVOU PGTFSWJOHT
    ίίφοπΦΠϧೖΓ΄͏ΕΜ૲ΧϨʔ
    4QJOBDI$VSSZXJUI$PDPOVU0JM

    ࡼ෼
    QMBUFT


    ͜ΜͿɾͻ͖͡ɾ΄ͨͯ͝͸Μ
    .JYFE3JDFXJUI,PNCVɾ)JKJLJɾ4DBMMPQ

    ߹෼
    HP


    άϥϊʔϥΫοΩʔ
    (SBOPMB$PPLJFT


    QJFDFT


    ύϯϓΩϯλϧτ
    1VNQLJO5BSU

    DNλϧτܕ
    DNUBSUNPME


    &YBNQMFTPGTFSWJOHFTUJNBUJPO
    "MUIPVHIUIFSFBSFTFWFSBMBOTXFSTGPSTPNFSFDJQFT XFEPOPUQVSTVFUIJTJTTVFJOPVSTFSWJDF

    View full-size slide

  11. 4FSWJOH&TUJNBUJPO

    "QQMZUXPOFVSBMDMBTTJpDBUJPONPEFMT
    5IFFODPEFSUBLFTBUJUMFBOEPSBNPVOUJOBSFDJQFBTBOJOQVU

    5IFEFOTFMBZFSTUSBOTGPSNUIFPVUQVUPGUIFFODPEFS T
    JOUPUIFOVNCFSPG

    TFSWJOHT
    dense
    title or amount
    encoder
    xa1
    xa2
    xaI

    softmax
    y
    serving
    dense
    amount
    encoder
    xa1
    xa2
    xaJ

    title
    encoder
    xt1
    xt2
    xtI

    softmax
    y
    serving
    4JOHMFTPVSDFNPEFM .VMUJTPVSDFNPEFM

    View full-size slide

  12. &YQFSJNFOU*OHSFEJFOU/PSNBMJ[BUJPO
    %BUB
    8FDPMMFDUFE EJTUJODUJOHSFEJFOUTGSPN$PPLQBE OPSNBMJ[FEUIFN

    NBOVBMMZ UIFOSFNPWFE XIPTFOPSNBMJ[FEWFSTJPOTXFSFUIFTBNFBT

    UIFJSPSJHJOBMTUSJOHT
    8FVTFEUIFSFNBJOJOH JOHSFEJFOUT
    .FUIPET
    3&SFHVMBSFYQSFTTJPOCBTFENFUIPEXIJDISFNPWFTTZNCPMTBOE

    QBSFOUIFTFTGSPNUIFPSJHJOBMTUSJOHT
    .-PVS.-CBTFENFUIPE

    View full-size slide

  13. &YQFSJNFOU*OHSFEJFOU/PSNBMJ[BUJPO
    3FTVMUT
    5IF.-CBTFENFUIPETVDDFTTGVMMZOPSNBMJ[FENPSF

    UIBOPGUIFJOHSFEJFOUT
    0SJHJOBMGPSN $BOPOJDBMGPSN
    0SUIPHSBQIJD
    WBSJBOUT
    ͔ͨ͘Γค
    QPUBUPTUBSDI

    ย܀ค
    QPUBUPTUBSDI

    "CCSFWJBUJPOT
    ڇͻ͖
    HSPVOECFFG

    ڇͻ͖೑
    HSPVOECFFG

    4QFMMJOH
    NJTUBLFT
    ਫ໊
    NJ[VOB

    ਫࡊ
    NJ[VOB

    0UIFS
    ࢳ͏Γ
    TQBHIFUUJTRVBTI

    ͦ͏ΊΜ͔΅ͪΌ
    TQBHIFUUJTRVBTI

    0SJHJOBMGPSN $BOPOJDBMGPSN
    4ZNCPMT
    ˑখഴค
    ˑqPVS

    খഴค
    qPVS

    1BSFOUIFTFT
    Ϩʔζϯ ͓޷ΈͰ

    SBJTJO BTZPVMJLF

    Ϩʔζϯ
    SBJTJO

    1SFpY
    ͓౾෗
    UPGV

    ౾෗
    UPGV

    0UIFS
    ྫྷ͑ͨτϚτ
    DIJMMFEUPNBUP

    τϚτ
    UPNBUP

    4VDDFTTGVMFYBNQMFTPGJOHSFEJFOUOPSNBMJ[BUJPO
    "DDVSBDZ
    3&
    .-

    View full-size slide

  14. 5XPUZQFTPGOPSNBMJ[BUJPOFSSPS
    8SPOHMZHFOFSBUJOHOPOFYJTUFOUJOHSFEJFOUT
    8SPOHMZHFOFSBUJOHEJ⒎FSFOUJOHSFEJFOUT
    'VUVSFXPSLSFRVJSFE
    5PDPOTJEFSUIFFYJTUFODFPGUIFPVUQVUBOEJUTTJNJMBSJUZUPUIFJOQVU

    )BSBTIJNBBOE:BNBEB

    5PFYQBOEUIFUSBJOJOHTFU
    &YQFSJNFOU*OHSFEJFOU/PSNBMJ[BUJPO
    *OQVU )VNBOPVUQVU 4ZTUFNPVUQVU
    χΨ΢Ϧ
    CJUUFSNFMPO

    ΰʔϠ
    CJUUFSNFMPO

    ʹ͕͕Γ
    OPOFYJTUFOUXPSE

    ḕᤸ
    ZBN

    ࢁҶ
    ZBN

    ͡Ό͕͍΋
    QPUBUP

    View full-size slide

  15. &YQFSJNFOU4FSWJOH&TUJNBUJPO
    %BUB
    8FDPMMFDUFE SFDJQFTGSPN$PPLQBEUIFOFTUJNBUFEUIFOVNCFSPG

    TFSWJOHTGSPNUIFNNBOVBMMZ
    .FUIPET
    3&SFHVMBSFYQSFTTJPOCBTFENFUIPEXIJDIFYUSBDUTBWBMVFGSPNUIF

    BNPVOUJOFBDISFDJQF
    .-PVS.-CBTFENFUIPEXIJDIVTFTDBUFHPSJFT GPSQFSTPO ʜ VQUP

    QFPQMF

    View full-size slide

  16. &YQFSJNFOU4FSWJOH&TUJNBUJPO
    3FTVMUT
    5IF.-CBTFENFUIPEBDIJFWFEBOBDDVSBDZPGNPSFUIBO
    5IFTJOHMFTPVSDFNPEFMTMJHIUMZPVUQFSGPSNFEUIFNVMUJTPVSDFNPEFM

    &TTFOUJBMMZ UIFOVNCFSPGTFSWJOHTGPSBSFDJQFDBOOPUCFFTUJNBUFEXJUIPVU

    JUTUJUMFJOGPSNBUJPO

    5IFOVNCFSPGFYBNQMFTJOPVSUSBJOJOHTFUXBTUPPTNBMMUPUSBJOUXP

    FODPEFSTTPXFXJMMFYQBOEUIFUSBJOJOHTFUJOGVUVSFXPSL
    4PVSDF 5JUMF "NPVOU "DDVSBDZ
    3& TJOHMF 㾎
    .-
    TJOHMF 㾎
    TJOHMF 㾎
    NVMUJ 㾎 㾎

    View full-size slide

  17. &YQFSJNFOU4FSWJOH&TUJNBUJPO
    ✕'BJMFECZVTJOHUIFUJUMFJOGPSNBUJPO

    0VSUSBJOJOHTFUDPOUBJOFETPNFSFDJQFTJOXIJDIPOF,PSFBOQBODBLFTFSWFEUXPQFPQMF

    ✕'BJMFECZOPUVTJOHUIFUJUMFJOGPSNBUJPO

    0OFTNBMMDSPRVFUUFJTHFOFSBMMZOPUTV⒏DJFOUGPSPOFQFSTPO

    ✕'BJMFECZOPUVTJOHUIFRVBOUJUZJOGPSNBUJPOPOUIFJOHSFEJFOUT

    'VUVSFXPSLXJMMCFOFFEFE
    5JUMF "NPVOU )VNBO
    4ZTUFN
    TJOHMF

    4ZTUFN
    NVMUJ

    γϡΨʔύϯ
    4VHBS#SFBE

    ݸ
    QJFDFT


    ૉ๿ͳϨʔζϯύϯ
    4JNQMF3BJTJO#SFBE

    ی
    MPBG


    ΋ͪ΋ͪ໺ࡊνξϛ
    $IFXZ,PSFBO7FHFUBCMF1BODBLF

    ϑϥΠύϯຕ෼
    GSZJOHQBOT


    πφϙςτͷϛχίϩοέˑ͓ห౰ʹ΋
    4NBMM5VOBBOE1PUBUP$SPRVFUUFTˑBMTPGPS-VODI#PY

    ݸ෼
    QJFDFT


    ؁͞߇͑ΊͷΫοΩʔ
    $PPLJFTXJUI-FTT4VHBS

    మ൘ຕ෼
    JSPOQMBUF


    4VDDFTTGVMBOEGBJMFEFYBNQMFTPGTFSWJOHFTUJNBUJPO


    ✕ ✕

    View full-size slide

  18. %FQMPZ
    "TBSJ$MBN3JDF

    HP

    ɹɹɹɹɹΞαϦɹ
    ɹɹɹɹɹถɹɹɹ
    ɹɹɹɹɹԘɹɹɹ
    ɹɹɹɹɹञɹɹɹ
    ɹɹɹɹɹ͠ΐ͏Ώ
    ɹɹɹɹɹΈΓΜɹ
    SJDF

    BTBSJDMBN

    TBMU

    TBLF

    TPZTBVDF

    TXFFUTBLF







    8FIBWFFTUJNBUFEUIFOVNCFSPGDBMPSJFTJOPWFS
    SFDJQFTBOEBDUVBMMZVTFUIFNJOPVSSFDJQFTFSWJDF
    6TFUIFTJOHMFTPVSDFNPEFMGPSTFSWJOHFTUJNBUJPO

    c(r) =

    i∈Ir
    c(i) ⋅ q(i)/100
    s(r)
    = 306.6

    *OHSFEJFOU

    View full-size slide

  19. 4VNNBSZ
    8FIBWFJOUSPEVDFEUIFGPMMPXJOHUISFFUBTLT

    JOUP$PPLQBE
    $BMPSJFFTUJNBUJPO NBJOUBTL

    *OHSFEJFOUOPSNBMJ[BUJPO TVCUBTL

    4FSWJOHFTUJNBUJPO TVCUBTL

    8FIBWFBQQMJFE.-CBTFENFUIPETGPSUIFTF
    UBTLTBOEEFQMPZFEUIFNJOPVSSFBMXPSMESFDJQF
    TFSWJDF
    5IFNFUIPETQFSGPSNFETV⒏DJFOUMZXFMMUPCFVTFEJOPVSTFSWJDF
    'VUVSFXPSL
    5PFYQBOEUIFOVNCFSPGJOHSFEJFOUTBOESFDJQFTUPXIJDIPVSNFUIPETDBOCF

    BQQMJFE

    View full-size slide