Calorie Estimation in a Real-World Recipe Service/iaai-20

Transcript

1. $BMPSJF&TUJNBUJPO
   JOB3FBM8PSME3FDJQF4FSWJDF
   +VO)BSBTIJNB .BLPUP)JSBNBUTV 4BUPTIJ4BOKP
   $PPLQBE*OD
   

   View Slide

2. 4VNNBSZ
   8FIBWFJOUSPEVDFEUIFGPMMPXJOHUISFFUBTLT
   JOUP$PPLQBE
   $BMPSJFFTUJNBUJPO NBJOUBTL
   

   *OHSFEJFOUOPSNBMJ[BUJPO TVCUBTL
   

   4FSWJOHFTUJNBUJPO TVCUBTL
   
   8FIBWFBQQMJFENBDIJOFMFBSOJOH .-
   CBTFE
   NFUIPETGPSUIFTFUBTLTBOEEFQMPZFEUIFNJO
   PVSSFBMXPSMESFDJQFTFSWJDF
   

   View Slide

3. $PPLQBE
   1MBUGPSNGPSQFPQMFUPTIBSF

   SFDJQFJEFBTBOEDPPLJOHUJQT
   NJMMJPOVTFSTJODPVOUSJFT

   NJMMJPOSFDJQFT IBMGBSFJO+BQBOFTF
   
   

   View Slide

4. $BMPSJF&TUJNBUJPO
   
   &TUJNBUFUIFOVNCFSPGDBMPSJFT

   QFSTFSWJOHJOBSFDJQF
   DBMPSJF XIPMF

   BNPVOU
   

   View Slide

5. 6TFJOHSFEJFOUTBOEBNPVOU
   TVNPGUIFDBMPSJFTJOUIFJOHSFEJFOUT
   OVNCFSPGTFSWJOHTPGUIFSFDJQF
   $BMPSJF&TUJNBUJPO
   
   c(r) =
   ∑
   i∈Ir
   c(i) ⋅ q(i)/100
   s(r)
   

   View Slide

6. HSBNTPGUIFJOHSFEJFOU

   JOUIFSFDJQF
   )PXEPXFPCUBJOc(i)
   γϝδ

   TIJNFKJNVTISPPN
   
   
   χϯδϯ

   DBSSPU
   
   
   ʜ ʜ
   ✕
   JOHSFEJFOUEJDUJPOBSZ
   ͠Ί͡
   TIJNFKJNVTISPPN
   
   ͠Ί͡JTO`UUIFTBNF

   BTγϝδ
   c(r) =
   ∑
   i∈Ir
   c(i) ⋅ q(i)/100
   s(r)
   *OHSFEJFOU/PSNBMJ[BUJPO
   
   DBMPSJFTQFSHSBNT

   PGBOJOHSFEJFOU
   

   View Slide

7. 4FWFSBMJTTVFTNVTUCFTPMWFE
   1FPQMFDBOEFTDSJCFUIFJOHSFEJFOUTJOUIFJSSFDJQFTGSFFMZ JONBOZEJ⒎FSFOU

   XBZT
   FH ͠ΐ͏Ώ TPZTBVDF
   IBTNPSFUIBOFYQSFTTJPOTJO$PPLQBE TVDIBT
   γϣ΢Ϣ ͠ΐ͏༉ ͓ে༉ BOEὑ͠ΐ͏༉
   *OHSFEJFOU/PSNBMJ[BUJPO
   
   0SJHJOBMGPSN $BOPOJDBMGPSN
   0SUIPHSBQIJD
   WBSJBOUT
   ਓࢀ
   DBSSPU
   
   ʹΜ͡Μ
   DBSSPU
   
   "CCSFWJBUJPOT
   ৽ͨ·
   TQSJOHPOJPO
   
   ৽ͨ·Ͷ͗
   TQSJOHPOJPO
   
   4QFMMJOH
   NJTUBLFT
   ΞϘΨυ
   BWPDBEP
   
   ΞϘΧυ
   BWPDBEP
   
   0UIFS
   അླḕ
   QPUBUP
   
   ͡Ό͕͍΋
   QPUBUP
   
   0SJHJOBMGPSN $BOPOJDBMGPSN
   4ZNCPMT
   ὑຯḩ
   ὑNJTP
   
   ຯḩ
   NJTP
   
   1BSFOUIFTFT
   ༉ ᖱΊ༻
   
   PJM GPSGSZJOH
   
   ༉
   PJM
   
   1SFpY
   ͓࠭౶
   TVHBS
   
   ࠭౶
   TVHBS
   
   0UIFS
   ͋Ε͹ϩʔζϚϦʔ
   SPTFNBSZ JGBOZ
   
   ϩʔζϚϦʔ

   SPTFNBSZ
   
   &YBNQMFTPGJOHSFEJFOUOPSNBMJ[BUJPO
   

   View Slide

8. "QQMZBOFODPEFSEFDPEFSNPEFM
   5IFFODPEFSUBLFTUIFPSJHJOBMJOHSFEJFOUOBNFBTBOJOQVU

   5IFEFDPEFSHFOFSBUFTBDBOPOJDBMGPSNBTBOPVUQVU
   %PO`UVTFBUUFOUJPONFDIBOJTNTCFDBVTFUIFJOHSFEJFOUOBNFTBSFTIPSUTP

   UIFNFUIPEXPVMEOPUCFF⒎FDUJWF
   *OHSFEJFOU/PSNBMJ[BUJPO
   
   y1
   yJ
   y2
   …
   canonical form of the ingredient
   decoder
   original string of an ingredient
   encoder
   x1
   x2
   xI
   …
   softmax softmax softmax
   …
   

   View Slide

9. )PXEPXFPCUBJOs(r)
   ✕
   OVNCFSPGTFSWJOHTPGUIFSFDJQF
   খുݸ෼
   TNBMMCPXMT
   
   IPXNBOZQFPQMFJT
   UIJTSFDJQFGPS
   c(r) =
   ∑
   i∈Ir
   c(i) ⋅ q(i)/100
   s(r)
   4FSWJOH&TUJNBUJPO
   
   

   View Slide

10. 4FSWJOH&TUJNBUJPO
    
    4FWFSBMJTTVFTNVTUCFTPMWFE
    1FPQMFBMTPGSFFMZFYQSFTTBOBNPVOUJOUIFJSSFDJQFT
    5JUMF "NPVOU PGTFSWJOHT
    ίίφοπΦΠϧೖΓ΄͏ΕΜ૲ΧϨʔ
    4QJOBDI$VSSZXJUI$PDPOVU0JM
    
    ࡼ෼
    QMBUFT
    
    
    ͜ΜͿɾͻ͖͡ɾ΄ͨͯ͝͸Μ
    .JYFE3JDFXJUI,PNCVɾ)JKJLJɾ4DBMMPQ
    
    ߹෼
    HP
    
    
    άϥϊʔϥΫοΩʔ
    (SBOPMB$PPLJFT
    
    ຕ
    QJFDFT
    
    
    ύϯϓΩϯλϧτ
    1VNQLJO5BSU
    
    DNλϧτܕ
    DNUBSUNPME
    
    
    &YBNQMFTPGTFSWJOHFTUJNBUJPO
    "MUIPVHIUIFSFBSFTFWFSBMBOTXFSTGPSTPNFSFDJQFT XFEPOPUQVSTVFUIJTJTTVFJOPVSTFSWJDF
    

    View Slide

11. 4FSWJOH&TUJNBUJPO
    
    "QQMZUXPOFVSBMDMBTTJpDBUJPONPEFMT
    5IFFODPEFSUBLFTBUJUMFBOEPSBNPVOUJOBSFDJQFBTBOJOQVU

    5IFEFOTFMBZFSTUSBOTGPSNUIFPVUQVUPGUIFFODPEFS T
    JOUPUIFOVNCFSPG

    TFSWJOHT
    dense
    title or amount
    encoder
    xa1
    xa2
    xaI
    …
    softmax
    y
    serving
    dense
    amount
    encoder
    xa1
    xa2
    xaJ
    …
    title
    encoder
    xt1
    xt2
    xtI
    …
    softmax
    y
    serving
    4JOHMFTPVSDFNPEFM .VMUJTPVSDFNPEFM
    

    View Slide

12. &YQFSJNFOU*OHSFEJFOU/PSNBMJ[BUJPO
    %BUB
    8FDPMMFDUFE EJTUJODUJOHSFEJFOUTGSPN$PPLQBE OPSNBMJ[FEUIFN

    NBOVBMMZ UIFOSFNPWFE XIPTFOPSNBMJ[FEWFSTJPOTXFSFUIFTBNFBT

    UIFJSPSJHJOBMTUSJOHT
    8FVTFEUIFSFNBJOJOH JOHSFEJFOUT
    .FUIPET
    3&SFHVMBSFYQSFTTJPOCBTFENFUIPEXIJDISFNPWFTTZNCPMTBOE

    QBSFOUIFTFTGSPNUIFPSJHJOBMTUSJOHT
    .-PVS.-CBTFENFUIPE
    

    View Slide

13. &YQFSJNFOU*OHSFEJFOU/PSNBMJ[BUJPO
    3FTVMUT
    5IF.-CBTFENFUIPETVDDFTTGVMMZOPSNBMJ[FENPSF

    UIBOPGUIFJOHSFEJFOUT
    0SJHJOBMGPSN $BOPOJDBMGPSN
    0SUIPHSBQIJD
    WBSJBOUT
    ͔ͨ͘Γค
    QPUBUPTUBSDI
    
    ย܀ค
    QPUBUPTUBSDI
    
    "CCSFWJBUJPOT
    ڇͻ͖
    HSPVOECFFG
    
    ڇͻ͖೑
    HSPVOECFFG
    
    4QFMMJOH
    NJTUBLFT
    ਫ໊
    NJ[VOB
    
    ਫࡊ
    NJ[VOB
    
    0UIFS
    ࢳ͏Γ
    TQBHIFUUJTRVBTI
    
    ͦ͏ΊΜ͔΅ͪΌ
    TQBHIFUUJTRVBTI
    
    0SJHJOBMGPSN $BOPOJDBMGPSN
    4ZNCPMT
    ˑখഴค
    ˑqPVS
    
    খഴค
    qPVS
    
    1BSFOUIFTFT
    Ϩʔζϯ ͓޷ΈͰ
    
    SBJTJO BTZPVMJLF
    
    Ϩʔζϯ
    SBJTJO
    
    1SFpY
    ͓౾෗
    UPGV
    
    ౾෗
    UPGV
    
    0UIFS
    ྫྷ͑ͨτϚτ
    DIJMMFEUPNBUP
    
    τϚτ
    UPNBUP
    
    4VDDFTTGVMFYBNQMFTPGJOHSFEJFOUOPSNBMJ[BUJPO
    "DDVSBDZ
    3&
    .-
    

    View Slide

14. 5XPUZQFTPGOPSNBMJ[BUJPOFSSPS
    8SPOHMZHFOFSBUJOHOPOFYJTUFOUJOHSFEJFOUT
    8SPOHMZHFOFSBUJOHEJ⒎FSFOUJOHSFEJFOUT
    'VUVSFXPSLSFRVJSFE
    5PDPOTJEFSUIFFYJTUFODFPGUIFPVUQVUBOEJUTTJNJMBSJUZUPUIFJOQVU

    )BSBTIJNBBOE:BNBEB
    

    5PFYQBOEUIFUSBJOJOHTFU
    &YQFSJNFOU*OHSFEJFOU/PSNBMJ[BUJPO
    *OQVU )VNBOPVUQVU 4ZTUFNPVUQVU
    χΨ΢Ϧ
    CJUUFSNFMPO
    
    ΰʔϠ
    CJUUFSNFMPO
    
    ʹ͕͕Γ
    OPOFYJTUFOUXPSE
    
    ḕᤸ
    ZBN
    
    ࢁҶ
    ZBN
    
    ͡Ό͕͍΋
    QPUBUP
    
    

    View Slide

15. &YQFSJNFOU4FSWJOH&TUJNBUJPO
    %BUB
    8FDPMMFDUFE SFDJQFTGSPN$PPLQBEUIFOFTUJNBUFEUIFOVNCFSPG

    TFSWJOHTGSPNUIFNNBOVBMMZ
    .FUIPET
    3&SFHVMBSFYQSFTTJPOCBTFENFUIPEXIJDIFYUSBDUTBWBMVFGSPNUIF

    BNPVOUJOFBDISFDJQF
    .-PVS.-CBTFENFUIPEXIJDIVTFTDBUFHPSJFT GPSQFSTPO ʜ VQUP

    QFPQMF
    
    

    View Slide

16. &YQFSJNFOU4FSWJOH&TUJNBUJPO
    3FTVMUT
    5IF.-CBTFENFUIPEBDIJFWFEBOBDDVSBDZPGNPSFUIBO
    5IFTJOHMFTPVSDFNPEFMTMJHIUMZPVUQFSGPSNFEUIFNVMUJTPVSDFNPEFM

    &TTFOUJBMMZ UIFOVNCFSPGTFSWJOHTGPSBSFDJQFDBOOPUCFFTUJNBUFEXJUIPVU

    JUTUJUMFJOGPSNBUJPO

    5IFOVNCFSPGFYBNQMFTJOPVSUSBJOJOHTFUXBTUPPTNBMMUPUSBJOUXP

    FODPEFSTTPXFXJMMFYQBOEUIFUSBJOJOHTFUJOGVUVSFXPSL
    4PVSDF 5JUMF "NPVOU "DDVSBDZ
    3& TJOHMF 㾎
    .-
    TJOHMF 㾎
    TJOHMF 㾎
    NVMUJ 㾎 㾎
    

    View Slide

17. &YQFSJNFOU4FSWJOH&TUJNBUJPO
    ✕'BJMFECZVTJOHUIFUJUMFJOGPSNBUJPO

    0VSUSBJOJOHTFUDPOUBJOFETPNFSFDJQFTJOXIJDIPOF,PSFBOQBODBLFTFSWFEUXPQFPQMF

    ✕'BJMFECZOPUVTJOHUIFUJUMFJOGPSNBUJPO

    0OFTNBMMDSPRVFUUFJTHFOFSBMMZOPUTV⒏DJFOUGPSPOFQFSTPO

    ✕'BJMFECZOPUVTJOHUIFRVBOUJUZJOGPSNBUJPOPOUIFJOHSFEJFOUT

    'VUVSFXPSLXJMMCFOFFEFE
    5JUMF "NPVOU )VNBO
    4ZTUFN
    TJOHMF
    
    4ZTUFN
    NVMUJ
    
    γϡΨʔύϯ
    4VHBS#SFBE
    
    ݸ
    QJFDFT
    
    
    ૉ๿ͳϨʔζϯύϯ
    4JNQMF3BJTJO#SFBE
    
    ی
    MPBG
    
    
    ΋ͪ΋ͪ໺ࡊνξϛ
    $IFXZ,PSFBO7FHFUBCMF1BODBLF
    
    ϑϥΠύϯຕ෼
    GSZJOHQBOT
    
    
    πφϙςτͷϛχίϩοέˑ͓ห౰ʹ΋
    4NBMM5VOBBOE1PUBUP$SPRVFUUFTˑBMTPGPS-VODI#PY
    
    ݸ෼
    QJFDFT
    
    
    ؁͞߇͑ΊͷΫοΩʔ
    $PPLJFTXJUI-FTT4VHBS
    
    మ൘ຕ෼
    JSPOQMBUF
    
    
    4VDDFTTGVMBOEGBJMFEFYBNQMFTPGTFSWJOHFTUJNBUJPO
    ✕
    ✕
    ✕ ✕
    

    View Slide

18. %FQMPZ
    "TBSJ$MBN3JDF
    
    HP
    
    ɹɹɹɹɹΞαϦɹ
    ɹɹɹɹɹถɹɹɹ
    ɹɹɹɹɹԘɹɹɹ
    ɹɹɹɹɹञɹɹɹ
    ɹɹɹɹɹ͠ΐ͏Ώ
    ɹɹɹɹɹΈΓΜɹ
    SJDF
    
    BTBSJDMBN
    
    TBMU
    
    TBLF
    
    TPZTBVDF
    
    TXFFUTBLF
    
    㾎
    㾎
    㾎
    㾎
    㾎
    㾎
    8FIBWFFTUJNBUFEUIFOVNCFSPGDBMPSJFTJOPWFS
    SFDJQFTBOEBDUVBMMZVTFUIFNJOPVSSFDJQFTFSWJDF
    6TFUIFTJOHMFTPVSDFNPEFMGPSTFSWJOHFTUJNBUJPO
    
    c(r) =
    ∑
    i∈Ir
    c(i) ⋅ q(i)/100
    s(r)
    = 306.6
    㾎
    *OHSFEJFOU
    

    View Slide

19. 4VNNBSZ
    8FIBWFJOUSPEVDFEUIFGPMMPXJOHUISFFUBTLT

    JOUP$PPLQBE
    $BMPSJFFTUJNBUJPO NBJOUBTL
    

    *OHSFEJFOUOPSNBMJ[BUJPO TVCUBTL
    

    4FSWJOHFTUJNBUJPO TVCUBTL
    
    8FIBWFBQQMJFE.-CBTFENFUIPETGPSUIFTF
    UBTLTBOEEFQMPZFEUIFNJOPVSSFBMXPSMESFDJQF
    TFSWJDF
    5IFNFUIPETQFSGPSNFETV⒏DJFOUMZXFMMUPCFVTFEJOPVSTFSWJDF
    'VUVSFXPSL
    5PFYQBOEUIFOVNCFSPGJOHSFEJFOUTBOESFDJQFTUPXIJDIPVSNFUIPETDBOCF

    BQQMJFE
    

    View Slide

20. 5IBOLZPV
    

    View Slide