Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up
for free
Calorie Estimation in a Real-World Recipe Service/iaai-20
j.harashima
February 09, 2020
0
6.7k
Calorie Estimation in a Real-World Recipe Service/iaai-20
j.harashima
February 09, 2020
Tweet
Share
More Decks by j.harashima
See All by j.harashima
junharashima
1
470
junharashima
0
1.9k
junharashima
0
44
junharashima
0
3.6k
junharashima
1
9k
junharashima
0
2.5k
junharashima
0
5.5k
junharashima
1
55
junharashima
4
3.5k
Featured
See All Featured
robhawkes
52
2.8k
jakevdp
775
200k
ammeep
656
54k
eitanlees
112
10k
andyhume
63
3.7k
tammielis
237
23k
dougneiner
118
7.9k
lynnandtonic
272
16k
philnash
9
590
geoffreycrofte
21
930
kneath
294
39k
davidbonilla
70
3.6k
Transcript
$BMPSJF&TUJNBUJPO JOB3FBM8PSME3FDJQF4FSWJDF +VO)BSBTIJNB .BLPUP)JSBNBUTV 4BUPTIJ4BOKP $PPLQBE*OD
4VNNBSZ 8FIBWFJOUSPEVDFEUIFGPMMPXJOHUISFFUBTLT JOUP$PPLQBE $BMPSJFFTUJNBUJPO NBJOUBTL *OHSFEJFOUOPSNBMJ[BUJPO TVCUBTL 4FSWJOHFTUJNBUJPO
TVCUBTL 8FIBWFBQQMJFENBDIJOFMFBSOJOH .- CBTFE NFUIPETGPSUIFTFUBTLTBOEEFQMPZFEUIFNJO PVSSFBMXPSMESFDJQFTFSWJDF
$PPLQBE 1MBUGPSNGPSQFPQMFUPTIBSF SFDJQFJEFBTBOEDPPLJOHUJQT NJMMJPOVTFSTJODPVOUSJFT NJMMJPOSFDJQFT IBMGBSFJO+BQBOFTF
$BMPSJF&TUJNBUJPO &TUJNBUFUIFOVNCFSPGDBMPSJFT QFSTFSWJOHJOBSFDJQF DBMPSJF XIPMF BNPVOU
6TFJOHSFEJFOUTBOEBNPVOU TVNPGUIFDBMPSJFTJOUIFJOHSFEJFOUT OVNCFSPGTFSWJOHTPGUIFSFDJQF $BMPSJF&TUJNBUJPO c(r) = ∑ i∈Ir c(i)
⋅ q(i)/100 s(r)
HSBNTPGUIFJOHSFEJFOU JOUIFSFDJQF )PXEPXFPCUBJOc(i) γϝδ TIJNFKJNVTISPPN χϯδϯ DBSSPU ʜ
ʜ ✕ JOHSFEJFOUEJDUJPOBSZ ͠Ί͡ TIJNFKJNVTISPPN ͠Ί͡JTO`UUIFTBNF BTγϝδ c(r) = ∑ i∈Ir c(i) ⋅ q(i)/100 s(r) *OHSFEJFOU/PSNBMJ[BUJPO DBMPSJFTQFSHSBNT PGBOJOHSFEJFOU
4FWFSBMJTTVFTNVTUCFTPMWFE 1FPQMFDBOEFTDSJCFUIFJOHSFEJFOUTJOUIFJSSFDJQFTGSFFMZ JONBOZEJ⒎FSFOU XBZT FH ͠ΐ͏Ώ TPZTBVDF IBTNPSFUIBOFYQSFTTJPOTJO$PPLQBE TVDIBT γϣϢ
͠ΐ͏༉ ͓ে༉ BOEὑ͠ΐ͏༉ *OHSFEJFOU/PSNBMJ[BUJPO 0SJHJOBMGPSN $BOPOJDBMGPSN 0SUIPHSBQIJD WBSJBOUT ਓࢀ DBSSPU ʹΜ͡Μ DBSSPU "CCSFWJBUJPOT ৽ͨ· TQSJOHPOJPO ৽ͨ·Ͷ͗ TQSJOHPOJPO 4QFMMJOH NJTUBLFT ΞϘΨυ BWPDBEP ΞϘΧυ BWPDBEP 0UIFS അླḕ QPUBUP ͡Ό͕͍ QPUBUP 0SJHJOBMGPSN $BOPOJDBMGPSN 4ZNCPMT ὑຯḩ ὑNJTP ຯḩ NJTP 1BSFOUIFTFT ༉ ᖱΊ༻ PJM GPSGSZJOH ༉ PJM 1SFpY ͓࠭ TVHBS ࠭ TVHBS 0UIFS ͋ΕϩʔζϚϦʔ SPTFNBSZ JGBOZ ϩʔζϚϦʔ SPTFNBSZ &YBNQMFTPGJOHSFEJFOUOPSNBMJ[BUJPO
"QQMZBOFODPEFSEFDPEFSNPEFM 5IFFODPEFSUBLFTUIFPSJHJOBMJOHSFEJFOUOBNFBTBOJOQVU 5IFEFDPEFSHFOFSBUFTBDBOPOJDBMGPSNBTBOPVUQVU %PO`UVTFBUUFOUJPONFDIBOJTNTCFDBVTFUIFJOHSFEJFOUOBNFTBSFTIPSUTP UIFNFUIPEXPVMEOPUCFF⒎FDUJWF *OHSFEJFOU/PSNBMJ[BUJPO y1 yJ y2
… canonical form of the ingredient decoder original string of an ingredient encoder x1 x2 xI … softmax softmax softmax …
)PXEPXFPCUBJOs(r) ✕ OVNCFSPGTFSWJOHTPGUIFSFDJQF খുݸ TNBMMCPXMT IPXNBOZQFPQMFJT UIJTSFDJQFGPS c(r) = ∑
i∈Ir c(i) ⋅ q(i)/100 s(r) 4FSWJOH&TUJNBUJPO
4FSWJOH&TUJNBUJPO 4FWFSBMJTTVFTNVTUCFTPMWFE 1FPQMFBMTPGSFFMZFYQSFTTBOBNPVOUJOUIFJSSFDJQFT 5JUMF "NPVOU PGTFSWJOHT ίίφοπΦΠϧೖΓ΄͏ΕΜΧϨʔ 4QJOBDI$VSSZXJUI$PDPOVU0JM ࡼ
QMBUFT ͜ΜͿɾͻ͖͡ɾ΄ͨͯ͝Μ .JYFE3JDFXJUI,PNCVɾ)JKJLJɾ4DBMMPQ ߹ HP άϥϊʔϥΫοΩʔ (SBOPMB$PPLJFT ຕ QJFDFT ύϯϓΩϯλϧτ 1VNQLJO5BSU DNλϧτܕ DNUBSUNPME &YBNQMFTPGTFSWJOHFTUJNBUJPO "MUIPVHIUIFSFBSFTFWFSBMBOTXFSTGPSTPNFSFDJQFT XFEPOPUQVSTVFUIJTJTTVFJOPVSTFSWJDF
4FSWJOH&TUJNBUJPO "QQMZUXPOFVSBMDMBTTJpDBUJPONPEFMT 5IFFODPEFSUBLFTBUJUMFBOEPSBNPVOUJOBSFDJQFBTBOJOQVU 5IFEFOTFMBZFSTUSBOTGPSNUIFPVUQVUPGUIFFODPEFS T JOUPUIFOVNCFSPG TFSWJOHT dense title
or amount encoder xa1 xa2 xaI … softmax y serving dense amount encoder xa1 xa2 xaJ … title encoder xt1 xt2 xtI … softmax y serving 4JOHMFTPVSDFNPEFM .VMUJTPVSDFNPEFM
&YQFSJNFOU*OHSFEJFOU/PSNBMJ[BUJPO %BUB 8FDPMMFDUFE EJTUJODUJOHSFEJFOUTGSPN$PPLQBE OPSNBMJ[FEUIFN NBOVBMMZ UIFOSFNPWFE XIPTFOPSNBMJ[FEWFSTJPOTXFSFUIFTBNFBT UIFJSPSJHJOBMTUSJOHT 8FVTFEUIFSFNBJOJOH
JOHSFEJFOUT .FUIPET 3&SFHVMBSFYQSFTTJPOCBTFENFUIPEXIJDISFNPWFTTZNCPMTBOE QBSFOUIFTFTGSPNUIFPSJHJOBMTUSJOHT .-PVS.-CBTFENFUIPE
&YQFSJNFOU*OHSFEJFOU/PSNBMJ[BUJPO 3FTVMUT 5IF.-CBTFENFUIPETVDDFTTGVMMZOPSNBMJ[FENPSF UIBOPGUIFJOHSFEJFOUT 0SJHJOBMGPSN $BOPOJDBMGPSN 0SUIPHSBQIJD WBSJBOUT ͔ͨ͘Γค QPUBUPTUBSDI
ย܀ค QPUBUPTUBSDI "CCSFWJBUJPOT ڇͻ͖ HSPVOECFFG ڇͻ͖ HSPVOECFFG 4QFMMJOH NJTUBLFT ਫ໊ NJ[VOB ਫࡊ NJ[VOB 0UIFS ࢳ͏Γ TQBHIFUUJTRVBTI ͦ͏ΊΜ͔΅ͪΌ TQBHIFUUJTRVBTI 0SJHJOBMGPSN $BOPOJDBMGPSN 4ZNCPMT ˑখഴค ˑqPVS খഴค qPVS 1BSFOUIFTFT Ϩʔζϯ ͓ΈͰ SBJTJO BTZPVMJLF Ϩʔζϯ SBJTJO 1SFpY ͓౾ UPGV ౾ UPGV 0UIFS ྫྷ͑ͨτϚτ DIJMMFEUPNBUP τϚτ UPNBUP 4VDDFTTGVMFYBNQMFTPGJOHSFEJFOUOPSNBMJ[BUJPO "DDVSBDZ 3& .-
5XPUZQFTPGOPSNBMJ[BUJPOFSSPS 8SPOHMZHFOFSBUJOHOPOFYJTUFOUJOHSFEJFOUT 8SPOHMZHFOFSBUJOHEJ⒎FSFOUJOHSFEJFOUT 'VUVSFXPSLSFRVJSFE 5PDPOTJEFSUIFFYJTUFODFPGUIFPVUQVUBOEJUTTJNJMBSJUZUPUIFJOQVU )BSBTIJNBBOE:BNBEB 5PFYQBOEUIFUSBJOJOHTFU &YQFSJNFOU*OHSFEJFOU/PSNBMJ[BUJPO
*OQVU )VNBOPVUQVU 4ZTUFNPVUQVU χΨϦ CJUUFSNFMPO ΰʔϠ CJUUFSNFMPO ʹ͕͕Γ OPOFYJTUFOUXPSE ḕᤸ ZBN ࢁҶ ZBN ͡Ό͕͍ QPUBUP
&YQFSJNFOU4FSWJOH&TUJNBUJPO %BUB 8FDPMMFDUFE SFDJQFTGSPN$PPLQBEUIFOFTUJNBUFEUIFOVNCFSPG TFSWJOHTGSPNUIFNNBOVBMMZ .FUIPET 3&SFHVMBSFYQSFTTJPOCBTFENFUIPEXIJDIFYUSBDUTBWBMVFGSPNUIF BNPVOUJOFBDISFDJQF .-PVS.-CBTFENFUIPEXIJDIVTFTDBUFHPSJFT GPSQFSTPO
ʜ VQUP QFPQMF
&YQFSJNFOU4FSWJOH&TUJNBUJPO 3FTVMUT 5IF.-CBTFENFUIPEBDIJFWFEBOBDDVSBDZPGNPSFUIBO 5IFTJOHMFTPVSDFNPEFMTMJHIUMZPVUQFSGPSNFEUIFNVMUJTPVSDFNPEFM &TTFOUJBMMZ UIFOVNCFSPGTFSWJOHTGPSBSFDJQFDBOOPUCFFTUJNBUFEXJUIPVU JUTUJUMFJOGPSNBUJPO 5IFOVNCFSPGFYBNQMFTJOPVSUSBJOJOHTFUXBTUPPTNBMMUPUSBJOUXP FODPEFSTTPXFXJMMFYQBOEUIFUSBJOJOHTFUJOGVUVSFXPSL 4PVSDF
5JUMF "NPVOU "DDVSBDZ 3& TJOHMF 㾎 .- TJOHMF 㾎 TJOHMF 㾎 NVMUJ 㾎 㾎
&YQFSJNFOU4FSWJOH&TUJNBUJPO ✕'BJMFECZVTJOHUIFUJUMFJOGPSNBUJPO 0VSUSBJOJOHTFUDPOUBJOFETPNFSFDJQFTJOXIJDIPOF,PSFBOQBODBLFTFSWFEUXPQFPQMF ✕'BJMFECZOPUVTJOHUIFUJUMFJOGPSNBUJPO 0OFTNBMMDSPRVFUUFJTHFOFSBMMZOPUTV⒏DJFOUGPSPOFQFSTPO ✕'BJMFECZOPUVTJOHUIFRVBOUJUZJOGPSNBUJPOPOUIFJOHSFEJFOUT 'VUVSFXPSLXJMMCFOFFEFE 5JUMF "NPVOU )VNBO
4ZTUFN TJOHMF 4ZTUFN NVMUJ γϡΨʔύϯ 4VHBS#SFBE ݸ QJFDFT ૉͳϨʔζϯύϯ 4JNQMF3BJTJO#SFBE ی MPBG ͪͪࡊνξϛ $IFXZ,PSFBO7FHFUBCMF1BODBLF ϑϥΠύϯຕ GSZJOHQBOT πφϙςτͷϛχίϩοέˑ͓หʹ 4NBMM5VOBBOE1PUBUP$SPRVFUUFTˑBMTPGPS-VODI#PY ݸ QJFDFT ͞߇͑ΊͷΫοΩʔ $PPLJFTXJUI-FTT4VHBS మ൘ຕ JSPOQMBUF 4VDDFTTGVMBOEGBJMFEFYBNQMFTPGTFSWJOHFTUJNBUJPO ✕ ✕ ✕ ✕
%FQMPZ "TBSJ$MBN3JDF HP ɹɹɹɹɹΞαϦɹ ɹɹɹɹɹถɹɹɹ ɹɹɹɹɹԘɹɹɹ ɹɹɹɹɹञɹɹɹ ɹɹɹɹɹ͠ΐ͏Ώ ɹɹɹɹɹΈΓΜɹ SJDF
BTBSJDMBN TBMU TBLF TPZTBVDF TXFFUTBLF 㾎 㾎 㾎 㾎 㾎 㾎 8FIBWFFTUJNBUFEUIFOVNCFSPGDBMPSJFTJOPWFS SFDJQFTBOEBDUVBMMZVTFUIFNJOPVSSFDJQFTFSWJDF 6TFUIFTJOHMFTPVSDFNPEFMGPSTFSWJOHFTUJNBUJPO c(r) = ∑ i∈Ir c(i) ⋅ q(i)/100 s(r) = 306.6 㾎 *OHSFEJFOU
4VNNBSZ 8FIBWFJOUSPEVDFEUIFGPMMPXJOHUISFFUBTLT JOUP$PPLQBE $BMPSJFFTUJNBUJPO NBJOUBTL *OHSFEJFOUOPSNBMJ[BUJPO TVCUBTL 4FSWJOHFTUJNBUJPO
TVCUBTL 8FIBWFBQQMJFE.-CBTFENFUIPETGPSUIFTF UBTLTBOEEFQMPZFEUIFNJOPVSSFBMXPSMESFDJQF TFSWJDF 5IFNFUIPETQFSGPSNFETV⒏DJFOUMZXFMMUPCFVTFEJOPVSTFSWJDF 'VUVSFXPSL 5PFYQBOEUIFOVNCFSPGJOHSFEJFOUTBOESFDJQFTUPXIJDIPVSNFUIPETDBOCF BQQMJFE
5IBOLZPV