2023年2月2日 知識転移グラフによる深層共同学習 知識転移グラフによる最適な半教師あり学習の探索 素人発想玄人実行2.0
%FFQ/FVSBM/FUXPSLͷڞಉֶश౻٢߂ʢத෦େֶɾػց֮ϩϘςΟΫεάϧʔϓʣIUUQNQSHKQ
View Slide
݄ɿۚग़༤ઌੜ͔Β͍͓ͨݴ༿2த෦େֶϩΰத෦େֶϩΰ
w ʮൃ୯७ɼૉɼࣗ༝ɼ؆୯でͳ͚ΕばͳΒͳ͍ɽ͔͠͠ɼൃΛ࣮ߦʹҠ͢ʹࣝが͍Δɼख़࿅͞Εٕͨが͍Δʯʢۚग़༤ʣʮૉਓൃݰਓ࣮ߦʯͱʁ3த෦େֶϩΰத෦େֶϩΰIUUQTXXXBNB[PODPKQEQ#11428/IUUQNQSHKQ5,CPPL
w ʮݴ͏қ͘ɼߦ͏͠ʯͷయܕw ͔ΒΛܦͯʮૉਓൃݰਓ࣮ߦʯ͚ۙͮͨݚڀ ඇํੑ-P(ϑΟϧλʹΑΔෳͷΞϑΟϯྖҬͷਪఆ *$$7> ࣝసҠグϥϑʹΑΔڞಉֶश
ਂֶशͷωοτϫʔΫߏத෦େֶϩΰத෦େֶϩΰw *-473$ʹ͓͚ΔωοτϫʔΫߏͷมભ2012SuperVision GoogLeNetKonvolüzasyonPoolingSoftmaxDiğer[Krizhevsky NIPS 2012] [Szegedy arxiv 2014]-22 [Sim"MFY/FU MBZFST*-473$2014GoogLeNetKonvolüzasyonPoolingSoftmaxDiğerVGG MSRA[Szegedy arxiv 2014]-22 [Simonyan arxiv 2014] -19 [He arxiv 2014]n2014GoogLeNetKonvolüzasyonPoolingSoftmaxDiğerVGG MSRA35/36 tderin öğkullanm20/36 taçık-kayCaffeuygulakullanm012] [Szegedy arxiv 2014]-22 [Simonyan arxiv 2014] -19 [He arxiv 2014]7(( MBZFST*-473$(PPHMF/FU MBZFST*-473$143FT/FU MBZFST*-473$ˠ ສύϥϝʔλˠԯύϥϝʔλˠ ສύϥϝʔλ
,OPXMFEHF%JTUJMMBUJPO ,%)JOUPO >த෦େֶϩΰத෦େֶϩΰw ࣝৠཹ ֶशࡁΈ-BSHFωοτϫʔΫ͔Β4NBMMωοτϫʔΫʹࣝసҠ ੑೳΛอͪͭͭɺύϥϝʔλͱܭࢉίετΛݮՄೳ5FBDIFS/FUXPSL4UVEFOU/FUXPSL-BSHFQSFUSBJOFE4NBMM,OPXMFEHFUSBOTGFSࣝৠཹࣝ
,OPXMFEHF%JTUJMMBUJPO ,%)JOUPO >த෦େֶϩΰத෦େֶϩΰw ࣝৠཹɿ5FBDIFSˠ4UVEFOU ֶशࡁΈ-BSHFωοτϫʔΫ͔Β4NBMMωοτϫʔΫʹࣝసҠ ੑೳΛอͪͭͭɺύϥϝʔλͱܭࢉίετΛݮՄೳ ֶशํ๏ɿ)BSEUBSHFUͱ4PGUUBSHFUͰ4UVEFOUωοτϫʔΫΛֶश%BSL,OPXMFEHFʢӅΕͨࣝʣ5FBDIFS4UVEFOU$SPTT&OUSPQZ$SPTT&OUSPQZMBCFMQSFUSBJOFE4PGUUBSHFU)BSEUBSHFUʢਖ਼ղϥϕϧʣp1p2ʢ֬ʣ#BDLQSPQ
%FFQ.VUVBM-FBSOJOH %.-
ωοτϫʔΫؒͷࣝసҠத෦େֶϩΰத෦େֶϩΰw ͍୩ʹམ͍ͪͯΔͱԿނྑ͍ͷ͔ʁࣝΛ͑Δ͜ͱͰೝࣝੑೳ্͕ ࣝৠཹɿ,OPXMFEHF%JTUJMMBUJPO)JOUPO > ૬ޓֶशɿ%FFQ.VUVBM-FBSOJOH
ࣝৠཹɾ૬ޓֶशͷੜख๏த෦େֶϩΰத෦େֶϩΰ4UVEFOU4UVEFOU4UVEFOU4UVEFOU4UVEFOU4UVEFOU4UVEFOU5FBDIFS4UVEFOU5FBDIFS4UVEFOU5FBDIFS5"4UVEFOU,OPXMFEHF%JTUJMMBUJPO%FFQ.VUVBM-FBSOJOH,OPXMFEHF%JTUJMMBUJPO)JOUPO >#PSO"HBJO >5FBDIFS"TTJTUBOU
ࣝৠཹɾ૬ޓֶशͷੜख๏த෦େֶϩΰத෦େֶϩΰ,OPXMFEHF%JTUJMMBUJPO)JOUPO >#PSO"HBJO >5FBDIFS"TTJTUBOU
ຊݚڀͷඪத෦େֶϩΰத෦େֶϩΰw ڞಉֶशΛΫϥεϧʔϜεέʔϧ֦ு ଟ༷ੑͷߴ͍ڞಉֶशΛ࣮ݱʮૉਓൃʯ ˠڭࣨͰͷֶशͷΑ͏ʹઌੜ͔ΒͰͳ͘ଟ͘ͷੜె͕ෳࡶʹڭ͑͋͏ֶश
w ڞಉֶशΛΫϥεϧʔϜεέʔϧ֦ு ଟ༷ੑͷߴ͍ڞಉֶशΛ࣮ݱw ࣝసҠάϥϑͷఏҊ
w άϥϑΛ༻͍ͯ,%ͱ%.-Λදݱ ϊʔυɿਂֶशϞσϧ Τοδɿࣝৠཹͷଛࣦάϥϑදݱͷมத෦େֶϩΰத෦େֶϩΰ,OPXMFEHF%JTUJMMBUJPO,%5FBDIFS4UVEFOU4UVEFOU4UVEFOU%FFQ.VUVBM-FBSOJOH%.-p1p2p1p2-BSHF 4NBMMm1m2ํͷΤοδ-BSHF 4NBMMm1m2ํͷΤοδάϥϑදݱ
w ิॿϊʔυ͕ධՁରϊʔυͷֶशΛαϙʔτ͢Δ ϊʔυɿਂֶशϞσϧ ΤοδɿࣝৠཹͷଛࣦࣝసҠάϥϑ ϊʔυ͕ͷ߹த෦େֶϩΰத෦େֶϩΰ𝑚3𝑚1𝑚2𝐿^𝑦,1𝐿^𝑦,2𝐿^𝑦,3𝐿1,2𝐿1,3𝐿2,1𝐿3,1𝐿3,2𝐿2,3^𝑦^𝑦^𝑦ਖ਼ղϥϕϧධՁରϊʔυ3FT/FUิॿϊʔυ3FT/FU8JEF3FT/FU%FOTF/FU…
w ֤ΤοδʹҟͳΔଛࣦؔΛఆٛ ଛࣦؔͷΈ߹ΘͤΛ୳ࡧ͢Δ͜ͱͰ৽ͨͳֶशํ๏Λ࣮ݱࣝసҠάϥϑ ϊʔυ͕ͷ߹த෦େֶϩΰத෦େֶϩΰ𝑚3𝑚1𝑚2𝐿^𝑦,1𝐿^𝑦,2𝐿^𝑦,3𝐿1,2𝐿1,3𝐿2,1𝐿3,1𝐿3,2𝐿2,3^𝑦^𝑦^𝑦ଛࣦؔ𝐿=𝐻(𝑝^𝑦,𝑝𝑛)𝐿=𝐾𝐿(𝑝𝑛||𝑝𝑚)𝐿= 0…ˠଟ༷ͳࣝసҠΛදݱ͢ΔϑϨʔϜϫʔΫΛઃܭʮݰਓ࣮ߦᶃʯ
w ϊʔυ̎ TPVSDF͔Βϊʔυ̍ EFTUJOBUJPOͷࣝసҠΤοδͷࣝసҠͷଛࣦܭࢉத෦େֶϩΰத෦େֶϩΰ𝑚3𝑚1𝑚2𝐿^𝑦,1𝐿^𝑦,2𝐿^𝑦,3𝐿1,2𝐿1,3𝐿2,1𝐿3,1𝐿3,2𝐿2,3^𝑦^𝑦^𝑦𝑚1𝑚2𝐿2,1-PTTGVOD4PVSDF %FTUJOBUJPO𝑚2𝑚1p2(c|x) p1(c|x)L2,1(p2, p1)'PSXBSE
w ϊʔυ̎ TPVSDF͔Βϊʔυ̍ EFTUJOBUJPOͷࣝసҠΤοδͷࣝసҠͷଛࣦܭࢉத෦େֶϩΰத෦େֶϩΰ𝑚3𝑚1𝑚2𝐿^𝑦,1𝐿^𝑦,2𝐿^𝑦,3𝐿1,2𝐿1,3𝐿2,1𝐿3,1𝐿3,2𝐿2,3^𝑦^𝑦^𝑦𝑚1𝑚2𝐿2,1-PTTGVOD4PVSDF %FTUJOBUJPO𝑚2𝑚1L2,1(p2, p1)#BDLQSPQ%FUBDI#BDLXBSE
w ϊʔυ̎ TPVSDF͔Βϊʔυ̍ EFTUJOBUJPOͷࣝసҠΤοδͷࣝసҠͷଛࣦܭࢉத෦େֶϩΰத෦େֶϩΰ𝑚3𝑚1𝑚2𝐿^𝑦,1𝐿^𝑦,2𝐿^𝑦,3𝐿1,2𝐿1,3𝐿2,1𝐿3,1𝐿3,2𝐿2,3^𝑦^𝑦^𝑦𝑚1𝑚2𝐿2,1-PTTGVOD4PVSDF %FTUJOBUJPO𝑚2𝑚1L2,1(p2, p1)GateKL div'PSXBSEp2(c|x) p1(c|x)
w ͲͷΑ͏ʹࣝసҠ͢Δ͔Λ੍ޚήʔτؔத෦େֶϩΰத෦େֶϩΰ𝑚3𝑚1𝑚2𝐿^𝑦,1𝐿^𝑦,2𝐿^𝑦,3𝐿1,2𝐿1,3𝐿2,1𝐿3,1𝐿3,2𝐿2,3^𝑦^𝑦^𝑦𝑚1𝑚2𝐿2,14PVSDF %FTUJOBUJPO𝑚2𝑚1L2,1(p2, p1)'PSXBSEp2(c|x) p1(c|x)GateKL div$VUPGG(BUF-JOFBS(BUF5ISPVHI(BUF$PSSFDU(BUF
w ೖྗ͞ΕͨΛͦͷ··ग़ྗ͢Δήʔτؔɿ5ISPVHI(BUFத෦େֶϩΰத෦େֶϩΰ𝑚3𝑚1𝑚2𝐿^𝑦,1𝐿^𝑦,2𝐿^𝑦,3𝐿1,2𝐿1,3𝐿2,1𝐿3,1𝐿3,2𝐿2,3^𝑦^𝑦^𝑦𝑚1𝑚2𝐿2,14PVSDF %FTUJOBUJPO𝑚2𝑚1L2,1(p2, p1)$VUPGG(BUF-JOFBS(BUF$PSSFDU(BUF5ISPVHI(BUF𝐺(𝐷𝐾 𝐿) =𝐷𝐾 𝐿มߋΛՃ͑ͣɺͦͷ··ୡ'PSXBSEp2(c|x) p1(c|x)GateKL div
w ೖྗʹରͯ͠ৗʹΛग़ྗήʔτؔɿ$VUPff(BUFத෦େֶϩΰத෦େֶϩΰ𝑚3𝑚1𝑚2𝐿^𝑦,1𝐿^𝑦,2𝐿^𝑦,3𝐿1,2𝐿1,3𝐿2,1𝐿3,1𝐿3,2𝐿2,3^𝑦^𝑦^𝑦𝑚1𝑚2𝐿2,14PVSDF %FTUJOBUJPO𝑚2𝑚1L2,1(p2, p1)$VUPGG(BUF-JOFBS(BUF$PSSFDU(BUF5ISPVHI(BUFৗʹΛग़ྗΤοδͷஅ𝐺(𝐷𝐾 𝐿) = 0'PSXBSEp2(c|x) p1(c|x)GateKL div
w ֶश͕࣌ؒܦաͱͱʹग़ྗ͕ঃʑʹେ͖͘ͳΔήʔτؔɿ-JOFBS(BUFத෦େֶϩΰத෦େֶϩΰ𝑚3𝑚1𝑚2𝐿^𝑦,1𝐿^𝑦,2𝐿^𝑦,3𝐿1,2𝐿1,3𝐿2,1𝐿3,1𝐿3,2𝐿2,3^𝑦^𝑦^𝑦𝑚1𝑚2𝐿2,14PVSDF %FTUJOBUJPO𝑚2𝑚1L2,1(p2, p1)$VUPGG(BUF-JOFBS(BUF$PSSFDU(BUF5ISPVHI(BUFGateKL div࣌ؒͱڞʹग़ྗ͕େ͖͘ͳΔ𝐺(𝐷𝐾 𝐿) =𝑡𝑡𝑚 𝑎𝑥∙𝐷𝐾𝐿'PSXBSEp2(c|x) p1(c|x)
w ιʔεϊʔυ͕ਖ਼ղͨ͠߹ͷΈग़ྗήʔτؔɿ$PSSFDUHBUFத෦େֶϩΰத෦େֶϩΰ𝑚3𝑚1𝑚2𝐿^𝑦,1𝐿^𝑦,2𝐿^𝑦,3𝐿1,2𝐿1,3𝐿2,1𝐿3,1𝐿3,2𝐿2,3^𝑦^𝑦^𝑦𝑚1𝑚2𝐿2,14PVSDF %FTUJOBUJPO𝑚2𝑚1L2,1(p2, p1)$VUPGG(BUF-JOFBS(BUF$PSSFDU(BUF5ISPVHI(BUFGateKL divਖ਼ղͨ͠αϯϓϧͷใͷΈୡ𝐺(𝐷𝐾𝐿) =𝛿^𝑦,𝑦𝑚2∙𝐷𝐾 𝐿'PSXBSEp2(c|x) p1(c|x)
w ϋΠύʔύϥϝʔλαʔνͰࣝసҠάϥϑΛ࠷దԽ ࠷దԽख๏"TZODISPOPVT4VDDFTTJWF)BMWJOH"MHPSJUIN "4)" ύϥϝʔλήʔτؔ ิॿϊʔυࣝసҠάϥϑͷ࠷దԽத෦େֶϩΰத෦େֶϩΰ𝑚3𝑚1𝑚2𝐿^𝑦,1𝐿^𝑦,2𝐿^𝑦,3𝐿1,2𝐿1,3𝐿2,1𝐿3,1𝐿3,2𝐿2,3^𝑦^𝑦^𝑦ήʔτؔ• 5ISPVHI(BUF• $VUPGG(BUF• -JOFBS(BUF• $PSSFDU(BUF• 3FT/FUධՁରϊʔυ• 3FT/FU• 3FT/FU• 8JEF3FT/FUิॿϊʔυશΈ߹Θͤɿ ௨Γʢϊʔυͷ߹ʣ
ࣝసҠάϥϑͷ࠷దԽத෦େֶϩΰத෦େֶϩΰαʔόɹɹɹɹɿ୳ࡧճɹɹɹɹɿ ճϑϨʔϜϫʔΫɹɿ0QUVOB
w ධՁରϊʔυɿ 3FT/FUw 7BOJMMBϞσϧɿ࠷దԽʹΑͬͯ֫ಘͨࣝ͠సҠάϥϑʢୈҐʣத෦େֶϩΰத෦େֶϩΰڭࢣϥϕϧධՁରϊʔυิॿϊʔυQSFUSBJOFEิॿϊʔυڭࢣϥϕϧ
w ධՁରϊʔυɿ 3FT/FUw 7BOJMMBϞσϧɿ࠷దԽʹΑͬͯ֫ಘͨࣝ͠సҠάϥϑʢୈҐʣத෦େֶϩΰத෦େֶϩΰิॿϊʔυQSFUSBJOFEิॿϊʔυධՁରϊʔυࣝৠཹ
w ධՁରϊʔυɿ 3FT/FUw 7BOJMMBϞσϧɿ࠷దԽʹΑͬͯ֫ಘͨࣝ͠సҠάϥϑʢୈҐʣத෦େֶϩΰத෦େֶϩΰิॿϊʔυQSFUSBJOFEิॿϊʔυධՁରϊʔυॳΊ,%ϥΠΫͳֶशɼ࣍ୈʹ,%ʴ%.-ͳֶश͕ߦΘΕΔɹ૬ޓֶशɹ
w ࣮ݧ֓ཁ σʔληοτɿ$*'"3 ֶशϊʔυɿ ࠷దԽରϊʔυɿ3FT/FUैདྷख๏ ,% %.-ͱͷൺֱத෦େֶϩΰத෦େֶϩΰख๏ ೝࣝ> ิॿϊʔυͷϞσϧ*OEFQFOEFOU r,% 3FT/FU QSFUSBJOFE%.- 3FT/FU 3FT/FU0VST 3FT/FU QSFUSBJOFE 3FT/FU
࠷దԽʹΑͬͯ֫ಘͨࣝ͠సҠάϥϑ $*'"3த෦େֶϩΰத෦େֶϩΰϊʔυɿϊʔυɿϊʔυɿϊʔυɿϊʔυɿϊʔυɿ*OEFQFOEFOU3FT/FU
࠷దԽʹΑͬͯ֫ಘͨࣝ͠సҠάϥϑ $*'"3த෦େֶϩΰத෦େֶϩΰ*OEFQFOEFOU3FT/FU ϊʔυɿ
w ࣝసҠάϥϑʹΞϯαϯϒϧϊʔυͱΞςϯγϣϯϩεΛಋೖ ΞςϯγϣϯϩεɿΤοδͷϩεʹΞςϯγϣϯϩεΛՃ Ξϯαϯϒϧϊʔυɿ֤ϊʔυͷग़ྗΛΞϯαϯϒϧ͢ΔػߏࣝసҠάϥϑΛ༻͍ͨΞϯαϯϒϧֶश<0LBNPUP &$$7>த෦େֶϩΰத෦େֶϩΰΞϯαϯϒϧϊʔυ^𝑦𝐿^𝑦,𝑒𝑒𝑛𝑠𝑚1𝑚2𝑚3^𝑦^𝑦^𝑦𝐿^𝑦,3𝐿^𝑦,2𝐿^𝑦,1𝐿2,1𝐿1,2𝐿3,1𝐿1,3𝐿2,3𝐿3,2𝐿i,j=𝐾 𝐿(𝒑i,𝒑j) ±𝐿𝐴𝑇(𝑸i,𝑸j)Ξςϯγϣϯϩε
w ϊʔυ͔̎Βϊʔυ̍ͷΞςϯγϣϯϩε ೋͭͷϞσϧؒͷΞςϯγϣϯΛ͚ۙͮͨΓ͢ޮՌΞςϯγϣϯϩεத෦େֶϩΰத෦େֶϩΰ4PVSDF %FTUJOBUJPO𝑚2𝑚1p2(c|x) p1(c|x)'PSXBSE𝑚1𝑚2𝑚3^𝑦^𝑦^𝑦𝐿^𝑦,3𝐿^𝑦,2𝐿^𝑦,1𝐿2,1𝐿1,2𝐿3,1𝐿1,3𝐿2,3𝐿3,2,-EJWx x"UUFOUJPO "UUFOUJPO𝑸=𝐶∑𝑖=1𝑨𝑖𝑝 ɿಛϚοϓ ɿνϟωϧ ɿϊϧϜ𝑨𝑖𝐶𝑝ैདྷͷଛࣦ𝐿𝐴𝑇𝐿𝐴𝑇(𝑸2,𝑸1) =1𝐽𝐽∑𝑗𝑸𝑗2𝑸𝑗22−𝑸𝑗1𝑸𝑗122"UUFOUJPOଛࣦ(BUF𝐿2,1= G(𝐾𝐿(𝒑2,𝒑1) ±𝐿𝐴 𝑇(𝑸2,𝑸1))͚ۙͮΔ͢+−
w ,-EJWFSHFODFͱΞςϯγϣϯϩεΛ#BDLQSPQͯ͠ Λߋ৽𝑚1ϊʔυ̎ TPVSDF͔Βϊʔυ̍ EFTUJOBUJPOͷࣝసҠத෦େֶϩΰத෦େֶϩΰ𝑚2𝑚1𝑚1𝑚2𝑚3^𝑦^𝑦^𝑦𝐿^𝑦,3𝐿^𝑦,2𝐿^𝑦,1𝐿1,2𝐿3,1𝐿1,3𝐿2,3𝐿3,2x x4PVSDF %FTUJOBUJPO͚ۙͮΔ͢+−𝐿2,1#BDLXBSE𝐿2,1= G(𝐾𝐿(𝒑2,𝒑1) ±𝐿𝐴 𝑇(𝑸2,𝑸1)),-EJW𝐿𝐴𝑇(BUF"UUFOUJPO "UUFOUJPODetachBack-prop
w ΞϯαϯϒϧϊʔυΛλʔήοτϊʔυͱͯ͠࠷େԽ͢ΔΑ͏ʹ࠷దԽ ֤ϊʔυͷग़ྗΛฏۉʹΑΓΞϯαϯϒϧ͢ΔػߏΞϯαϯϒϧϊʔυத෦େֶϩΰத෦େֶϩΰ^𝑦𝐿^𝑦,𝑒𝑒𝑛𝑠𝑚1𝑚2𝑚3^𝑦^𝑦^𝑦𝐿^𝑦,3𝐿^𝑦,2𝐿^𝑦,1𝐿2,1𝐿1,2𝐿3,1𝐿1,3𝐿2,3𝐿3,2𝑚1𝑚2𝑒𝑛𝑠Ξϯαϯϒϧϊʔυ𝑚3Ξϯαϯϒϧػߏ𝑝(𝑐 𝑥) =𝑝1(𝑐 𝑥) +𝑝2(𝑐 𝑥) +𝑝3(𝑐|𝑥)𝑝1(𝑐 𝑥)𝑝2(𝑐 𝑥)𝑝3(𝑐 𝑥)𝑝(𝑐 𝑥)
ࣝసҠάϥϑͷΞϯαϯϒϧޮՌத෦େֶϩΰத෦େֶϩΰˠࣝసҠάϥϑʹ͓͍ͯΞϯαϯϒϧਫ਼্͕
࠷దԽͨ͠ΞϯαϯϒϧࣝసҠάϥϑ ϊʔυɿʣத෦େֶϩΰத෦େֶϩΰˠҟͳΔΞςϯγϣϯϚοϓʢΞϯαϯϒϧʹదͨ͠ଟ༷ੑʣΛ֫ಘw Ξϯαϯϒϧϊʔυɿw 7BOJMMBϞσϧɿೖྗը૾
࠷దԽͨ͠ΞϯαϯϒϧࣝసҠάϥϑ ϊʔυɿʣத෦େֶϩΰத෦େֶϩΰʹ͚ۙͮΔʹ͚ۙͮΔ͔Β͢ˠҟͳΔΞςϯγϣϯϚοϓʢΞϯαϯϒϧʹదͨ͠ଟ༷ੑʣΛ֫ಘʹ͚ۙͮΔ͔Β͓͢ޓ͍ʹ͚ۙͮΔʹ͚ۙͮΔʹ͚ۙͮΔʹ͚ۙͮΔೖྗը૾w Ξϯαϯϒϧϊʔυɿw 7BOJMMBϞσϧɿ
ଟ༷ͳΞϯαϯϒϧϞσϧ͔Βͷࣝৠཹத෦େֶϩΰத෦େֶϩΰw ࣝసҠάϥϑͷΞϯαϯϒϧΛڭࢣͱͯࣝ͠ৠཹ ڭࢣωοτϫʔΫɿࣝసҠάϥϑͰֶशͨ͠ෳͷ"#/ʢ3FT/FUʣ ੜెωοτϫʔΫɿ"#/ʢ3FT/FUʣೖྗը૾𝒙4UVEFOU/FUXPSLωοτϫʔΫ𝑚1𝒍1(𝒙)𝒍3(𝒙)ωοτϫʔΫ𝑚3𝒍𝑒𝑛𝑠(𝒙)𝒑𝑠(𝒙) ڭࢣϥϕϧ ^𝑦ࣝసҠ𝒑𝑒𝑛𝑠(𝒙)Թ͖4PGUNBYؔ5FBDIFS/FUXPSL𝑒 𝑛 𝑠𝑚1𝑚2𝑚3ࣝసҠάϥϑ
ଟ༷ͳΞϯαϯϒϧϞσϧ͔Βͷࣝৠཹத෦େֶϩΰத෦େֶϩΰࣝసҠάϥϑʹΑΔΞϯαϯϒϧΛৠཹ͢Δ͜ͱͰಉ͡ύϥϝʔλͰߴ͍ೝࣝੑೳΛൃش
ࣝసҠάϥϑʹΑΔڞಉֶशத෦େֶϩΰத෦େֶϩΰw ࣝసҠάϥϑΛఏҊ ̐छྨͷ(BUFؔʹΑΓɺࣝసҠΛ੍ޚ͢Δ͜ͱͰଟ༷ͳڞಉֶश ϋΠύʔύϥϝʔλαʔνʹΑΔ࠷దͳࣝసҠάϥϑΛ୳ࡧ ࣝసҠάϥϑʹΑΔΞϯαϯϒϧֶशw ൃݟͨ͜͠ͱʢϊʔυͷ߹ʣ ,%ͱ%.-͕༥߹ͨࣝ͠సҠάϥϑैདྷ๏Λ͑Δਫ਼Λୡ,%ͱ%.-ͷ༥߹ͨࣝ͠సҠάϥϑ𝑚3𝑚1𝑚2𝐿^𝑦,1𝐿^𝑦,2𝐿^𝑦,3𝐿1,2𝐿1,3𝐿2,1𝐿3,1𝐿3,2𝐿2,3^𝑦^𝑦^𝑦
ૉਓൃݰਓ࣮ߦத෦େֶϩΰத෦େֶϩΰw ࣝసҠάϥϑΛఏҊࣝసҠグϥϑʹΑΔڞಉֶश
47த෦େֶϩΰத෦େֶϩΰࣝసҠάϥϑʹΑΔ࠷దͳڭࢣ͋Γਂڞಉֶशͷ୳ࡧ +4"*>ʮૉਓൃݰਓ࣮ߦʯ୳ࡧʹΑΓ֫ಘͨ͠৽ͨͳݟが࣍ͷ৽ͨͳݚڀͷॹͱͳΔ͜ͱΛظ
w ϥϕϧ͋Γσʔλͱϥϕϧͳ͠σʔλΛֶशʹར༻ Ξϊςʔγϣϯʢϥϕϧ͚ʣʹ͔͔Δίετݮ ֶश༻σʔλͷ֬อ͕༰қڭࢣ͋Γֶशʢ4FNJTVQFSWJTFEMFBSOJOH 44-ֶश༻σʔλʹର͢Δσʔλͷׂ߹
w Ұகੑਖ਼ଇԽʢ$POTJTUFODZSFHVMBSJ[BUJPOʣ ϥϕϧͳ͠σʔλʹઁಈΛ༩͠ɼͦͷը૾ʹର͢ΔҰகੑΛֶश ैདྷ๏ɿ NPEFM *$-3>ɼ.FBO5FBDIFS<5BSWBJOFO /FVS*14>ͳͲw ٖࣅϥϕϦϯάʢ1TFVEPMBCFMJOHʣ ༧ଌ݁ՌΛPOFIPUԽٖͯ͠ࣅϥϕϧΛϥϕϧͳ͠σʔλʹ༩ ϥϕϧ͋Γσʔλͱٖࣅϥϕϧ͋Γσʔλͷࠞ߹ηοτΛ༻͍ͯڭࢣ͋Γֶश ैདྷ๏ɿ1TFVEP-BCFM *$.->ͳͲΠڭࢣ͋Γֶशͷදతͳํ๏ϥϕϧͳ͠σʔλ༧ଌ/FUXPSLٖࣅϥϕϧ͋Γσʔλʢϥϕϧͳ͠σʔλʣ)BSEUBSHFUラベルありデータNetwork正解情報予測誤差ラベルなしデータ予測１摂動を付与ٖࣅϥϕϦϯάラベルありデータNetwork正解情報予測誤差ラベルなしデータ予測１摂動を付与Ұ؏ੑਖ਼ଇԽϥϕϧͳ͠σʔλラベルありデータNetwork正解情報予測誤差ラベルなしデータ予測１摂動を付与ʴઁಈʴઁಈ༧ଌ༧ଌޡࠩ/FUXPSL
w ٖࣅϥϕϦϯάɿऑม࣌ͷ༧ଌ͕ᮢΛ͑ͨ߹ͷΈٖࣅϥϕϧΛੜ ऑมɿࠨӈస ฏߦҠಈw Ұகੑଛࣦɿੜٖͨ͠ࣅϥϕϧͱڧม࣌ͷ༧ଌͷޡࠩ ڧมɿෳछͷը૾มʹΑΔڧ͍ઁಈʢ3BOE"VHNFOU $713>ʣҰகੑਖ਼ଇԽͱٖࣅϥϕϦϯάͷϋΠϒϦουɿ'JY.BUDI<4PIO /FVS*14>Ұ؏ੑଛࣦ༧ଌ/FUXPSLڧมऑมڭࢣ͋Γଛࣦϥϕϧ༧ଌٖࣅϥϕϦϯάϥϕϧ͋ΓσʔλラベルありデータNetwork正解情報予測誤差ラベルなしデータNetwork予測１予測２摂動を付与ϥϕϧͳ͠σʔλ)BSEUBSHFUˠਓ͕ઃܭͨ͠Έ߹ΘͤͰ͋ΔͨΊ࠷దͳֶश๏ͱݶΒͳ͍
w ਓखʹΑΒͳ͍৽͍͠ڭࢣ͋Γڞಉֶश๏ͷ֫ಘw Ξϓϩʔν ֤ैདྷ๏ΛͦΕͧΕάϥϑͰ౷Ұతʹදݱ άϥϑදݱͷߏཁૉΛϥϯμϜʹΈ߹Θͤͯߴਫ਼ͳֶश๏Λ୳ࡧຊݚڀͷతɾɾɾάϥϑදݱNPEFMΠ.FBO5FBDIFS!!!"Parameterfor ExponentialMoving AverageKL-divɾɾɾNPEFMΠ.FBO5FBDIFSैདྷ๏Network!!"+$! Network!!+$!′ &(!!, " + $!′)BackPropLossGraphicalrepresentation!!!!!!&(!!, " + $!)KL-divKL-divNetworkEMA(%!)'+)!ExponentialMoving AverageNetwork%!+)!′ +(EMA(%!), ' + )!)+(%!, ' + )!′)LossBackProp!!!"Parameterfor ExponentialMoving AverageGraphicalrepresentationKL-divྫɿNPEFM.FBO5FBDIFSΠάϥϑߏͷ୳ࡧ!!KL-div!"Parameterfor ExponentialMoving AverageKL-divɾɾɾ
ɾɾɾάϥϑදݱNPEFMΠ.FBO5FBDIFS!!!"Parameterfor ExponentialMoving AverageKL-divɾɾɾNPEFMΠ.FBO5FBDIFSैདྷ๏Network!!"+$! Network!!+$!′ &(!!, " + $!′)BackPropLossGraphicalrepresentation!!!!!!&(!!, " + $!)KL-divKL-divNetworkEMA(%!)'+)!ExponentialMoving AverageNetwork%!+)!′ +(EMA(%!), ' + )!)+(%!, ' + )!′)LossBackProp!!!"Parameterfor ExponentialMoving AverageGraphicalrepresentationKL-divྫɿNPEFM.FBO5FBDIFSΠάϥϑߏͷ୳ࡧ!!KL-div!"Parameterfor ExponentialMoving AverageKL-divɾɾɾw ֤ैདྷ๏ΛͦΕͧΕάϥϑͰ౷Ұతʹදݱ Έ߹Θ͕ͤ༰қʹͳΓϋΠύʔύϥϝʔλͷΑ͏ʹௐՄೳ ϊʔυɿωοτϫʔΫ Τοδɿଛࣦܭࢉैདྷͷڭࢣ͋Γֶश๏ΛάϥϑͰදݱ
w ڭࢣ͋ΓଛࣦͱҰகੑଛࣦ͕খ͘͞ͳΔΑ͏ʹֶश ڭࢣ͋Γଛࣦɿϥϕϧ͋Γσʔλͷ༧ଌͱϥϕϧͷޡࠩ Ұ؏ੑଛࣦɹɿϥϕϧͳ͠σʔλʹҟͳΔઁಈΛ༩ͨ࣌͠ͷ༧ଌؒͷޡࠩઁಈɿ%SPQPVUɼը૾มҰகਖ਼ଇԽͷදతͳख๏ɿ NPEFM *$-3>ΠҰகੑଛࣦ༧ଌ/FUXPSLڭࢣ͋Γଛࣦϥϕϧ༧ଌϥϕϧ͋ΓσʔλラベルありデータNetwork正解情報予測誤差ラベルなしデータNetwork予測１予測２摂動を付与ϥϕϧͳ͠σʔλʴઁಈʴઁಈ)BSEUBSHFUラベルありデータNetwork正解情報予測誤差ラベルなしデータ予測１摂動を付与
w Ұகੑଛࣦɿ࢝ͷϊʔυͱऴͷϊʔυ͕ಉ͡ΤοδͰදݱ ,-EJWFSHFODFʢ,-EJWʣͰ༧ଌؒͷޡࠩΛܭࢉNPEFMΛάϥϑͰදݱΠKL(f(x), f(x′)) =C∑ifi(x)logfi(x)fi(x′)NPEFMΠ άϥϑදݱɿ֬ʢ༧ଌ֬ʣɿΫϥεf(x), f(x′)CNetwork!!"+$! Network!!+$!′ &(!!, " + $!′)BackPropLossGraphicalrepresentation!!!!!!&(!!, " + $!)KL-divKL-div
w NPEFMʹࢦҠಈฏۉʢ&."ʣωοτϫʔΫΛಋೖֶͯ͠श ҰகੑଛࣦɿωοτϫʔΫͱ&."ωοτϫʔΫʹ͓͚Δ༧ଌؒͷޡࠩ &."ωοτϫʔΫͷॏΈωοτϫʔΫͷॏΈΛՃࢉ͢Δ͜ͱͰߋ৽ॏΈύϥϝʔλͷΞϯαϯϒϧʹΑΓߴ͍ੑೳΛൃش͠ɼֶशΛิॿΠҰகੑਖ਼ଇԽͷදతͳख๏ɿ.FBO5FBDIFS<5BSWBJOFO /FVS*14>ڭࢣ͋Γଛࣦϥϕϧϥϕϧ͋ΓσʔλラベルありデータNetwork正解情報予測誤差ラベルなしデータNetwork予測１予測２摂動を付与)BSEUBSHFUҰகੑଛࣦ༧ଌ/FUXPSL༧ଌϥϕϧͳ͠σʔλラベルありデータNetwork正解情報予測誤差ラベルなしデータNetwork予測１摂動を付与ڭࢣ͋Γଛࣦϥϕϧ! " #$ % & ' () *+, -./㎯1㌪刷卡䇦6〱! " #8 9 & ' () *+, -./卡䇦:卡䇦;㏗俍> 冊卥ϥϕϧ͋ΓσʔλPOFIPU! " #$ % & ' () *+, -./㎯1㌪刷卡䇦6〱㏗俍> 冊卥ϥϕϧͳ͠σʔλ/FUXPSL! " #$ % & ' () *+, -./㎯1㌪刷卡䇦6〱! " #8 9 & ' () *+, -./卡䇦:㏗俍> 冊卥! " #$ % & ' () *+, -./㎯1㌪刷卡䇦6〱! " #$ % & ' () *+, -./㎯1㌪刷卡䇦6〱༧ଌ༧ଌҰ؏ੑଛࣦڭࢣ͋Γଛࣦϥϕϧ! " #$ % & ' () *+, -./㎯1㌪刷卡䇦6〱! " #8 9 & ' () *+, -./卡䇦:卡䇦;㏗俍> 冊卥ϥϕϧ͋ΓσʔλPOFIPU! " #$ % & ' () *+, -./㎯1㌪刷卡䇦6〱! " #8 9 & ' () *+, -./卡䇦:卡䇦;㏗俍> 冊卥ϥϕϧͳ͠σʔλ/FUXPSL! " #$ % & ' () *+, -./㎯1㌪刷卡䇦6〱! " #8 9 & ' () *+, -./卡䇦:卡䇦;㏗俍> 冊卥! " #$ % & ' () *+, -./㎯1㌪刷卡䇦6〱! " #8 9 & ' (卡䇦:㏗俍> 冊卥! " #$ % & ' () *+, -./㎯1㌪刷卡䇦6〱! " #8 9 & ' (卡䇦:㏗俍> 冊卥༧ଌ༧ଌҰ؏ੑଛࣦʴઁಈʴઁಈ/FUXPSL
w &."ωοτϫʔΫʹՃࢉ͢ΔύϥϝʔλͷํΛΤοδͰදݱ &."ωοτϫʔΫɿύϥϝʔλ ͷࢦҠಈฏۉ Ͱߋ৽θ1EMA(θ1).FBO5FBDIFSΛάϥϑͰදݱEMA(θ1,t) = αEMA(θ1,t−1) + (1 − α)θ1,tɿϋΠύʔύϥϝʔλɿֶशεςοϓαt.FBO5FBDIFS άϥϑදݱNetworkEMA(%!)'+)!ExponentialMoving AverageNetwork%!+)!′ +(EMA(%!), ' + )!′)+(%!, ' + )!)LossBackProp!!!"Parameterfor ExponentialMoving AverageGraphicalrepresentationKL-div
w ϥϕϧͳ͠σʔλʹٖࣅϥϕϧΛ༩ֶͯ͠श ֶशং൫ϥϕϧ͋ΓσʔλͷΈΛ༻͍ͯڭࢣ͋Γֶश ༧ଌΛͱʹϥϕϧͳ͠σʔλʹٖࣅϥϕϧΛ༩ʢ·ͨߋ৽ʣ ϥϕϧ͋Γσʔλͱٖࣅϥϕϧ͋Γσʔλͷࠞ߹ηοτΛ༻͍ͯڭࢣ͋Γֶश ̎ͱ̏Λ܁Γฦ͢ϥϕϧ༧ଌޡࠩ/FUXPSLٖࣅϥϕϧ͋Γσʔλࠞ߹ηοτラベルありデータNetwork正解情報予測誤差ラベルなしデータNetwork予測１摂動を付与ϥϕϧ͋ΓσʔλラベルありデータNetwork正解情報予測誤差ラベルなしデータNetwork予測１予測２摂動を付与ٖࣅϥϕϧ͋Γσʔλ)BSEUBSHFUラベルありデータNetwork正解情報予測誤差ラベルなしデータNetwork予測１予測２摂動を付与ϥϕϧͳ͠σʔλ༧ଌ/FUXPSLラベルありデータNetwork正解情報予測誤差ラベルなしデータNetwork予測１予測２摂動を付与ٖࣅϥϕϦϯάͷදతͳख๏ɿ1TFVEP-BCFM *$.->
w ٖࣅϥϕϧʹର͢Δ༧ଌͷޡࠩΛٻΊΔ1TFVEP-PTTʹΑΓΤοδͰදݱ 1TFVEP-PTTɿٖࣅϥϕϧͱ༧ଌ֬ͷޡࠩ1TFVEP-BCFMΛάϥϑͰදݱAAACqXichVFNTxNBGH5Yv6B+UORi4qWxQUqAZtYQNCQmRDTx4IGChUaWbGaHabth9iO706Zls3+AP+DBEybGqD/Di0cuHOo/MB4x8eLBt9tNCBL1nczMM8+8zzvPzDihcmPN2GDMuHT5ytVr4xOF6zdu3posTt3eioNOJGRdBCqIGg6PpXJ9WdeuVrIRRpJ7jpLbzv7acH+7K6PYDfyXuh/KXY+3fLfpCq6JsotPLY/rtuAqeZHaSRjLtNKbe7yYsY6TPEvtXn82sSKvpIJW2qxYui01t82FUm/eOsjg7JxdLLMqy6J0EZg5KCOP9aD4Hhb2EECgAw8SPjRhBY6Y2g5MMITE7SIhLiLkZvsSKQqk7VCWpAxO7D6NLVrt5KxP62HNOFMLOkVRj0hZwgw7YR/YKfvCPrFv7NdfayVZjaGXPs3OSCtDe/LwzubP/6o8mjXaZ6p/etZo4lHm1SXvYcYMbyFG+u7B69PNlY2Z5D57y76T/yM2YJ/pBn73h3hXkxtvUKAPMP987otg60HVXK4u1ZbKq0/yrxjHXdxDhd77IVbxHOuo07kfcYwBvhrzRs1oGK9GqcZYrpnGuTDEb06spCc=Lpse(x) = Exy0logf(✓1, x + ⇣01)AAACqXichVFNTxNBGH5Yv6B+UORi4qWxQUqAZtYQNCQmRDTx4IGChUaWbGaHabth9iO706Zls3+AP+DBEybGqD/Di0cuHOo/MB4x8eLBt9tNCBL1nczMM8+8zzvPzDihcmPN2GDMuHT5ytVr4xOF6zdu3posTt3eioNOJGRdBCqIGg6PpXJ9WdeuVrIRRpJ7jpLbzv7acH+7K6PYDfyXuh/KXY+3fLfpCq6JsotPLY/rtuAqeZHaSRjLtNKbe7yYsY6TPEvtXn82sSKvpIJW2qxYui01t82FUm/eOsjg7JxdLLMqy6J0EZg5KCOP9aD4Hhb2EECgAw8SPjRhBY6Y2g5MMITE7SIhLiLkZvsSKQqk7VCWpAxO7D6NLVrt5KxP62HNOFMLOkVRj0hZwgw7YR/YKfvCPrFv7NdfayVZjaGXPs3OSCtDe/LwzubP/6o8mjXaZ6p/etZo4lHm1SXvYcYMbyFG+u7B69PNlY2Z5D57y76T/yM2YJ/pBn73h3hXkxtvUKAPMP987otg60HVXK4u1ZbKq0/yrxjHXdxDhd77IVbxHOuo07kfcYwBvhrzRs1oGK9GqcZYrpnGuTDEb06spCc=Lpse(x) = Exy0logf(✓1, x + ⇣01)AAACqXichVFNTxNBGH5Yv6B+UORi4qWxQUqAZtYQNCQmRDTx4IGChUaWbGaHabth9iO706Zls3+AP+DBEybGqD/Di0cuHOo/MB4x8eLBt9tNCBL1nczMM8+8zzvPzDihcmPN2GDMuHT5ytVr4xOF6zdu3posTt3eioNOJGRdBCqIGg6PpXJ9WdeuVrIRRpJ7jpLbzv7acH+7K6PYDfyXuh/KXY+3fLfpCq6JsotPLY/rtuAqeZHaSRjLtNKbe7yYsY6TPEvtXn82sSKvpIJW2qxYui01t82FUm/eOsjg7JxdLLMqy6J0EZg5KCOP9aD4Hhb2EECgAw8SPjRhBY6Y2g5MMITE7SIhLiLkZvsSKQqk7VCWpAxO7D6NLVrt5KxP62HNOFMLOkVRj0hZwgw7YR/YKfvCPrFv7NdfayVZjaGXPs3OSCtDe/LwzubP/6o8mjXaZ6p/etZo4lHm1SXvYcYMbyFG+u7B69PNlY2Z5D57y76T/yM2YJ/pBn73h3hXkxtvUKAPMP987otg60HVXK4u1ZbKq0/yrxjHXdxDhd77IVbxHOuo07kfcYwBvhrzRs1oGK9GqcZYrpnGuTDEb06spCc=Lpse(x) = Exy0logf(✓1, x + ⇣01)1TFVEP-BCFM άϥϑදݱωοτϫʔΫ͕̎ͭͷ߹Network!! or !" Pseudo-Labelingone-hotNetwork!!"+$!+$!′&(!!, " + $!)LossPseudoLoss!!!!!"PseudoLossGraphicalrepresentation orBackProp
w ධՁରϊʔυͷਫ਼͕࠷େԽ͢ΔΑ͏ʹάϥϑߏΛ࠷దԽ ิॿϊʔυɿධՁରϊʔυͷֶशΛαϙʔτLosses:・KL-divergence・PseudoLossGate functions:・Through Gate・Cutoff Gate・Linear Gate・Threshold GateExplore space:Models:・ResNet32・WideResNet28-2・WideResNet28-6・EMA modelEdgeNodeBackpropDetachGateLoss⋅ධՁରϊʔυิॿϊʔυάϥϑ࠷దԽʹΑΔڭࢣ͋Γਂڞಉֶश๏ͷ୳ࡧ
ධՁ݁Ռϥϕϧ͋Γσʔλ BMM4VQFSWJTFE 1TFVEP-BCFM .FBO5FBDIFS 'JY.BUDI 0VST ϊʔυ 0VST ϊʔυ NPEFMΠਖ਼ղ>ˠϥϕϧ͋ΓσʔλʹΑͬͯ࠷దͳڭࢣ͋Γֶश๏ҟͳΔଟ͍߹ʢ ʣɿ1TFVEP-BCFM͕ߴਫ਼গͳ͍߹ʢ ʙ ʣɿ NPEFM .FBO5FBDIFS͕ߴਫ਼Π
ධՁ݁Ռϥϕϧ͋Γσʔλ BMM4VQFSWJTFE 1TFVEP-BCFM .FBO5FBDIFS 'JY.BUDI 0VST ϊʔυ 0VST ϊʔυ NPEFMΠਖ਼ղ>ˠैདྷ๏ͱൺ୳ࡧͨ͠ख๏ʢ0VSTʣͷਫ਼͕࠷ߴ͍ʴ̒ʴ
w ࠷దԽͨ͠άϥϑߏΛௐࠪ ڭࢣ͋Γֶशͷw ௐࠪ ֶशܦաʹ͓͚Δڭࢣ͋Γڞಉֶशͷ ϥϕϧ͋Γσʔλʹ͓͚Δڭࢣ͋Γڞಉֶशͷ ϊʔυͷมԽʹ͓͚Δڭࢣ͋Γڞಉֶशͷ࠷దԽͨ͠άϥϑߏ
w ϊʔυɿ̎ɼϥϕϧ͋Γσʔλɿ ʢগͳ͍ʣ ਖ਼ղɿ>࠷దԽͨ͠άϥϑߏɿֶशܦաʹ͓͚Δ>(BUFؔ ֶशํ๏1. ResNet32 (55.94%) Linear2. WRN28_6 (57.81%)LinearLinear LabelThroughLabelThrough1. ResNet32 (55.94%) Consistency2. WRN28_6 (57.81%)KDConsistency LabelSupervisedLabelSupervisedڭࢣ͋Γֶशڭࢣ͋Γֶशֶशং൫ɿݸʑͷϊʔυͰಠཱʹڭࢣ͋Γֶश
w ϊʔυɿ̎ɼϥϕϧ͋Γσʔλɿ ʢগͳ͍ʣ ਖ਼ղɿ>࠷దԽͨ͠άϥϑߏɿֶशܦաʹ͓͚Δ>(BUFؔ ֶशํ๏1. ResNet32 (55.94%) Linear2. WRN28_6 (57.81%)LinearLinear LabelThroughLabelThroughֶशং൫ɿݸʑͷϊʔυͰಠཱʹڭࢣ͋Γֶश1. ResNet32 (55.94%) Consistency2. WRN28_6 (57.81%)KDConsistency LabelSupervisedLabelSupervisedˠֶशલͱޙͰҟͳΔֶशઓུ͕ޮՌతֶशऴ൫ɿݸʑͷϞσϧ͕ NPEFMʹΑΔֶशʹมԽͭͭ͠ɼ̎൪ϊʔυ͔ΒৠཹΠNPEFMΠNPEFMΠৠཹ
w ϊʔυɿ̎ɼϥϕϧ͋Γσʔλɿ ʢଟ͍ʣ ਖ਼ղɿ>࠷దԽͨ͠άϥϑߏɿֶशܦաʹ͓͚Δ>(BUFؔ ֶशํ๏1. ResNet32 (62.76%) Linear2. WRN28_6 (60.4%)Through1. ResNet32 (62.76%) Consistency2. WRN28_6 (60.4%)PseudoLabelingֶशࡁΈϞσϧٖࣅϥϕϦϯάֶशং൫ɿࣄલֶशϞσϧͷٖࣅϥϕϦϯάͰֶशNPEFMΠֶशऴ൫ɿٖࣅϥϕϦϯάͱ NPEFMͰֶशΠˠֶशऴ൫ʹҰகੑਖ਼ଇԽʢ NPEFMʣͰֶश͢Δ͜ͱ͕ޮՌతΠ
ֶशܦաʹ͓͚Δɿ6."1ʹΑΔՄࢹԽFQPDIFQPDI1. ResNet32 (62.76%) Consistency2. WRN28_6 (60.4%)PseudoLabelingֶशࡁΈϞσϧٖࣅϥϕϦϯάNPEFMΠˠֶशޙͷҰகੑਖ਼ଇԽʢ NPEFMʣʹΑΓΫϥελԽ͕ਐΉΠ1. ResNet32 (62.76%) Consistency2. WRN28_6 (60.4%)PseudoLabelingֶशࡁΈϞσϧٖࣅϥϕϦϯά
w ϥϕϧ͋Γσʔλ͕গͳ͍ʢ ຕʣ߹࠷దԽͨ͠άϥϑߏɿϥϕϧ͋Γσʔλʹ͓͚Δ>4,000 label2,000 label 6,000 label1. ResNet32 (55.94%) Consistency2. WRN28_6 (57.81%)KDConsistency LabelSupervisedLabelSupervised> >ֶशํ๏ʢϊʔυ̎ʣ ֶशํ๏ʢϊʔυ̏ʣϥϕϧ͋Γσʔλ͕গͳ͍߹ɿҰகੑਖ਼ଇԽʢ NPEFMʣͱৠཹʢ૬ޓֶशʣ͕ޮՌతΠৠཹNPEFMΠNPEFMΠNPEFMΠ.FBO5FBDIFS૬ޓֶश
w ϥϕϧ͋Γσʔλ͕ଟ͍ʢ ຕʣ߹࠷దԽͨ͠άϥϑߏɿϥϕϧ͋Γσʔλʹ͓͚Δ>ֶशํ๏ʢϊʔυ̎ʣ ֶशํ๏ʢϊʔυ̏ʣ> >Gate関数 学習方法Gate function Learning method1. ResNet32 (62.76%) Consistency2. WRN28_6 (60.4%)PseudoLabelingֶशࡁΈϞσϧٖࣅϥϕϦϯάٖࣅϥϕϦϯάϥϕϧ͋Γσʔλ͕ଟ͍߹ɿˠධՁରϊʔυٖࣅϥϕϦϯάΛ༻ֶ͍ͯश
w ϊʔυͷ߹࠷దԽͨ͠άϥϑߏɿϊʔυ͕ଟ͍࣌ͷֶशํ๏ʢϥϕϧ͋Γσʔλ ຕʣֶशํ๏ʢϥϕϧ͋Γσʔλ ຕʣ> >Gate関数 学習方法Gate function Learning methodlabel 6,000 label 10,000 label8,000 label.FBO5FBDIFS.FBO5FBDIFS૬ޓֶशิॿϊʔυ.FBO5FBDIFSΛแˠࢦҠಈฏۉϞσϧͰΑΓྑ͍5FBDIFSΛֶशϊʔυ͕ଟ͍߹.FBO5FBDIFSͰิॿϊʔυΛվળ͢Δ͜ͱ͕ޮՌతɹˠڭࢣ͋Γڞಉֶश
w ධՁରϊʔυʹରͯ͠खಈͰΤοδΛՃ ϥϕϧ͔ΒͷΤοδɹɹɹɹɹɹɿڭࢣ͋Γֶश ධՁରϊʔυ͔Β̎ͷΤοδɿ.FBO5FBDIFSͱͷ૬ޓֶश୳ࡧͰಘͨݟΛ׆͔ͨ͠खಈઃܭʹΑΔߋͳΔվળख࡞ۀͰઃܭͨ͠άϥϑʢʣ୳ࡧͰ֫ಘͨ͠άϥϑʢʣQU্୳ࡧͰಘͨάϥϑͱݟΛ׆͔͠खಈઃܭͰ͞Βʹվળ.FBO5FBDIFSͰิॿϊʔυΛվળٖͭͭ͠ࣅϥϕϦϯάͰֶश.FBO5FBDIFSͰิॿϊʔυΛվળٖͭͭ͠ࣅϥϕϦϯάͰ NPEFMͱ૬ޓֶशΠֶशํ๏ ֶशํ๏学習方法n Learning method.FBO5FBDIFSٖࣅϥϕϦϯάGate関数 学習方法Gate function Learning method.FBO5FBDIFSٖࣅϥϕϦϯάҰ؏ੑਖ਼ଇԽ૬ޓֶश 'FFECBDLڭࢣ͋Γֶश-BCFM
w άϥϑ୳ࡧʹΑΓ৽͍͠ڭࢣ͋Γڞಉֶश๏Λ୳ࡧ άϥϑ୳ࡧʹΑΓಘΒΕͨޮՌతͳڭࢣ͋Γڞಉֶशʹ͓͚Δݟ ֶशͷܦաͱͱʹֶशઓུΛมԽͤ͞Δ͜ͱͰߴਫ਼ԽֶशޙʹҰ؏ੑਖ਼ଇԽʢ NPEFMʣΛߦ͏͜ͱ͕༗ޮ ϥϕϧ͋Γσʔλ͝ͱʹ࠷దͳֶशઓུҟͳΔϥϕϧ͋Γσʔλ͕ଟ͍߹ٖࣅϥϕϦϯάͰͷֶश͕༗ޮɹ ෳϞσϧΛ༻͍ͨڞಉֶशڭࢣ͋Γֶशʹ༗ޮϊʔυ͕ଟ͍߹.FBO5FBDIFSͰิॿϊʔυΛվળ͢Δ͜ͱͰޮՌతͳڭࢣ͋ΓڞಉֶशΛ࣮ݱ ୳ࡧͰಘͨάϥϑͱݟΛ׆͔͠खಈઃܭͰ͞Βʹվળϊʔυɼϥϕϧ͋Γσʔλ ͷ୳ࡧͰಘͨάϥϑͷਫ਼ΛखಈઃܭͰQUվળΠ·ͱΊɿࣝసҠάϥϑʹΑΔ࠷దͳڭࢣ͋Γਂڞಉֶशͷ୳ࡧ
ૉਓൃݰਓ࣮ߦத෦େֶϩΰத෦େֶϩΰʲϑΣϩʔ͔Βͷϝοηʔδʳ ใɾγεςϜιαΠΤςΟࢽ ୈ 26 רୈ 4 ߸ʢ௨ר 105 ߸ʣૉਓൃݰਓ࣮ߦ 2.0ϑΣϩʔ ౻٢ ߂த෦େֶʮண؟େہணखখہɼૉਓൃݰਓ࣮ߦʯɼ2006 8 ݄ʹࡏ֎ݚڀͰถࠃΧʔωΪʔϝϩϯେֶϩϘοτֶݚڀॴʹ 1 ؒࡏ͠ɼؼࠃͷࡍʹۚग़༤ઌੜ͔Β͍ͨݴ༿Ͱ͋Δɽʮૉਓൃݰਓ࣮ߦʯͱɼۚग़ઌੜͷஶॻ [1] ʹΑΔͱɼʮൃ୯७ɼૉɼࣗ༝ɼ؆୯Ͱͳ͚ΕͳΒͳ͍ɽ͔͠͠ɼൃΛ࣮ߦʹҠ͢ʹ͕͍ࣝΔɼख़࿅͞Εٕ͕͍ͨΔʯͱ͍͏͜ͱͰ͋ΔɽචऀͦΕҎདྷɼ͜ͷݴ༿ΛϞοτʔʹͯ͠ݚڀʹऔΓΜͰ͍Δɽ͔͠͠ɼʮݴ͏қ͘ɼߦ͏͠ʯͷయܕͰ͋Γɼ࣮ફ͢Δͷͳ͔ͳ͔͍͠ɽଟ͘ͷจΛಡΜͰ͍͘ͱ͕ࣝਂ·Γઐੑߴ͘ͳΔ͕ɼͦΕ͕োนͱͳͬͯɼຊ࣭Ͱͳ͘খ͞ͳ͜ͱʹணͨ͠ઃఆΛߦ͍͕ͪͰ͋Δɽ·ͨɼຊ࣭Λଊ͑ͯΛ۪ͤͣʹ࣮͢Δͱ͏·͘ಈ͔ͳ͍͜ͱ͕͋ΔɽຊߘͰɼ 10 Λܦͯʮૉਓൃݰਓ࣮ߦʯʹগ͚͚ͩۙͮͨ͠ͷͰͱࢥ͏චऀΒͷݚڀ 2 ྫʹ͍ͭͯհ͠ɼ࠷ۙɼࣗͳΓʹࢥ͍ඳ͘ʮૉਓൃݰਓ࣮ߦʯͷΞοϓσʔτΛڞ༗͍ͨ͠ɽ2010 ࠒɼը૾ؒͷରԠϚονϯάͷͨΊͷಛݕग़ɾهड़ͷݚڀ͕ଟ͘औΓ·Ε͍ͯͨɽதͰɼࣹӨมԽΛ͏ը૾ؒͷରԠϚονϯάɼΩʔϙΠϯτͷಛΛදݱ͢ΔΞϑΟϯྖҬΛٻΊΔඞཁ͕͋Γɼ͍͠Ͱ͋ͬͨɽैདྷख๏ͰɼΩʔϙΠϯτʹରͯ͠ҰͭͷΞϑΟϯྖҬ͔͠ਪఆ͠ͳ͍ͨΊɼը૾ͷมܗΩʔϙΠϯτͷҐஔͣΕͷӨڹʹΑΓҟͳΔΞϑΟϯྖҬΛਪఆͯ͠͠·͏ͱ͍͏͕͋ͬͨɽ͜Εɼہॴత୳ࡧΛߦ͏͜ͱ͕ݪҼͰ͋Γɼʮண؟খہணखখہʯͱݴ͑Δɽ2015 ʹචऀΒ͕ࠃࡍձٞ ICCV ʹͯൃදͨ͠ʮඇํੑ LoG ϑΟϧλʹΑΔෳͷΞϑΟϯྖҬͷਪఆʯ[2] Ͱɼ༷ʑͳପԁܗঢ়ͷඇํੑ LoG ϑΟϧλΛ༻͍ͯෳͷΞϑΟϯྖҬΛਪఆ͢Δ͜ͱΛఏҊͨ͠ɽγϯϓϧʹɼҰͭͰͳ͘ෳͷྖҬ͕͋ͬͯΑ͍ͷͰɼͱ͍͏ʮૉਓൃʯͰ͋Δɽ͔͠͠ɼ͍࣮͟ͱͳΔͱɼඇํੑ LoG ϑΟϧλʹ x ํͷεέʔϧɼy ํͷεέʔϧɼճస֯ͷ 3 ύϥϝʔλ͕͋ΓɼͦͷΈ߹ΘͤઍछྨͱͳΔɽෳͷΞϑΟϯྖҬΛਪఆ͢ΔͨΊɼযΔ͕༨Γ͜ͷઍछྨͷϑΟϧλશͯΛΈࠐΉॲཧΛ͜ͷ··ߦ͏ͱɼେͳܭࢉίετ͕ඞཁͱͳΔɽͦ͜Ͱɼʮݰਓ࣮ߦʯͱͯ͠ɼઍछྨͷඇํੑ LoG ϑΟϧλ܈ΛಛҟղʹΑΓٻΊͨ 14 छྨͷݻ༗ϑΟϧλͰۙࣅ͠ɼΈࠐΈॲཧΛޮతʹܭࢉ͢Δ͜ͱʹͨ͠ɽ͜ΕʹΑΓɼෳͷΞϑΟϯྖҬΛޮతʹٻΊΔ͜ͱ͕Ͱ͖ɼࣹӨมԽΛ͏ը૾ؒͷରԠϚονϯάͷߴਫ਼ԽΛ࣮ݱͨ͠ɽ͜ͷݚڀʹ͓͍ͯɼʮૉਓൃݰਓ࣮ߦʯͷݴ༿͕ݚڀͷํੑਐΊํΛܾΊΔखॿ͚Λͯ͘͠ΕͨΑ͏ʹࢥ͑ɼ2006 ͔Β 10 ΛܦͯɼΑ͏͘ʮૉਓൃݰਓ࣮ߦʯʹҰา͚ۙͮͨͱࢥ͑ΔݚڀͰ͋ͬͨɽ͜ͷݚڀҎޙɼৗʹɼૉਓൃͰݰਓ࣮ߦʹͳ͍ͬͯΔ͔Λࣗࣗ͠ͳ͕ΒݚڀʹऔΓΜͰདྷͨɽ2012 Ҏ߱ɼਂֶश͕ओମͱͳͬͨίϯϐ16[email protected]@BSUJDMFDIBSKB
ίϯϐϡʔλϏδϣϯ࠷લઢத෦େֶϩΰத෦େֶϩΰIUUQTXXXLZPSJUTVQVCDPKQCPPLEFUBJMίϯϐϡʔλϏδϣϯ࠷લઢ Spring 2022 ʗר಄ݴ5ר಄ݴ Spring 2022Visual HullతνϡʔτϦΞϧͷεεϝ˙౻٢߂2000 લޙʹऔΓ·Ε͍ͯͨίϯϐϡʔλϏδϣϯͷΞϧΰϦζϜͰ͋Δࢹମੵަࠩ๏ʢvisual hullʣΛ͝ଘͩΖ͏͔ɻࢹମੵަࠩ๏ɼLaurentini͕ఏҊͨ͠ Shape-from-silhouette ʹΑΔ 3D ࠶ߏख๏Ͱ͋Δɻ·ͣɼΧϝϥࢹ͔ΒγϧΤοτը૾1) Λ༻͍ͯ෮ݩରͷΦϒδΣΫτΛӨͨ͠γϧ 1) γϧΤοτը૾ͷલܠϚεΫɼ෮ݩରԠͰ͋ΔΦϒδΣΫτͷ 2 ࣍ݩӨͰ͋ΔɻΤοτԁਲ਼Λ࡞͢ΔɻҟͳΔࢹͰࡱӨͨ͠γϧΤοτը૾͔Βੜ͞Εͨԁਲ਼ͷަ Visual Hull ͱݺΕɼ͜ͷަΛٻΊΔ͜ͱͰΦϒδΣΫτͷ 3࣍ݩܗঢ়ͷ෮ݩ͕ՄೳͱͳΔɻਤ 1 ɼCV ͳΒͼʹ CG քͰ༗໊ͳ “StanfordBunny”ʢhttp://graphics.stanford.edu/data/3Dscanrep/ʣͱݺΕΔ 3D ΦϒδΣΫτΛ෮ݩͨ͠ྫͰ͋Δɻগͳ͍ࢹͷγϧΤοτը૾͔Β෮ݩͨ͠ 3࣍ݩܗঢ়ɼຊདྷͷόχʔͷ 3 ࣍ݩܗঢ়ʹͳ͍ͬͯͳ͍ɻҰํͰɼଟͷҟͳΔࢹͷγϧΤοτը૾Λ༻͍Δͱɼਖ਼֬ͳ 3 ࣍ݩܗঢ়Λ෮ݩ͢Δ͜ͱ͕Ͱ͖Δɻ͜ΕɼݪஶจΛಡΉ͜ͱʹ͓͍ͯಉ༷Ͱ͋Δͱࢲࢥ͏ɻจͷຊ࣭͕Ͳ͜ʹ͋Δ͔Λਂ͘ཧղ͢ΔʹɼҰࢹ͔ΒಡΈࠐΉͷͰͳ͘ɼҟͳΔ(a) 3 ࢹ(b) 80 ࢹʜABCABCγϧΤοτը૾ ෮ݩ݁Ռਤ 1 ࢹମੵަࠩ๏ʢvisual hullʣɻhttp://www.sanko-shoko.net/note.php?id=tjly ͷίʔυΛར༻ͯ͠࡞ɻίϯϐϡʔλϏδϣϯ࠷લઢɹ4QSJOHҪ৲ળٱɾڇٱɾยԬ༟༤ɾ౻٢߂ฤIUUQTXXXLZPSJUTVQVCDPKQCPPLEFUBJM
.13(5PVS74த෦େֶϩΰத෦େֶϩΰIUUQTXXXZPVUVCFDPNXBUDI W(LV,'5&
ػց֮ϩϘςΟΫεݚڀάϧʔϓத෦େֶϩΰத෦େֶϩΰڭत౻٢߂ Hironobu Fujiyoshi E-mail: [email protected]1997 த෦େֶେֶӃത࢜ޙظ՝ఔमྃ, 1997 ถΧʔωΪʔϝϩϯେֶϩϘοτֶݚڀॴPostdoctoral Fellow, 2000 த෦େֶֶ෦ใֶՊߨࢣ, 2004 த෦େֶ।ڭत,2005 ถΧʔωΪʔϝϩϯେֶϩϘοτֶݚڀॴ٬һݚڀһ(ʙ2006), 2010 த෦େֶڭत, 2014໊ݹେֶ٬һڭत. ܭࢉػࢹ֮ɼಈը૾ॲཧɼύλʔϯೝࣝɾཧղͷݚڀʹैࣄɽ ϩϘΧοϓݚڀ(2005)ɼใॲཧֶձจࢽCVIM༏लจ(2009)ɼใॲཧֶձࢁԼه೦ݚڀ(2009)ɼը૾ηϯγϯάγϯϙδϜ༏लֶज़(2010, 2013, 2014) ɼిࢠใ௨৴ֶձ ใɾγεςϜιαΠΤςΟจ(2013)ଞڭतࢁԼོٛ Takayoshi Yamashita E-mail:[email protected]2002 ಸྑઌՊֶٕज़େֶӃେֶത࢜લظ՝ఔमྃ, 2002 ΦϜϩϯגࣜձࣾೖࣾ, 2009 த෦େֶେֶӃത࢜ޙظ՝ఔमྃ(ࣾձਓυΫλʔ), 2014 த෦େֶߨࢣɼ2017 த෦େֶ।ڭतɼ2021 த෦େֶڭतɽ ਓͷཧղʹ͚ͨಈը૾ॲཧɼύλʔϯೝࣝɾػցֶशͷݚڀʹैࣄɽ ը૾ηϯγϯάγϯϙδϜߴ(2009)ɼిࢠใ௨৴ֶձ ใɾγεςϜιαΠΤςΟจ(2013)ɼిࢠใ௨৴ֶձPRMUݚڀձݚڀྭ(2013)डɽߨࢣฏཌྷ Tsubasa Hirakawa E-mail:[email protected]2013 ౡେֶେֶӃത࢜՝ఔલظऴྃɼ2014 ౡେֶେֶӃത࢜՝ఔޙظೖֶɼ2017 த෦େֶݚڀһ (ʙ2019)ɼ2017 ౡେֶେֶӃത࢜ޙظ՝ఔमྃɽ2019 த෦େֶಛॿڭɼ2021 த෦େֶߨࢣɽ2014 ಠཱߦ๏ਓຊֶज़ৼڵձಛผݚڀһDC1ɽ2014 ESIEE Paris٬һݚڀһ (ʙ2015)ɽίϯϐϡʔλϏδϣϯɼύλʔϯೝࣝɼҩ༻ը૾ॲཧͷݚڀʹैࣄ
hf149
nomu
arthur1
kenichirokimura
piacerex
sakaitakeshi
ikuyamada
ytsuzaki
ayanonanjo
colinfay
visional_engineering_and_design
sbtechnight
lara
addyosmani
destraynor
mthomps
62gerente
mraible
chriscoyier
sugarenia
iamctodd
phodgson
denniskardys
malarkey