3D CNNによる動画像の時空間特徴表現 / Spatiotemporal Feature Representation Using 3D CNN

ݪ݈ᠳ %$//ʹΑΔಈը૾ͷۭ࣌ؒಛ௃දݱ ࢈ۀٕज़૯߹ݚڀॴίϯϐϡʔλϏδϣϯݚڀάϧʔϓ

ࣗݾ঺հ ˔ ݪ݈ᠳ ˔ ത࢜ ʢ৘ใՊֶʣ ˔ ࢈ۀٕज़૯߹ݚڀॴಛผݚڀһʢϙευΫʣ
˔ ʙ໊ݹ԰େֶ৘ใՊֶݚڀՊത࢜՝ఔ ˔ ʙ࢈૯ݚϙευΫ ݸਓ8FCIUUQTLFOTIPIBSBHJUIVCJP (JU)VCIUUQTHJUIVCDPNLFOTIPIBSB

ߨԋ಺༰ ˔ ಈըʹର͢Δۭ࣌ؒ%$//ͷൃలͷྲྀΕ ˔ $713Ͱൃද༧ఆͷ࿦จ঺հ ˔ %$//ͷߋͳΔൃలΛࣔࠦ ˔
࠷ۙͷؔ࿈ݚڀ঺հ ˔ ࡢ೥຤͋ͨΓ͔Βͷ࿦จΛ঺հ

ಈըೝࣝʹ͓͚Δ%$// ˔ ۭؒ%ʴ࣌ؒ%ͷ  ࣍ݩۭؒͰ৞ΈࠐΈ ˔ ಈըͷೖྗ͔Β  ௚઀ۭ࣌ؒಛ௃Λநग़Մೳ ˔
ύϥϝʔλ਺͕ଟ͍ͨΊ  ֶश͕େม

ॳظͷ%$//ͷ୅දݚڀ େن໛ͳ4QPSUT.σʔληοτʹΑΔֶशͰߴਫ਼౓ͳೝࣝΛ࣮ݱ $%ͱ͍͏7((ϕʔεͷ%$// %5SBOFUBM l-FBSOJOH4QBUJPUFNQPSBM'FBUVSFTXJUI%$POWPMVUJPOBM/FUXPSLTz *$$7 %DPOW
YY

$%ʹΑΔਓ෺ߦಈೝࣝ ˔ $%Ͱ͸ೖྗαΠζ͸  ϑϨʔϜºըૉºըૉ ˔ ϑϨʔϜ୯ҐͰಈըΛ  ෼ׂͯ͠%$//ʹೖྗ ˔
ग़ྗείΞͷฏۉ͕  ࠷େͱͳΔΫϥεΛ  ಈըͷϥϕϧͱͯ͠ग़ྗ ೖྗಈը t $% είΞ ಈըϥϕϧ ฏۉ

ೖྗͷϑϨʔϜ਺ʹ͍ͭͯ ˔ ϑϨʔϜ͔ΒϑϨʔϜ·Ͱେ͖͍ͯ͘͘͠ͱ  جຊతʹ͸େ͖͍΄Ͳߴਫ਼౓ 0QUJDBM'MPXͷར༻ʹ͍ͭͯ ˔ %$POWPMVUJPOͯ͠΋0QUJDBM'MPXͱͷ  5XPTUSFBNͷํ͕ߴਫ਼౓ $%ʹ͍ͭͯͷৄࡉͳݕ౼
(7BSPMFUBM l-POHUFSN5FNQPSBM$POWPMVUJPOTGPS"DUJPO3FDPHOJUJPOz 51".* ʢॳग़͸BS9JWͰʣ

͜ͷลΓ·Ͱͷ%$//ͷੑೳ ˔ %$//ͷਫ਼౓͸%$//ʹ ൺ΂ͯ༏ҐͰ͸ͳ͍ ˔ ΑΓෳࡶͳॲཧΛ͍ͯ͠Δ %$//ϕʔεͷख๏͸  ͞Βʹߴ͍ਫ਼౓
˔ ಉఔ౓ͷੑೳͳΒ%$//ͷ ํ͕ߴ଎͔ͭ௿ϝϞϦ ೝࣝ཰ 7FSZEFFQ  UXPTUSFBN%$// %-5$ 3(# %-5$ 3(# 'MPX

ύϥϝʔλ਺͕๲େͰֶश͕ࠔ೉ ˔ ৞ΈࠐΈ͕Y͔ΒYYʹͳ͍ͬͯΔ෼  ύϥϝʔλ਺͕%$//ʹൺ΂ͯ๲େ ˔ ଟ਺ͷύϥϝʔλΛ࣋ͭϞσϧͷֶशʹ͸  ΑΓେن໛ͳσʔληοτ͕ඞཁ ˔ ಈըσʔληοτ͸ը૾ͱൺֱ͢Δͱখن໛ ˔
%$//ͩͱ*NBHF/FUͰͷ1SFUSBJOJOH͕͠ʹ͍͘ %$//ͷ໰୊఺

,JOFUJDTσʔληοτ ˔ ೥ʹ%FFQ.JOEʹΑΓެ։ ˔ ߦಈΫϥεɼສಈըͷσʔληοτ ˔ ͦΕҎલ͸͍͍ͤͥ਺ສಈը ˔ ,JOFUJDTͰͷֶशʹΑΓ  %$//ͷੑೳ͕େ෯ʹ޲্͠ɼ࠷ߴਫ਼౓Λୡ੒
େن໛ಈըσʔληοτͷొ৔ 8,BZFUBM l5IF,JOFUJDT)VNBO"DUJPO7JEFP%BUBTFUz BS9JW

࠷ߴਫ਼౓Λୡ੒ͨ͠%$//ʛ*% ˔ *ODFQUJPOϕʔεͷߏ଄ ˔ (PPH-F/FU ˔ ,JOFUJDTͰֶश ˔
*NBHF/FUͰ1SFUSBJOJOH  ͢ΔͨΊʹ%ͷॏΈΛ  %ʹίϐʔ *OGMBUJPO ˔ ۚଠ࿠᷒తʹίϐʔ +$BSSFJSBFUBM l2VP7BEJT "DUJPO3FDPHOJUJPO "/FX.PEFMBOEUIF,JOFUJDT%BUBTFUz $713

͜͜·Ͱͷ%$// ˔ ,JOFUJDTͰֶश͢Δͱߴਫ਼౓ ˔ େن໛σʔλͰͷֶश͕ॏཁ ˔ ը૾ೝࣝͰ΋*NBHF/FU͕ॏཁͳͷͱಉ༷ ˔
ͱ͸͍͑Ϟσϧ͸*ODFQUJPOW·Ͱ͔͠ະݕ౼ ˔ ૚ͷωοτϫʔΫߏ଄ ˔ ը૾ೝࣝͰ͸3FT/FUͳͲߋʹਂ૚Ϟσϧ͕༗ޮ

ը૾λεΫͰͷਐల *NBHF/FU %$// b b b b ૚਺
૚ ෺ମݕग़ จষੜ੒ ສը૾ɼ Ϋϥε ը૾ೝࣝͰ੒ޭ ਂ͍΄Ͳߴਫ਼౓ ଞ΁ͷసҠ΋੒ޭ ෼໺Λେ͖͘ਐల

*NBHF/FU %$// b b b b ૚਺ ૚ ෺ମݕग़ จষੜ੒
ສը૾ɼ Ϋϥε ಈըλεΫͰͷਐల ߦಈݕग़ Ҡಈਪఆ ,JOFUJDT %$// ສಈըɼ Ϋϥε b b b b ૚਺ ૚ b b ૚਺ b b b ૚ ઙ͍ϞσϧͷΈݕ౼ ఆ൪σʔληοτ͸ະ֬ఆ

ಈըσʔληοτͷن໛͸े෼͔෼ੳ ˔ ֶशՄೳͳϞσϧͷ૚਺͔Βن໛Λ෼ੳ ˔ 3FT/FUΛϕʔεʹ༷ʑͳ૚਺ͷ  %$//ϞσϧΛֶश͢Δେن໛࣮ݧ ˔ ݱঢ়ͷಈըσʔληοτͷن໛ײΛࣔ͢͜ͱͰ  ࠓޙͷಈըೝࣝ %$//
ͷൃలՄೳੑΛࣔࠦ ݚڀ໨త ,)BSBFUBM l$BO4QBUJPUFNQPSBM%$//T3FUSBDFUIF)JTUPSZPG%$//TBOE*NBHF/FU z $713

࣮ݧͰݕূͨ͠Ϟσϧ conv, 33, F BN ReLU conv, 33,
F BN ReLU ResNet (basic) conv, 13, F BN ReLU conv, 33, F BN ReLU ReLU conv, 13, 4F BN ResNet (bottleneck) conv, 13, F BN ReLU conv, 33, F BN ReLU ReLU conv, 13, 4F BN ResNet (pre-act) conv, 13, F BN ReLU conv, 33, F, group=32 BN ReLU ReLU BN conv, 13, 2F ResNeXt conv, 13, 128 BN ReLU conv, 33, 32 BN ReLU DenseNet Concat

ֶश ˔ 4(%XJUINPNFOUVNͰֶश ˔ YYYͷೖྗαϯϓϧΛੜ੒ ˔ ϑϨʔϜΛϥϯμϜ੾Γग़͠ ˔
࢛۱தԝͷ୒͔ΒۭؒҐஔΛϥϯμϜબ୒ ˔ ۭؒ੾Γग़͠εέʔϧΛϥϯμϜબ୒ ˔ Ͱ)PSJ[POUBM'MJQ ˔ "DUJWJUZ/FUͷฏۉͰ.FBO4VCUSBDUJPO

ֶशʛύϥϝʔλ ˔ *OJUJBM-FBSOJOH3BUF '5࣌ɿ ˔ 7BMJEBUJPO-PTT͕ΤϙοΫߋ৽͞Εͳ͚Ε͹ ˔
.PNFOUVN ˔ 8FJHIUEFDBZF '5࣌ɿF

Ϛγϯ୆࢈૯ݚεύίϯ ""*$ ར༻ ˔ /7*%*"5*"/9º ˔ /7*%*"5*5"/9º ˔ /7*%*"5*5"/9º ˔
εύίϯϊʔυɿ5FTMB1º ˔ ͜ΕΛϊʔυ͘Β͍ฒྻͰར༻ ֶश؀ڥ

શମͰϲ݄൒ఔ౓ ˔ ,JOFUJDTͰϞσϧΛ  εΫϥονͰֶश͢Δͷʹdिؒ ˔ ߹ܭϞσϧֶश ˔ ͦͷଞผσʔληοτͰͷֶश΋࣮ࢪ ˔ 6$'ͳͲͰͷ'JOF5VOJOH͸೔͘Β͍Ͱ׬ྃ
࣮ݧʹཁͨ࣌ؒ͠

σʔληοτ σʔληοτ ެ։೥ Ϋϥε਺ Πϯελϯε਺ ).%#
6$' "DUJWJUZ/FU ,JOFUJDT

֤σʔληοτͰͷֶशͷ෼ੳ ˔ σʔληοτͦΕͧΕͰ3FT/FUΛֶश ˔ 6$' ).%# "DUJWJUZ/FUͰ͸  %$//ͷֶश͸ࠔ೉ͱ͍͏ैདྷݚڀͷใࠂΛ࠶ݱ ࣮ݧ

࣮ݧʛখن໛σʔληοτͰ͸աֶश 3FT/FUͰ͋ͬͯ΋,JOFUJDTҎ֎Ͱ͸େ͖͘աֶश ֶशσʔλ͚ͩ  ͏·͍͘͘աֶश

ΑΓਂ͍Ϟσϧͷ෼ੳ ˔ ,JOFUJDTͰ3FT/FU͔Β·ͰΛ  ֶशͯ݁͠ՌΛൺֱ ˔ ͦͷଞϞσϧͷ݁Ռͱ΋ൺֱ ࣮ݧ

࣮ݧʛ,JOFUJDT͸े෼ʹେن໛ Ϟσϧ૚਺
ೝ ࣝ ਫ਼ ౓ ಈըʛ%$// Ϟσϧ૚਺ ೝ ࣝ ਫ਼ ౓ ը૾ʛ%$// ,JOFUJDTར༻Ͱ*NBHF/FUͱಉ౳ͷ૚਺·ͰֶशՄೳ

࣮ݧʛ݁Ռ ˔ 1SFBDUJWBUJPO͸  ਫ਼౓্͕Βͣ ˔ 3FT/F9U͕  ࠷ߴਫ਼౓Λୡ੒ ˔
%FOTF/FU΋ਫ਼౓௿Ί Ϟσϧ 5PQ 5PQ 3FT/FU 3FT/FU QSFBDU 8JEF3FT/FU 3FT/F9U %FOTF/FU

'JOFUVOJOHͷ෼ੳ ˔ ,JOFUJDTͰֶशͨ͠ϞσϧΛ  6$' ).%#Ͱ'JOFUVOJOH ˔ 'JOFUVOJOHͰ͸DPOW@ͱGDͷΈֶश ࣮ݧ

࣮ݧʛ݁Ռ ˔ %$//͸୯७ͳϞσϧͰ  ߴਫ਼౓Λ࣮ݱՄೳ ˔ 5XPTUSFBN͸  %$//Ͱ΋ਫ਼౓Λ޲্ ˔
*%͕τοϓͷੑೳ ˔ ೖྗαΠζ͸  ϑϨʔϜºըૉºըૉ Ϟσϧ 6$' ).%# 3FT/F9U 3FT/F9U  G $% 5XPTUSFBN*% 54/  %ͷ405"

1Z5PSDIίʔυͱֶशࡁΈϞσϧΛެ։த ֶशɾςετΛ؆୯ʹ࣮ߦͰ͖Δଞɼ  ಈըೝ͚ࣝͩࢼ͍ͨ͠ਓ༻ͷίʔυ΋༻ҙ (JU)VCͰެ։ʢελʔҎ্֫ಘʣ IUUQTHJUIVCDPNLFOTIPIBSB%3FT/FUT1Z5PSDI

GT: shot put  Pr: shot put GT: zumba 
Pr: zumba GT: egg hunting  Pr: egg hunting GT: arm wrestling  Pr: arm wrestling GT: ironing  Pr: ironing GT: high kick  Pr: high kick GT: digging  Pr: digging GT: use computer  Pr: use computer GT: washing hair  Pr: washing hair GT: abseiling  Pr: bungee jumping

%$//ʹΑΔಈըೝࣝͷؔ࿈ݚڀ ࠷ۙͷؔ࿈ݚڀ঺հ

σʔληοτͷߋͳΔେن໛Խ ˔ ,JOFUJDT "DUJWJUZ/FU$IBMMFOHF5BTL" ˔ ,JOFUJDTΛΫϥεɼ ສಈըʹ֦େ
˔ .PNFOUTJO5JNF "DUJWJUZ/FU$IBMMFOHF5BTL$ ˔ Ϋϥεɼ ສಈըʢ֤ඵʣ ˔ 4-"$ ˔ Ϋϥεɼ ສಈըɼ ສΞϊςʔγϣϯ );IBPFUBM l4-"$"4QBSTFMZ-BCFMFE%BUBTFUGPS"DUJPO$MBTTJGJDBUJPOBOE-PDBMJ[BUJPOz BS9JW ..POGPSUFUBM l.PNFOUTJO5JNF%BUBTFUPOFNJMMJPOWJEFPTGPSFWFOUVOEFSTUBOEJOHz BS9JW

%ͱ%ͷ૊Έ߹Θͤ ۭؒํ޲ͷ%$POWͱ࣌ؒํ޲ͷ%$POWΛ  ૊Έ߹Θٖͤͯࣅతʹ%$POW ٖࣅతʹ%$POWPMVUJPOΛ࠶ݱ ;2JVFUBM l-FBSOJOH4QBUJP5FNQPSBM3FQSFTFOUBUJPOXJUI1TFVEP%3FTJEVBM/FUXPSLTz *$$7
ۭؒ% ࣌ؒ%

%ͱ%ͷ૊Έ߹Θͤ̎ શ෦%ΑΓ΋% %૊Έ߹Θͤͷํ͕ྑ͍ਫ਼౓Ͱɼ  Ұ൪ྑ͍ͷ͸ %ʢલεϥΠυͷΑ͏ͳٖࣅత%$POWʣ %ͱ%ͷ৭ʑͳ૊Έ߹ΘͤΛݕ౼
%5SBOFUBM l"$MPTFS-PPLBU4QBUJPUFNQPSBM$POWPMVUJPOTGPS"DUJPO3FDPHOJUJPOz $713

0QUJDBM'MPXΛೖΕΔͱͳͥྑ͘ͳΔ͔෼ੳ ˔ 0QUJDBM'MPX͸ΞϐΞϥϯεͷࡉ͔͍มԽʹ  ෆมͳͷ͕ྑ͍ͱ͍͏݁࿦ ˔ ಈ͖৘ใ͕औΕ͍ͯΔͱ͍͏ͷ͸͋·ΓॏཁͰ͸ͳ͍ 5XPTUSFBNͷղੳ -4FWJMMB-BSBFUBM
l0OUIF*OUFHSBUJPOPG0QUJDBM'MPXBOE"DUJPO3FDPHOJUJPOz BS9JW

ࠓޙͷల๬ˍؾʹͳ͍ͬͯΔ఺ͳͲ ˔ %Ͱͷֶश͸ࠓޙ΋Ճ଎ ˔ %$//Ҏ্ʹ%$//Λ࢖͏࿦จ͕Ұؾʹ૿͍͑ͯΔ ˔ 0QUJDBM'MPX͸%$//ʹຊ౰ʹඞཁ͔Ͳ͏͔ ˔
಺෦Ͱ'MPXʹ૬౰͢Δಛ௃நग़ΛࣗಈͰֶशͯ͠΋͍͍͸ͣ ˔ -PDBMJ[BUJPOख๏͸·ͩ·ͩະཱ֬ ˔ ೝࣝਫ਼౓͸͋Δఔ౓্͕͖͍ͬͯͯΔ΋ͷͷ  ݕग़ͳͲͷઃఆͩͱҰؾʹਫ਼౓͸མͪͯ͠·͏

ࢀߟจݙ %5SBOFUBM l-FBSOJOH4QBUJPUFNQPSBM'FBUVSFTXJUI%$POWPMVUJPOBM/FUXPSLTz *$$7 -4FWJMMB-BSBFUBM l0OUIF*OUFHSBUJPOPG0QUJDBM'MPXBOE"DUJPO3FDPHOJUJPOz BS9JW
%5SBOFUBM l"$MPTFS-PPLBU4QBUJPUFNQPSBM$POWPMVUJPOTGPS"DUJPO3FDPHOJUJPOz $713 ;2JVFUBM l-FBSOJOH4QBUJP5FNQPSBM3FQSFTFOUBUJPOXJUI1TFVEP%3FTJEVBM/FUXPSLTz *$$7 );IBPFUBM l4-"$"4QBSTFMZ-BCFMFE%BUBTFUGPS"DUJPO$MBTTJGJDBUJPOBOE-PDBMJ[BUJPOz BS9JW ,)BSBFUBM l$BO4QBUJPUFNQPSBM%$//T3FUSBDFUIF)JTUPSZPG%$//TBOE*NBHF/FU z $713 +$BSSFJSBFUBM l2VP7BEJT "DUJPO3FDPHOJUJPO "/FX.PEFMBOEUIF,JOFUJDT%BUBTFUz $713 8,BZFUBM l5IF,JOFUJDT)VNBO"DUJPO7JEFP%BUBTFUz BS9JW (7BSPMFUBM l-POHUFSN5FNQPSBM$POWPMVUJPOTGPS"DUJPO3FDPHOJUJPOz 51".* ..POGPSUFUBM l.PNFOUTJO5JNF%BUBTFUPOFNJMMJPOWJEFPTGPSFWFOUVOEFSTUBOEJOHz BS9JW

3D CNNによる動画像の時空間特徴表現 / Spatiotemporal Feature R...

3D CNNによる動画像の時空間特徴表現 / Spatiotemporal Feature Representation Using 3D CNN

More Decks by Kensho Hara

Other Decks in Technology

Featured

Transcript