論文紹介：Neural Architecture Search with Reinforcement Learning

#;PQI BOE27-F
l/FVSBM"SDIJUFDUVSF4FBSDI
XJUI3FJOGPSDFNFOU-FBSOJOH z
1SPDFFEJOHTPGUI*OUFSOBUJPOBM$POGFSFODFPO
-FBSOJOH3FQSFTFOUBUJPOT *$-3

೥݄೔
؀ڥ৘ใֶ෎
৘ใϝσΟΞ؀ڥֶઐ߈
৘ใϝσΟΞֶίʔε ന઒ݚڀࣨ೥
੪౻ ᠳଡ

View Slide

•#;PQI BOE27-F
l/FVSBM"SDIJUFDUVSF4FBSDIXJUI
3FJOGPSDFNFOU-FBSOJOH z
1SPDFFEJOHTPGUI*OUFSOBUJPOBM
$POGFSFODFPO-FBSOJOH
3FQSFTFOUBUJPOT *$-3

•//ͷߏ଄࠷దԽʹؔ͢ΔݚڀͰɼ
࠷৽ͷ΋ͷͰ͋ͬͨͨΊ
࿦จ঺հɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH
จݙ৘ใɾ͜ͷ࿦จΛબΜͩཧ༝

View Slide

•3FDVSSFOU/FVSBM/FUXPSL 3//
Λ
༻͍ͯɼ//ߏ଄ͷੜ੒ϞσϧΛֶश
•ςετσʔλʹର͢Δਫ਼౓ BDDVSBDZ

ͷظ଴஋࠷େԽΛڧԽֶशʹΑͬͯղ͘
• ޯ഑ͷਪఆͰ͸3&*/'03$&ΞϧΰϦζϜ
Λ࢖༻
•$*'"3ɿఏҊख๏ͰಘΒΕͨϞσϧ
͸ɼଞͷϞσϧΑΓͷਫ਼౓޲্ͱ
ഒͷֶश଎౓ͷվળ
•1FOO5SFFCBOLͰ΋ݕূ
֓ཁ

View Slide

•//ʹΑΔޯ഑๏ʹجͮ͘ߏ଄୳ࡧ
*OUSPEVDUJPO
͋Δ֬཰Qͷ΋ͱͰ
ߏ଄"ΛαϯϓϦϯά
ߏ଄"Λ࣋ͭ//Λ
܇࿅͠ɼਫ਼౓3Λܭࢉ
ਫ਼౓3ʹج͍ͮͯɼ
֬཰Qͷޯ഑Λܭࢉ
ޯ഑Λ༻͍ͯɼ
֬཰QΛߋ৽

View Slide

•//ͷߏ଄͓Αͼ઀ଓؔ܎͸ɼ
Մม௕ͷจࣈྻʹΑͬͯࢦఆՄೳ
• จࣈܥྻͷऔΓѻ͍͕Մೳͳ3//Λ࢖༻
• ߏ଄Λੜ੒͢Δ//ɿDPOUSPMMFS
• ੜ੒͞Εͨߏ଄Λ࣋ͭ//ɿDIJMEOFUXPSL
•ੜ੒͞Εͨߏ଄Λ࣋ͭ//ͷςετਫ਼౓
Λใुͱஔ͘
• ͜ͷͱ͖ɼ3//ͷํࡦޯ഑͕ܭࢉՄೳ
•ࠓճ͸-45.Λ࢖ͬͯɼ3//Λߏ੒
*OUSPEVDUJPO

View Slide

•ϋΠύʔύϥϝʔλ࠷దԽͷख๏͸༷ʑ
•ߏ଄Ҏ֎ͷϋΠύʔύϥϝʔλ
࠷దԽ͸ɼطʹز͔ͭఏҊ͞Ε͍ͯΔ
• 3BOEPNTFBSDI #BZFTJBO0QUJNJ[BUJPO
•ߏ଄ͷ࠷దԽ
• #BZFTJBO0QUJNJ[BUJPO
• /FVSP&WPMVUJPOɿ("ʹΑΔߏ଄୳ࡧ
3FMBUFE8PSL
ҰൠతͰͳ͘
ॊೈੑʹ͚ܽΔ
ॊೈੑ͸͋Δ͕ɼ
େن໛ͳϞσϧͷ৔߹
͕͔͔࣌ؒΔ

View Slide

•લճͷ༧ଌ݁ՌΛࠓճͷೖྗͱ͢Δ
ࣗݾ࠶ؼܕͷDPOUSPMMFS 3//
Λ࢖༻
•ࠓճͷ໰୊Ͱ͸໨తؔ਺͕ඍ෼ෆՄೳ
• ໨తؔ਺ɿςετਫ਼౓
•ಘΒΕͨςετਫ਼౓ͷ஋Λ࢖༻ֶͯ͠श
3FMBUFE8PSL
•/FVSBM"SDIJUFDUVSF4FBSDI
• //ʹجͮ͘ޯ഑๏ʹΑΔߏ଄୳ࡧ
Ξϓϩʔν

View Slide

•චऀΒʹΑͬͯఏҊ͞Ε͍ͯΔͷ͸ͭ
ํࡦޯ഑ʹΑΔ3//ͷֶश๏
3//ͷֶशΛߴ଎Խͤ͞ΔͨΊͷ෼ࢄֶश
ෳࡶͳϞσϧΛੜ੒͢ΔͨΊͷ3//΁ͷ
TLJQDPOOFDUJPOͷಋೖ
-45.ͷΑ͏ͳ࠶ؼతͳߏ଄Λ࣋ͭDFMMͷ
ੜ੒ํ๏
.FUIPET

View Slide

•ࠓճ࢖༻ͨ͠3//ͷߏ଄ ষΑΓ

•ݸͷγʔέϯεͰͭͷ૚Λܗ੒
• 'JMUFS)FJHIU͔Β/VNCFSPG'JMUFST·Ͱ
(FOFSBUF.PEFM%FTDSJQUJPOTXJUIB$POUSPMMFS
3FDVSSFOU/FVSBM/FUXPSL
-45.
DFMM
-45.
DFMM
TPGU
NBY
ग़ྗ
ೖྗ
/૚໨ͷ

/VNCFS
PG'JMUFST
/૚໨ͷ

'JMUFS
)FJHIU

View Slide

•$//ͷߏ଄Λग़ྗ͢Δ3//
• 'JHVSF͸3//Λ࣌ؒํ޲ʹల։ͨ͠ਤ
લճͷग़ྗ͕
࣍ͷೖྗͱͳΔ
ͦΕͧΕͷ஋͸TPGUNBY
ʹΑͬͯ֬཰ͱͯ͠දݱ

View Slide

•૚਺͕نఆͷ஋ʹ౸ୡͨ͠Βɼ3//ʹ
ΑΔߏ଄ͷੜ੒Λऴྃ
• ੜ੒ͨ͠Ϟσϧͷ܇࿅Λߦ͏
• ܇࿅ͨ͠Ϟσϧ͔Βɼςετਫ਼౓Λܭࢉ
•ੜ੒ͨ͠Ϟσϧͷςετਫ਼౓͔Βɼ
ςετਫ਼౓ͷظ଴஋ͷ࠷େԽํ޲΁
3//ͷॏΈύϥϝʔλ Λߋ৽
✓c
͜Ε͕ํࡦޯ഑

View Slide

•3//͕ੜ੒ͨ͠ߏ଄ͷϦετɿ
• ɿ༧ଌ͢΂͖ϋΠύʔύϥϝʔλͷ਺
•ੜ੒ͨ͠ϞσϧʹΑΔςετਫ਼౓ɿ
•ظ଴ςετਫ਼౓ɿ
•ظ଴ςετਫ਼౓ͷޯ഑ɿ
5SBJOJOHXJUI3&*/'03$&
a1:T
R
J(✓c
) = EP(a1:T
;✓c
)
[R]
r✓c
J
(
✓c
)
=
T
X
t=
1
EP
(
a
1:
T
;
✓c
)[
r✓c log
P
(
at | a
(
t
1):1;
✓c
)
R
]
3&*/'03$&ɿ
ߦಈՁ஋ΛใुͰۙࣅ
T

View Slide

•ޯ഑͸ϞϯςΧϧϩ๏ʹΑͬͯਪఆɿ
• ͸ճ͋ͨΓʹੜ੒ͨ͠ߏ଄ͷݸ਺
• 3//ͷύϥϝʔλߋ৽
ˠNݸͷαϯϓϧʢߏ଄ʣΛαϯϓϦϯά
ˠͦΕͧΕͷςετਫ਼౓ ͱ্ࣜʹ
Αͬͯ3//ͷύϥϝʔλߋ৽ˠʜ
m
R1 ⇠ Rm
r✓c
J
(
✓c
)
⇡ 1
m
m
X
k=
1
T
X
t=
1
r✓c log
P
(
at | a
(
t
1):1;
✓c
)
Rk
=
0
B
B
B
B
B
@ 1
m
m
X
k=
1
Rk
1
C
C
C
C
C
A ⇥
0
B
B
B
B
B
@
T
X
t=
1
r✓c log
P
(
at | a
(
t
1):1;
✓c
)
1
C
C
C
C
C
A

View Slide

•ޯ഑ͷਪఆ஋͸ෆภਪఆྔ
•ͨͩ͠ɼ෼ࢄ͕ඇৗʹେ͖͍ͷͰ
ϕʔεϥΠϯΛ࢖༻
•ϕʔεϥΠϯ ʹ͸ɼաڈͷςετਫ਼౓
ͷࢦ਺ҠಈฏۉΛ࢖༻
r✓c
J
(
✓c
)
⇡ 1
m
m
X
k=
1
T
X
t=
1
r✓c log
P
(
at | a
(
t
1):1;
✓c
)(
Rk
b
)
b
bt = ↵R + (1 ↵)bt 1

View Slide

•ฒྻԽͱඇಉظߋ৽ʹΑΔ3//ͷ܇࿅
DIJME͸ͦΕͧΕςετਫ਼౓Λܭࢉ
3FQMJDB͸ύϥϝʔλ
Λ4FSWFSʹฦ͢
4FSWFS͸3FQMJDBʹ
ύϥϝʔλΛ౉͢

View Slide

•(PPH-F/FU΍3FTJEVBM/FUʹ
୅ද͞ΕΔΑ͏ͳ૚Λ·͙ͨ݁߹΋
୳ࡧͷର৅ͱ͢ΔΑ͏ʹ֦ு͢Δ
•૚Λ·͙ͨ݁߹Λ؅ཧ͢Δ"OLFSQPJOU
Λγʔέϯεʹ௥Ճ
•૚Λ·͙ͨ݁߹Λੜ੒͢Δ֬཰͸
TJHNPJEؔ਺ʹΑܾͬͯఆ
*ODSFBTF"SDIJUFDUVSF$PNQMFYJUZXJUI4LJQ
$POOFDUJPOTBOE0UIFS-BZFS5ZQFT
P
(Layer j is an input to layer i)
=
sigmoid(
vT
tanh(
W
prev
⇤ hj + W
curr
⇤ hi
))
IϨΠϠʔʹؔ͢ΔӅΕঢ়ଶ IҎ֎͸܇࿅Մೳͳύϥϝʔλ

View Slide

•૚Λ·͙ͨ݁߹΋֬཰ͱͯ͠දݱ͞ΕΔ
ͨΊ3&*/'03$&ʹΑΔֶश͕Մೳ
•֦ு͞Εͨ3//ͷߏ੒ਤ
*ODSFBTF"SDIJUFDUVSF$PNQMFYJUZXJUI4LJQ
$POOFDUJPOTBOE0UIFS-BZFS5ZQFT

View Slide

•-45.ͷΑ͏ͳ࠶ؼܕͰ಺෦ঢ়ଶΛ࣋ͭ
ηϧͷߏ଄Λ୳ࡧ͢Δ
•ηϧ΁ͷೖྗ͸ͭ
• εςοϓ໨ͷग़ྗɿ
• ηϧͷ಺෦ঢ়ଶɿ
• લͷ૚͔Βͷग़ྗɿ
•ग़ྗ͸ ͱ ͷͭ
(FOFSBUF3FDVSSFOU$FMM"SDIJUFDUVSFT
t 1
xt
ht 1
ct
ht
ct 1

View Slide

•ॲཧΛෳ਺ͷϊʔυʹ෼ׂ
• ֤ϊʔυͰߦ͏ॲཧΛ3//ʹΑͬͯ֬཰త
ʹܾΊ͍ͯ͘

View Slide

•'JHVSFͰߦΘΕΔߏ଄ͷ༧ଌɿ
• 5SFF*OEFYɿ"EEԋࢉΛͨ͠ޙʹ5BOI
Λ͔͚Δ͔
• 5SFF*OEFYɿཁૉ͝ͱʹੵΛͱͬͨޙʹ
3F-6Λ͔͚Δ͔
• $FMM*OKFDUɿ5SFF*OEFYͷग़ྗʹ಺෦
ঢ়ଶΛ଍ͯ͠ɼ3F-6Λ͔͚Δ͔
• 5SFF*OEFYɿ$FMM*OKFDUͷग़ྗͱ5SFF
*OEFYͷग़ྗΛɼཁૉ͝ͱʹੵΛͱͬͨ
ޙʹ4JHNPJEΛ͔͚Δ͔

View Slide

•࣮ݧʹ࢖༻ͨ͠σʔληοτ͸ͭ
• $*'"3
• 1FOO5SFFCBOL
•$*'"3͸4FDUJPOʹج͍ͮͯ
৞ΈࠐΈ//ͷߏ଄୳ࡧ
•1FOO5SFFCBOL͸4FDUJPOʹ
ج͍ͮͯ࠶ؼܕηϧͷߏ଄୳ࡧ
•࠷ྑͷςετΤϥʔΛใࠂ
&YQFSJNFOUTBOE3FTVMUT

View Slide

• 3//ͷߏ੒
• -45.ͭɼϢχοτ਺͸ݸ
• < >ͷҰ༷ཚ਺ͰॳظԽ
• "EBN ֶश཰ɿ
Ͱ࠷దԽ
• ใुɿ࠷ޙͷΤϙοΫͷςετਫ਼౓ͷ࠷େ஋
• ୆ͷ(16Λಉ࣌ʹ࢖༻
• ύϥϝʔλαʔό୆ɼϨϓϦΧ୆
• ϨϓϦΧͭ͋ͨΓੜ੒͢Δߏ଄ͭ
• ݸͷߏ଄Λੜ੒͢Δ͝ͱʹ૚૿Ճ
• ࠷ॳ͸૚͔Βελʔτ
-FBSOJOH$POWPMVUJPOBM"SDIJUFDUVSFTGPS$*'"3
$POUSPMMFS
ͷઃఆ

View Slide

• ΤϙοΫ਺ɿ
• /FTUFSPW .PNFOUVN4(%Ͱ࠷దԽ
• ֶश཰ɼ׳ੑ߲
• ਖ਼ଇԽ߲ɿ8FJHIU%FDBZ ܎਺

• ୳ࡧۭؒɿ
• 'JMUFSIFJHIUXJEUIɿ< >
• /VNCFSPGGJMUFSTɿ< >
• 4USJEF IFJHIUXJEUIɿ< >
• ׆ੑԽؔ਺ɿ3F-6
• #BUDI/PSNBMJ[BUJPOΛ࢖༻
ϛχόον͝ͱʹग़ྗΛਖ਼نԽ
$IJME
ͷઃఆ

View Slide

•܇࿅σʔλɿຕ
•ςετσʔλɿຕ
•લॲཧ
• ന৭Խ
• ը૾ΛϥϯμϜʹ֦େ͓Αͼ൓స
σʔλ֦ு

܇࿅σʔλؒͷ૬ؔΛແ͘͠ɼ
ฏۉ̌ɾ෼ࢄʹਖ਼نԽ

View Slide

•ݸͷߏ଄͔Β࠷ྑͷ΋ͷΛબ୒
• ͦͷޙɼHJSETFBSDIͰֶश཰΍8FJHIU
%FDBZͷ܎਺ͳͲΛௐ੔
5BCMF͔ΒҰ෦Λൈਮ

View Slide

•Wͷ݁Ռʹ͍ͭͯɼैདྷ๏ΑΓ΋ઙ͘ɼ
͔ͭਫ਼౓͕ߴ͍ߏ଄Λ֫ಘ
• "QQFOEJY"ͷ'JHVSFͷߏ଄
•ϑΟϧλͷܗঢ়͸௕ํܗ͕ଟ͘ɼಛʹ
ग़ྗʹ͍ۙ૚Ͱ͸େ͖͍ϑΟϧλ
•ߏ଄ͷมߋʹऑ͍ɼہॴతͳ࠷దੑ
• શͯͷ૚ʹର͠ɼ૚Λ·͍ͨͨ݁߹Λ
Ճ͑ΔͱΤϥʔ͕ʹ૿Ճ
• ૚Λ·͍ͨͨ݁߹Λશͯ࡟আ͢ΔͱΤϥʔ
͕·Ͱ૿Ճ

View Slide

•Wͷ࣮ݧͰ͸ɼWʹՃ͑ͯετϥΠυ
αΠζͷ࠷దԽ΋ߦ͏Α͏ʹมߋ
•୳ࡧۭؒ͸େ͖͘ͳ͕ͬͨɼ
͞΄Ͳਫ਼౓͕ѱ͘ͳΔ͜ͱ͸ͳ͔ͬͨ
•Wͷ࣮ݧͰ͸ɼWʹՃ͑ͯ૚໨ͱ
૚໨ͷޙʹૠೖ͢ΔϓʔϦϯά૚ͷ
࠷దԽ΋ߦ͏Α͏ʹมߋ
• ૚ͷશ݁߹૚Λ࣋ͬͨ૚ͷϒϩοΫ
͝ͱʹߏ଄Λ୳ࡧʢ୳ࡧۭؒͷ੍ݶʣ

View Slide

•ͦͷ݁ՌɼΤϥʔ͸·Ͱվળ
•ಘΒΕͨߏ଄ͷϑΟϧλʔ਺ΛͦΕͧΕ
ݸ૿΍͢ͱΤϥʔ͸·Ͱվળ
•·ͨɼΤϥʔͷ%FOTF/FUϞσϧ
ͱൺֱͯ͠ഒͷ଎͞Ͱֶश͕ਐߦ

View Slide

•σʔληοτ͕খ͍ͨ͞Ίਖ਼ଇԽ͕ඞཁ
•छྨͷ%SPQPVUΛ࢖༻
• &NCFEEJOH%SPQPVUɿ૚ͷؒͷ݁߹ʹ
%SPQPVUΛ͔͚Δ<;BSFNCB FUBM >
• 3FDVSSFOU%SPQPVUɿ࣌ؒํ޲ͷ݁߹ʹ
ม෼ۙࣅͨ͠%SPQPVUΛ͔͚Δ<(BM >
•ೖྗͱग़ྗͷຒΊࠐΈΛڞ༗͢Δख๏
<1SFTT8PMG
>
Λ࢖༻
-FBSOJOH3FDVSSFOU$FMMTGPS1FOO5SFFCBOL

View Slide

•3// $POUSPMMFS
͸݁߹ํ๏ͱ
׆ੑԽؔ਺Λ༧ଌ
• ݁߹ํ๏ɿ"EEɼ&MFN@NVMU
• ׆ੑԽؔ਺ɿJEFOUJUZ UBOI TJHNPJE SFMV
•ೖྗϖΞʹର͢Δॲཧ͸छྨ༧ଌ
͜ͷϨϕϧͷϊʔυ͕ͭଘࡏ

View Slide

• $*'"3ͷઃఆͱ΄΅ಉ༷
• ҟͳΔ෦෼ɿ
• "EBNͷֶश཰͕ʹมߋ
• ෼ࢄॲཧͷߏ੒ɿϨϓϦΧ͕ɼ
ϨϓϦΧ͋ͨΓͷαϯϓϧ਺͕ʹมߋ
• ݸͷߏ଄Λಉ࣌ʹ$16Ͱ܇࿅
• ޯ഑͕ݸ஝ੵ͞ΕΔͱύϥϝʔλαʔόͷΈ
஋ͷߋ৽Λߦ͏
• ใु͸ʮWBMJEBUJPOQFSQMFYJUZͷ৐ʯ
ʹઃఆ
$POUSPMMFS
ͷઃఆ

View Slide

•ӅΕ૚ͷ਺͸૚
•ֶशՄೳͳύϥϝʔλ਺͕
ʮ-45.૚ɼϢχοτ਺ݸʯͷ
৔߹ͱҰக͢ΔΑ͏ʹϢχοτ਺Λௐ੔
•࠷ྑͷߏ଄Λݟ͚ͭͨ͋ͱɼֶश཰΍
%SPQPVU཰΍ॏΈͷॳظԽͳͲΛ
HSJETFBSDIͰ࠷దԽ
$IJME
ͷઃఆ

View Slide

•ଞͷϞσϧͱൺֱͯ͠ɼςετ
QFSQMFYJUZ͕࠷΋খ͍͞
•͞Βʹ;JMMZΒͷϞσϧΑΓ΋ഒૣ͘
ֶश͕Մೳ
5BCMF͔ΒҰ෦Λൈਮ

View Slide

•୳ࡧʹΑͬͯಘΒΕͨ$FMM͸-45.ͱ
ྨࣅͷߏ଄Λ͍࣋ͬͯͨ
• ಛʹ࠷ॳʹ"EEΛબ୒͍ͯ͠Δ෦෼
-45. ಘΒΕͨߏ଄

View Slide

•1FOO5SFFCBOLͷ࣮ݧͰಘΒΕͨߏ଄
ΛଞͷλεΫͰ࢖༻ʢసҠֶशʣ
•͸͡Ίʹ1FOO5SFFCBOLͰDIBSBDUFS
NPEFMJOHλεΫͰ࣮ݧ
• -45.΍ଞͷख๏ΑΓ΋ྑ͍݁Ռ

View Slide

•ӳޠˠυΠπޠͷ຋༁λεΫʹ͓͍ͯ΋
-45.Λ࢖༻ͨ͠৔߹ΑΓ΋ྑ͍݁Ռ
• #-&6஋͕޲্
•࣍ʹɼ୳ࡧۭؒͷ֦େʹର͢Δϩόετ
ੑΛ֬ೝ
• ݁߹ํ๏ʹNBYɼ׆ੑԽؔ਺ʹTJOΛ௥Ճ
• ݁Ռͱͯ͠ɼಉ౳ͷਫ਼౓Λ࣋ͭߏ଄Λ֫ಘ

View Slide

୳ࡧۭؒΛ֦ுͯ͠
ಘΒΕͨߏ଄

View Slide

•3BOEPNTFBSDIʹΑΔߏ଄୳ࡧͱͷ
ൺֱ
• 3BOEPNTFBSDI͸୯७͕ͩڧྗͳख๏
•3BOEPNTFBSDIʹΑΔ࠷ྑͷϞσϧ
͔ΒΈͯఏҊख๏ͷQFSQMFYJUZͷݮগ
ྔΛάϥϑԽ
•ํࡦޯ഑Λ༻͍ͨఏҊख๏ͷ΄͏͕
3BOEPNTFBSDIΑΓ΋QFSQMFYJUZ͕
վળ

View Slide


View Slide

•3//ʹΑͬͯ//ͷߏ଄୳ࡧΛߦ͏ख๏
ΛఏҊ
• 3//Λ࢖༻͢Δ͜ͱʹΑͬͯɼΑΓॊೈʹ
ߏ଄୳ࡧΛߦ͏͜ͱ͕Մೳ
•ఏҊख๏ʹΑͬͯੜ੒͞Εͨߏ଄͸ྑ޷
ͳੑೳΛ࣋ͭ
•ࠓճͷख๏͸ߏ଄୳ࡧʹ͓͚Δ৽ͨͳ
ݚڀͷํ޲Λ͍ࣔͯ͠Δ
$PODMVTJPO

View Slide

ࢀߟจݙ
• 8PKDJFDI;BSFNCB *MZB4VUTLFWFS BOE0SJPM7JOZBMTl3FDVSSFOU
OFVSBMOFUXPSLSFHVMBSJ[BUJPOz BS9JWQSFQSJOUBS9JW
• :BSJO (BMl"UIFPSFUJDBMMZHSPVOEFEBQQMJDBUJPOPGESPQPVUJO
SFDVSSFOUOFVSBMOFUXPSLTzBS9JW QSFQSJOUBS9JW
• 0GJS 1SFTTBOE-JPS8PMGl6TJOHUIFPVUQVUFNCFEEJOHUPJNQSPWF
MBOHVBHFNPEFMTzBS9JW QSFQSJOUBS9JW

View Slide

論文紹介：Neural Architecture Search with Reinforcement Learning

論文紹介：Neural Architecture Search with Reinforcement Learning

S.Shota

More Decks by S.Shota

Other Decks in Technology

Featured

Transcript