Share
Embed
論文紹介:Neural Architecture Search with Reinforcement Learning
2017年06月16日の論文紹介で使用したスライドです.
題材にした論文:
・B. Zoph and Q. V Le, “Neural Architecture Search with Reinforcement Learning,” Proceedings of 5th International Conference on Learning Representations (ICLR'17), 2017.
・https://openreview.net/forum?id=r1Ue8Hcxg¬eId=r1Ue8Hcxg
S.Shota
June 16, 2017
Transcript
41 slides
-
#;PQI BOE27-F l/FVSBM"SDIJUFDUVSF4FBSDI XJUI3FJOGPSDFNFOU-FBSOJOH z 1SPDFFEJOHTPGUI*OUFSOBUJPOBM$POGFSFODFPO -FBSOJOH3FQSFTFOUBUJPOT *$-3
݄ ڥใֶ ใϝσΟΞڥֶઐ߈ ใϝσΟΞֶίʔε നݚڀࣨ ੪౻ ᠳଡ -
•#;PQI BOE27-F l/FVSBM"SDIJUFDUVSF4FBSDIXJUI 3FJOGPSDFNFOU-FBSOJOH z 1SPDFFEJOHTPGUI*OUFSOBUJPOBM $POGFSFODFPO-FBSOJOH 3FQSFTFOUBUJPOT
*$-3 •//ͷߏ࠷దԽʹؔ͢ΔݚڀͰɼ ࠷৽ͷͷͰ͋ͬͨͨΊ จհɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH จݙใɾ͜ͷจΛબΜͩཧ༝ -
•3FDVSSFOU/FVSBM/FUXPSL 3// Λ ༻͍ͯɼ//ߏͷੜϞσϧΛֶश •ςετσʔλʹର͢Δਫ਼ BDDVSBDZ ͷظ࠷େԽΛڧԽֶशʹΑͬͯղ͘ • ޯͷਪఆͰ3&*/'03$&ΞϧΰϦζϜ
Λ༻ •$*'"3ɿఏҊख๏ͰಘΒΕͨϞσϧ ɼଞͷϞσϧΑΓͷਫ਼্ͱ ഒͷֶशͷվળ •1FOO5SFFCBOLͰݕূ จհɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH ֓ཁ -
•//ͷߏ͓Αͼଓؔɼ ՄมͷจࣈྻʹΑͬͯࢦఆՄೳ • จࣈܥྻͷऔΓѻ͍͕Մೳͳ3//Λ༻ • ߏΛੜ͢Δ//ɿDPOUSPMMFS • ੜ͞ΕͨߏΛ࣋ͭ//ɿDIJMEOFUXPSL •ੜ͞ΕͨߏΛ࣋ͭ//ͷςετਫ਼
Λใुͱஔ͘ • ͜ͷͱ͖ɼ3//ͷํࡦޯ͕ܭࢉՄೳ •ࠓճ-45.Λͬͯɼ3//Λߏ จհɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH *OUSPEVDUJPO -
•ϋΠύʔύϥϝʔλ࠷దԽͷख๏༷ʑ •ߏҎ֎ͷϋΠύʔύϥϝʔλ ࠷దԽɼطʹز͔ͭఏҊ͞Ε͍ͯΔ • 3BOEPNTFBSDI #BZFTJBO0QUJNJ[BUJPO •ߏͷ࠷దԽ • #BZFTJBO0QUJNJ[BUJPO
• /FVSP&WPMVUJPOɿ("ʹΑΔߏ୳ࡧ จհɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH 3FMBUFE8PSL ҰൠతͰͳ͘ ॊೈੑʹ͚ܽΔ ॊೈੑ͋Δ͕ɼ େنͳϞσϧͷ߹ ͕͔͔࣌ؒΔ -
•લճͷ༧ଌ݁ՌΛࠓճͷೖྗͱ͢Δ ࣗݾ࠶ؼܕͷDPOUSPMMFS 3// Λ༻ •ࠓճͷͰత͕ؔඍෆՄೳ • తؔɿςετਫ਼ •ಘΒΕͨςετਫ਼ͷΛ༻ֶͯ͠श
จհɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH 3FMBUFE8PSL •/FVSBM"SDIJUFDUVSF4FBSDI • //ʹجͮ͘ޯ๏ʹΑΔߏ୳ࡧ Ξϓϩʔν -
•චऀΒʹΑͬͯఏҊ͞Ε͍ͯΔͷͭ ํࡦޯʹΑΔ3//ͷֶश๏ 3//ͷֶशΛߴԽͤ͞ΔͨΊͷࢄֶश ෳࡶͳϞσϧΛੜ͢ΔͨΊͷ3//ͷ TLJQDPOOFDUJPOͷಋೖ
-45.ͷΑ͏ͳ࠶ؼతͳߏΛ࣋ͭDFMMͷ ੜํ๏ จհɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH .FUIPET -
•ࠓճ༻ͨ͠3//ͷߏ ষΑΓ •ݸͷγʔέϯεͰͭͷΛܗ • 'JMUFS)FJHIU͔Β/VNCFSPG'JMUFST·Ͱ จհɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH (FOFSBUF.PEFM%FTDSJQUJPOTXJUIB$POUSPMMFS 3FDVSSFOU/FVSBM/FUXPSL
-45. DFMM -45. DFMM TPGU NBY ग़ྗ ೖྗ /ͷ /VNCFS PG'JMUFST /ͷ 'JMUFS )FJHIU -
•͕نఆͷʹ౸ୡͨ͠Βɼ3//ʹ ΑΔߏͷੜΛऴྃ • ੜͨ͠Ϟσϧͷ܇࿅Λߦ͏ • ܇࿅ͨ͠Ϟσϧ͔Βɼςετਫ਼Λܭࢉ •ੜͨ͠Ϟσϧͷςετਫ਼͔Βɼ ςετਫ਼ͷظͷ࠷େԽํ 3//ͷॏΈύϥϝʔλ
Λߋ৽ จհɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH (FOFSBUF.PEFM%FTDSJQUJPOTXJUIB$POUSPMMFS 3FDVSSFOU/FVSBM/FUXPSL ✓c ͜Ε͕ํࡦޯ -
•3//͕ੜͨ͠ߏͷϦετɿ • ɿ༧ଌ͖͢ϋΠύʔύϥϝʔλͷ •ੜͨ͠ϞσϧʹΑΔςετਫ਼ɿ •ظςετਫ਼ɿ •ظςετਫ਼ͷޯɿ จհɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH 5SBJOJOHXJUI3&*/'03$&
a1:T R J(✓c ) = EP(a1:T ;✓c ) [R] r✓c J ( ✓c ) = T X t= 1 EP ( a 1: T ; ✓c )[ r✓c log P ( at | a ( t 1):1; ✓c ) R ] 3&*/'03$&ɿ ߦಈՁΛใुͰۙࣅ T -
•ޯϞϯςΧϧϩ๏ʹΑͬͯਪఆɿ • ճ͋ͨΓʹੜͨ͠ߏͷݸ • 3//ͷύϥϝʔλߋ৽ ˠNݸͷαϯϓϧʢߏʣΛαϯϓϦϯά ˠͦΕͧΕͷςετਫ਼ ͱ্ࣜʹ Αͬͯ3//ͷύϥϝʔλߋ৽ˠʜ
จհɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH 5SBJOJOHXJUI3&*/'03$& m R1 ⇠ Rm r✓c J ( ✓c ) ⇡ 1 m m X k= 1 T X t= 1 r✓c log P ( at | a ( t 1):1; ✓c ) Rk = 0 B B B B B @ 1 m m X k= 1 Rk 1 C C C C C A ⇥ 0 B B B B B @ T X t= 1 r✓c log P ( at | a ( t 1):1; ✓c ) 1 C C C C C A -
•ޯͷਪఆෆภਪఆྔ •ͨͩ͠ɼࢄ͕ඇৗʹେ͖͍ͷͰ ϕʔεϥΠϯΛ༻ •ϕʔεϥΠϯ ʹɼաڈͷςετਫ਼ ͷࢦҠಈฏۉΛ༻ จհɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH 5SBJOJOHXJUI3&*/'03$&
r✓c J ( ✓c ) ⇡ 1 m m X k= 1 T X t= 1 r✓c log P ( at | a ( t 1):1; ✓c )( Rk b ) b bt = ↵R + (1 ↵)bt 1 -
•(PPH-F/FU3FTJEVBM/FUʹ ද͞ΕΔΑ͏ͳΛ·͙ͨ݁߹ ୳ࡧͷରͱ͢ΔΑ͏ʹ֦ு͢Δ •Λ·͙ͨ݁߹Λཧ͢Δ"OLFSQPJOU ΛγʔέϯεʹՃ •Λ·͙ͨ݁߹Λੜ͢Δ֬ TJHNPJEؔʹΑܾͬͯఆ จհɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH
*ODSFBTF"SDIJUFDUVSF$PNQMFYJUZXJUI4LJQ $POOFDUJPOTBOE0UIFS-BZFS5ZQFT P (Layer j is an input to layer i) = sigmoid( vT tanh( W prev ⇤ hj + W curr ⇤ hi )) IϨΠϠʔʹؔ͢ΔӅΕঢ়ଶ IҎ֎܇࿅Մೳͳύϥϝʔλ -
•-45.ͷΑ͏ͳ࠶ؼܕͰ෦ঢ়ଶΛ࣋ͭ ηϧͷߏΛ୳ࡧ͢Δ •ηϧͷೖྗͭ • εςοϓͷग़ྗɿ • ηϧͷ෦ঢ়ଶɿ • લͷ͔Βͷग़ྗɿ
•ग़ྗ ͱ ͷͭ จհɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH (FOFSBUF3FDVSSFOU$FMM"SDIJUFDUVSFT t 1 xt ht 1 ct ht ct 1 -
•'JHVSFͰߦΘΕΔߏͷ༧ଌɿ • 5SFF*OEFYɿ"EEԋࢉΛͨ͠ޙʹ5BOI Λ͔͚Δ͔ • 5SFF*OEFYɿཁૉ͝ͱʹੵΛͱͬͨޙʹ 3F-6Λ͔͚Δ͔ • $FMM*OKFDUɿ5SFF*OEFYͷग़ྗʹ෦
ঢ়ଶΛͯ͠ɼ3F-6Λ͔͚Δ͔ • 5SFF*OEFYɿ$FMM*OKFDUͷग़ྗͱ5SFF *OEFYͷग़ྗΛɼཁૉ͝ͱʹੵΛͱͬͨ ޙʹ4JHNPJEΛ͔͚Δ͔ จհɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH (FOFSBUF3FDVSSFOU$FMM"SDIJUFDUVSFT -
•࣮ݧʹ༻ͨ͠σʔληοτͭ • $*'"3 • 1FOO5SFFCBOL •$*'"34FDUJPOʹج͍ͮͯ ΈࠐΈ//ͷߏ୳ࡧ •1FOO5SFFCBOL4FDUJPOʹ ج͍ͮͯ࠶ؼܕηϧͷߏ୳ࡧ
•࠷ྑͷςετΤϥʔΛใࠂ จհɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH &YQFSJNFOUTBOE3FTVMUT -
• 3//ͷߏ • -45.ͭɼϢχοτݸ • < >ͷҰ༷ཚͰॳظԽ • "EBN
ֶशɿ Ͱ࠷దԽ • ใुɿ࠷ޙͷΤϙοΫͷςετਫ਼ͷ࠷େ • ͷ(16Λಉ࣌ʹ༻ • ύϥϝʔλαʔόɼϨϓϦΧ • ϨϓϦΧͭ͋ͨΓੜ͢Δߏͭ • ݸͷߏΛੜ͢Δ͝ͱʹ૿Ճ • ࠷ॳ͔Βελʔτ จհɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH -FBSOJOH$POWPMVUJPOBM"SDIJUFDUVSFTGPS$*'"3 $POUSPMMFS ͷઃఆ -
• ΤϙοΫɿ • /FTUFSPW .PNFOUVN4(%Ͱ࠷దԽ • ֶशɼ׳ੑ߲ • ਖ਼ଇԽ߲ɿ8FJHIU%FDBZ
• ୳ࡧۭؒɿ • 'JMUFSIFJHIUXJEUIɿ< > • /VNCFSPGGJMUFSTɿ< > • 4USJEF IFJHIUXJEUIɿ< > • ׆ੑԽؔɿ3F-6 • #BUDI/PSNBMJ[BUJPOΛ༻ จհɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH -FBSOJOH$POWPMVUJPOBM"SDIJUFDUVSFTGPS$*'"3 ϛχόον͝ͱʹग़ྗΛਖ਼نԽ $IJME ͷઃఆ -
•܇࿅σʔλɿຕ •ςετσʔλɿຕ •લॲཧ • ന৭Խ • ը૾ΛϥϯμϜʹ֦େ͓Αͼస σʔλ֦ு
จհɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH -FBSOJOH$POWPMVUJPOBM"SDIJUFDUVSFTGPS$*'"3 ܇࿅σʔλؒͷ૬ؔΛແ͘͠ɼ ฏۉ̌ɾࢄʹਖ਼نԽ -
•Wͷ݁Ռʹ͍ͭͯɼैདྷ๏ΑΓઙ͘ɼ ͔ͭਫ਼͕ߴ͍ߏΛ֫ಘ • "QQFOEJY"ͷ'JHVSFͷߏ •ϑΟϧλͷܗঢ়ํܗ͕ଟ͘ɼಛʹ ग़ྗʹ͍ۙͰେ͖͍ϑΟϧλ •ߏͷมߋʹऑ͍ɼہॴతͳ࠷దੑ • શͯͷʹର͠ɼΛ·͍ͨͨ݁߹Λ
Ճ͑ΔͱΤϥʔ͕ʹ૿Ճ • Λ·͍ͨͨ݁߹Λશͯআ͢ΔͱΤϥʔ ͕·Ͱ૿Ճ จհɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH -FBSOJOH$POWPMVUJPOBM"SDIJUFDUVSFTGPS$*'"3 -
•Wͷ࣮ݧͰɼWʹՃ͑ͯετϥΠυ αΠζͷ࠷దԽߦ͏Α͏ʹมߋ •୳ࡧۭؒେ͖͘ͳ͕ͬͨɼ ͞΄Ͳਫ਼͕ѱ͘ͳΔ͜ͱͳ͔ͬͨ •Wͷ࣮ݧͰɼWʹՃ͑ͯͱ ͷޙʹૠೖ͢ΔϓʔϦϯάͷ ࠷దԽߦ͏Α͏ʹมߋ • ͷશ݁߹Λ࣋ͬͨͷϒϩοΫ
͝ͱʹߏΛ୳ࡧʢ୳ࡧۭؒͷ੍ݶʣ จհɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH -FBSOJOH$POWPMVUJPOBM"SDIJUFDUVSFTGPS$*'"3 -
•σʔληοτ͕খ͍ͨ͞Ίਖ਼ଇԽ͕ඞཁ •छྨͷ%SPQPVUΛ༻ • &NCFEEJOH%SPQPVUɿͷؒͷ݁߹ʹ %SPQPVUΛ͔͚Δ<;BSFNCB FUBM > • 3FDVSSFOU%SPQPVUɿ࣌ؒํͷ݁߹ʹ
มۙࣅͨ͠%SPQPVUΛ͔͚Δ<(BM > •ೖྗͱग़ྗͷຒΊࠐΈΛڞ༗͢Δख๏ <1SFTT8PMG > Λ༻ จհɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH -FBSOJOH3FDVSSFOU$FMMTGPS1FOO5SFFCBOL -
•3// $POUSPMMFS ݁߹ํ๏ͱ ׆ੑԽؔΛ༧ଌ • ݁߹ํ๏ɿ"EEɼ&MFN@NVMU • ׆ੑԽؔɿJEFOUJUZ UBOI
TJHNPJE SFMV •ೖྗϖΞʹର͢Δॲཧछྨ༧ଌ จհɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH -FBSOJOH3FDVSSFOU$FMMTGPS1FOO5SFFCBOL ͜ͷϨϕϧͷϊʔυ͕ͭଘࡏ -
• $*'"3ͷઃఆͱ΄΅ಉ༷ • ҟͳΔ෦ɿ • "EBNͷֶश͕ʹมߋ • ࢄॲཧͷߏɿϨϓϦΧ͕ɼ ϨϓϦΧ͋ͨΓͷαϯϓϧ͕ʹมߋ
• ݸͷߏΛಉ࣌ʹ$16Ͱ܇࿅ • ޯ͕ݸੵ͞ΕΔͱύϥϝʔλαʔόͷΈ ͷߋ৽Λߦ͏ • ใुʮWBMJEBUJPOQFSQMFYJUZͷʯ ʹઃఆ จհɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH -FBSOJOH3FDVSSFOU$FMMTGPS1FOO5SFFCBOL $POUSPMMFS ͷઃఆ -
•ӅΕͷ •ֶशՄೳͳύϥϝʔλ͕ ʮ-45.ɼϢχοτݸʯͷ ߹ͱҰக͢ΔΑ͏ʹϢχοτΛௐ •࠷ྑͷߏΛݟ͚ͭͨ͋ͱɼֶश %SPQPVUॏΈͷॳظԽͳͲΛ HSJETFBSDIͰ࠷దԽ จհɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH
-FBSOJOH3FDVSSFOU$FMMTGPS1FOO5SFFCBOL $IJME ͷઃఆ -
•ӳޠˠυΠπޠͷ༁λεΫʹ͓͍ͯ -45.Λ༻ͨ͠߹ΑΓྑ͍݁Ռ • #-&6্͕ •࣍ʹɼ୳ࡧۭؒͷ֦େʹର͢Δϩόετ ੑΛ֬ೝ • ݁߹ํ๏ʹNBYɼ׆ੑԽؔʹTJOΛՃ •
݁Ռͱͯ͠ɼಉͷਫ਼Λ࣋ͭߏΛ֫ಘ จհɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH -FBSOJOH3FDVSSFOU$FMMTGPS1FOO5SFFCBOL -
•3BOEPNTFBSDIʹΑΔߏ୳ࡧͱͷ ൺֱ • 3BOEPNTFBSDI୯७͕ͩڧྗͳख๏ •3BOEPNTFBSDIʹΑΔ࠷ྑͷϞσϧ ͔ΒΈͯఏҊख๏ͷQFSQMFYJUZͷݮগ ྔΛάϥϑԽ •ํࡦޯΛ༻͍ͨఏҊख๏ͷ΄͏͕ 3BOEPNTFBSDIΑΓQFSQMFYJUZ͕
վળ จհɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH -FBSOJOH3FDVSSFOU$FMMTGPS1FOO5SFFCBOL -
•3//ʹΑͬͯ//ͷߏ୳ࡧΛߦ͏ख๏ ΛఏҊ • 3//Λ༻͢Δ͜ͱʹΑͬͯɼΑΓॊೈʹ ߏ୳ࡧΛߦ͏͜ͱ͕Մೳ •ఏҊख๏ʹΑͬͯੜ͞Εͨߏྑ ͳੑೳΛ࣋ͭ •ࠓճͷख๏ߏ୳ࡧʹ͓͚Δ৽ͨͳ ݚڀͷํΛ͍ࣔͯ͠Δ
จհɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH $PODMVTJPO -
จհɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH ࢀߟจݙ • 8PKDJFDI;BSFNCB *MZB4VUTLFWFS BOE0SJPM7JOZBMTl3FDVSSFOU OFVSBMOFUXPSLSFHVMBSJ[BUJPOz BS9JWQSFQSJOUBS9JW
• :BSJO (BMl"UIFPSFUJDBMMZHSPVOEFEBQQMJDBUJPOPGESPQPVUJO SFDVSSFOUOFVSBMOFUXPSLTzBS9JW QSFQSJOUBS9JW • 0GJS 1SFTTBOE-JPS8PMGl6TJOHUIFPVUQVUFNCFEEJOHUPJNQSPWF MBOHVBHFNPEFMTzBS9JW QSFQSJOUBS9JW