Upgrade to Pro — share decks privately, control downloads, hide ads and more …

論文紹介:Neural Architecture Search with Reinforcement Learning

S.Shota
June 16, 2017

論文紹介:Neural Architecture Search with Reinforcement Learning

2017年06月16日の論文紹介で使用したスライドです.

題材にした論文:
・B. Zoph and Q. V Le, “Neural Architecture Search with Reinforcement Learning,” Proceedings of 5th International Conference on Learning Representations (ICLR'17), 2017.
https://openreview.net/forum?id=r1Ue8Hcxg&noteId=r1Ue8Hcxg

S.Shota

June 16, 2017
Tweet

More Decks by S.Shota

Other Decks in Technology

Transcript

  1. #;PQI BOE27-F  l/FVSBM"SDIJUFDUVSF4FBSDI XJUI3FJOGPSDFNFOU-FBSOJOH z 1SPDFFEJOHTPGUI*OUFSOBUJPOBM$POGFSFODFPO -FBSOJOH3FQSFTFOUBUJPOT *$-3 

    ೥݄೔ ؀ڥ৘ใֶ෎ ৘ใϝσΟΞ؀ڥֶઐ߈ ৘ใϝσΟΞֶίʔε ന઒ݚڀࣨ೥ ੪౻ ᠳଡ
  2.  •#;PQI BOE27-F  l/FVSBM"SDIJUFDUVSF4FBSDIXJUI 3FJOGPSDFNFOU-FBSOJOH z 1SPDFFEJOHTPGUI*OUFSOBUJPOBM $POGFSFODFPO-FBSOJOH 3FQSFTFOUBUJPOT

    *$-3  •//ͷߏ଄࠷దԽʹؔ͢ΔݚڀͰɼ ࠷৽ͷ΋ͷͰ͋ͬͨͨΊ  ࿦จ঺հɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH จݙ৘ใɾ͜ͷ࿦จΛબΜͩཧ༝
  3.  •3FDVSSFOU/FVSBM/FUXPSL 3// Λ ༻͍ͯɼ//ߏ଄ͷੜ੒ϞσϧΛֶश •ςετσʔλʹର͢Δਫ਼౓ BDDVSBDZ ͷظ଴஋࠷େԽΛڧԽֶशʹΑͬͯղ͘ • ޯ഑ͷਪఆͰ͸3&*/'03$&ΞϧΰϦζϜ

    Λ࢖༻ •$*'"3ɿఏҊख๏ͰಘΒΕͨϞσϧ ͸ɼଞͷϞσϧΑΓͷਫ਼౓޲্ͱ ഒͷֶश଎౓ͷվળ •1FOO5SFFCBOLͰ΋ݕূ  ࿦จ঺հɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH ֓ཁ
  4.  •//ͷߏ଄͓Αͼ઀ଓؔ܎͸ɼ Մม௕ͷจࣈྻʹΑͬͯࢦఆՄೳ • จࣈܥྻͷऔΓѻ͍͕Մೳͳ3//Λ࢖༻ • ߏ଄Λੜ੒͢Δ//ɿDPOUSPMMFS • ੜ੒͞Εͨߏ଄Λ࣋ͭ//ɿDIJMEOFUXPSL •ੜ੒͞Εͨߏ଄Λ࣋ͭ//ͷςετਫ਼౓

    Λใुͱஔ͘ • ͜ͷͱ͖ɼ3//ͷํࡦޯ഑͕ܭࢉՄೳ •ࠓճ͸-45.Λ࢖ͬͯɼ3//Λߏ੒  ࿦จ঺հɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH *OUSPEVDUJPO
  5.  •ϋΠύʔύϥϝʔλ࠷దԽͷख๏͸༷ʑ •ߏ଄Ҏ֎ͷϋΠύʔύϥϝʔλ ࠷దԽ͸ɼطʹز͔ͭఏҊ͞Ε͍ͯΔ • 3BOEPNTFBSDI #BZFTJBO0QUJNJ[BUJPO •ߏ଄ͷ࠷దԽ • #BZFTJBO0QUJNJ[BUJPO

    • /FVSP&WPMVUJPOɿ("ʹΑΔߏ଄୳ࡧ  ࿦จ঺հɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH 3FMBUFE8PSL ҰൠతͰͳ͘ ॊೈੑʹ͚ܽΔ ॊೈੑ͸͋Δ͕ɼ େن໛ͳϞσϧͷ৔߹ ͕͔͔࣌ؒΔ
  6.  •લճͷ༧ଌ݁ՌΛࠓճͷೖྗͱ͢Δ ࣗݾ࠶ؼܕͷDPOUSPMMFS 3// Λ࢖༻ •ࠓճͷ໰୊Ͱ͸໨తؔ਺͕ඍ෼ෆՄೳ • ໨తؔ਺ɿςετਫ਼౓ •ಘΒΕͨςετਫ਼౓ͷ஋Λ࢖༻ֶͯ͠श 

    ࿦จ঺հɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH 3FMBUFE8PSL •/FVSBM"SDIJUFDUVSF4FBSDI • //ʹجͮ͘ޯ഑๏ʹΑΔߏ଄୳ࡧ Ξϓϩʔν
  7.  •චऀΒʹΑͬͯఏҊ͞Ε͍ͯΔͷ͸ͭ  ํࡦޯ഑ʹΑΔ3//ͷֶश๏  3//ͷֶशΛߴ଎Խͤ͞ΔͨΊͷ෼ࢄֶश  ෳࡶͳϞσϧΛੜ੒͢ΔͨΊͷ3//΁ͷ TLJQDPOOFDUJPOͷಋೖ 

    -45.ͷΑ͏ͳ࠶ؼతͳߏ଄Λ࣋ͭDFMMͷ ੜ੒ํ๏  ࿦จ঺հɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH .FUIPET
  8.  •૚਺͕نఆͷ஋ʹ౸ୡͨ͠Βɼ3//ʹ ΑΔߏ଄ͷੜ੒Λऴྃ • ੜ੒ͨ͠Ϟσϧͷ܇࿅Λߦ͏ • ܇࿅ͨ͠Ϟσϧ͔Βɼςετਫ਼౓Λܭࢉ •ੜ੒ͨ͠Ϟσϧͷςετਫ਼౓͔Βɼ ςετਫ਼౓ͷظ଴஋ͷ࠷େԽํ޲΁ 3//ͷॏΈύϥϝʔλ

    Λߋ৽  ࿦จ঺հɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH (FOFSBUF.PEFM%FTDSJQUJPOTXJUIB$POUSPMMFS 3FDVSSFOU/FVSBM/FUXPSL ✓c ͜Ε͕ํࡦޯ഑
  9.  •ޯ഑͸ϞϯςΧϧϩ๏ʹΑͬͯਪఆɿ • ͸ճ͋ͨΓʹੜ੒ͨ͠ߏ଄ͷݸ਺ • 3//ͷύϥϝʔλߋ৽ ˠNݸͷαϯϓϧʢߏ଄ʣΛαϯϓϦϯά ˠͦΕͧΕͷςετਫ਼౓ ͱ্ࣜʹ Αͬͯ3//ͷύϥϝʔλߋ৽ˠʜ

     ࿦จ঺հɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH 5SBJOJOHXJUI3&*/'03$& m R1 ⇠ Rm r✓c J ( ✓c ) ⇡ 1 m m X k= 1 T X t= 1 r✓c log P ( at | a ( t 1):1; ✓c ) Rk = 0 B B B B B @ 1 m m X k= 1 Rk 1 C C C C C A ⇥ 0 B B B B B @ T X t= 1 r✓c log P ( at | a ( t 1):1; ✓c ) 1 C C C C C A
  10.  •(PPH-F/FU΍3FTJEVBM/FUʹ ୅ද͞ΕΔΑ͏ͳ૚Λ·͙ͨ݁߹΋ ୳ࡧͷର৅ͱ͢ΔΑ͏ʹ֦ு͢Δ •૚Λ·͙ͨ݁߹Λ؅ཧ͢Δ"OLFSQPJOU Λγʔέϯεʹ௥Ճ •૚Λ·͙ͨ݁߹Λੜ੒͢Δ֬཰͸ TJHNPJEؔ਺ʹΑܾͬͯఆ  ࿦จ঺հɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH

    *ODSFBTF"SDIJUFDUVSF$PNQMFYJUZXJUI4LJQ $POOFDUJPOTBOE0UIFS-BZFS5ZQFT P (Layer j is an input to layer i) = sigmoid( vT tanh( W prev ⇤ hj + W curr ⇤ hi )) IϨΠϠʔʹؔ͢ΔӅΕঢ়ଶ IҎ֎͸܇࿅Մೳͳύϥϝʔλ
  11.  •-45.ͷΑ͏ͳ࠶ؼܕͰ಺෦ঢ়ଶΛ࣋ͭ ηϧͷߏ଄Λ୳ࡧ͢Δ •ηϧ΁ͷೖྗ͸ͭ • εςοϓ໨ͷग़ྗɿ • ηϧͷ಺෦ঢ়ଶɿ • લͷ૚͔Βͷग़ྗɿ

    •ग़ྗ͸ ͱ ͷͭ  ࿦จ঺հɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH (FOFSBUF3FDVSSFOU$FMM"SDIJUFDUVSFT t 1 xt ht 1 ct ht ct 1
  12.  •'JHVSFͰߦΘΕΔߏ଄ͷ༧ଌɿ • 5SFF*OEFYɿ"EEԋࢉΛͨ͠ޙʹ5BOI Λ͔͚Δ͔ • 5SFF*OEFYɿཁૉ͝ͱʹੵΛͱͬͨޙʹ 3F-6Λ͔͚Δ͔ • $FMM*OKFDUɿ5SFF*OEFYͷग़ྗʹ಺෦

    ঢ়ଶΛ଍ͯ͠ɼ3F-6Λ͔͚Δ͔ • 5SFF*OEFYɿ$FMM*OKFDUͷग़ྗͱ5SFF *OEFYͷग़ྗΛɼཁૉ͝ͱʹੵΛͱͬͨ ޙʹ4JHNPJEΛ͔͚Δ͔  ࿦จ঺հɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH (FOFSBUF3FDVSSFOU$FMM"SDIJUFDUVSFT
  13.  •࣮ݧʹ࢖༻ͨ͠σʔληοτ͸ͭ • $*'"3 • 1FOO5SFFCBOL •$*'"3͸4FDUJPOʹج͍ͮͯ ৞ΈࠐΈ//ͷߏ଄୳ࡧ •1FOO5SFFCBOL͸4FDUJPOʹ ج͍ͮͯ࠶ؼܕηϧͷߏ଄୳ࡧ

    •࠷ྑͷςετΤϥʔΛใࠂ  ࿦จ঺հɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH &YQFSJNFOUTBOE3FTVMUT
  14.  • 3//ͷߏ੒ • -45.ͭɼϢχοτ਺͸ݸ • < >ͷҰ༷ཚ਺ͰॳظԽ • "EBN

    ֶश཰ɿ Ͱ࠷దԽ • ใुɿ࠷ޙͷΤϙοΫͷςετਫ਼౓ͷ࠷େ஋ • ୆ͷ(16Λಉ࣌ʹ࢖༻ • ύϥϝʔλαʔό୆ɼϨϓϦΧ୆ • ϨϓϦΧͭ͋ͨΓੜ੒͢Δߏ଄ͭ • ݸͷߏ଄Λੜ੒͢Δ͝ͱʹ૚૿Ճ • ࠷ॳ͸૚͔Βελʔτ  ࿦จ঺հɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH -FBSOJOH$POWPMVUJPOBM"SDIJUFDUVSFTGPS$*'"3 $POUSPMMFS ͷઃఆ
  15.  • ΤϙοΫ਺ɿ • /FTUFSPW .PNFOUVN4(%Ͱ࠷దԽ • ֶश཰ɼ׳ੑ߲ • ਖ਼ଇԽ߲ɿ8FJHIU%FDBZ

    ܎਺ • ୳ࡧۭؒɿ • 'JMUFSIFJHIUXJEUIɿ<   > • /VNCFSPGGJMUFSTɿ<   > • 4USJEF IFJHIUXJEUIɿ<  > • ׆ੑԽؔ਺ɿ3F-6 • #BUDI/PSNBMJ[BUJPOΛ࢖༻  ࿦จ঺հɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH -FBSOJOH$POWPMVUJPOBM"SDIJUFDUVSFTGPS$*'"3 ϛχόον͝ͱʹग़ྗΛਖ਼نԽ $IJME ͷઃఆ
  16.  •܇࿅σʔλɿຕ •ςετσʔλɿຕ •લॲཧ • ന৭Խ • ը૾ΛϥϯμϜʹ֦େ͓Αͼ൓స σʔλ֦ு 

    ࿦จ঺հɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH -FBSOJOH$POWPMVUJPOBM"SDIJUFDUVSFTGPS$*'"3 ܇࿅σʔλؒͷ૬ؔΛແ͘͠ɼ ฏۉ̌ɾ෼ࢄʹਖ਼نԽ
  17.  •Wͷ݁Ռʹ͍ͭͯɼैདྷ๏ΑΓ΋ઙ͘ɼ ͔ͭਫ਼౓͕ߴ͍ߏ଄Λ֫ಘ • "QQFOEJY"ͷ'JHVSFͷߏ଄ •ϑΟϧλͷܗঢ়͸௕ํܗ͕ଟ͘ɼಛʹ ग़ྗʹ͍ۙ૚Ͱ͸େ͖͍ϑΟϧλ •ߏ଄ͷมߋʹऑ͍ɼہॴతͳ࠷దੑ • શͯͷ૚ʹର͠ɼ૚Λ·͍ͨͨ݁߹Λ

    Ճ͑ΔͱΤϥʔ͕ʹ૿Ճ • ૚Λ·͍ͨͨ݁߹Λશͯ࡟আ͢ΔͱΤϥʔ ͕·Ͱ૿Ճ  ࿦จ঺հɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH -FBSOJOH$POWPMVUJPOBM"SDIJUFDUVSFTGPS$*'"3
  18.  •σʔληοτ͕খ͍ͨ͞Ίਖ਼ଇԽ͕ඞཁ •छྨͷ%SPQPVUΛ࢖༻ • &NCFEEJOH%SPQPVUɿ૚ͷؒͷ݁߹ʹ %SPQPVUΛ͔͚Δ<;BSFNCB FUBM > • 3FDVSSFOU%SPQPVUɿ࣌ؒํ޲ͷ݁߹ʹ

    ม෼ۙࣅͨ͠%SPQPVUΛ͔͚Δ<(BM > •ೖྗͱग़ྗͷຒΊࠐΈΛڞ༗͢Δख๏ <1SFTT8PMG  > Λ࢖༻  ࿦จ঺հɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH -FBSOJOH3FDVSSFOU$FMMTGPS1FOO5SFFCBOL
  19.  •3// $POUSPMMFS ͸݁߹ํ๏ͱ ׆ੑԽؔ਺Λ༧ଌ • ݁߹ํ๏ɿ"EEɼ&MFN@NVMU • ׆ੑԽؔ਺ɿJEFOUJUZ UBOI

    TJHNPJE SFMV •ೖྗϖΞʹର͢Δॲཧ͸छྨ༧ଌ  ࿦จ঺հɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH -FBSOJOH3FDVSSFOU$FMMTGPS1FOO5SFFCBOL ͜ͷϨϕϧͷϊʔυ͕ͭଘࡏ
  20.  • $*'"3ͷઃఆͱ΄΅ಉ༷ • ҟͳΔ෦෼ɿ • "EBNͷֶश཰͕ʹมߋ • ෼ࢄॲཧͷߏ੒ɿϨϓϦΧ͕ɼ ϨϓϦΧ͋ͨΓͷαϯϓϧ਺͕ʹมߋ

    • ݸͷߏ଄Λಉ࣌ʹ$16Ͱ܇࿅ • ޯ഑͕ݸ஝ੵ͞ΕΔͱύϥϝʔλαʔόͷΈ ஋ͷߋ৽Λߦ͏ • ใु͸ʮWBMJEBUJPOQFSQMFYJUZͷ৐ʯ ʹઃఆ  ࿦จ঺հɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH -FBSOJOH3FDVSSFOU$FMMTGPS1FOO5SFFCBOL $POUSPMMFS ͷઃఆ
  21.  •ӳޠˠυΠπޠͷ຋༁λεΫʹ͓͍ͯ΋ -45.Λ࢖༻ͨ͠৔߹ΑΓ΋ྑ͍݁Ռ • #-&6஋͕޲্ •࣍ʹɼ୳ࡧۭؒͷ֦େʹର͢Δϩόετ ੑΛ֬ೝ • ݁߹ํ๏ʹNBYɼ׆ੑԽؔ਺ʹTJOΛ௥Ճ •

    ݁Ռͱͯ͠ɼಉ౳ͷਫ਼౓Λ࣋ͭߏ଄Λ֫ಘ  ࿦จ঺հɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH -FBSOJOH3FDVSSFOU$FMMTGPS1FOO5SFFCBOL
  22.   ࿦จ঺հɿ/FVSBM"SDIJUFDUVSF4FBSDIXJUI3FJOGPSDFNFOU-FBSOJOH ࢀߟจݙ • 8PKDJFDI;BSFNCB *MZB4VUTLFWFS BOE0SJPM7JOZBMTl3FDVSSFOU OFVSBMOFUXPSLSFHVMBSJ[BUJPOz BS9JWQSFQSJOUBS9JW

     • :BSJO (BMl"UIFPSFUJDBMMZHSPVOEFEBQQMJDBUJPOPGESPQPVUJO SFDVSSFOUOFVSBMOFUXPSLTzBS9JW QSFQSJOUBS9JW  • 0GJS 1SFTTBOE-JPS8PMGl6TJOHUIFPVUQVUFNCFEEJOHUPJNQSPWF MBOHVBHFNPEFMTzBS9JW QSFQSJOUBS9JW