論文紹介：Speeding up Automatic Hyperparameter Optimization of Deep Neural Networks by Extrapolation of Learning Curves

5%PNIBO +54QSJOHFOCFSH BOE')VUUFS l4QFFEJOHVQ"VUPNBUJD )ZQFSQBSBNFUFS 0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG -FBSOJOH$VSWFT z 1SPDFFEJOHTPGUIF5XFOUZ'PVSUI*OUFSOBUJPOBM+PJOU
$POGFSFODFPO"SUJGJDJBM*OUFMMJHFODF *+$"* ೥݄೔ ؀ڥ৘ใֶ෎ ৘ใϝσΟΞ؀ڥֶઐ߈ ৘ใϝσΟΞֶίʔε ന઒ݚڀࣨ೥ ੪౻ ᠳଡ

• 5%PNIBO +54QSJOHFOCFSH BOE' )VUUFS l4QFFEJOHVQ"VUPNBUJD )ZQFSQBSBNFUFS 0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG
-FBSOJOH$VSWFT z1SPDFFEJOHTPGUIF 5XFOUZ'PVSUI*OUFSOBUJPOBM+PJOU $POGFSFODFPO"SUJGJDJBM*OUFMMJHFODF *+$"* ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT จݙ৘ใ

•$."&4ʹΑΔ%//ͷϋΠύʔ ύϥϝʔλ࠷దԽ ͷࢀߟจݙʹ ڍ͛ΒΕ͍ͯͨͨΊ • *-PTIDIJMPW BOE')VUUFS l$."&4GPS )ZQFSQBSBNFUFS
0QUJNJ[BUJPOPG%FFQ/FVSBM /FUXPSLT z1SPDFFEJOHPG*OUFSOBUJPOBM$POGFSFODFPO -FBSOJOH3FQSFTFOUBUJPOT •֬཰Ϟσϧʹجͮ͘%//ͷϋΠύʔ ύϥϝʔλ࠷దԽͷख๏ͷͭͰ͋Δ ͨΊ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS 0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT ͜ͷ࿦จΛબΜͩཧ༝

•ϋΠύʔύϥϝʔλ࠷దԽΛߴ଎Խ͢Δ ख๏ΛఏҊ •܇࿅தͷ࠷ॳͷ਺εςοϓͷ৘ใ͔Βɼ ੑೳΛ༧ଌ • ѱͦ͏ͩͬͨΒऴྃͯ࣍͠ͷ୳ࡧ΁ • ֶशۂઢͷ֬཰తͳϞσϧΛఆ࣮ٛͯ͠ݱ •%//Ͱ࣮ݧͨ͠ͱ͜Ζɼ໿ഒͷ଎౓Ͱ
ϋΠύʔύϥϝʔλͷ࠷దԽΛ࣮ݱ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT ֓ཁ

•%//ͷϋΠύʔύϥϝʔλ࠷దԽ͸ɼ ઐ໳ՈʹΑΔνϡʔχϯάͱಉ౳͔ͦΕ Ҏ্ͷੑೳΛҾ͖ग़͢έʔε΋͋Δ •ͨͩ͠ɼ%//͕େ͖͍΄ͲϋΠύʔ ύϥϝʔλ࠷దԽͷܭࢉίετ͕ߴ͍ ʹ͕͔͔࣌ؒΔ •ͦͷͨΊɼઐ໳ՈʹΑΔखಈͰͷ νϡʔχϯά͕Ұൠత
࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT *OUSPEVDUJPO

࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT *OUSPEVDUJPO ੑೳ͕ѱԽͦ͠͏ͳͱ͖͸܇࿅Λ ૣظऴྃͤ͞Δ͜ͱͰɼ࣌ؒΛઅ໿͢Δ ैདྷख๏͸ੑೳ͕ѱͯ͘΋࠷ޙ·Ͱ܇࿅ ໰୊఺ ໨త
ֶशۂઢΛ༧ଌ͢Δ֬཰ϞσϧΛ༻͍ͯ ੑೳͷѱԽΛ༧ଌ͠ɼૣظऴྃ͢Δ Ξϓϩʔν

• ػցֶशΞϧΰϦζϜɿ • ͷ࣋ͭϋΠύʔύϥϝʔλɿ • ͷఆٛҬɿ • ϋΠύʔύϥϝʔλશମ͕ͳۭؒ͢ɿ •
ϋΠύʔύϥϝʔλͷ૊ɿ • Λ࢖ͬͨ৔߹ͷΞϧΰϦζϜɿ • ܇࿅σʔλɿ • ςετσʔλɿ • Ͱ܇࿅ͨ͠ ͷ ʹର͢Δଛࣦɿ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS 0QUJNJ[BUJPOPG%FFQ/FVSBM /FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT )ZQFSQBSBNFUFS0QUJNJ[BUJPO.FUIPE A A i (i = 1, 2, . . . , n) i ⇤i (i = 1, 2, . . . , n) ⇤ = ⇤1 ⇥ · · · ⇥ ⇤n 2 ⇤ A Dtrain Dvalid Dtrain Dvalid A l( )

•୅දతͳख๏ • HSJETFBSDI • SBDJOHBMHPSJUIN • HSBEJFOUTFBSDI • SBOEPNTFBSDI
• #BZFTJBOPQUJNJ[BUJPO #0 ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT )ZQFSQBSBNFUFS0QUJNJ[BUJPO.FUIPE ϋΠύʔύϥϝʔλ࠷దԽ໰୊ arg min 2⇤ l( ) Ծఆͨ͠ࣄલ෼෍͔Βͷ αϯϓϧΛ༻͍ͯ ໨తؔ਺ͷ֬཰ϞσϧΛ ߋ৽͢Δख๏

•4QFBSNJOUɿ#BZFTJBO0QUJNJ[BUJPOͷ ୅දతͳ࣮૷ • ࣄલ෼෍ʹ(BVTTJBO1SPDFTT (1 Λ࢖༻ • ࿈ଓతͳ஋Λ࣋ͭϋΠύʔύϥϝʔλͷ ࠷దԽʹదͨ͠ख๏
•4."$ɼ51&ɿ໦ߏ଄ϕʔεͷ࠷దԽ ख๏ • %//ͷΑ͏ʹߴ࣍ݩɼ཭ࢄతͳϋΠύʔ ύϥϝʔλΛ͍࣋ͬͯΔ৔߹ʹ༗ޮ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT )ZQFSQBSBNFUFS0QUJNJ[BUJPO.FUIPE ͪ͜ΒΛ࠾༻

•ֶशۂઢ͕ࣔͭ͢ͷݱ৅ • 5ZQF൓෮ճ਺ʢ࣮ߦ࣌ؒʣͱ ػցֶशΞϧΰϦζϜͷੑೳͷؔ܎ • 5ZQF܇࿅ʹ࢖༻Մೳͳσʔληοτͷ αΠζͱػցֶशΞϧΰϦζϜͷੑೳͷؔ܎ •͜ͷݚڀ͸5ZQFͷֶशۂઢʹؔ܎
࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT .PEFMJOH-FBSOJOH$VSWFT

•5ZQFͷֶशۂઢ͸ɼҰൠతʹ͸ աֶशͷߟ͑ΛՄࢹԽ • աֶश͢Δલʹ܇࿅Λऴྃͤ͞Δ͜ͱΛ ໨తͱͯ͠ݚڀ •ֶशۂઢϞσϧΛؚΉ(1ϕʔεͷ#0΋ طʹߟҊ͞Ε͍ͯΔ • ߦྻ෼ղɼϩδεςΟοΫճؼɼ-BUFOU
%JSJDIMFU "MMPDBUJPO -%" ʹద༻ • %//Ͱ͸͏·͘ػೳ͠ͳ͍ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT .PEFMJOH-FBSOJOH$VSWFT

•֬཰తޯ഑߱Լ๏ʢ4UPDIBTUJD (SBEJFOU%FTDFOUʀ4(%ʣͷεςοϓ ਺ͱੑೳͷؔ਺ʹয఺Λ౰ͯΔ •ςετσʔλʹର͢Δ෼ྨਫ਼౓Λ ੑೳΛଌΔࢦඪͱͯ͠࢖༻ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT &YUSBQPMBUJPOPG-FBSOJOH$VSWFT

•ֶशॳظͷ৘ใΛ༻͍ͯɼֶशۂઢͷ ࢒Γͷ෦෼Λ֬཰తʹਪఆ •%//Ͱ4(%Λ༻͍Δ৔߹ɼఆظతʹ ςετσʔλʹର͢ΔੑೳΛଌఆ •࠷ॳͷOεςοϓʹ͓͍ͯଌఆ͞Εͨ ੑೳ஋ɿ • Λ༻͍ͯɼޙʹ౸ୡͦ͠͏ͳੑೳ Λ֬཰ϞσϧΛ༻͍ͯ༧ଌ
࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT -FBSOJOH$VSWF.PEFM y1:n y1:n ym (m n)

•؍ଌֶͨ͠शۂઢΛ,ݸͷؔ਺ ͰϞσϧԽ •ؔ਺ ͷ࣋ͭύϥϝʔλɿ •Ψ΢γΞϯϊΠζɿ •ؔ਺ ͷ΋ͱͰɼ࣌ؒ Ͱͷੑೳɿ •ؔ਺
ͷ΋ͱͰɼੑೳ ΛऔΔ֬཰ɿ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT 1BSBNFUSJD-FBSOJOH$VSWF.PEFMT {f1, . . . , fK} fk ✓k ✏ ⇠ N ⇣ 0, 2 ⌘ fk t fk yt p ⇣ yt | ✓k, 2 ⌘ = N ⇣ yt | fk (t | ✓k ), 2 ⌘ yt = fk (t | ✓k ) + ✏

•ࠓճ͸,ݸͷؔ਺Λ࢖༻ •͜ΕΒΛ֬཰తʹ૊Έ߹Θֶͤͯशۂઢ Λදݱ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT 1BSBNFUSJD-FBSOJOH$VSWF.PEFMT

•,ݸͷؔ਺ΛॏΈ෇͖ઢܗ࿨Ͱ݁߹ɿ • ͷύϥϝʔλϕΫτϧɿ •ෆ࣮֬ੑΛద੾ʹϞσϧԽ͢ΔͨΊʹ ϕΠζతͳਪఆΛ࢖༻ • ͷਪఆʹ͸.$.$Λ࢖༻ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ
/FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT "8FJHIUFE1SPCBCJMJTUJD-FBSOJOH$VSWF.PEFMT f comb ( t | ⇠ ) = K X k= 1 wk fk ( t | ✓k ) f comb ⇠ = ⇣ w1, . . . , wK, ✓1, . . . , ✓K, 2 ⌘ ym

•ͨͩ͠ɼॏΈ͸ਖ਼ʹ੍ݶ • ݮগ͢ΔۂઢΛಘΔͨΊ • ͢΂ͯͷϞσϧΛฏۉԽ͢Δͱ༧ଌͨ͠ۂ ઢ͸ฏୱʹͳΔͱ͍͏ޮՌ΋ •্ͷ੍໿ͷ΋ͱͰࣄલ෼෍Λ࣍ࣜͰఆٛ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ
/FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT "8FJHIUFE1SPCBCJMJTUJD-FBSOJOH$VSWF.PEFMT p ( ⇠ ) / 0 B B B B B @ K Y k= 1 p ( wk ) p ( ✓k ) 1 C C C C C A p ( 2 ) I ( f comb(1 | ⇠ ) < f comb( m | ⇠ )) 8k, p ( wk ) / 8 > > < > > : 1 if wk > 0 0 otherwise

•੍໿ΛೖΕͳ͍ࣄલ෼෍ ͱ ੍໿ΛೖΕͨࣄલ෼෍ͷൺֱ •੍໿ΛೖΕΔͱ๞࿨ۂઢ͕ଟ͘ಘΒΕΔ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS 0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT "8FJHIUFE1SPCBCJMJTUJD-FBSOJOH$VSWF.PEFMT
p(⇠) / 1

•4ݸͷαϯϓϧ ͸࣍ͷࣄޙ෼෍ ͔ΒαϯϓϦϯά •ͨͩ͠ɼ ͸࣍ࣜͰఆٛ •Ϟσϧύϥϝʔλ ͷॳظ஋͸ɼϞσϧ ͝ͱͷ࠷໬ਪఆ஋ʹઃఆ
࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT "8FJHIUFE1SPCBCJMJTUJD-FBSOJOH$VSWF.PEFMT ⇠1, . . . , ⇠S P(⇠ | y1:n ) / P(y1:n | ⇠)P(⇠) P(y1:n | ⇠) P ( y 1: n | ⇠ ) = n Y t= 1 N ⇣ yt | f comb( t | ✓k ) , 2 ⌘ ✓k

•ॏΈ ͷॳظ஋͸Ұ༷ʹઃఆ •෼ࢄͷॳظ஋͸࠷໬ਪఆྔʹઃఆ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT "8FJHIUFE1SPCBCJMJTUJD-FBSOJOH$VSWF.PEFMT wk wk
= 1 K 2 = 1 n n X t= 1 ( yt f comb( t | ⇠ ))2

• ͕༩͑ΒΕͨͱ͖ɼ ͕ Λ௒͑Δ ֬཰ɿ • ͸ਖ਼ن෼෍ͷྦྷੵີ౓ؔ਺ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ
/FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT "8FJHIUFE1SPCBCJMJTUJD-FBSOJOH$VSWF.PEFMT y1:n ym ˆ y P ( ym ˆ y | y 1: n ) = Z P ( ⇠ | y 1: n ) P ( ym > ˆ y | ⇠ )d ⇠ ⇡ 1 S S X s= 1 P ( ym > ˆ y | ⇠S ) = 1 S S X s= 1 ⇣ 1 ⇣ ˆ y | f comb( m | ⇠S ) , 2 S ⌘⌘ (· | µ, 2)

࠷ߴਫ਼౓ ͱ͢Δʢ ͷॳظ஋͸ ʣ ͋ΔϋΠύʔύϥϝʔλ Λ࢖ͬͯ%//Λ ܇࿅͠ɼΠςϨʔγϣϯ͝ͱʹ
ςετσʔλʹର͢ΔධՁ஋ Λอଘ ΤϙοΫ͝ͱʹɼ֬཰ Λܭࢉ ֬཰ ͕͖͍͠஋ Λ ௒͍͑ͯͨΒ܇࿅Λଓߦɽͦ͏Ͱͳ͍৔߹͸ தஅ͠ɼධՁ஋ͷظ଴஋ Λ ϋΠύʔύϥϝʔλͷ࠷దԽثʹ౉͢ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT 4QFFEJOHVQ)ZQFSQBSBNFUFS 0QUJNJ[BUJPO ΞϧΰϦζϜ ˆ y ˆ y 1 i yi P(ym ˆ y | y1:n ) p P(ym ˆ y | y1:n ) 1 E[ym | y1:n ] QSFEJDUJWFUFSNJOBUJPODSJUFSJPO

•ظ଴஋͸࣍ࣜͰۙࣅɿ •ఏҊख๏͸ɼ%//ͷ܇࿅Λதஅ͢Δ ඞཁ͕ͳ͍ •%//ͷ܇࿅Λ(16Ͱߦ͍ͬͯΔؒʹɼ .$.$ʹΑΔαϯϓϦϯά͸$16Ͱ ߦ͑͹ྑ͍ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT
4QFFEJOHVQ)ZQFSQBSBNFUFS 0QUJNJ[BUJPO E [ ym | y 1: n ] ⇡ 1 S S X s= 1 f comb( m | ⇠S )

•͍͔ͭ͘ͷ%//Ϟσϧɼσʔληοτ ʹ͍࣮ͭͯݧ •༻͍ΔϋΠύʔύϥϝʔλ࠷దԽख๏͸ ͭʢ4."$ͱ51&ʣ •ϞσϧͱϋΠύʔύϥϝʔλ࠷దԽख๏ Λ૊Έ߹Θ࣮ͤͯݧ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT
&YQFSJFODF

•࢖༻͢Δσʔληοτ • $*'"3 • $*'"3 • ./*45 •%//ͷ࣮૷͸$BGGFΛ࢖༻ •
(16ͭͷΈ࢖༻ •ϋΠύʔύϥϝʔλ࠷దԽͷ࣮૷͸ )10-*#Λ࢖༻ •.$.$ͷ࣮૷͸&.$&&Λ࢖༻ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT &YQFSJNFOUBM4FUVQ Ұൠը૾ೝࣝͷσʔληοτ खॻ͖਺ࣈͷσʔληοτ

•શͯͷ࣮ݧͰ͖͍͠஋ ͸ʹઃఆ •QSFEJDUJWFUFSNJOBUJPODSJUFSJPO͸ɼ ΤϙοΫ͝ͱʹ࣮ߦ •ςετσʔλʹΑΔධՁ͸Ϟσϧ͝ͱʹ ઃఆ • શ݁߹ωοτϫʔΫɿ •
৞ΈࠐΈωοτϫʔΫɿ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT &YQFSJNFOUBM4FUVQ k = 10 k = 2

•$*'"3Ͱ͸લॲཧΛ࣮ߦ • Yͷখ͞ͳը૾ΛϥϯμϜͰऔΓग़͠ɼ ॏ৺ݸͷLNFBOTΛ࣮ߦ • $*'"3ͷ֤ը૾Λ֤ॏ৺Ͱ৞ΈࠐΈɼ ը૾ͷԠ౴ΛฏۉԽ • ͜ͷ݁ՌΛωοτϫʔΫʹೖྗ
•ϋΠύʔύϥϝʔλͷ୳ࡧճ਺͸ճ •࠷େΤϙοΫ਺͸ΤϙοΫ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT 'VMMZ$POOFDUFE/FUXPSLT

•./*45͸લॲཧͳ͠ • ҰճͷωοτϫʔΫͷ܇࿅ʹ͸d෼ •ϋΠύʔύϥϝʔλͷ୳ࡧճ਺͸߹ܭ ճ·Ͱ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT 'VMMZ$POOFDUFE/FUXPSLT

•֬཰తޯ഑߱Լ๏͸.PNFOUVN4(% Λ࢖༻ •ֶश཰͸࣍ࣜͰεέδϡʔϦϯά • ͱ൒ݮظ͔Β ͕ܾఆ • ൒ݮظΛϋΠύʔύϥϝʔλͱͯ͠࠾༻ •׆ੑԽؔ਺͸3F-6ʢதؒ૚ʣͱ
4PGUNBYʢग़ྗ૚ʣΛ࢖༻ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT %//)ZQFSQBSBNFUFST ↵t = ↵0 (1 + t) p p

•֤ύϥϝʔλͷ ఆٛҬͱॳظ஋ •ܾఆ໦ϕʔεͷ ࠷దԽ๏ͳΒ͹ ੍໿͖ͭͷ ࠷దԽ΋Մೳ • Ϣχοτ਺ͳͲ ͸ϨΠϠʔ਺ʹ
ґଘ͢Δ ύϥϝʔλ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT %//)ZQFSQBSBNFUFST

•ϋΠύʔύϥϝʔλ࠷దԽث͝ͱʹ ϥϯμϜͳγʔυͰճ࣮ݧ •ճͷςετΤϥʔͷฏۉͱඪ४ภࠩΛ ՄࢹԽ •4."$ɼ51&ͷͲͪΒΛ࢖ͬͯ΋ɼ ఏҊख๏ͳ͠ʹൺ΂ͯ໿ഒͷߴ଎Խ • ఏҊख๏ͳ͠Ͱ࣌ؒޙͷੑೳͱ ఏҊख๏͋ΓͰ࣌ؒޙͷੑೳ͕ಉ͡
•࠷ऴతͳςετΤϥʔ͸4."$͕࠷খ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT 3FTVMUTGPSQSFQSPDFTTFE$*'"3

•ςετΤϥʔͱܦա࣌ؒͷਪҠ •&BSMZ4UPQQJOHWTఏҊख๏ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT 3FTVMUTGPSQSFQSPDFTTFE$*'"3

•֤छख๏Ͱݟ͚ͭͨ࠷ྑͷϞσϧʹ ର͢ΔςετΤϥʔͷൺֱ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT 3FTVMUTGPSQSFQSPDFTTFE$*'"3

•ఏҊख๏ͳ͋͠ΓͰͷֶशۂઢͷ༷ࢠ •ੑೳ͕ѱ͍ͱ͖͸ૣ͘ଧͪ੾ΒΕ͍ͯΔ ͜ͱ͕Θ͔Δ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT 3FTVMUTGPSQSFQSPDFTTFE$*'"3

•$*'"3Ͱͷ݁ՌΛ౿·͑ɼϋΠύʔ ύϥϝʔλ࠷దԽث͸4."$Λ࢖༻ •࣮ݧճ਺͸ճ •ఏҊख๏ͳ͠Ͱ4."$Λ࢖ͬͨ৔߹ͱ ൺֱͯ͠ɼ໿ͷ࣌ؒͰಉ͡ੑೳΛ ୡ੒ • QSFEJDUJWFUFSNJOBUJPODSJUFSJPOʹΑΔ ߴ଎Խ͕༗ޮ
࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT 3FTVMUGPS./*45

•4."$ͷΈ WTఏҊख๏ 4."$ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT 3FTVMUGPS./*45

•৞ΈࠐΈχϡʔϥϧωοτͰͷ࣮ݧ •$*'"3ɼ$*'"3Λ࢖༻ • ͲͪΒ΋ന৭Խͷલॲཧ •ଞͷ$//Ϟσϧ͸σʔλΛͦͷ·· ࢖༻ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT
4NBMM$POWPMVUJPOBM/FVSBM/FUXPSLT

•৞ΈࠐΈ૚ͷ͋ͱʹશ݁߹૚ΛೖΕͳ͍ ϞσϧΛ࢖༻ • ৞ΈࠐΈ૚ͷΧʔωϧαΠζ͸Y •৞ΈࠐΈ૚ͷޙʹ͸ϓʔϦϯά૚Λૠೖ • NBYQPPMJOHΛ࣮ߦ • ϓʔϦϯά૚ͷ૚਺͸࠷దԽͷର৅
•Ұճͷ܇࿅͸෼Ͱऴྃ͢Δখ͞ͳ Ϟσϧ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT 4NBMM$//)ZQFSQBSBNFUFST

•ϋΠύʔύϥϝʔλͷఆٛҬͱॳظ஋ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT 4NBMM$//)ZQFSQBSBNFUFST

•4."$ɼ51&ͰͦΕͧΕճ࣮ͣͭݧ •ϋΠύʔύϥϝʔλͷ୳ࡧճ਺͸ճ •࠷େΤϙοΫ਺͸ΤϙοΫ •ςετΤϥʔͷฏۉΛൺֱ͢Δͱɼ 4."$͕51&ΑΓ΋एׯྑ͍݁Ռ • 4."$ɿ •
51&ɿ ˋ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT 3FTVMUTGPS4NBMM$//TPO$*'"3

•4."$ͷΈ WTఏҊख๏ 4."$ B •ఏҊख๏ʹΑΓதஅ͞Εֶͨशɿ C ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT
3FTVMUTGPS4NBMM$//TPO$*'"3

•࠷ྑͷϋΠύʔύϥϝʔλʹ͓͚Δ ςετΤϥʔͷൺֱ • ϕʔεϥΠϯ͕ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT 3FTVMUTGPS4NBMM$//TPO$*'"3

•$*'"3Ͱͷ݁ՌΛ౿·͑ɼ 4."$ͷΈΛ࢖༻ •࣮ݧճ਺͸ճ •ϋΠύʔύϥϝʔλͷ୳ࡧճ਺͸ճ ·Ͱ •ఏҊख๏ͳ͠ͱൺֱͯ͠ɼ ಉ͡ςετΤϥʔʹୡ͢Δ଎͕͞໿ഒ •ςετσʔλʹର͢Δਪఆਫ਼౓΋޲্
࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT 3FTVMUTGPS4NBMM$//TPO$*'"3

•4."$ͷΈ WTఏҊख๏ 4."$ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT 3FTVMUTGPS4NBMM$//TPO$*'"3

•࠷ྑͷϋΠύʔύϥϝʔλʹ͓͚Δ ςετΤϥʔͷൺֱ • ϕʔεϥΠϯ͕ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT 3FTVMUTGPS4NBMM$//TPO$*'"3

•࠷ઌ୺ͷϞσϧͱͷൺֱ •$*'"3σʔληοτΛ࢖༻ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT -BSHF$POWPMVUJPOBM/FUXPSLT

•ଟ͘ͷϋΠύʔύϥϝʔλͷఆٛҬ͸ 5BCMFʹै͏ •࢖༻͢ΔϞσϧ͸"MM$//ϕʔε • ͨͩ͠ϓʔϦϯά૚͸௥Ճͷ৞ΈࠐΈ૚ʹ มߋ • ௥Ճͷ৞ΈࠐΈ૚ͷετϥΠυ͸ •৞ΈࠐΈ૚ͷΧʔωϧαΠζ͸Yʹ
มߋ •ग़ྗ૚ͷલʹ΋৞ΈࠐΈ૚ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT -BSHF$//)ZQFSQBSBNFUFST

•4."$ͷΈΛ࢖༻ •࣮ݧ͸ճ •࠷େΤϙοΫ਺͸ΤϙοΫ •ϋΠύʔύϥϝʔλͷ୳ࡧճ਺͸ճ •σʔλ֦ுͳ͠Ͱൺֱͨ͠ͱ͜Ζ ࠷ઌ୺ͷωοτϫʔΫͱಉ౳Ҏ্ͷੑೳ •ఏҊख๏͋Γˠ(16 5JUBO Ͱ೔ؒ
ͳ͠ˠ(16Ͱ೔ؒҎ্ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT 3FTVMUTGPS-BSHF$//TPO$*'"3

•࠷ઌ୺ͷωοτϫʔΫͱͷςετΤϥʔ ͷൺֱ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT 3FTVMUTGPS-BSHF$//TPO$*'"3

•ੑೳ͕վળ͠ͳ͍͜ͱΛࣗಈతʹݕ஌͠ ϋΠύʔύϥϝʔλͷධՁΛૣظऴྃ͢ Δख๏ΛఏҊ •ఏҊख๏ʹΑΓ%//ʹ͓͚ΔϋΠύʔ ύϥϝʔλ࠷దԽ͕ߴ଎Խ •ϋΠύʔύϥϝʔλ࠷దԽثʹґଘ ͠ͳ͍ख๏Ͱɼͭͷख๏ͷͲͪΒͱ΋ ߴ଎ԽΛ࣮ݱ •$*'"3ʹ͓͍ͯ࠷ઌ୺ͷ݁Ռ
࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT $PODMVTJPO

論文紹介：Speeding up Automatic Hyperparameter Optim...

論文紹介：Speeding up Automatic Hyperparameter Optimization of Deep Neural Networks by Extrapolation of Learning Curves

More Decks by S.Shota

Other Decks in Technology

Featured

Transcript