論文紹介：Speeding up Automatic Hyperparameter Optimization of Deep Neural Networks by Extrapolation of Learning Curves

5%PNIBO +54QSJOHFOCFSH BOE')VUUFS l4QFFEJOHVQ"VUPNBUJD )ZQFSQBSBNFUFS 0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG -FBSOJOH$VSWFT z 1SPDFFEJOHTPGUIF5XFOUZ'PVSUI*OUFSOBUJPOBM+PJOU
$POGFSFODFPO"SUJGJDJBM*OUFMMJHFODF *+$"* ೥݄೔ ؀ڥ৘ใֶ෎ ৘ใϝσΟΞ؀ڥֶઐ߈ ৘ใϝσΟΞֶίʔε ന઒ݚڀࣨ೥ ੪౻ ᠳଡ

• 5%PNIBO +54QSJOHFOCFSH BOE' )VUUFS l4QFFEJOHVQ"VUPNBUJD )ZQFSQBSBNFUFS 0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG
-FBSOJOH$VSWFT z1SPDFFEJOHTPGUIF 5XFOUZ'PVSUI*OUFSOBUJPOBM+PJOU $POGFSFODFPO"SUJGJDJBM*OUFMMJHFODF *+$"* ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT จݙ৘ใ

•$."&4ʹΑΔ%//ͷϋΠύʔ ύϥϝʔλ࠷దԽ ͷࢀߟจݙʹ ڍ͛ΒΕ͍ͯͨͨΊ • *-PTIDIJMPW BOE')VUUFS l$."&4GPS )ZQFSQBSBNFUFS
0QUJNJ[BUJPOPG%FFQ/FVSBM /FUXPSLT z1SPDFFEJOHPG*OUFSOBUJPOBM$POGFSFODFPO -FBSOJOH3FQSFTFOUBUJPOT •֬཰Ϟσϧʹجͮ͘%//ͷϋΠύʔ ύϥϝʔλ࠷దԽͷख๏ͷͭͰ͋Δ ͨΊ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS 0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT ͜ͷ࿦จΛબΜͩཧ༝

•ϋΠύʔύϥϝʔλ࠷దԽΛߴ଎Խ͢Δ ख๏ΛఏҊ •܇࿅தͷ࠷ॳͷ਺εςοϓͷ৘ใ͔Βɼ ੑೳΛ༧ଌ • ѱͦ͏ͩͬͨΒऴྃͯ࣍͠ͷ୳ࡧ΁ • ֶशۂઢͷ֬཰తͳϞσϧΛఆ࣮ٛͯ͠ݱ •%//Ͱ࣮ݧͨ͠ͱ͜Ζɼ໿ഒͷ଎౓Ͱ
ϋΠύʔύϥϝʔλͷ࠷దԽΛ࣮ݱ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT ֓ཁ

•%//ͷϋΠύʔύϥϝʔλ࠷దԽ͸ɼ ઐ໳ՈʹΑΔνϡʔχϯάͱಉ౳͔ͦΕ Ҏ্ͷੑೳΛҾ͖ग़͢έʔε΋͋Δ •ͨͩ͠ɼ%//͕େ͖͍΄ͲϋΠύʔ ύϥϝʔλ࠷దԽͷܭࢉίετ͕ߴ͍ ʹ͕͔͔࣌ؒΔ •ͦͷͨΊɼઐ໳ՈʹΑΔखಈͰͷ νϡʔχϯά͕Ұൠత
࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT *OUSPEVDUJPO

࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT *OUSPEVDUJPO ੑೳ͕ѱԽͦ͠͏ͳͱ͖͸܇࿅Λ ૣظऴྃͤ͞Δ͜ͱͰɼ࣌ؒΛઅ໿͢Δ ैདྷख๏͸ੑೳ͕ѱͯ͘΋࠷ޙ·Ͱ܇࿅ ໰୊఺ ໨త
ֶशۂઢΛ༧ଌ͢Δ֬཰ϞσϧΛ༻͍ͯ ੑೳͷѱԽΛ༧ଌ͠ɼૣظऴྃ͢Δ Ξϓϩʔν

• ػցֶशΞϧΰϦζϜɿ • ͷ࣋ͭϋΠύʔύϥϝʔλɿ • ͷఆٛҬɿ • ϋΠύʔύϥϝʔλશମ͕ͳۭؒ͢ɿ •
ϋΠύʔύϥϝʔλͷ૊ɿ • Λ࢖ͬͨ৔߹ͷΞϧΰϦζϜɿ • ܇࿅σʔλɿ • ςετσʔλɿ • Ͱ܇࿅ͨ͠ ͷ ʹର͢Δଛࣦɿ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS 0QUJNJ[BUJPOPG%FFQ/FVSBM /FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT )ZQFSQBSBNFUFS0QUJNJ[BUJPO.FUIPE A A i (i = 1, 2, . . . , n) i ⇤i (i = 1, 2, . . . , n) ⇤ = ⇤1 ⇥ · · · ⇥ ⇤n 2 ⇤ A Dtrain Dvalid Dtrain Dvalid A l( )

•୅දతͳख๏ • HSJETFBSDI • SBDJOHBMHPSJUIN • HSBEJFOUTFBSDI • SBOEPNTFBSDI
• #BZFTJBOPQUJNJ[BUJPO #0 ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT )ZQFSQBSBNFUFS0QUJNJ[BUJPO.FUIPE ϋΠύʔύϥϝʔλ࠷దԽ໰୊ arg min 2⇤ l( ) Ծఆͨ͠ࣄલ෼෍͔Βͷ αϯϓϧΛ༻͍ͯ ໨తؔ਺ͷ֬཰ϞσϧΛ ߋ৽͢Δख๏

•4QFBSNJOUɿ#BZFTJBO0QUJNJ[BUJPOͷ ୅දతͳ࣮૷ • ࣄલ෼෍ʹ(BVTTJBO1SPDFTT (1 Λ࢖༻ • ࿈ଓతͳ஋Λ࣋ͭϋΠύʔύϥϝʔλͷ ࠷దԽʹదͨ͠ख๏
•4."$ɼ51&ɿ໦ߏ଄ϕʔεͷ࠷దԽ ख๏ • %//ͷΑ͏ʹߴ࣍ݩɼ཭ࢄతͳϋΠύʔ ύϥϝʔλΛ͍࣋ͬͯΔ৔߹ʹ༗ޮ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT )ZQFSQBSBNFUFS0QUJNJ[BUJPO.FUIPE ͪ͜ΒΛ࠾༻

•ֶशۂઢ͕ࣔͭ͢ͷݱ৅ • 5ZQF൓෮ճ਺ʢ࣮ߦ࣌ؒʣͱ ػցֶशΞϧΰϦζϜͷੑೳͷؔ܎ • 5ZQF܇࿅ʹ࢖༻Մೳͳσʔληοτͷ αΠζͱػցֶशΞϧΰϦζϜͷੑೳͷؔ܎ •͜ͷݚڀ͸5ZQFͷֶशۂઢʹؔ܎
࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT .PEFMJOH-FBSOJOH$VSWFT

•5ZQFͷֶशۂઢ͸ɼҰൠతʹ͸ աֶशͷߟ͑ΛՄࢹԽ • աֶश͢Δલʹ܇࿅Λऴྃͤ͞Δ͜ͱΛ ໨తͱͯ͠ݚڀ •ֶशۂઢϞσϧΛؚΉ(1ϕʔεͷ#0΋ طʹߟҊ͞Ε͍ͯΔ • ߦྻ෼ղɼϩδεςΟοΫճؼɼ-BUFOU
%JSJDIMFU "MMPDBUJPO -%" ʹద༻ • %//Ͱ͸͏·͘ػೳ͠ͳ͍ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT .PEFMJOH-FBSOJOH$VSWFT

•֬཰తޯ഑߱Լ๏ʢ4UPDIBTUJD (SBEJFOU%FTDFOUʀ4(%ʣͷεςοϓ ਺ͱੑೳͷؔ਺ʹয఺Λ౰ͯΔ •ςετσʔλʹର͢Δ෼ྨਫ਼౓Λ ੑೳΛଌΔࢦඪͱͯ͠࢖༻ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT &YUSBQPMBUJPOPG-FBSOJOH$VSWFT

•ֶशॳظͷ৘ใΛ༻͍ͯɼֶशۂઢͷ ࢒Γͷ෦෼Λ֬཰తʹਪఆ •%//Ͱ4(%Λ༻͍Δ৔߹ɼఆظతʹ ςετσʔλʹର͢ΔੑೳΛଌఆ •࠷ॳͷOεςοϓʹ͓͍ͯଌఆ͞Εͨ ੑೳ஋ɿ • Λ༻͍ͯɼޙʹ౸ୡͦ͠͏ͳੑೳ Λ֬཰ϞσϧΛ༻͍ͯ༧ଌ
࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT -FBSOJOH$VSWF.PEFM y1:n y1:n ym (m n)

•؍ଌֶͨ͠शۂઢΛ,ݸͷؔ਺ ͰϞσϧԽ •ؔ਺ ͷ࣋ͭύϥϝʔλɿ •Ψ΢γΞϯϊΠζɿ •ؔ਺ ͷ΋ͱͰɼ࣌ؒ Ͱͷੑೳɿ •ؔ਺
ͷ΋ͱͰɼੑೳ ΛऔΔ֬཰ɿ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT 1BSBNFUSJD-FBSOJOH$VSWF.PEFMT {f1, . . . , fK} fk ✓k ✏ ⇠ N ⇣ 0, 2 ⌘ fk t fk yt p ⇣ yt | ✓k, 2 ⌘ = N ⇣ yt | fk (t | ✓k ), 2 ⌘ yt = fk (t | ✓k ) + ✏

•ࠓճ͸,ݸͷؔ਺Λ࢖༻ •͜ΕΒΛ֬཰తʹ૊Έ߹Θֶͤͯशۂઢ Λදݱ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT 1BSBNFUSJD-FBSOJOH$VSWF.PEFMT

•,ݸͷؔ਺ΛॏΈ෇͖ઢܗ࿨Ͱ݁߹ɿ • ͷύϥϝʔλϕΫτϧɿ •ෆ࣮֬ੑΛద੾ʹϞσϧԽ͢ΔͨΊʹ ϕΠζతͳਪఆΛ࢖༻ • ͷਪఆʹ͸.$.$Λ࢖༻ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ
/FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT "8FJHIUFE1SPCBCJMJTUJD-FBSOJOH$VSWF.PEFMT f comb ( t | ⇠ ) = K X k= 1 wk fk ( t | ✓k ) f comb ⇠ = ⇣ w1, . . . , wK, ✓1, . . . , ✓K, 2 ⌘ ym

•ͨͩ͠ɼॏΈ͸ਖ਼ʹ੍ݶ • ݮগ͢ΔۂઢΛಘΔͨΊ • ͢΂ͯͷϞσϧΛฏۉԽ͢Δͱ༧ଌͨ͠ۂ ઢ͸ฏୱʹͳΔͱ͍͏ޮՌ΋ •্ͷ੍໿ͷ΋ͱͰࣄલ෼෍Λ࣍ࣜͰఆٛ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ
/FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT "8FJHIUFE1SPCBCJMJTUJD-FBSOJOH$VSWF.PEFMT p ( ⇠ ) / 0 B B B B B @ K Y k= 1 p ( wk ) p ( ✓k ) 1 C C C C C A p ( 2 ) I ( f comb(1 | ⇠ ) < f comb( m | ⇠ )) 8k, p ( wk ) / 8 > > < > > : 1 if wk > 0 0 otherwise

•੍໿ΛೖΕͳ͍ࣄલ෼෍ ͱ ੍໿ΛೖΕͨࣄલ෼෍ͷൺֱ •੍໿ΛೖΕΔͱ๞࿨ۂઢ͕ଟ͘ಘΒΕΔ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS 0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT "8FJHIUFE1SPCBCJMJTUJD-FBSOJOH$VSWF.PEFMT
p(⇠) / 1

•4ݸͷαϯϓϧ ͸࣍ͷࣄޙ෼෍ ͔ΒαϯϓϦϯά •ͨͩ͠ɼ ͸࣍ࣜͰఆٛ •Ϟσϧύϥϝʔλ ͷॳظ஋͸ɼϞσϧ ͝ͱͷ࠷໬ਪఆ஋ʹઃఆ
࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT "8FJHIUFE1SPCBCJMJTUJD-FBSOJOH$VSWF.PEFMT ⇠1, . . . , ⇠S P(⇠ | y1:n ) / P(y1:n | ⇠)P(⇠) P(y1:n | ⇠) P ( y 1: n | ⇠ ) = n Y t= 1 N ⇣ yt | f comb( t | ✓k ) , 2 ⌘ ✓k

•ॏΈ ͷॳظ஋͸Ұ༷ʹઃఆ •෼ࢄͷॳظ஋͸࠷໬ਪఆྔʹઃఆ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT "8FJHIUFE1SPCBCJMJTUJD-FBSOJOH$VSWF.PEFMT wk wk
= 1 K 2 = 1 n n X t= 1 ( yt f comb( t | ⇠ ))2

• ͕༩͑ΒΕͨͱ͖ɼ ͕ Λ௒͑Δ ֬཰ɿ • ͸ਖ਼ن෼෍ͷྦྷੵີ౓ؔ਺ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ
/FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT "8FJHIUFE1SPCBCJMJTUJD-FBSOJOH$VSWF.PEFMT y1:n ym ˆ y P ( ym ˆ y | y 1: n ) = Z P ( ⇠ | y 1: n ) P ( ym > ˆ y | ⇠ )d ⇠ ⇡ 1 S S X s= 1 P ( ym > ˆ y | ⇠S ) = 1 S S X s= 1 ⇣ 1 ⇣ ˆ y | f comb( m | ⇠S ) , 2 S ⌘⌘ (· | µ, 2)

࠷ߴਫ਼౓ ͱ͢Δʢ ͷॳظ஋͸ ʣ ͋ΔϋΠύʔύϥϝʔλ Λ࢖ͬͯ%//Λ ܇࿅͠ɼΠςϨʔγϣϯ͝ͱʹ
ςετσʔλʹର͢ΔධՁ஋ Λอଘ ΤϙοΫ͝ͱʹɼ֬཰ Λܭࢉ ֬཰ ͕͖͍͠஋ Λ ௒͍͑ͯͨΒ܇࿅Λଓߦɽͦ͏Ͱͳ͍৔߹͸ தஅ͠ɼධՁ஋ͷظ଴஋ Λ ϋΠύʔύϥϝʔλͷ࠷దԽثʹ౉͢ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT 4QFFEJOHVQ)ZQFSQBSBNFUFS 0QUJNJ[BUJPO ΞϧΰϦζϜ ˆ y ˆ y 1 i yi P(ym ˆ y | y1:n ) p P(ym ˆ y | y1:n ) 1 E[ym | y1:n ] QSFEJDUJWFUFSNJOBUJPODSJUFSJPO

•ظ଴஋͸࣍ࣜͰۙࣅɿ •ఏҊख๏͸ɼ%//ͷ܇࿅Λதஅ͢Δ ඞཁ͕ͳ͍ •%//ͷ܇࿅Λ(16Ͱߦ͍ͬͯΔؒʹɼ .$.$ʹΑΔαϯϓϦϯά͸$16Ͱ ߦ͑͹ྑ͍ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT
4QFFEJOHVQ)ZQFSQBSBNFUFS 0QUJNJ[BUJPO E [ ym | y 1: n ] ⇡ 1 S S X s= 1 f comb( m | ⇠S )

•͍͔ͭ͘ͷ%//Ϟσϧɼσʔληοτ ʹ͍࣮ͭͯݧ •༻͍ΔϋΠύʔύϥϝʔλ࠷దԽख๏͸ ͭʢ4."$ͱ51&ʣ •ϞσϧͱϋΠύʔύϥϝʔλ࠷దԽख๏ Λ૊Έ߹Θ࣮ͤͯݧ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT
&YQFSJFODF

•࢖༻͢Δσʔληοτ • $*'"3 • $*'"3 • ./*45 •%//ͷ࣮૷͸$BGGFΛ࢖༻ •
(16ͭͷΈ࢖༻ •ϋΠύʔύϥϝʔλ࠷దԽͷ࣮૷͸ )10-*#Λ࢖༻ •.$.$ͷ࣮૷͸&.$&&Λ࢖༻ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT &YQFSJNFOUBM4FUVQ Ұൠը૾ೝࣝͷσʔληοτ खॻ͖਺ࣈͷσʔληοτ

•શͯͷ࣮ݧͰ͖͍͠஋ ͸ʹઃఆ •QSFEJDUJWFUFSNJOBUJPODSJUFSJPO͸ɼ ΤϙοΫ͝ͱʹ࣮ߦ •ςετσʔλʹΑΔධՁ͸Ϟσϧ͝ͱʹ ઃఆ • શ݁߹ωοτϫʔΫɿ •
৞ΈࠐΈωοτϫʔΫɿ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT &YQFSJNFOUBM4FUVQ k = 10 k = 2

•$*'"3Ͱ͸લॲཧΛ࣮ߦ • Yͷখ͞ͳը૾ΛϥϯμϜͰऔΓग़͠ɼ ॏ৺ݸͷLNFBOTΛ࣮ߦ • $*'"3ͷ֤ը૾Λ֤ॏ৺Ͱ৞ΈࠐΈɼ ը૾ͷԠ౴ΛฏۉԽ • ͜ͷ݁ՌΛωοτϫʔΫʹೖྗ
•ϋΠύʔύϥϝʔλͷ୳ࡧճ਺͸ճ •࠷େΤϙοΫ਺͸ΤϙοΫ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT 'VMMZ$POOFDUFE/FUXPSLT

•./*45͸લॲཧͳ͠ • ҰճͷωοτϫʔΫͷ܇࿅ʹ͸d෼ •ϋΠύʔύϥϝʔλͷ୳ࡧճ਺͸߹ܭ ճ·Ͱ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT 'VMMZ$POOFDUFE/FUXPSLT

•֬཰తޯ഑߱Լ๏͸.PNFOUVN4(% Λ࢖༻ •ֶश཰͸࣍ࣜͰεέδϡʔϦϯά • ͱ൒ݮظ͔Β ͕ܾఆ • ൒ݮظΛϋΠύʔύϥϝʔλͱͯ͠࠾༻ •׆ੑԽؔ਺͸3F-6ʢதؒ૚ʣͱ
4PGUNBYʢग़ྗ૚ʣΛ࢖༻ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT %//)ZQFSQBSBNFUFST ↵t = ↵0 (1 + t) p p

•֤ύϥϝʔλͷ ఆٛҬͱॳظ஋ •ܾఆ໦ϕʔεͷ ࠷దԽ๏ͳΒ͹ ੍໿͖ͭͷ ࠷దԽ΋Մೳ • Ϣχοτ਺ͳͲ ͸ϨΠϠʔ਺ʹ
ґଘ͢Δ ύϥϝʔλ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT %//)ZQFSQBSBNFUFST

•ϋΠύʔύϥϝʔλ࠷దԽث͝ͱʹ ϥϯμϜͳγʔυͰճ࣮ݧ •ճͷςετΤϥʔͷฏۉͱඪ४ภࠩΛ ՄࢹԽ •4."$ɼ51&ͷͲͪΒΛ࢖ͬͯ΋ɼ ఏҊख๏ͳ͠ʹൺ΂ͯ໿ഒͷߴ଎Խ • ఏҊख๏ͳ͠Ͱ࣌ؒޙͷੑೳͱ ఏҊख๏͋ΓͰ࣌ؒޙͷੑೳ͕ಉ͡
•࠷ऴతͳςετΤϥʔ͸4."$͕࠷খ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT 3FTVMUTGPSQSFQSPDFTTFE$*'"3

•ςετΤϥʔͱܦա࣌ؒͷਪҠ •&BSMZ4UPQQJOHWTఏҊख๏ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT 3FTVMUTGPSQSFQSPDFTTFE$*'"3

•֤छख๏Ͱݟ͚ͭͨ࠷ྑͷϞσϧʹ ର͢ΔςετΤϥʔͷൺֱ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT 3FTVMUTGPSQSFQSPDFTTFE$*'"3

•ఏҊख๏ͳ͋͠ΓͰͷֶशۂઢͷ༷ࢠ •ੑೳ͕ѱ͍ͱ͖͸ૣ͘ଧͪ੾ΒΕ͍ͯΔ ͜ͱ͕Θ͔Δ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT 3FTVMUTGPSQSFQSPDFTTFE$*'"3

•$*'"3Ͱͷ݁ՌΛ౿·͑ɼϋΠύʔ ύϥϝʔλ࠷దԽث͸4."$Λ࢖༻ •࣮ݧճ਺͸ճ •ఏҊख๏ͳ͠Ͱ4."$Λ࢖ͬͨ৔߹ͱ ൺֱͯ͠ɼ໿ͷ࣌ؒͰಉ͡ੑೳΛ ୡ੒ • QSFEJDUJWFUFSNJOBUJPODSJUFSJPOʹΑΔ ߴ଎Խ͕༗ޮ
࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT 3FTVMUGPS./*45

•4."$ͷΈ WTఏҊख๏ 4."$ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT 3FTVMUGPS./*45

•৞ΈࠐΈχϡʔϥϧωοτͰͷ࣮ݧ •$*'"3ɼ$*'"3Λ࢖༻ • ͲͪΒ΋ന৭Խͷલॲཧ •ଞͷ$//Ϟσϧ͸σʔλΛͦͷ·· ࢖༻ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT
4NBMM$POWPMVUJPOBM/FVSBM/FUXPSLT

•৞ΈࠐΈ૚ͷ͋ͱʹશ݁߹૚ΛೖΕͳ͍ ϞσϧΛ࢖༻ • ৞ΈࠐΈ૚ͷΧʔωϧαΠζ͸Y •৞ΈࠐΈ૚ͷޙʹ͸ϓʔϦϯά૚Λૠೖ • NBYQPPMJOHΛ࣮ߦ • ϓʔϦϯά૚ͷ૚਺͸࠷దԽͷର৅
•Ұճͷ܇࿅͸෼Ͱऴྃ͢Δখ͞ͳ Ϟσϧ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT 4NBMM$//)ZQFSQBSBNFUFST

•ϋΠύʔύϥϝʔλͷఆٛҬͱॳظ஋ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT 4NBMM$//)ZQFSQBSBNFUFST

•4."$ɼ51&ͰͦΕͧΕճ࣮ͣͭݧ •ϋΠύʔύϥϝʔλͷ୳ࡧճ਺͸ճ •࠷େΤϙοΫ਺͸ΤϙοΫ •ςετΤϥʔͷฏۉΛൺֱ͢Δͱɼ 4."$͕51&ΑΓ΋एׯྑ͍݁Ռ • 4."$ɿ •
51&ɿ ˋ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT 3FTVMUTGPS4NBMM$//TPO$*'"3

•4."$ͷΈ WTఏҊख๏ 4."$ B •ఏҊख๏ʹΑΓதஅ͞Εֶͨशɿ C ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT
3FTVMUTGPS4NBMM$//TPO$*'"3

•࠷ྑͷϋΠύʔύϥϝʔλʹ͓͚Δ ςετΤϥʔͷൺֱ • ϕʔεϥΠϯ͕ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT 3FTVMUTGPS4NBMM$//TPO$*'"3

•$*'"3Ͱͷ݁ՌΛ౿·͑ɼ 4."$ͷΈΛ࢖༻ •࣮ݧճ਺͸ճ •ϋΠύʔύϥϝʔλͷ୳ࡧճ਺͸ճ ·Ͱ •ఏҊख๏ͳ͠ͱൺֱͯ͠ɼ ಉ͡ςετΤϥʔʹୡ͢Δ଎͕͞໿ഒ •ςετσʔλʹର͢Δਪఆਫ਼౓΋޲্
࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT 3FTVMUTGPS4NBMM$//TPO$*'"3

•4."$ͷΈ WTఏҊख๏ 4."$ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT 3FTVMUTGPS4NBMM$//TPO$*'"3

•࠷ྑͷϋΠύʔύϥϝʔλʹ͓͚Δ ςετΤϥʔͷൺֱ • ϕʔεϥΠϯ͕ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT 3FTVMUTGPS4NBMM$//TPO$*'"3

•࠷ઌ୺ͷϞσϧͱͷൺֱ •$*'"3σʔληοτΛ࢖༻ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT -BSHF$POWPMVUJPOBM/FUXPSLT

•ଟ͘ͷϋΠύʔύϥϝʔλͷఆٛҬ͸ 5BCMFʹै͏ •࢖༻͢ΔϞσϧ͸"MM$//ϕʔε • ͨͩ͠ϓʔϦϯά૚͸௥Ճͷ৞ΈࠐΈ૚ʹ มߋ • ௥Ճͷ৞ΈࠐΈ૚ͷετϥΠυ͸ •৞ΈࠐΈ૚ͷΧʔωϧαΠζ͸Yʹ
มߋ •ग़ྗ૚ͷલʹ΋৞ΈࠐΈ૚ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT -BSHF$//)ZQFSQBSBNFUFST

•4."$ͷΈΛ࢖༻ •࣮ݧ͸ճ •࠷େΤϙοΫ਺͸ΤϙοΫ •ϋΠύʔύϥϝʔλͷ୳ࡧճ਺͸ճ •σʔλ֦ுͳ͠Ͱൺֱͨ͠ͱ͜Ζ ࠷ઌ୺ͷωοτϫʔΫͱಉ౳Ҏ্ͷੑೳ •ఏҊख๏͋Γˠ(16 5JUBO Ͱ೔ؒ
ͳ͠ˠ(16Ͱ೔ؒҎ্ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT 3FTVMUTGPS-BSHF$//TPO$*'"3

•࠷ઌ୺ͷωοτϫʔΫͱͷςετΤϥʔ ͷൺֱ ࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT 3FTVMUTGPS-BSHF$//TPO$*'"3

•ੑೳ͕վળ͠ͳ͍͜ͱΛࣗಈతʹݕ஌͠ ϋΠύʔύϥϝʔλͷධՁΛૣظऴྃ͢ Δख๏ΛఏҊ •ఏҊख๏ʹΑΓ%//ʹ͓͚ΔϋΠύʔ ύϥϝʔλ࠷దԽ͕ߴ଎Խ •ϋΠύʔύϥϝʔλ࠷దԽثʹґଘ ͠ͳ͍ख๏Ͱɼͭͷख๏ͷͲͪΒͱ΋ ߴ଎ԽΛ࣮ݱ •$*'"3ʹ͓͍ͯ࠷ઌ୺ͷ݁Ռ
࿦จ঺հɿ4QFFEJOHVQ"VUPNBUJD)ZQFSQBSBNFUFS0QUJNJ[BUJPOPG%FFQ /FVSBM/FUXPSLTCZ&YUSBQPMBUJPOPG-FBSOJOH$VSWFT $PODMVTJPO

論文紹介：Speeding up Automatic Hyperparameter Optimization of Deep Neural Networks by Extrapolation of Learning Curves

論文紹介：Speeding up Automatic Hyperparameter Optimization of Deep Neural Networks by Extrapolation of Learning Curves

More Decks by S.Shota

Other Decks in Technology

Featured

Transcript