ブラックボックス最適化とその応用

by gree_tech

Embed

Start on current slide

Slide 1

Slide 1 text

Slide 2

Slide 2 text

Copyright © GREE, Inc. All Rights Reserved. ॴଐ • άϦʔגࣜձࣾ  AIϦαʔννʔϜ ΤϯδχΞ • ࢈ۀٕज़૯߹ݚڀॴ ਓ޻஌ೳݚڀηϯλʔ  ಛఆूதݚڀઐ໳һʢ݉຿ʣ ݚڀ෼໺ • ඍ෼ϑϦʔ࠷దԽɾϒϥοΫϘοΫε࠷దԽ • Automated Machine Learning (AutoML) ඌ࡚ Յ඙ https://y0z.github.io/about/

Slide 3

Slide 3 text

Copyright © GREE, Inc. All Rights Reserved. • ԿΒ͔ͷ໨తؔ਺Λಛఆ੍໿ԼͰ࠷খԽʢͳ͍͠࠷େԽʣ͢Δ໰୊    !   • Ұൠʹ! ʹؔͯ͠ಘΒΕΔ৘ใ΍ɼ͓͚ΔԾఆ͕ଟ͍΄Ͳޮ཰తʹղ͚Δ • Ұ࣍ͷޯ഑৘ใɼೋ࣍ͷޯ഑৘ใ • ತੑɼϦϓγοπ࿈ଓੑɼྼϞδϡϥੑ Minimize f(x) subject to x ∈ X f(x) ਺ཧ࠷దԽ

Slide 4

Slide 4 text

Copyright © GREE, Inc. All Rights Reserved. • ήʔϜͷόϥϯεΛࠨӈ͢Δύϥϝʔλͷࣗಈௐ੔ • ԿΒ͔ͷείΞ! (ྫ͑͹ɼউ཰)͸ήʔϜγϛϡϨʔλΛಈ࡞ͤ͞Δ ͜ͱͰಘΒΕΔ͕ɼ! ͷৄࡉ͸खʹෛ͑ͳ͍΄Ͳෳࡶ • ػցֶशϞσϧͷϋΠύύϥϝʔλ࠷దԽ • AutoMLͷத৺త՝୊ͷ1ͭ (Feurer and Hutter, 2019) • Ϟσϧੑೳ! ͕࠷ྑͱͳΔϋΠύύϥϝʔλ! ͷ୳ࡧʢؔ܎͸ඇࣗ໌ʣ f(x) f(x) f(x) x ݱ৔ʹ͸”ϒϥοΫϘοΫε”ؔ਺͕ଟ਺

Slide 5

Slide 5 text

Copyright © GREE, Inc. All Rights Reserved. • యܕతͳ໰୊ઃఆ • ໨తؔ਺஋! ͷΈ͕؍ଌՄೳ • ݪଇͱͯ͠ޯ഑৘ใ΍ؔ਺ͷੑ࣭ͳͲΛར༻Ͱ͖ͳ͍ • ؔ਺ධՁίετ͕ߴ͍ʢήʔϜγϛϡϨʔγϣϯ΍Ϟσϧͷֶशʣ • ໨తؔ਺ΛධՁͰ͖Δճ਺ʹݶΓ͕͋Δ • ؍ଌ͸ϊΠζΛ൐͏ʢήʔϜͷ݁Ռ΍ֶशͷ݁Ռʹ͸ཚ਺͕Өڹʣ • ΋͏গ͠ϦονͳઃఆΛάϨΠϘοΫε࠷దԽͱݺͿ͜ͱ͕͋Δ • ϚϧνϑΟσϦςΟ࠷దԽ • ࢀߟɿGrey-box Bayesian Optimization for AutoML  https://slideslive.com/38916582/keynote-greybox-bayesian- optimization-for-automl f(x) ϒϥοΫϘοΫε࠷దԽ

Slide 6

Slide 6 text

Copyright © GREE, Inc. All Rights Reserved. • ϕΠζ࠷దԽɾόϯσΟοτΞϧΰϦζϜ  ػցֶशܥݚڀऀΒ͕੝ΜʹݚڀɼGP-EIɼSMACɼTPEͳͲ • ਐԽܭࢉ  Population-based methodsͱ΋ɼCMA-ESͳͲ • ௚઀୳ࡧ๏  Nelder–Mead๏ɼMADSͳͲ • اۀ΋ϒϥοΫϘοΫε࠷దԽιϑτ΢ΣΞ։ൃʹਚྗ • Google Vizier (Google) • Optuna (PFN) • Nevergrad (Facebook) ϒϥοΫϘοΫε࠷దԽख๏

Slide 7

Slide 7 text

Slide 8

Slide 8 text

Copyright © GREE, Inc. All Rights Reserved. • ؔ਺ධՁͱ୯ମͷมܗΛ܁Γฦ͢ඍ෼ϑϦʔہॴ୳ࡧώϡʔϦεςΟοΫ • ϋΠύύϥϝʔλ࠷దԽΛؚΉɼ࣮༻্ͷଟ͘ͷ໰୊Ͱ্ख͘ಇ͘ (Cohen et al., 2005; Ozaki et al., 2017) Nelder–Mead๏ Nelder and Mead, 1965 CNNͷϋΠύύϥϝʔλ࠷దԽ (Ozaki et al., 2017)

Slide 9

Slide 9 text

Slide 10

Slide 10 text

Copyright © GREE, Inc. All Rights Reserved. • Nelder–Mead๏ͷ୳ࡧ఺͸֤఺ͷධՁ஋ʹج͖ͮஞ࣍తʹܾ·ΔͨΊɼ ͜ͷख๏͸ฒྻԽʹෆ޲͖Ͱ͋Γɼ࣮༻্େ͖ͳ՝୊ • ໨తؔ਺ͷαϩήʔτ্ͰɼNelder–Mead๏Λ࣮ߦ͢ΔϞϯςΧϧϩ๏ʹ ΑΓɼධՁ͞ΕΔݟࠐΈͷߴ͍఺Λ༧ଌ͠ɼ౤ػతʹධՁ ༧ଌʹجͮ͘ฒྻධՁʹΑΔNelder–Mead๏ͷߴ଎Խ Accelerating the Nelder–Mead Method with Predictive Parallel Evaluation Yoshihiko Ozaki, Shuhei Watanabe, and Masaki Onishi  6th ICML Workshop on Automated Machine Learning, Jun 2019. ! ΛԾఆ͠ɼ! ͸Ψ΢εաఔ͔ΒͷαϯϓϧΛද͢ f(x) ∼ GP(m(x), k(x, x′)) g(x)

Slide 11

Slide 11 text

Copyright © GREE, Inc. All Rights Reserved. 1.ॳظ୯ମʹؚ·ΕΔ఺ΛฒྻධՁ 2.ະධՁ఺ʹ౸ୡ͢Δ·Ͱɼଓ͖͔ΒNelder–Mead๏Λ࣮ߦ 3.ϞϯςΧϧϩ๏Λ࣮ߦ͠ɼ౤ػతʹධՁ͢ΔPݸͷީิ఺ΛٻΊɼฒྻධՁ 4.࠷దԽͷఀࢭ৚݅Λຬ͍ͨͯ͠Ε͹݁ՌΛฦ͠ɼͦ͏Ͱͳ͚Ε͹2.ʹ໭Δ ༧ଌʹجͮ͘ฒྻධՁʹΑΔNelder–Mead๏ͷߴ଎Խ ఏҊख๏ ! ΛԾఆ͠ɼ! ͸Ψ΢εաఔ͔ΒͷαϯϓϧΛද͢ f(x) ∼ GP(m(x), k(x, x′)) g(x)

Slide 12

Slide 12 text

Copyright © GREE, Inc. All Rights Reserved. •࣮ݧઃఆ • 6छྨͷ਺஋ϋΠύύϥϝʔλΛ࠷దԽ͢ΔϕϯνϚʔΫ໰୊ (Klein et al., 2018) • ฒྻ਺! Ͱݻఆ͠ɼઌಡΈΠςϨʔγϣϯ਺! Ͱ࣮ݧ • Baseline 1͸ɼॳظԽͱshrinkૢ࡞ͷΈฒྻධՁ (ࣗ໌ͳฒྻԽ) • Baseline 2͸ɼ࣍ΠςϨʔγϣϯͰධՁ͞ΕಘΔશͯͷ఺ΛฒྻධՁ •݁Ռ • Baseline 1ʹൺ΂49%ߴ଎Խɼ2ʹൺ΂13%ߴ଎Խ͔ͭগͳ͍ධՁ਺ P = 10 J = 1,2,3,4,5 ༧ଌʹجͮ͘ฒྻධՁʹΑΔNelder–Mead๏ͷߴ଎Խ ܭࢉ࣮ݧ Method J Average # of eval steps Average # of evaluations Baseline 1 - 590.27 (±141.42) 614.10 (±142.82) Baseline 2 - 347.27 (±89.32) 3469.67 (±893.21) Proposed 1 406.20 (±97.24) 1534.20 (±427.69) 2 314.13 (±72.26) 2307.83 (±558.02) 3 304.97 (±54.57) 2679.13 (±464.80) 4 310.60 (±67.58) 2948.20 (±642.62) 5 301.90 (±58.70) 2942.33 (±567.27)

Slide 13

Slide 13 text

Copyright © GREE, Inc. All Rights Reserved. • ฒྻ਺! ɼઌಡΈΠςϨʔγϣϯ਺! Λ࣮ݧ • ߴ଎ԽͷޮՌ͸͋Δఔ౓ͷ! ·Ͱεέʔϧ͢Δ͕ɼͦΕҎ্͸མͪண͘ ʢઌͷΠςϨʔγϣϯʹͳΔ΄ͲɼධՁ͞ΕΔ఺ͷ༧ଌ͸೉͘͠ͳΔʣ P = 10,20,30,40 J = 1,2,3,4,5 P, J ༧ଌʹجͮ͘ฒྻධՁʹΑΔNelder–Mead๏ͷߴ଎Խ ܭࢉ࣮ݧ

Slide 14

Slide 14 text

Copyright © GREE, Inc. All Rights Reserved. • ϒϥοΫϘοΫε࠷దԽ͸ۃΊͯ༗༻ • ϋΠύύϥϝʔλ࠷దԽɼήʔϜͷύϥϝʔλࣗಈௐ੔ͳͲԠ༻ଟ਺ • ٳܜ࣌ؒʹσΟεΧογϣϯ׻ܴ • 8݄5೔ͷKDD AutoML Workshopʹͯ࠷৽ͷݚڀʹ͍ͭͯൃද༧ఆ • Yoshihiko Ozaki and Masaki Onishi,  “Practical Deep Neural Network Performance Prediction for Hyperparameter Optimization,”  To appear. • https://sites.google.com/view/automl2019-workshop/ ·ͱΊ