Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
ブラックボックス最適化とその応用
Search
gree_tech
PRO
July 19, 2019
Technology
0
78
ブラックボックス最適化とその応用
『CCSE2019』で発表された資料です。
https://ccse.jp/2019/
gree_tech
PRO
July 19, 2019
Tweet
Share
More Decks by gree_tech
See All by gree_tech
kustomizeをいい感じに使う方法
gree_tech
PRO
3
1.1k
スケーラビリティとコスト管理 Google Cloud Spanner 費用最適化の取り組み
gree_tech
PRO
0
540
「アナザーエデン 時空を超える猫」の5年前のログを引っ越してデータドリブンで事業運用プロセスを改善した話
gree_tech
PRO
0
380
Unity,PHP+Jenkins+GAS 多言語対応を意識させない開発を目指したシステム構築
gree_tech
PRO
0
830
全社総会における「REALITY Spaces」の活用と、Addressableを用いたコンテンツ配信技術について
gree_tech
PRO
0
500
AWSのEKS環境でログ機能を構築/リリースしたお話
gree_tech
PRO
0
380
「ヘブンバーンズレッド」の大規模アップデートにおける国内及び翻訳QAの取り組み
gree_tech
PRO
0
450
アプリ「REALITY」の12言語対応プロセスの仕組みと品質向上の取り組み
gree_tech
PRO
0
700
REALITYアプリのメンテナンスなしでの機能リリースを実現する、Istio導入とB/Gデプロイ実現の取り組み
gree_tech
PRO
0
570
Other Decks in Technology
See All in Technology
SLOいつ決めましょう?
abnoumaru
3
760
汎用ポリシー言語Rego + OPAと認可・検証事例の紹介 / Introduction Rego & OPA for authorization and validation
mizutani
1
160
B2C、B2B プロダクトマネジメントの違い(および思考の罠) / B2C, B2B PM and reduction fallacy
ykmc09
5
2.4k
#phpconkagawa レガシーコードにもオブザーバビリティを 〜少しずつ始めるサービス監視〜
yamato_sorariku
0
560
5分で分かる(かもしれない) Vector engine for OpenSearch Serverless
tsukuboshi
1
420
cgroup v2 で何が変わったのか / TechFeed Experts Night #28
tenforward
2
160
生成AI活用推進の為にやったこと/やらなかったこと
ktc_wada
0
180
My road to OSEE Part1
yunolay
0
120
AWSの生成AI入門書を執筆しました🎉
minorun365
PRO
0
150
YJIT Makes Rails 1.7x faster / RubyKaigi 2024
k0kubun
3
480
[PyconUS 2024] Having fun with Pydantic and pattern matching
enforcerpl
0
190
Blazor WASM × Code-first gRPC で始める C# ⼤統⼀理論
sansantech
PRO
1
750
Featured
See All Featured
Music & Morning Musume
bryan
41
5.6k
5 minutes of I Can Smell Your CMS
philhawksworth
199
19k
Side Projects
sachag
451
41k
In The Pink: A Labor of Love
frogandcode
138
21k
GraphQLとの向き合い方2022年版
quramy
33
13k
The Cult of Friendly URLs
andyhume
74
5.7k
How to train your dragon (web standard)
notwaldorf
75
5.2k
From Idea to $5000 a Month in 5 Months
shpigford
377
45k
Embracing the Ebb and Flow
colly
80
4.2k
[RailsConf 2023] Rails as a piece of cake
palkan
29
4.1k
Fireside Chat
paigeccino
22
2.7k
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
323
20k
Transcript
Copyright © GREE, Inc. All Rights Reserved. ϒϥοΫϘοΫε࠷దԽͱͦͷԠ༻ ඌ࡚ Յ
Copyright © GREE, Inc. All Rights Reserved. ॴଐ • άϦʔגࣜձࣾ
AIϦαʔννʔϜ ΤϯδχΞ • ࢈ۀٕज़૯߹ݚڀॴ ਓೳݚڀηϯλʔ ಛఆूதݚڀઐһʢ݉ʣ ݚڀ • ඍϑϦʔ࠷దԽɾϒϥοΫϘοΫε࠷దԽ • Automated Machine Learning (AutoML) ඌ࡚ Յ https://y0z.github.io/about/
Copyright © GREE, Inc. All Rights Reserved. • ԿΒ͔ͷతؔΛಛఆ੍ԼͰ࠷খԽʢͳ͍͠࠷େԽʣ͢Δ
! • Ұൠʹ! ʹؔͯ͠ಘΒΕΔใɼ͓͚ΔԾఆ͕ଟ͍΄Ͳޮతʹղ͚Δ • Ұ࣍ͷޯใɼೋ࣍ͷޯใ • ತੑɼϦϓγοπ࿈ଓੑɼྼϞδϡϥੑ Minimize f(x) subject to x ∈ X f(x) ཧ࠷దԽ
Copyright © GREE, Inc. All Rights Reserved. • ήʔϜͷόϥϯεΛࠨӈ͢Δύϥϝʔλͷࣗಈௐ •
ԿΒ͔ͷείΞ! (ྫ͑ɼউ)ήʔϜγϛϡϨʔλΛಈ࡞ͤ͞Δ ͜ͱͰಘΒΕΔ͕ɼ! ͷৄࡉखʹෛ͑ͳ͍΄Ͳෳࡶ • ػցֶशϞσϧͷϋΠύύϥϝʔλ࠷దԽ • AutoMLͷத৺త՝ͷ1ͭ (Feurer and Hutter, 2019) • Ϟσϧੑೳ! ͕࠷ྑͱͳΔϋΠύύϥϝʔλ! ͷ୳ࡧʢؔඇࣗ໌ʣ f(x) f(x) f(x) x ݱʹ”ϒϥοΫϘοΫε”͕ؔଟ
Copyright © GREE, Inc. All Rights Reserved. • యܕతͳઃఆ •
తؔ! ͷΈ͕؍ଌՄೳ • ݪଇͱͯ͠ޯใؔͷੑ࣭ͳͲΛར༻Ͱ͖ͳ͍ • ؔධՁίετ͕ߴ͍ʢήʔϜγϛϡϨʔγϣϯϞσϧͷֶशʣ • తؔΛධՁͰ͖ΔճʹݶΓ͕͋Δ • ؍ଌϊΠζΛ͏ʢήʔϜͷ݁Ռֶशͷ݁Ռʹཚ͕Өڹʣ • ͏গ͠ϦονͳઃఆΛάϨΠϘοΫε࠷దԽͱݺͿ͜ͱ͕͋Δ • ϚϧνϑΟσϦςΟ࠷దԽ • ࢀߟɿGrey-box Bayesian Optimization for AutoML https://slideslive.com/38916582/keynote-greybox-bayesian- optimization-for-automl f(x) ϒϥοΫϘοΫε࠷దԽ
Copyright © GREE, Inc. All Rights Reserved. • ϕΠζ࠷దԽɾόϯσΟοτΞϧΰϦζϜ ػցֶशܥݚڀऀΒ͕ΜʹݚڀɼGP-EIɼSMACɼTPEͳͲ
• ਐԽܭࢉ Population-based methodsͱɼCMA-ESͳͲ • ୳ࡧ๏ Nelder–Mead๏ɼMADSͳͲ • اۀϒϥοΫϘοΫε࠷దԽιϑτΣΞ։ൃʹਚྗ • Google Vizier (Google) • Optuna (PFN) • Nevergrad (Facebook) ϒϥοΫϘοΫε࠷దԽख๏
Copyright © GREE, Inc. All Rights Reserved. • ϕΠζ࠷దԽɾόϯσΟοτΞϧΰϦζϜ ػցֶशܥݚڀऀΒ͕ΜʹݚڀɼGP-EIɼSMACɼTPEͳͲ
• ਐԽܭࢉ Population-based methodsͱɼCMA-ESͳͲ • ୳ࡧ๏ Nelder–Mead๏ɼMADSͳͲ • اۀϒϥοΫϘοΫε࠷దԽιϑτΣΞ։ൃʹਚྗ • Google Vizier (Google) • Optuna (PFN) • Nevergrad (Facebook) ϒϥοΫϘοΫε࠷దԽख๏
Copyright © GREE, Inc. All Rights Reserved. • ؔධՁͱ୯ମͷมܗΛ܁Γฦ͢ඍϑϦʔہॴ୳ࡧώϡʔϦεςΟοΫ •
ϋΠύύϥϝʔλ࠷దԽΛؚΉɼ࣮༻্ͷଟ͘ͷͰ্ख͘ಇ͘ (Cohen et al., 2005; Ozaki et al., 2017) Nelder–Mead๏ Nelder and Mead, 1965 CNNͷϋΠύύϥϝʔλ࠷దԽ (Ozaki et al., 2017)
Copyright © GREE, Inc. All Rights Reserved. Nelder–Mead๏ reflect, expand,
inside contract, outside contract, shrinkͷ5छྨͷૢ࡞Λ෮తʹద༻ reflect, expand, inside contract, outside contract shrink
Copyright © GREE, Inc. All Rights Reserved. • Nelder–Mead๏ͷ୳ࡧ֤ͷධՁʹج͖ͮஞ࣍తʹܾ·ΔͨΊɼ ͜ͷख๏ฒྻԽʹෆ͖Ͱ͋Γɼ࣮༻্େ͖ͳ՝
• తؔͷαϩήʔτ্ͰɼNelder–Mead๏Λ࣮ߦ͢ΔϞϯςΧϧϩ๏ʹ ΑΓɼධՁ͞ΕΔݟࠐΈͷߴ͍Λ༧ଌ͠ɼػతʹධՁ ༧ଌʹجͮ͘ฒྻධՁʹΑΔNelder–Mead๏ͷߴԽ Accelerating the Nelder–Mead Method with Predictive Parallel Evaluation Yoshihiko Ozaki, Shuhei Watanabe, and Masaki Onishi 6th ICML Workshop on Automated Machine Learning, Jun 2019. ! ΛԾఆ͠ɼ! Ψεաఔ͔ΒͷαϯϓϧΛද͢ f(x) ∼ GP(m(x), k(x, x′)) g(x)
Copyright © GREE, Inc. All Rights Reserved. 1.ॳظ୯ମʹؚ·ΕΔΛฒྻධՁ 2.ະධՁʹ౸ୡ͢Δ·Ͱɼଓ͖͔ΒNelder–Mead๏Λ࣮ߦ 3.ϞϯςΧϧϩ๏Λ࣮ߦ͠ɼػతʹධՁ͢ΔPݸͷީิΛٻΊɼฒྻධՁ
4.࠷దԽͷఀࢭ݅Λຬ͍ͨͯ͠Ε݁ՌΛฦ͠ɼͦ͏Ͱͳ͚Ε2.ʹΔ ༧ଌʹجͮ͘ฒྻධՁʹΑΔNelder–Mead๏ͷߴԽ ఏҊख๏ ! ΛԾఆ͠ɼ! Ψεաఔ͔ΒͷαϯϓϧΛද͢ f(x) ∼ GP(m(x), k(x, x′)) g(x)
Copyright © GREE, Inc. All Rights Reserved. •࣮ݧઃఆ • 6छྨͷϋΠύύϥϝʔλΛ࠷దԽ͢ΔϕϯνϚʔΫ
(Klein et al., 2018) • ฒྻ! Ͱݻఆ͠ɼઌಡΈΠςϨʔγϣϯ! Ͱ࣮ݧ • Baseline 1ɼॳظԽͱshrinkૢ࡞ͷΈฒྻධՁ (ࣗ໌ͳฒྻԽ) • Baseline 2ɼ࣍ΠςϨʔγϣϯͰධՁ͞ΕಘΔશͯͷΛฒྻධՁ •݁Ռ • Baseline 1ʹൺ49%ߴԽɼ2ʹൺ13%ߴԽ͔ͭগͳ͍ධՁ P = 10 J = 1,2,3,4,5 ༧ଌʹجͮ͘ฒྻධՁʹΑΔNelder–Mead๏ͷߴԽ ܭࢉ࣮ݧ Method J Average # of eval steps Average # of evaluations Baseline 1 - 590.27 (±141.42) 614.10 (±142.82) Baseline 2 - 347.27 (±89.32) 3469.67 (±893.21) Proposed 1 406.20 (±97.24) 1534.20 (±427.69) 2 314.13 (±72.26) 2307.83 (±558.02) 3 304.97 (±54.57) 2679.13 (±464.80) 4 310.60 (±67.58) 2948.20 (±642.62) 5 301.90 (±58.70) 2942.33 (±567.27)
Copyright © GREE, Inc. All Rights Reserved. • ฒྻ! ɼઌಡΈΠςϨʔγϣϯ!
Λ࣮ݧ • ߴԽͷޮՌ͋Δఔͷ! ·Ͱεέʔϧ͢Δ͕ɼͦΕҎ্མͪண͘ ʢઌͷΠςϨʔγϣϯʹͳΔ΄ͲɼධՁ͞ΕΔͷ༧ଌ͘͠ͳΔʣ P = 10,20,30,40 J = 1,2,3,4,5 P, J ༧ଌʹجͮ͘ฒྻධՁʹΑΔNelder–Mead๏ͷߴԽ ܭࢉ࣮ݧ
Copyright © GREE, Inc. All Rights Reserved. • ϒϥοΫϘοΫε࠷దԽۃΊͯ༗༻ •
ϋΠύύϥϝʔλ࠷దԽɼήʔϜͷύϥϝʔλࣗಈௐͳͲԠ༻ଟ • ٳܜ࣌ؒʹσΟεΧογϣϯܴ • 8݄5ͷKDD AutoML Workshopʹͯ࠷৽ͷݚڀʹ͍ͭͯൃද༧ఆ • Yoshihiko Ozaki and Masaki Onishi, “Practical Deep Neural Network Performance Prediction for Hyperparameter Optimization,” To appear. • https://sites.google.com/view/automl2019-workshop/ ·ͱΊ
Copyright © GREE, Inc. All Rights Reserved.