Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
ブラックボックス最適化とその応用
Search
gree_tech
PRO
July 19, 2019
Technology
0
280
ブラックボックス最適化とその応用
『CCSE2019』で発表された資料です。
https://ccse.jp/2019/
gree_tech
PRO
July 19, 2019
Tweet
Share
More Decks by gree_tech
See All by gree_tech
コミュニケーションに鍵を見いだす、エンジニア1年目の経験談
gree_tech
PRO
0
130
REALITY株式会社における開発生産性向上の取り組み: 失敗と成功から学んだこと
gree_tech
PRO
2
1.8k
『ヘブンバーンズレッド』におけるフィールドギミックの裏側
gree_tech
PRO
2
600
セキュリティインシデント対応の体制・運用の試行錯誤 / greetechcon2024-session-a1
gree_tech
PRO
1
610
『アナザーエデン 時空を超える猫』国内海外同時運営実現への道のり ~別々で開発されたアプリを安定して同時リリースするまでの取り組み~
gree_tech
PRO
1
570
『アサルトリリィ Last Bullet』におけるクラウドストリーミング技術を用いたブラウザゲーム化の紹介
gree_tech
PRO
1
650
UnityによるPCアプリの新しい選択肢。「PC版 Google Play Games」への対応について
gree_tech
PRO
1
1k
実機ビルドのエラーによる検証ブロッカーを0に!『ヘブンバーンズレッド』のスモークテスト自動化の取り組み
gree_tech
PRO
1
680
"ゲームQA業界の技術向上を目指す! 会社を超えた研究会の取り組み"
gree_tech
PRO
1
800
Other Decks in Technology
See All in Technology
P2P ではじめる WebRTC のつまづきどころ
tnoho
1
210
ML Pipelineの開発と運用を OpenTelemetryで繋ぐ @ OpenTelemetry Meetup 2025-07
getty708
0
250
大規模組織にAIエージェントを迅速に導入するためのセキュリティの勘所 / AI agents for large-scale organizations
i35_267
6
220
データ駆動経営の道しるべ:プロダクト開発指標の戦略的活用法
ham0215
2
230
Microsoft Defender XDRで疲弊しないためのインシデント対応
sophiakunii
3
400
なぜAI時代に 「イベント」を中心に考えるのか? / Why focus on "events" in the age of AI?
ytake
2
540
室長の逆襲 :データ活用の陣地を増やすためのヒント
masatoshi0205
0
180
Recoil脱却の現状と挑戦
kirik
2
340
An introduction to Claude Code SDK
choplin
3
3.3k
エンジニアリングマネージャー“お悩み相談”パネルセッション
ar_tama
1
660
The Madness of Multiple Gemini CLIs Developing Simultaneously with Jujutsu
gunta
1
2.5k
メモ整理が苦手な者による頑張らないObsidian活用術
optim
0
120
Featured
See All Featured
The Invisible Side of Design
smashingmag
301
51k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
48
2.9k
The Straight Up "How To Draw Better" Workshop
denniskardys
235
140k
[RailsConf 2023] Rails as a piece of cake
palkan
55
5.7k
Fireside Chat
paigeccino
37
3.5k
Building Better People: How to give real-time feedback that sticks.
wjessup
367
19k
Building an army of robots
kneath
306
45k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
507
140k
Imperfection Machines: The Place of Print at Facebook
scottboms
267
13k
RailsConf 2023
tenderlove
30
1.2k
The Language of Interfaces
destraynor
158
25k
A designer walks into a library…
pauljervisheath
207
24k
Transcript
Copyright © GREE, Inc. All Rights Reserved. ϒϥοΫϘοΫε࠷దԽͱͦͷԠ༻ ඌ࡚ Յ
Copyright © GREE, Inc. All Rights Reserved. ॴଐ • άϦʔגࣜձࣾ
AIϦαʔννʔϜ ΤϯδχΞ • ࢈ۀٕज़૯߹ݚڀॴ ਓೳݚڀηϯλʔ ಛఆूதݚڀઐһʢ݉ʣ ݚڀ • ඍϑϦʔ࠷దԽɾϒϥοΫϘοΫε࠷దԽ • Automated Machine Learning (AutoML) ඌ࡚ Յ https://y0z.github.io/about/
Copyright © GREE, Inc. All Rights Reserved. • ԿΒ͔ͷతؔΛಛఆ੍ԼͰ࠷খԽʢͳ͍͠࠷େԽʣ͢Δ
! • Ұൠʹ! ʹؔͯ͠ಘΒΕΔใɼ͓͚ΔԾఆ͕ଟ͍΄Ͳޮతʹղ͚Δ • Ұ࣍ͷޯใɼೋ࣍ͷޯใ • ತੑɼϦϓγοπ࿈ଓੑɼྼϞδϡϥੑ Minimize f(x) subject to x ∈ X f(x) ཧ࠷దԽ
Copyright © GREE, Inc. All Rights Reserved. • ήʔϜͷόϥϯεΛࠨӈ͢Δύϥϝʔλͷࣗಈௐ •
ԿΒ͔ͷείΞ! (ྫ͑ɼউ)ήʔϜγϛϡϨʔλΛಈ࡞ͤ͞Δ ͜ͱͰಘΒΕΔ͕ɼ! ͷৄࡉखʹෛ͑ͳ͍΄Ͳෳࡶ • ػցֶशϞσϧͷϋΠύύϥϝʔλ࠷దԽ • AutoMLͷத৺త՝ͷ1ͭ (Feurer and Hutter, 2019) • Ϟσϧੑೳ! ͕࠷ྑͱͳΔϋΠύύϥϝʔλ! ͷ୳ࡧʢؔඇࣗ໌ʣ f(x) f(x) f(x) x ݱʹ”ϒϥοΫϘοΫε”͕ؔଟ
Copyright © GREE, Inc. All Rights Reserved. • యܕతͳઃఆ •
తؔ! ͷΈ͕؍ଌՄೳ • ݪଇͱͯ͠ޯใؔͷੑ࣭ͳͲΛར༻Ͱ͖ͳ͍ • ؔධՁίετ͕ߴ͍ʢήʔϜγϛϡϨʔγϣϯϞσϧͷֶशʣ • తؔΛධՁͰ͖ΔճʹݶΓ͕͋Δ • ؍ଌϊΠζΛ͏ʢήʔϜͷ݁Ռֶशͷ݁Ռʹཚ͕Өڹʣ • ͏গ͠ϦονͳઃఆΛάϨΠϘοΫε࠷దԽͱݺͿ͜ͱ͕͋Δ • ϚϧνϑΟσϦςΟ࠷దԽ • ࢀߟɿGrey-box Bayesian Optimization for AutoML https://slideslive.com/38916582/keynote-greybox-bayesian- optimization-for-automl f(x) ϒϥοΫϘοΫε࠷దԽ
Copyright © GREE, Inc. All Rights Reserved. • ϕΠζ࠷దԽɾόϯσΟοτΞϧΰϦζϜ ػցֶशܥݚڀऀΒ͕ΜʹݚڀɼGP-EIɼSMACɼTPEͳͲ
• ਐԽܭࢉ Population-based methodsͱɼCMA-ESͳͲ • ୳ࡧ๏ Nelder–Mead๏ɼMADSͳͲ • اۀϒϥοΫϘοΫε࠷దԽιϑτΣΞ։ൃʹਚྗ • Google Vizier (Google) • Optuna (PFN) • Nevergrad (Facebook) ϒϥοΫϘοΫε࠷దԽख๏
Copyright © GREE, Inc. All Rights Reserved. • ϕΠζ࠷దԽɾόϯσΟοτΞϧΰϦζϜ ػցֶशܥݚڀऀΒ͕ΜʹݚڀɼGP-EIɼSMACɼTPEͳͲ
• ਐԽܭࢉ Population-based methodsͱɼCMA-ESͳͲ • ୳ࡧ๏ Nelder–Mead๏ɼMADSͳͲ • اۀϒϥοΫϘοΫε࠷దԽιϑτΣΞ։ൃʹਚྗ • Google Vizier (Google) • Optuna (PFN) • Nevergrad (Facebook) ϒϥοΫϘοΫε࠷దԽख๏
Copyright © GREE, Inc. All Rights Reserved. • ؔධՁͱ୯ମͷมܗΛ܁Γฦ͢ඍϑϦʔہॴ୳ࡧώϡʔϦεςΟοΫ •
ϋΠύύϥϝʔλ࠷దԽΛؚΉɼ࣮༻্ͷଟ͘ͷͰ্ख͘ಇ͘ (Cohen et al., 2005; Ozaki et al., 2017) Nelder–Mead๏ Nelder and Mead, 1965 CNNͷϋΠύύϥϝʔλ࠷దԽ (Ozaki et al., 2017)
Copyright © GREE, Inc. All Rights Reserved. Nelder–Mead๏ reflect, expand,
inside contract, outside contract, shrinkͷ5छྨͷૢ࡞Λ෮తʹద༻ reflect, expand, inside contract, outside contract shrink
Copyright © GREE, Inc. All Rights Reserved. • Nelder–Mead๏ͷ୳ࡧ֤ͷධՁʹج͖ͮஞ࣍తʹܾ·ΔͨΊɼ ͜ͷख๏ฒྻԽʹෆ͖Ͱ͋Γɼ࣮༻্େ͖ͳ՝
• తؔͷαϩήʔτ্ͰɼNelder–Mead๏Λ࣮ߦ͢ΔϞϯςΧϧϩ๏ʹ ΑΓɼධՁ͞ΕΔݟࠐΈͷߴ͍Λ༧ଌ͠ɼػతʹධՁ ༧ଌʹجͮ͘ฒྻධՁʹΑΔNelder–Mead๏ͷߴԽ Accelerating the Nelder–Mead Method with Predictive Parallel Evaluation Yoshihiko Ozaki, Shuhei Watanabe, and Masaki Onishi 6th ICML Workshop on Automated Machine Learning, Jun 2019. ! ΛԾఆ͠ɼ! Ψεաఔ͔ΒͷαϯϓϧΛද͢ f(x) ∼ GP(m(x), k(x, x′)) g(x)
Copyright © GREE, Inc. All Rights Reserved. 1.ॳظ୯ମʹؚ·ΕΔΛฒྻධՁ 2.ະධՁʹ౸ୡ͢Δ·Ͱɼଓ͖͔ΒNelder–Mead๏Λ࣮ߦ 3.ϞϯςΧϧϩ๏Λ࣮ߦ͠ɼػతʹධՁ͢ΔPݸͷީิΛٻΊɼฒྻධՁ
4.࠷దԽͷఀࢭ݅Λຬ͍ͨͯ͠Ε݁ՌΛฦ͠ɼͦ͏Ͱͳ͚Ε2.ʹΔ ༧ଌʹجͮ͘ฒྻධՁʹΑΔNelder–Mead๏ͷߴԽ ఏҊख๏ ! ΛԾఆ͠ɼ! Ψεաఔ͔ΒͷαϯϓϧΛද͢ f(x) ∼ GP(m(x), k(x, x′)) g(x)
Copyright © GREE, Inc. All Rights Reserved. •࣮ݧઃఆ • 6छྨͷϋΠύύϥϝʔλΛ࠷దԽ͢ΔϕϯνϚʔΫ
(Klein et al., 2018) • ฒྻ! Ͱݻఆ͠ɼઌಡΈΠςϨʔγϣϯ! Ͱ࣮ݧ • Baseline 1ɼॳظԽͱshrinkૢ࡞ͷΈฒྻධՁ (ࣗ໌ͳฒྻԽ) • Baseline 2ɼ࣍ΠςϨʔγϣϯͰධՁ͞ΕಘΔશͯͷΛฒྻධՁ •݁Ռ • Baseline 1ʹൺ49%ߴԽɼ2ʹൺ13%ߴԽ͔ͭগͳ͍ධՁ P = 10 J = 1,2,3,4,5 ༧ଌʹجͮ͘ฒྻධՁʹΑΔNelder–Mead๏ͷߴԽ ܭࢉ࣮ݧ Method J Average # of eval steps Average # of evaluations Baseline 1 - 590.27 (±141.42) 614.10 (±142.82) Baseline 2 - 347.27 (±89.32) 3469.67 (±893.21) Proposed 1 406.20 (±97.24) 1534.20 (±427.69) 2 314.13 (±72.26) 2307.83 (±558.02) 3 304.97 (±54.57) 2679.13 (±464.80) 4 310.60 (±67.58) 2948.20 (±642.62) 5 301.90 (±58.70) 2942.33 (±567.27)
Copyright © GREE, Inc. All Rights Reserved. • ฒྻ! ɼઌಡΈΠςϨʔγϣϯ!
Λ࣮ݧ • ߴԽͷޮՌ͋Δఔͷ! ·Ͱεέʔϧ͢Δ͕ɼͦΕҎ্མͪண͘ ʢઌͷΠςϨʔγϣϯʹͳΔ΄ͲɼධՁ͞ΕΔͷ༧ଌ͘͠ͳΔʣ P = 10,20,30,40 J = 1,2,3,4,5 P, J ༧ଌʹجͮ͘ฒྻධՁʹΑΔNelder–Mead๏ͷߴԽ ܭࢉ࣮ݧ
Copyright © GREE, Inc. All Rights Reserved. • ϒϥοΫϘοΫε࠷దԽۃΊͯ༗༻ •
ϋΠύύϥϝʔλ࠷దԽɼήʔϜͷύϥϝʔλࣗಈௐͳͲԠ༻ଟ • ٳܜ࣌ؒʹσΟεΧογϣϯܴ • 8݄5ͷKDD AutoML Workshopʹͯ࠷৽ͷݚڀʹ͍ͭͯൃද༧ఆ • Yoshihiko Ozaki and Masaki Onishi, “Practical Deep Neural Network Performance Prediction for Hyperparameter Optimization,” To appear. • https://sites.google.com/view/automl2019-workshop/ ·ͱΊ
Copyright © GREE, Inc. All Rights Reserved.