Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
ブラックボックス最適化とその応用
Search
gree_tech
PRO
July 19, 2019
Technology
0
270
ブラックボックス最適化とその応用
『CCSE2019』で発表された資料です。
https://ccse.jp/2019/
gree_tech
PRO
July 19, 2019
Tweet
Share
More Decks by gree_tech
See All by gree_tech
コミュニケーションに鍵を見いだす、エンジニア1年目の経験談
gree_tech
PRO
0
120
REALITY株式会社における開発生産性向上の取り組み: 失敗と成功から学んだこと
gree_tech
PRO
2
1.7k
『ヘブンバーンズレッド』におけるフィールドギミックの裏側
gree_tech
PRO
2
560
セキュリティインシデント対応の体制・運用の試行錯誤 / greetechcon2024-session-a1
gree_tech
PRO
1
560
『アナザーエデン 時空を超える猫』国内海外同時運営実現への道のり ~別々で開発されたアプリを安定して同時リリースするまでの取り組み~
gree_tech
PRO
1
530
『アサルトリリィ Last Bullet』におけるクラウドストリーミング技術を用いたブラウザゲーム化の紹介
gree_tech
PRO
1
620
UnityによるPCアプリの新しい選択肢。「PC版 Google Play Games」への対応について
gree_tech
PRO
1
970
実機ビルドのエラーによる検証ブロッカーを0に!『ヘブンバーンズレッド』のスモークテスト自動化の取り組み
gree_tech
PRO
1
640
"ゲームQA業界の技術向上を目指す! 会社を超えた研究会の取り組み"
gree_tech
PRO
1
760
Other Decks in Technology
See All in Technology
Amazon S3標準/ S3 Tables/S3 Express One Zoneを使ったログ分析
shigeruoda
5
590
変化する開発、進化する体系時代に適応するソフトウェアエンジニアの知識と考え方(JaSST'25 Kansai)
mizunori
1
260
無意味な開発生産性の議論から抜け出すための予兆検知とお金とAI
i35_267
0
860
AIとともに進化するエンジニアリング / Engineering-Evolving-with-AI_final.pdf
lycorptech_jp
PRO
0
140
Tech-Verse 2025 Keynote
lycorptech_jp
PRO
0
1.3k
Github Copilot エージェントモードで試してみた
ochtum
0
130
CursorによるPMO業務の代替 / Automating PMO Tasks with Cursor
motoyoshi_kakaku
2
780
生成AI開発案件におけるClineの業務活用事例とTips
shinya337
0
180
生まれ変わった AWS Security Hub (Preview) を紹介 #reInforce_osaka / reInforce New Security Hub
masahirokawahara
0
360
PHPでWebブラウザのレンダリングエンジンを実装する
dip_tech
PRO
0
220
ハッカソン by 生成AIハッカソンvol.05
1ftseabass
PRO
0
130
asken AI勉強会(Android)
tadashi_sato
0
140
Featured
See All Featured
Keith and Marios Guide to Fast Websites
keithpitt
411
22k
Designing for Performance
lara
609
69k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
161
15k
How to train your dragon (web standard)
notwaldorf
94
6.1k
The Art of Programming - Codeland 2020
erikaheidi
54
13k
Gamification - CAS2011
davidbonilla
81
5.3k
Java REST API Framework Comparison - PWX 2021
mraible
31
8.7k
The Language of Interfaces
destraynor
158
25k
Fantastic passwords and where to find them - at NoRuKo
philnash
51
3.3k
The Invisible Side of Design
smashingmag
300
51k
Designing Experiences People Love
moore
142
24k
Six Lessons from altMBA
skipperchong
28
3.9k
Transcript
Copyright © GREE, Inc. All Rights Reserved. ϒϥοΫϘοΫε࠷దԽͱͦͷԠ༻ ඌ࡚ Յ
Copyright © GREE, Inc. All Rights Reserved. ॴଐ • άϦʔגࣜձࣾ
AIϦαʔννʔϜ ΤϯδχΞ • ࢈ۀٕज़૯߹ݚڀॴ ਓೳݚڀηϯλʔ ಛఆूதݚڀઐһʢ݉ʣ ݚڀ • ඍϑϦʔ࠷దԽɾϒϥοΫϘοΫε࠷దԽ • Automated Machine Learning (AutoML) ඌ࡚ Յ https://y0z.github.io/about/
Copyright © GREE, Inc. All Rights Reserved. • ԿΒ͔ͷతؔΛಛఆ੍ԼͰ࠷খԽʢͳ͍͠࠷େԽʣ͢Δ
! • Ұൠʹ! ʹؔͯ͠ಘΒΕΔใɼ͓͚ΔԾఆ͕ଟ͍΄Ͳޮతʹղ͚Δ • Ұ࣍ͷޯใɼೋ࣍ͷޯใ • ತੑɼϦϓγοπ࿈ଓੑɼྼϞδϡϥੑ Minimize f(x) subject to x ∈ X f(x) ཧ࠷దԽ
Copyright © GREE, Inc. All Rights Reserved. • ήʔϜͷόϥϯεΛࠨӈ͢Δύϥϝʔλͷࣗಈௐ •
ԿΒ͔ͷείΞ! (ྫ͑ɼউ)ήʔϜγϛϡϨʔλΛಈ࡞ͤ͞Δ ͜ͱͰಘΒΕΔ͕ɼ! ͷৄࡉखʹෛ͑ͳ͍΄Ͳෳࡶ • ػցֶशϞσϧͷϋΠύύϥϝʔλ࠷దԽ • AutoMLͷத৺త՝ͷ1ͭ (Feurer and Hutter, 2019) • Ϟσϧੑೳ! ͕࠷ྑͱͳΔϋΠύύϥϝʔλ! ͷ୳ࡧʢؔඇࣗ໌ʣ f(x) f(x) f(x) x ݱʹ”ϒϥοΫϘοΫε”͕ؔଟ
Copyright © GREE, Inc. All Rights Reserved. • యܕతͳઃఆ •
తؔ! ͷΈ͕؍ଌՄೳ • ݪଇͱͯ͠ޯใؔͷੑ࣭ͳͲΛར༻Ͱ͖ͳ͍ • ؔධՁίετ͕ߴ͍ʢήʔϜγϛϡϨʔγϣϯϞσϧͷֶशʣ • తؔΛධՁͰ͖ΔճʹݶΓ͕͋Δ • ؍ଌϊΠζΛ͏ʢήʔϜͷ݁Ռֶशͷ݁Ռʹཚ͕Өڹʣ • ͏গ͠ϦονͳઃఆΛάϨΠϘοΫε࠷దԽͱݺͿ͜ͱ͕͋Δ • ϚϧνϑΟσϦςΟ࠷దԽ • ࢀߟɿGrey-box Bayesian Optimization for AutoML https://slideslive.com/38916582/keynote-greybox-bayesian- optimization-for-automl f(x) ϒϥοΫϘοΫε࠷దԽ
Copyright © GREE, Inc. All Rights Reserved. • ϕΠζ࠷దԽɾόϯσΟοτΞϧΰϦζϜ ػցֶशܥݚڀऀΒ͕ΜʹݚڀɼGP-EIɼSMACɼTPEͳͲ
• ਐԽܭࢉ Population-based methodsͱɼCMA-ESͳͲ • ୳ࡧ๏ Nelder–Mead๏ɼMADSͳͲ • اۀϒϥοΫϘοΫε࠷దԽιϑτΣΞ։ൃʹਚྗ • Google Vizier (Google) • Optuna (PFN) • Nevergrad (Facebook) ϒϥοΫϘοΫε࠷దԽख๏
Copyright © GREE, Inc. All Rights Reserved. • ϕΠζ࠷దԽɾόϯσΟοτΞϧΰϦζϜ ػցֶशܥݚڀऀΒ͕ΜʹݚڀɼGP-EIɼSMACɼTPEͳͲ
• ਐԽܭࢉ Population-based methodsͱɼCMA-ESͳͲ • ୳ࡧ๏ Nelder–Mead๏ɼMADSͳͲ • اۀϒϥοΫϘοΫε࠷దԽιϑτΣΞ։ൃʹਚྗ • Google Vizier (Google) • Optuna (PFN) • Nevergrad (Facebook) ϒϥοΫϘοΫε࠷దԽख๏
Copyright © GREE, Inc. All Rights Reserved. • ؔධՁͱ୯ମͷมܗΛ܁Γฦ͢ඍϑϦʔہॴ୳ࡧώϡʔϦεςΟοΫ •
ϋΠύύϥϝʔλ࠷దԽΛؚΉɼ࣮༻্ͷଟ͘ͷͰ্ख͘ಇ͘ (Cohen et al., 2005; Ozaki et al., 2017) Nelder–Mead๏ Nelder and Mead, 1965 CNNͷϋΠύύϥϝʔλ࠷దԽ (Ozaki et al., 2017)
Copyright © GREE, Inc. All Rights Reserved. Nelder–Mead๏ reflect, expand,
inside contract, outside contract, shrinkͷ5छྨͷૢ࡞Λ෮తʹద༻ reflect, expand, inside contract, outside contract shrink
Copyright © GREE, Inc. All Rights Reserved. • Nelder–Mead๏ͷ୳ࡧ֤ͷධՁʹج͖ͮஞ࣍తʹܾ·ΔͨΊɼ ͜ͷख๏ฒྻԽʹෆ͖Ͱ͋Γɼ࣮༻্େ͖ͳ՝
• తؔͷαϩήʔτ্ͰɼNelder–Mead๏Λ࣮ߦ͢ΔϞϯςΧϧϩ๏ʹ ΑΓɼධՁ͞ΕΔݟࠐΈͷߴ͍Λ༧ଌ͠ɼػతʹධՁ ༧ଌʹجͮ͘ฒྻධՁʹΑΔNelder–Mead๏ͷߴԽ Accelerating the Nelder–Mead Method with Predictive Parallel Evaluation Yoshihiko Ozaki, Shuhei Watanabe, and Masaki Onishi 6th ICML Workshop on Automated Machine Learning, Jun 2019. ! ΛԾఆ͠ɼ! Ψεաఔ͔ΒͷαϯϓϧΛද͢ f(x) ∼ GP(m(x), k(x, x′)) g(x)
Copyright © GREE, Inc. All Rights Reserved. 1.ॳظ୯ମʹؚ·ΕΔΛฒྻධՁ 2.ະධՁʹ౸ୡ͢Δ·Ͱɼଓ͖͔ΒNelder–Mead๏Λ࣮ߦ 3.ϞϯςΧϧϩ๏Λ࣮ߦ͠ɼػతʹධՁ͢ΔPݸͷީิΛٻΊɼฒྻධՁ
4.࠷దԽͷఀࢭ݅Λຬ͍ͨͯ͠Ε݁ՌΛฦ͠ɼͦ͏Ͱͳ͚Ε2.ʹΔ ༧ଌʹجͮ͘ฒྻධՁʹΑΔNelder–Mead๏ͷߴԽ ఏҊख๏ ! ΛԾఆ͠ɼ! Ψεաఔ͔ΒͷαϯϓϧΛද͢ f(x) ∼ GP(m(x), k(x, x′)) g(x)
Copyright © GREE, Inc. All Rights Reserved. •࣮ݧઃఆ • 6छྨͷϋΠύύϥϝʔλΛ࠷దԽ͢ΔϕϯνϚʔΫ
(Klein et al., 2018) • ฒྻ! Ͱݻఆ͠ɼઌಡΈΠςϨʔγϣϯ! Ͱ࣮ݧ • Baseline 1ɼॳظԽͱshrinkૢ࡞ͷΈฒྻධՁ (ࣗ໌ͳฒྻԽ) • Baseline 2ɼ࣍ΠςϨʔγϣϯͰධՁ͞ΕಘΔશͯͷΛฒྻධՁ •݁Ռ • Baseline 1ʹൺ49%ߴԽɼ2ʹൺ13%ߴԽ͔ͭগͳ͍ධՁ P = 10 J = 1,2,3,4,5 ༧ଌʹجͮ͘ฒྻධՁʹΑΔNelder–Mead๏ͷߴԽ ܭࢉ࣮ݧ Method J Average # of eval steps Average # of evaluations Baseline 1 - 590.27 (±141.42) 614.10 (±142.82) Baseline 2 - 347.27 (±89.32) 3469.67 (±893.21) Proposed 1 406.20 (±97.24) 1534.20 (±427.69) 2 314.13 (±72.26) 2307.83 (±558.02) 3 304.97 (±54.57) 2679.13 (±464.80) 4 310.60 (±67.58) 2948.20 (±642.62) 5 301.90 (±58.70) 2942.33 (±567.27)
Copyright © GREE, Inc. All Rights Reserved. • ฒྻ! ɼઌಡΈΠςϨʔγϣϯ!
Λ࣮ݧ • ߴԽͷޮՌ͋Δఔͷ! ·Ͱεέʔϧ͢Δ͕ɼͦΕҎ্མͪண͘ ʢઌͷΠςϨʔγϣϯʹͳΔ΄ͲɼධՁ͞ΕΔͷ༧ଌ͘͠ͳΔʣ P = 10,20,30,40 J = 1,2,3,4,5 P, J ༧ଌʹجͮ͘ฒྻධՁʹΑΔNelder–Mead๏ͷߴԽ ܭࢉ࣮ݧ
Copyright © GREE, Inc. All Rights Reserved. • ϒϥοΫϘοΫε࠷దԽۃΊͯ༗༻ •
ϋΠύύϥϝʔλ࠷దԽɼήʔϜͷύϥϝʔλࣗಈௐͳͲԠ༻ଟ • ٳܜ࣌ؒʹσΟεΧογϣϯܴ • 8݄5ͷKDD AutoML Workshopʹͯ࠷৽ͷݚڀʹ͍ͭͯൃද༧ఆ • Yoshihiko Ozaki and Masaki Onishi, “Practical Deep Neural Network Performance Prediction for Hyperparameter Optimization,” To appear. • https://sites.google.com/view/automl2019-workshop/ ·ͱΊ
Copyright © GREE, Inc. All Rights Reserved.