Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
ブラックボックス最適化とその応用
Search
gree_tech
PRO
July 19, 2019
Technology
0
340
ブラックボックス最適化とその応用
『CCSE2019』で発表された資料です。
https://ccse.jp/2019/
gree_tech
PRO
July 19, 2019
Tweet
Share
More Decks by gree_tech
See All by gree_tech
LLM翻訳ツールの開発と海外のお客様対応等への社内導入事例
gree_tech
PRO
0
850
ヘブンバーンズレッドのレンダリングパイプライン刷新
gree_tech
PRO
0
870
ヘブンバーンズレッドにおける、世界観を活かしたミニゲーム企画の作り方
gree_tech
PRO
0
860
「魔法少女まどか☆マギカ Magia Exedra」のグローバル展開を支える、開発チームと翻訳チームの「意識しない協創」を実現するローカライズシステム
gree_tech
PRO
0
850
「魔法少女まどか☆マギカ Magia Exedra」での負荷試験の実践と学び
gree_tech
PRO
0
940
「魔法少女まどか☆マギカ Magia Exedra」の必殺技演出を徹底解剖! -キャラクターの魅力を最大限にファンに届けるためのこだわり-
gree_tech
PRO
0
860
ヒューリスティック評価を用いたゲームQA実践事例
gree_tech
PRO
0
850
ライブサービスゲームQAのパフォーマンス検証による品質改善の取り組み
gree_tech
PRO
0
860
コミュニケーションに鍵を見いだす、エンジニア1年目の経験談
gree_tech
PRO
0
150
Other Decks in Technology
See All in Technology
[2025-09-30] Databricks Genie を利用した分析基盤とデータモデリングの IVRy の現在地
wxyzzz
0
480
pprof vs runtime/trace (FlightRecorder)
task4233
0
170
業務自動化プラットフォーム Google Agentspace に入門してみる #devio2025
maroon1st
0
190
生成AI_その前_に_マルチクラウド時代の信頼できるデータを支えるSnowflakeメタデータ活用術.pdf
cm_mikami
0
120
AI駆動開発を推進するためにサービス開発チームで 取り組んでいること
noayaoshiro
0
180
SOC2取得の全体像
shonansurvivors
1
390
Goに育てられ開発者向けセキュリティ事業を立ち上げた僕が今向き合う、AI × セキュリティの最前線 / Go Conference 2025
flatt_security
0
350
Goにおける 生成AIによるコード生成の ベンチマーク評価入門
daisuketakeda
2
100
OpenAI gpt-oss ファインチューニング入門
kmotohas
2
1k
Findy Team+のSOC2取得までの道のり
rvirus0817
0
340
コンテキストエンジニアリングとは? 考え方と応用方法
findy_eventslides
4
900
Oracle Base Database Service 技術詳細
oracle4engineer
PRO
11
77k
Featured
See All Featured
Faster Mobile Websites
deanohume
310
31k
Six Lessons from altMBA
skipperchong
28
4k
Unsuck your backbone
ammeep
671
58k
Measuring & Analyzing Core Web Vitals
bluesmoon
9
610
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
irinanazarova
9
960
Building Applications with DynamoDB
mza
96
6.6k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
33
2.5k
How To Stay Up To Date on Web Technology
chriscoyier
791
250k
4 Signs Your Business is Dying
shpigford
185
22k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
162
15k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
36
2.5k
Code Reviewing Like a Champion
maltzj
525
40k
Transcript
Copyright © GREE, Inc. All Rights Reserved. ϒϥοΫϘοΫε࠷దԽͱͦͷԠ༻ ඌ࡚ Յ
Copyright © GREE, Inc. All Rights Reserved. ॴଐ • άϦʔגࣜձࣾ
AIϦαʔννʔϜ ΤϯδχΞ • ࢈ۀٕज़૯߹ݚڀॴ ਓೳݚڀηϯλʔ ಛఆूதݚڀઐһʢ݉ʣ ݚڀ • ඍϑϦʔ࠷దԽɾϒϥοΫϘοΫε࠷దԽ • Automated Machine Learning (AutoML) ඌ࡚ Յ https://y0z.github.io/about/
Copyright © GREE, Inc. All Rights Reserved. • ԿΒ͔ͷతؔΛಛఆ੍ԼͰ࠷খԽʢͳ͍͠࠷େԽʣ͢Δ
! • Ұൠʹ! ʹؔͯ͠ಘΒΕΔใɼ͓͚ΔԾఆ͕ଟ͍΄Ͳޮతʹղ͚Δ • Ұ࣍ͷޯใɼೋ࣍ͷޯใ • ತੑɼϦϓγοπ࿈ଓੑɼྼϞδϡϥੑ Minimize f(x) subject to x ∈ X f(x) ཧ࠷దԽ
Copyright © GREE, Inc. All Rights Reserved. • ήʔϜͷόϥϯεΛࠨӈ͢Δύϥϝʔλͷࣗಈௐ •
ԿΒ͔ͷείΞ! (ྫ͑ɼউ)ήʔϜγϛϡϨʔλΛಈ࡞ͤ͞Δ ͜ͱͰಘΒΕΔ͕ɼ! ͷৄࡉखʹෛ͑ͳ͍΄Ͳෳࡶ • ػցֶशϞσϧͷϋΠύύϥϝʔλ࠷దԽ • AutoMLͷத৺త՝ͷ1ͭ (Feurer and Hutter, 2019) • Ϟσϧੑೳ! ͕࠷ྑͱͳΔϋΠύύϥϝʔλ! ͷ୳ࡧʢؔඇࣗ໌ʣ f(x) f(x) f(x) x ݱʹ”ϒϥοΫϘοΫε”͕ؔଟ
Copyright © GREE, Inc. All Rights Reserved. • యܕతͳઃఆ •
తؔ! ͷΈ͕؍ଌՄೳ • ݪଇͱͯ͠ޯใؔͷੑ࣭ͳͲΛར༻Ͱ͖ͳ͍ • ؔධՁίετ͕ߴ͍ʢήʔϜγϛϡϨʔγϣϯϞσϧͷֶशʣ • తؔΛධՁͰ͖ΔճʹݶΓ͕͋Δ • ؍ଌϊΠζΛ͏ʢήʔϜͷ݁Ռֶशͷ݁Ռʹཚ͕Өڹʣ • ͏গ͠ϦονͳઃఆΛάϨΠϘοΫε࠷దԽͱݺͿ͜ͱ͕͋Δ • ϚϧνϑΟσϦςΟ࠷దԽ • ࢀߟɿGrey-box Bayesian Optimization for AutoML https://slideslive.com/38916582/keynote-greybox-bayesian- optimization-for-automl f(x) ϒϥοΫϘοΫε࠷దԽ
Copyright © GREE, Inc. All Rights Reserved. • ϕΠζ࠷దԽɾόϯσΟοτΞϧΰϦζϜ ػցֶशܥݚڀऀΒ͕ΜʹݚڀɼGP-EIɼSMACɼTPEͳͲ
• ਐԽܭࢉ Population-based methodsͱɼCMA-ESͳͲ • ୳ࡧ๏ Nelder–Mead๏ɼMADSͳͲ • اۀϒϥοΫϘοΫε࠷దԽιϑτΣΞ։ൃʹਚྗ • Google Vizier (Google) • Optuna (PFN) • Nevergrad (Facebook) ϒϥοΫϘοΫε࠷దԽख๏
Copyright © GREE, Inc. All Rights Reserved. • ϕΠζ࠷దԽɾόϯσΟοτΞϧΰϦζϜ ػցֶशܥݚڀऀΒ͕ΜʹݚڀɼGP-EIɼSMACɼTPEͳͲ
• ਐԽܭࢉ Population-based methodsͱɼCMA-ESͳͲ • ୳ࡧ๏ Nelder–Mead๏ɼMADSͳͲ • اۀϒϥοΫϘοΫε࠷దԽιϑτΣΞ։ൃʹਚྗ • Google Vizier (Google) • Optuna (PFN) • Nevergrad (Facebook) ϒϥοΫϘοΫε࠷దԽख๏
Copyright © GREE, Inc. All Rights Reserved. • ؔධՁͱ୯ମͷมܗΛ܁Γฦ͢ඍϑϦʔہॴ୳ࡧώϡʔϦεςΟοΫ •
ϋΠύύϥϝʔλ࠷దԽΛؚΉɼ࣮༻্ͷଟ͘ͷͰ্ख͘ಇ͘ (Cohen et al., 2005; Ozaki et al., 2017) Nelder–Mead๏ Nelder and Mead, 1965 CNNͷϋΠύύϥϝʔλ࠷దԽ (Ozaki et al., 2017)
Copyright © GREE, Inc. All Rights Reserved. Nelder–Mead๏ reflect, expand,
inside contract, outside contract, shrinkͷ5छྨͷૢ࡞Λ෮తʹద༻ reflect, expand, inside contract, outside contract shrink
Copyright © GREE, Inc. All Rights Reserved. • Nelder–Mead๏ͷ୳ࡧ֤ͷධՁʹج͖ͮஞ࣍తʹܾ·ΔͨΊɼ ͜ͷख๏ฒྻԽʹෆ͖Ͱ͋Γɼ࣮༻্େ͖ͳ՝
• తؔͷαϩήʔτ্ͰɼNelder–Mead๏Λ࣮ߦ͢ΔϞϯςΧϧϩ๏ʹ ΑΓɼධՁ͞ΕΔݟࠐΈͷߴ͍Λ༧ଌ͠ɼػతʹධՁ ༧ଌʹجͮ͘ฒྻධՁʹΑΔNelder–Mead๏ͷߴԽ Accelerating the Nelder–Mead Method with Predictive Parallel Evaluation Yoshihiko Ozaki, Shuhei Watanabe, and Masaki Onishi 6th ICML Workshop on Automated Machine Learning, Jun 2019. ! ΛԾఆ͠ɼ! Ψεաఔ͔ΒͷαϯϓϧΛද͢ f(x) ∼ GP(m(x), k(x, x′)) g(x)
Copyright © GREE, Inc. All Rights Reserved. 1.ॳظ୯ମʹؚ·ΕΔΛฒྻධՁ 2.ະධՁʹ౸ୡ͢Δ·Ͱɼଓ͖͔ΒNelder–Mead๏Λ࣮ߦ 3.ϞϯςΧϧϩ๏Λ࣮ߦ͠ɼػతʹධՁ͢ΔPݸͷީิΛٻΊɼฒྻධՁ
4.࠷దԽͷఀࢭ݅Λຬ͍ͨͯ͠Ε݁ՌΛฦ͠ɼͦ͏Ͱͳ͚Ε2.ʹΔ ༧ଌʹجͮ͘ฒྻධՁʹΑΔNelder–Mead๏ͷߴԽ ఏҊख๏ ! ΛԾఆ͠ɼ! Ψεաఔ͔ΒͷαϯϓϧΛද͢ f(x) ∼ GP(m(x), k(x, x′)) g(x)
Copyright © GREE, Inc. All Rights Reserved. •࣮ݧઃఆ • 6छྨͷϋΠύύϥϝʔλΛ࠷దԽ͢ΔϕϯνϚʔΫ
(Klein et al., 2018) • ฒྻ! Ͱݻఆ͠ɼઌಡΈΠςϨʔγϣϯ! Ͱ࣮ݧ • Baseline 1ɼॳظԽͱshrinkૢ࡞ͷΈฒྻධՁ (ࣗ໌ͳฒྻԽ) • Baseline 2ɼ࣍ΠςϨʔγϣϯͰධՁ͞ΕಘΔશͯͷΛฒྻධՁ •݁Ռ • Baseline 1ʹൺ49%ߴԽɼ2ʹൺ13%ߴԽ͔ͭগͳ͍ධՁ P = 10 J = 1,2,3,4,5 ༧ଌʹجͮ͘ฒྻධՁʹΑΔNelder–Mead๏ͷߴԽ ܭࢉ࣮ݧ Method J Average # of eval steps Average # of evaluations Baseline 1 - 590.27 (±141.42) 614.10 (±142.82) Baseline 2 - 347.27 (±89.32) 3469.67 (±893.21) Proposed 1 406.20 (±97.24) 1534.20 (±427.69) 2 314.13 (±72.26) 2307.83 (±558.02) 3 304.97 (±54.57) 2679.13 (±464.80) 4 310.60 (±67.58) 2948.20 (±642.62) 5 301.90 (±58.70) 2942.33 (±567.27)
Copyright © GREE, Inc. All Rights Reserved. • ฒྻ! ɼઌಡΈΠςϨʔγϣϯ!
Λ࣮ݧ • ߴԽͷޮՌ͋Δఔͷ! ·Ͱεέʔϧ͢Δ͕ɼͦΕҎ্མͪண͘ ʢઌͷΠςϨʔγϣϯʹͳΔ΄ͲɼධՁ͞ΕΔͷ༧ଌ͘͠ͳΔʣ P = 10,20,30,40 J = 1,2,3,4,5 P, J ༧ଌʹجͮ͘ฒྻධՁʹΑΔNelder–Mead๏ͷߴԽ ܭࢉ࣮ݧ
Copyright © GREE, Inc. All Rights Reserved. • ϒϥοΫϘοΫε࠷దԽۃΊͯ༗༻ •
ϋΠύύϥϝʔλ࠷దԽɼήʔϜͷύϥϝʔλࣗಈௐͳͲԠ༻ଟ • ٳܜ࣌ؒʹσΟεΧογϣϯܴ • 8݄5ͷKDD AutoML Workshopʹͯ࠷৽ͷݚڀʹ͍ͭͯൃද༧ఆ • Yoshihiko Ozaki and Masaki Onishi, “Practical Deep Neural Network Performance Prediction for Hyperparameter Optimization,” To appear. • https://sites.google.com/view/automl2019-workshop/ ·ͱΊ
Copyright © GREE, Inc. All Rights Reserved.