Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
ブラックボックス最適化とその応用
Search
gree_tech
PRO
July 19, 2019
Technology
0
350
ブラックボックス最適化とその応用
『CCSE2019』で発表された資料です。
https://ccse.jp/2019/
gree_tech
PRO
July 19, 2019
Tweet
Share
More Decks by gree_tech
See All by gree_tech
今この時代に技術とどう向き合うべきか
gree_tech
PRO
2
2.2k
生成AIを開発組織にインストールするために: REALITYにおけるガバナンス・技術・文化へのアプローチ
gree_tech
PRO
0
75
安く・手軽に・現場発 既存資産を生かすSlack×AI検索Botの作り方
gree_tech
PRO
0
66
生成AIを安心して活用するために──「情報セキュリティガイドライン」策定とポイント
gree_tech
PRO
1
460
あうもんと学ぶGenAIOps
gree_tech
PRO
0
55
MVP開発における生成AIの活用と導入事例
gree_tech
PRO
0
74
機械学習・生成AIが拓く事業価値創出の最前線
gree_tech
PRO
0
95
コンテンツモデレーションにおける適切な監査範囲の考察
gree_tech
PRO
0
50
新サービス立ち上げの裏側 - QUANT for Shopsで実践した開発から運用まで
gree_tech
PRO
0
60
Other Decks in Technology
See All in Technology
アウトプットから始めるOSSコントリビューション 〜eslint-plugin-vueの場合〜 #vuefes
bengo4com
3
1.8k
20251027_findyさん_音声エージェントLT
almondo_event
2
480
webpack依存からの脱却!快適フロントエンド開発をViteで実現する #vuefes
bengo4com
4
3.6k
SCONE - 動画配信の帯域を最適化する新プロトコル
kazuho
1
400
abema-trace-sampling-observability-cost-optimization
tetsuya28
0
220
プロダクト開発と社内データ活用での、BI×AIの現在地 / Data_Findy
sansan_randd
1
540
様々なファイルシステム
sat
PRO
0
260
仕様駆動開発を実現する上流工程におけるAIエージェント活用
sergicalsix
2
540
東京大学「Agile-X」のFPGA AIデザインハッカソンを制したソニーのAI最適化
sony
0
140
20251024_TROCCO/COMETAアップデート紹介といくつかデモもやります!_#p_UG 東京:データ活用が進む組織の作り方
soysoysoyb
0
120
頭部ふわふわ浄酔器
uyupun
0
230
SOTA競争から人間を超える画像認識へ
shinya7y
0
600
Featured
See All Featured
What’s in a name? Adding method to the madness
productmarketing
PRO
24
3.7k
How to train your dragon (web standard)
notwaldorf
97
6.3k
Six Lessons from altMBA
skipperchong
29
4k
The Pragmatic Product Professional
lauravandoore
36
7k
A better future with KSS
kneath
239
18k
It's Worth the Effort
3n
187
28k
Connecting the Dots Between Site Speed, User Experience & Your Business [WebExpo 2025]
tammyeverts
10
620
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
panda_program
116
20k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
31
2.7k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
132
19k
Stop Working from a Prison Cell
hatefulcrawdad
272
21k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
PRO
190
55k
Transcript
Copyright © GREE, Inc. All Rights Reserved. ϒϥοΫϘοΫε࠷దԽͱͦͷԠ༻ ඌ࡚ Յ
Copyright © GREE, Inc. All Rights Reserved. ॴଐ • άϦʔגࣜձࣾ
AIϦαʔννʔϜ ΤϯδχΞ • ࢈ۀٕज़૯߹ݚڀॴ ਓೳݚڀηϯλʔ ಛఆूதݚڀઐһʢ݉ʣ ݚڀ • ඍϑϦʔ࠷దԽɾϒϥοΫϘοΫε࠷దԽ • Automated Machine Learning (AutoML) ඌ࡚ Յ https://y0z.github.io/about/
Copyright © GREE, Inc. All Rights Reserved. • ԿΒ͔ͷతؔΛಛఆ੍ԼͰ࠷খԽʢͳ͍͠࠷େԽʣ͢Δ
! • Ұൠʹ! ʹؔͯ͠ಘΒΕΔใɼ͓͚ΔԾఆ͕ଟ͍΄Ͳޮతʹղ͚Δ • Ұ࣍ͷޯใɼೋ࣍ͷޯใ • ತੑɼϦϓγοπ࿈ଓੑɼྼϞδϡϥੑ Minimize f(x) subject to x ∈ X f(x) ཧ࠷దԽ
Copyright © GREE, Inc. All Rights Reserved. • ήʔϜͷόϥϯεΛࠨӈ͢Δύϥϝʔλͷࣗಈௐ •
ԿΒ͔ͷείΞ! (ྫ͑ɼউ)ήʔϜγϛϡϨʔλΛಈ࡞ͤ͞Δ ͜ͱͰಘΒΕΔ͕ɼ! ͷৄࡉखʹෛ͑ͳ͍΄Ͳෳࡶ • ػցֶशϞσϧͷϋΠύύϥϝʔλ࠷దԽ • AutoMLͷத৺త՝ͷ1ͭ (Feurer and Hutter, 2019) • Ϟσϧੑೳ! ͕࠷ྑͱͳΔϋΠύύϥϝʔλ! ͷ୳ࡧʢؔඇࣗ໌ʣ f(x) f(x) f(x) x ݱʹ”ϒϥοΫϘοΫε”͕ؔଟ
Copyright © GREE, Inc. All Rights Reserved. • యܕతͳઃఆ •
తؔ! ͷΈ͕؍ଌՄೳ • ݪଇͱͯ͠ޯใؔͷੑ࣭ͳͲΛར༻Ͱ͖ͳ͍ • ؔධՁίετ͕ߴ͍ʢήʔϜγϛϡϨʔγϣϯϞσϧͷֶशʣ • తؔΛධՁͰ͖ΔճʹݶΓ͕͋Δ • ؍ଌϊΠζΛ͏ʢήʔϜͷ݁Ռֶशͷ݁Ռʹཚ͕Өڹʣ • ͏গ͠ϦονͳઃఆΛάϨΠϘοΫε࠷దԽͱݺͿ͜ͱ͕͋Δ • ϚϧνϑΟσϦςΟ࠷దԽ • ࢀߟɿGrey-box Bayesian Optimization for AutoML https://slideslive.com/38916582/keynote-greybox-bayesian- optimization-for-automl f(x) ϒϥοΫϘοΫε࠷దԽ
Copyright © GREE, Inc. All Rights Reserved. • ϕΠζ࠷దԽɾόϯσΟοτΞϧΰϦζϜ ػցֶशܥݚڀऀΒ͕ΜʹݚڀɼGP-EIɼSMACɼTPEͳͲ
• ਐԽܭࢉ Population-based methodsͱɼCMA-ESͳͲ • ୳ࡧ๏ Nelder–Mead๏ɼMADSͳͲ • اۀϒϥοΫϘοΫε࠷దԽιϑτΣΞ։ൃʹਚྗ • Google Vizier (Google) • Optuna (PFN) • Nevergrad (Facebook) ϒϥοΫϘοΫε࠷దԽख๏
Copyright © GREE, Inc. All Rights Reserved. • ϕΠζ࠷దԽɾόϯσΟοτΞϧΰϦζϜ ػցֶशܥݚڀऀΒ͕ΜʹݚڀɼGP-EIɼSMACɼTPEͳͲ
• ਐԽܭࢉ Population-based methodsͱɼCMA-ESͳͲ • ୳ࡧ๏ Nelder–Mead๏ɼMADSͳͲ • اۀϒϥοΫϘοΫε࠷దԽιϑτΣΞ։ൃʹਚྗ • Google Vizier (Google) • Optuna (PFN) • Nevergrad (Facebook) ϒϥοΫϘοΫε࠷దԽख๏
Copyright © GREE, Inc. All Rights Reserved. • ؔධՁͱ୯ମͷมܗΛ܁Γฦ͢ඍϑϦʔہॴ୳ࡧώϡʔϦεςΟοΫ •
ϋΠύύϥϝʔλ࠷దԽΛؚΉɼ࣮༻্ͷଟ͘ͷͰ্ख͘ಇ͘ (Cohen et al., 2005; Ozaki et al., 2017) Nelder–Mead๏ Nelder and Mead, 1965 CNNͷϋΠύύϥϝʔλ࠷దԽ (Ozaki et al., 2017)
Copyright © GREE, Inc. All Rights Reserved. Nelder–Mead๏ reflect, expand,
inside contract, outside contract, shrinkͷ5छྨͷૢ࡞Λ෮తʹద༻ reflect, expand, inside contract, outside contract shrink
Copyright © GREE, Inc. All Rights Reserved. • Nelder–Mead๏ͷ୳ࡧ֤ͷධՁʹج͖ͮஞ࣍తʹܾ·ΔͨΊɼ ͜ͷख๏ฒྻԽʹෆ͖Ͱ͋Γɼ࣮༻্େ͖ͳ՝
• తؔͷαϩήʔτ্ͰɼNelder–Mead๏Λ࣮ߦ͢ΔϞϯςΧϧϩ๏ʹ ΑΓɼධՁ͞ΕΔݟࠐΈͷߴ͍Λ༧ଌ͠ɼػతʹධՁ ༧ଌʹجͮ͘ฒྻධՁʹΑΔNelder–Mead๏ͷߴԽ Accelerating the Nelder–Mead Method with Predictive Parallel Evaluation Yoshihiko Ozaki, Shuhei Watanabe, and Masaki Onishi 6th ICML Workshop on Automated Machine Learning, Jun 2019. ! ΛԾఆ͠ɼ! Ψεաఔ͔ΒͷαϯϓϧΛද͢ f(x) ∼ GP(m(x), k(x, x′)) g(x)
Copyright © GREE, Inc. All Rights Reserved. 1.ॳظ୯ମʹؚ·ΕΔΛฒྻධՁ 2.ະධՁʹ౸ୡ͢Δ·Ͱɼଓ͖͔ΒNelder–Mead๏Λ࣮ߦ 3.ϞϯςΧϧϩ๏Λ࣮ߦ͠ɼػతʹධՁ͢ΔPݸͷީิΛٻΊɼฒྻධՁ
4.࠷దԽͷఀࢭ݅Λຬ͍ͨͯ͠Ε݁ՌΛฦ͠ɼͦ͏Ͱͳ͚Ε2.ʹΔ ༧ଌʹجͮ͘ฒྻධՁʹΑΔNelder–Mead๏ͷߴԽ ఏҊख๏ ! ΛԾఆ͠ɼ! Ψεաఔ͔ΒͷαϯϓϧΛද͢ f(x) ∼ GP(m(x), k(x, x′)) g(x)
Copyright © GREE, Inc. All Rights Reserved. •࣮ݧઃఆ • 6छྨͷϋΠύύϥϝʔλΛ࠷దԽ͢ΔϕϯνϚʔΫ
(Klein et al., 2018) • ฒྻ! Ͱݻఆ͠ɼઌಡΈΠςϨʔγϣϯ! Ͱ࣮ݧ • Baseline 1ɼॳظԽͱshrinkૢ࡞ͷΈฒྻධՁ (ࣗ໌ͳฒྻԽ) • Baseline 2ɼ࣍ΠςϨʔγϣϯͰධՁ͞ΕಘΔશͯͷΛฒྻධՁ •݁Ռ • Baseline 1ʹൺ49%ߴԽɼ2ʹൺ13%ߴԽ͔ͭগͳ͍ධՁ P = 10 J = 1,2,3,4,5 ༧ଌʹجͮ͘ฒྻධՁʹΑΔNelder–Mead๏ͷߴԽ ܭࢉ࣮ݧ Method J Average # of eval steps Average # of evaluations Baseline 1 - 590.27 (±141.42) 614.10 (±142.82) Baseline 2 - 347.27 (±89.32) 3469.67 (±893.21) Proposed 1 406.20 (±97.24) 1534.20 (±427.69) 2 314.13 (±72.26) 2307.83 (±558.02) 3 304.97 (±54.57) 2679.13 (±464.80) 4 310.60 (±67.58) 2948.20 (±642.62) 5 301.90 (±58.70) 2942.33 (±567.27)
Copyright © GREE, Inc. All Rights Reserved. • ฒྻ! ɼઌಡΈΠςϨʔγϣϯ!
Λ࣮ݧ • ߴԽͷޮՌ͋Δఔͷ! ·Ͱεέʔϧ͢Δ͕ɼͦΕҎ্མͪண͘ ʢઌͷΠςϨʔγϣϯʹͳΔ΄ͲɼධՁ͞ΕΔͷ༧ଌ͘͠ͳΔʣ P = 10,20,30,40 J = 1,2,3,4,5 P, J ༧ଌʹجͮ͘ฒྻධՁʹΑΔNelder–Mead๏ͷߴԽ ܭࢉ࣮ݧ
Copyright © GREE, Inc. All Rights Reserved. • ϒϥοΫϘοΫε࠷దԽۃΊͯ༗༻ •
ϋΠύύϥϝʔλ࠷దԽɼήʔϜͷύϥϝʔλࣗಈௐͳͲԠ༻ଟ • ٳܜ࣌ؒʹσΟεΧογϣϯܴ • 8݄5ͷKDD AutoML Workshopʹͯ࠷৽ͷݚڀʹ͍ͭͯൃද༧ఆ • Yoshihiko Ozaki and Masaki Onishi, “Practical Deep Neural Network Performance Prediction for Hyperparameter Optimization,” To appear. • https://sites.google.com/view/automl2019-workshop/ ·ͱΊ
Copyright © GREE, Inc. All Rights Reserved.