Share
Embed
入門 Automated Machine Learning
機械学習は現在様々な領域で活用されていますが、機械学習モデルの構築には高度な訓練を受けたエンジニアが必要です。しかし、近年、機械学習プロセスを自動化する技術である「Automated Machine Learning(AutoML)」が注目を集めています。AutoMLを使用することにより、プロセスを自動化し、生産性向上や専門的なエンジニアの助けなしにタスクに応じた機械学習モデルを構築できます。本資料では、従来の機械学習からAutoMLまでの道のりを紹介することで、AutoMLの可能性や課題について考察したいと思います。
Hiroki Nakayama
December 21, 2018
Transcript
73 slides
-
Table of contents Ұൠతͳػցֶशϓϩηε ͜͜ͰҰൠతͳػցֶशϓϩηεͰߦ ΘΕΔ͜ͱʹ͍ͭͯཧղΛڞ༗͠·͢ɻ AutoMLͱʁ ͜͜ͰAutoMLͱԿ͔ʹ͍ͭͯɺ ੜͨ͠എܠ͔Βઆ໌͠·͢ɻ AutoMLͰߦΘΕΔ͜ͱ
͜͜Ͱ֤ػցֶशϓϩηεͰैདྷߦΘΕ ͍ͯͨ͜ͱͱAutoMLͰߦΘΕΔ͜ͱΛ հ͠·͢ɻ AutoMLͷιϑτΣΞ ͜͜ͰAutoMLΛߦ͏ͨΊͷιϑτ ΣΞͱαʔϏεʹ͍ͭͯհ͠·͢ɻ AutoMLͷະདྷ ͜͜ͰAutoMLͷະདྷʹ͍ͭͯ ड़·͢ɻ ·ͱΊ ࠷ޙʹ͜Ε·Ͱͷ༰Λ·ͱΊ·͢ɻ -
Ұൠతͳػցֶशϓϩηε ग़య: Evaluation of a Tree-based Pipeline Optimization Tool for
Automating Data Science -
σʔλΫϦʔχϯά σʔλதͷෆશͰ͋ͬͨΓɺෆਖ਼֬ɺແؔͳ෦Λ আڈɾमਖ਼͢Δϓϩηε ςʔϒϧσʔλͷ߹ Gender Age City Income Male 18
Chicago $53,000 Female 25 Cicago $27,000 Female Chicago $89,000 Male 54 Tokyo ¥5,000,000 ܽଛ ϛεεϖϧ ҟͳΔ୯Ґ -
σʔλΫϦʔχϯάͷඞཁੑ ҰൠతʹɺσʔλΫϦʔχϯάΛͨ͠ํ͕Ϟσϧͷ ύϑΥʔϚϯε্͕͢Δɻ Gender Age City Income Male 18 Chicago
$53,000 Female 25 Cicago $27,000 Female Chicago $89,000 Male 54 Tokyo ¥5,000,000 ܽଛ ϛεεϖϧ ҟͳΔ୯Ґ ෆશɾෆਖ਼֬ͳσʔλͰਖ਼ֶ͘͠शͰ͖ͳ͍ -
Ϟσϧબͷඞཁੑ ग़య: Data-driven Advice for Applying Machine Learning to Bioinformatics
Problems Logistic Regression vs RF 10%ͷσʔληοτͰ Logistic Regression͕উར -
Ϟσϧબͷඞཁੑ ग़య: Data-driven Advice for Applying Machine Learning to Bioinformatics
Problems Ұ൪ऑ͍ϞσϧͰҰ൪ڧ͍ Ϟσϧʹউͭ͜ͱ͋ΔͷͰɺ༷ʑͳ ϞσϧΛߟྀ͢Δඞཁ͕͋Δ -
ϋΠύʔύϥϝʔλνϡʔχϯά ϞσϧͷϋΠύʔύϥϝʔλΛ࠷దͳʹௐ͢Δϓϩηε ɹɹɹɹˠֶशΛߦ͏લʹઃఆ͢Δύϥϝʔλ ϩδεςΟοΫճؼͷ߹ ϥϯμϜϑΥϨετͷ߹ RandomForestClassifier(n_estimators=’warn’, criterion=’gini’, max_depth=None, min_sa mples_split=2,
min_samples_leaf=1, min_weight_fraction_leaf=0.0, max_features=’auto’, max_leaf_nodes=None, min_impurity_decrease=0.0, min_impurity_split=None, bootstrap= True, oob_score=False, n_jobs=None, random_state=None, verbose=0, warm_start=False , class_weight=None) LogisticRegression(penalty=’l2’, dual=False, tol=0.0001, C=1.0, fit_intercept=True, intercept_scaling=1, class_weight=None, random_state=None, solver=’warn’, max_iter=100, multi_class=’warn’, verbose=0, warm_start=False, n_jobs=None) -
ϋΠύʔύϥϝʔλνϡʔχϯάͷඞཁੑ ग़య: Data-driven Advice for Applying Machine Learning to Bioinformatics
Problems νϡʔχϯάʹΑͬͯฏۉతʹ ਖ਼ղͰ3ʙ5%ఔͷվળ -
Table of contents Ұൠతͳػցֶशϓϩηε ͜͜ͰҰൠతͳػցֶशϓϩηεͰߦ ΘΕΔ͜ͱʹ͍ͭͯཧղΛڞ༗͠·͢ɻ AutoMLͱʁ ͜͜ͰAutoMLͱԿ͔ʹ͍ͭͯɺ ੜͨ͠എܠ͔Βઆ໌͠·͢ɻ AutoMLͰߦΘΕΔ͜ͱ
͜͜Ͱ֤ػցֶशϓϩηεͰैདྷߦΘΕ ͍ͯͨ͜ͱͱAutoMLͰߦΘΕΔ͜ͱΛ հ͠·͢ɻ AutoMLͷιϑτΣΞ ͜͜ͰAutoMLΛߦ͏ͨΊͷιϑτ ΣΞͱαʔϏεʹ͍ͭͯհ͠·͢ɻ AutoMLͷະདྷ ͜͜ͰAutoMLͷະདྷʹ͍ͭͯ ड़·͢ɻ ·ͱΊ ࠷ޙʹ͜Ε·Ͱͷ༰Λ·ͱΊ·͢ɻ -
AutoMLͱʁ AutoMLʢAutomated Machine LearningʣɺػցֶशϓϩηεΛ ࣗಈԽ͢ΔͨΊͷٕज़ ग़య: Evaluation of a Tree-based
Pipeline Optimization Tool for Automating Data Science -
ੜ࢈ੑ্͕ඞཁͳഎܠ ग़య: Study Shows That the Number of Data Scientists
Has Doubled in 4 Years 2ഒ σʔλαΠΤϯςΟετͷ ٸܹʹ૿͍͑ͯΔ -
ੜ࢈ੑ্͕ඞཁͳഎܠ ʢग़యʣMcKinsey Global InstituteʮBig data: The next frontier for innovation,
competition, and productivityʯ ٸܹʹ૿͍͑ͯΔ͕ धཁʹ͍͍͍ͭͯͳ͍ -
ػցֶशͷຽओԽͷഎܠ • ΞϧΰϦζϜ͕ίϞσΟςΟԽ͢ΔͱɺΞϧΰϦζϜͷ ੑೳͷࠩͰউෛ͢Δͷେม • ެ։͞Ε͍ͯΔΞϧΰϦζϜΛͬͯαʔϏεΛੜΈग़͢ ॏཁੑ͕૿͖ͯͨ͠ • ͨͩɺͦ͏͍ͬͨΞϧΰϦζϜPythonͷΑ͏ͳϓϩά ϥϛϯάݴޠΛѻ͑ͳ͍ͱ͑ͳ͍ɻதΛཧղ͠Α͏
ͱ͢Εߴͳֶͷ͕ࣝඞཁɻ • ͜͏͍ͬͨͷΛѻ͑ΔਓࡐΛͱͬͯ͘Δͷେม • ׆༻Λ͛ΔͨΊʹ؆୯ʹ͑ΔΑ͏ʹ͢Δඞཁ͕͋Δ -
Table of contents Ұൠతͳػցֶशϓϩηε ͜͜ͰҰൠతͳػցֶशϓϩηεͰߦ ΘΕΔ͜ͱʹ͍ͭͯཧղΛڞ༗͠·͢ɻ AutoMLͱʁ ͜͜ͰAutoMLͱԿ͔ʹ͍ͭͯɺ ੜͨ͠എܠ͔Βઆ໌͠·͢ɻ AutoMLͰߦΘΕΔ͜ͱ
͜͜Ͱ֤ػցֶशϓϩηεͰैདྷߦΘΕ ͍ͯͨ͜ͱͱAutoMLͰߦΘΕΔ͜ͱΛ հ͠·͢ɻ AutoMLͷιϑτΣΞ ͜͜ͰAutoMLΛߦ͏ͨΊͷιϑτ ΣΞͱαʔϏεʹ͍ͭͯհ͠·͢ɻ AutoMLͷະདྷ ͜͜ͰAutoMLͷະདྷʹ͍ͭͯ ड़·͢ɻ ·ͱΊ ࠷ޙʹ͜Ε·Ͱͷ༰Λ·ͱΊ·͢ɻ -
ϋΠύʔύϥϝʔλνϡʔχϯά ϞσϧͷϋΠύʔύϥϝʔλΛ࠷దͳʹௐ͢Δϓϩηε ɹɹɹɹˠֶशΛߦ͏લʹઃఆ͢Δύϥϝʔλ ϩδεςΟοΫճؼͷ߹ ϥϯμϜϑΥϨετͷ߹ RandomForestClassifier(n_estimators=’warn’, criterion=’gini’, max_depth=None, min_sa mples_split=2,
min_samples_leaf=1, min_weight_fraction_leaf=0.0, max_features=’auto’, max_leaf_nodes=None, min_impurity_decrease=0.0, min_impurity_split=None, bootstrap= True, oob_score=False, n_jobs=None, random_state=None, verbose=0, warm_start=False , class_weight=None) LogisticRegression(penalty=’l2’, dual=False, tol=0.0001, C=1.0, fit_intercept=True, intercept_scaling=1, class_weight=None, random_state=None, solver=’warn’, max_iter=100, multi_class=’warn’, verbose=0, warm_start=False, n_jobs=None) -
खಈʹΑΔνϡʔχϯάͷ 1 1 2 3 1 2 3 1 2
3 1 2 3 1 2 3 1 2 3 ộ ộ ộ ộ Parameter1 Parameter2 Parameter3 Parameter4 Parameter5 ϋΠύʔύϥϝʔλΛखಈͰνϡʔχϯά͢Δͷݱ࣮తͰͳ͍ ͨͱ͑ɺϋΠύʔύϥϝʔλ͕5ͭ͋Γɺ֤ϋΠύʔύϥϝʔλ ʹରͯ͠3ͭͷΛςετ͢Δ߹ɺΈ߹Θͤ3ͷ5ʢ=273ʣ ௨Γଘࡏ -
άϦουαʔν ͯ͢ͷΈ߹Θͤ (10, 0.1) (10, 0.2) (10, 0.5) (100, 0.1)
(100, 0.2) (100, 0.5) ֤ϋΠύʔύϥϝʔλͷީิΛෳ༻ҙͯ͠ɺͯ͢ͷ߹ͤΛࢼ͢ख๏ ͨͱ͑ɺC ͱ γ ͱ͍͏2ͭͷύϥϝʔλ͕͋ΓɺͦΕͧΕɺC∈ {10, 100}, γ∈ {0.1, 0.2, 0.5}ͱ͍͏ީิΛઃఆͨ͠߹ɺ2x3=6ͷΈ߹ΘͤΛࢼ͢ ग़య: Random Search for Hyper-Parameter Optimization -
ύϥϝʔλʹର͢ΔΛࢦఆ͠ɺ͔ͦ͜ΒΛαϯϓϦϯά͢Δख๏ ͨͱ͑ɺάϦουαʔνͰ γ ʹରͯ͠ γ∈ {0.1, 0.2, 0.5} ͷΑ͏ͳΛ ༩͍͑ͯͨͷʹରͯ͠ɺϥϯμϜαʔνͰɺࢦͷΑ͏ͳ֬Λ
༩͑ɺ͔ͦ͜ΒΛαϯϓϦϯά͠·͢ɻগͷϋΠύʔύϥϝʔλ͕ੑೳ ʹେ͖͘ӨڹΛ༩͑Δ߹ʹޮՌతͳख๏ ϥϯμϜαʔν ग़య: Random Search for Hyper-Parameter Optimization -
• ϕΠζ࠷దԽΛ༻͍ͨϋΠύʔύϥϝʔλνϡʔχϯάɺҎલͷ݁ՌΛ ͬͯ࣍ʹ୳ࡧ͢ΔϋΠύʔύϥϝʔλΛબͿ • ͜ΕʹΑΓɺ༗ͦ͏ͳͱ͜ΖΛத৺ʹϋΠύʔύϥϝʔλΛ୳ࡧͰ͖Δ ˠਓ͕ؒߦ͏୳ࡧʹ͍ۙ • σΟʔϓϥʔχϯάΛؚΉػցֶशͷϞσϧʹରͯ͠ɺൺֱతྑ͍ ϋΠύʔύϥϝʔλΛ୳ࡧͰ͖Δ͜ͱ͕ΒΕ͍ͯΔ ϕΠζ࠷దԽʹΑΔνϡʔχϯά
Optuna Hyperopt ϕΠζ࠷దԽʹΑΔϋΠύʔύϥϝʔλνϡʔχϯάΛߦ͑ΔOSS -
• χϡʔϥϧωοτϫʔΫͷΞʔΩςΫνϟΛઃܭ͢Δͷ ߴͳઐ͕ࣝඞཁͰඇৗʹ͍͠ • Α͍ΞʔΩςΫνϟΛ࡞Δʹࢼߦࡨޡ͕ඞཁͰɺ͓͔͔࣌ؒۚΔ • ΤϯδχΞͷڅྉ͕ߴ͍ • GPUϚγϯͷ༻ྉۚ •
ֶ͍श࣌ؒ • ͜ΕͰ׆༻Ͱ͖Δͷ͕গͷݚڀऀΤϯδχΞ͚ͩʹݶΒΕΔ ઃܭΛࣗಈԽ͍ͨ͠ཧ༝ -
χϡʔϥϧΞʔΩςΫνϟαʔνͷ՝ͱͯ͠ܭࢉྔͷଟ͕͋͞Δ →ωοτϫʔΫΛ࡞͢Δͨͼʹɺֶशͯ݁͠ՌΛ֬ೝ͢Δඞཁ͕͋Δ͔Β Neural Architecture Search with Reinforcement Learning 800 GPUͰ28ؒ
Learning Transferable Architectures for Scalable Image Recognition 500 GPU Ͱ4ؒ Ұൠͷݚڀऀ։ൃऀ͕ར༻͢Δͷݱ࣮తͰͳ͍ χϡʔϥϧΞʔΩςΫνϟαʔνͷ՝ -
खಈʹΑΔಛΤϯδχΞϦϯάͷ՝ ܟশ(Mr, Mrs, SirͳͲ)Λ நग़ͯ͠͏ ધͷ্ͷํͳͷ͔ɺ Լͷํͳͷ͔ ಛΛࢥ͍͍ͭͨΒ ऴΘΓͰͳ͍ ਓؒʹΑΔಛΤϯδχΞϦϯάͷ
→ྑ͍ಛΛࢥ͍ͭ͘ͷ͕͍͠ →ݕূ͢Δͷʹ͕͔͔࣌ؒΔ -
DataRobotͰͷํ๏ DataRobotͰΤΩεύʔτγεςϜΛߏங͢Δ͜ͱͰ ಛΤϯδχΞϦϯάΛࣗಈԽ͍ͯ͠Δ 1. ಛͷੜ 2. ಛΤϯδχΞϦϯά͕ඞཁͳϞσϧΛΔ 3. ֤Ϟσϧʹ༗ޮͳಛΤϯδχΞϦϯάͷछྨΛΔ 4.
γεςϚςΟοΫʹϞσϧΛൺֱͯ͠ɺಛΤϯδχΞϦ ϯάͱϞσϧͷ࠷ྑ͍Έ߹ΘͤΛΔ -
͜ΕΒͷૢ࡞ΛDataRobotͰ model blueprint Λͬͯߦ͍ͬͯ·͢ɻ͜͜ Ͱɺmodel blueprint ͱɺͪ͜ΒͷهࣄʹΑΔͱɺલॲཧɺಛΤϯδχΞ Ϧϯάɺֶशɺνϡʔχϯάͱ͍ͬͨॲཧͷγʔέϯεͷ͜ͱͷΑ͏Ͱ͢ɻ ҎԼ͕ model
blueprint ͷྫͰ͢ɻ DataRobotͰͷํ๏ ग़య: Automated Feature Engineering -
͜ΕΒͷૢ࡞ΛDataRobotͰ model blueprint Λͬͯߦ͍ͬͯ·͢ɻ͜͜ Ͱɺmodel blueprint ͱɺͪ͜ΒͷهࣄʹΑΔͱɺલॲཧɺಛΤϯδχΞ Ϧϯάɺֶशɺνϡʔχϯάͱ͍ͬͨॲཧͷγʔέϯεͷ͜ͱͷΑ͏Ͱ͢ɻ ҎԼ͕ model
blueprint ͷྫͰ͢ɻ DataRobotͰͷํ๏ ग़య: Automated Feature Engineering -
featuretools Ͱ Deep Feature Synthesis(DFS) ͱݺΕΔํ๏Ͱ৽ͨͳಛ Λੜ͍ͯ͠·͢ɻDFSͰ primitive ͱݺΕΔؔΛͬͯσʔλͷू ͱมΛߦ͍·͢ɻprimitive
ͷྫͱͯ͠ɺྻͷฏۉ࠷େΛऔΔؔ Λڍ͛Δ͜ͱ͕Ͱ͖·͢ɻ·ͨࣗͰఆٛͨؔ͠Λ primitive ͱͯ͠͏͜ ͱͰ͖·͢ɻ featuretoolsͰͷํ๏ -
Table of contents Ұൠతͳػցֶशϓϩηε ͜͜ͰҰൠతͳػցֶशϓϩηεͰߦ ΘΕΔ͜ͱʹ͍ͭͯཧղΛڞ༗͠·͢ɻ AutoMLͱʁ ͜͜ͰAutoMLͱԿ͔ʹ͍ͭͯɺ ੜͨ͠എܠ͔Βઆ໌͠·͢ɻ AutoMLͰߦΘΕΔ͜ͱ
͜͜Ͱ֤ػցֶशϓϩηεͰैདྷߦΘΕ ͍ͯͨ͜ͱͱAutoMLͰߦΘΕΔ͜ͱΛ հ͠·͢ɻ AutoMLͷιϑτΣΞ ͜͜ͰAutoMLΛߦ͏ͨΊͷιϑτ ΣΞͱαʔϏεʹ͍ͭͯհ͠·͢ɻ AutoMLͷະདྷ ͜͜ͰAutoMLͷະདྷʹ͍ͭͯ ड़·͢ɻ ·ͱΊ ࠷ޙʹ͜Ε·Ͱͷ༰Λ·ͱΊ·͢ɻ -
AutoMLͷιϑτΣΞ Auto-Keras scikit-learnϥΠΫͳΠϯλʔϑΣʔεͰ χϡʔϥϧΞʔΩςΫνϟαʔνΛߦ͑Δ ϥΠϒϥϦ auto-sklearn scikit-learnϥΠΫͳΠϯλʔϑΣʔεͰϞσ ϧબͱϋΠύʔύϥϝʔλνϡʔχϯά Λߦ͑ΔϥΠϒϥϦ optuna
ϋΠύʔύϥϝʔλνϡʔχϯάͷͨΊ ͷϥΠϒϥϦɻϕΠζ࠷దԽʹΑΔख ๏Λαϙʔτ TPOT scikit-learnϥΠΫͳΠϯλʔϑΣʔεͰϞσ ϧબͱϋΠύʔύϥϝʔλνϡʔχϯά Λߦ͑ΔϥΠϒϥϦ -
Table of contents Ұൠతͳػցֶशϓϩηε ͜͜ͰҰൠతͳػցֶशϓϩηεͰߦ ΘΕΔ͜ͱʹ͍ͭͯཧղΛڞ༗͠·͢ɻ AutoMLͱʁ ͜͜ͰAutoMLͱԿ͔ʹ͍ͭͯɺ ੜͨ͠എܠ͔Βઆ໌͠·͢ɻ AutoMLͰߦΘΕΔ͜ͱ
͜͜Ͱ֤ػցֶशϓϩηεͰैདྷߦΘΕ ͍ͯͨ͜ͱͱAutoMLͰߦΘΕΔ͜ͱΛ հ͠·͢ɻ AutoMLͷιϑτΣΞ ͜͜ͰAutoMLΛߦ͏ͨΊͷιϑτ ΣΞͱαʔϏεʹ͍ͭͯհ͠·͢ɻ AutoMLͷະདྷ ͜͜ͰAutoMLͷະདྷʹ͍ͭͯ ड़·͢ɻ ·ͱΊ ࠷ޙʹ͜Ε·Ͱͷ༰Λ·ͱΊ·͢ɻ -
AutoMLͷະདྷ σʔλΫϦʔχϯάՄೳʹͳΔ ͨͱ͑ɺςΩετͷΑ͏ͳඇߏԽσʔλΛ ੳʹ͙͢ʹ͑ΔΑ͏ʹςʔϒϧσʔλʹม͢Δ େنσʔλʹεέʔϧ͢Δ ݱࡏαϯϓϧͷখ͞ͳσʔλʹରͯ͠Ͱ͑͞ܭࢉ͕࣌ؒ ݁ߏ͔͔Δɻকདྷతʹ͍ΘΏΔϏοάσʔλʹରͯ͠ ͑ΔΑ͏ʹͳΔͰ͠ΐ͏ɻ ੑೳ͕ਓؒΛ্ճΔ ݱࡏͰҰ෦ͷσʔληοτͰਓؒʹඖఢ͢ΔੑೳΛग़͍ͯ͠·͕͢ɺকདྷ
తʹਓ͕ؒߟ͔͑ͭͳ͍Α͏ͳಛͰ͋Δͱ͔ωοτϫʔΫΞʔΩςΫνϟ ΛੜΈग़ͤΔΑ͏ʹͳΔͰ͠ΐ͏ -
ࢀߟࢿྉ ಛΤϯδχΞϦϯά • Why Automated Feature Engineering Will Change the
Way You Do Machine Learning • Deep Feature Synthesis: How Automated Feature Engineering Works • Automated Feature Engineering χϡʔϥϧΞʔΩςΫνϟαʔν • Neural Architecture Search with Reinforcement Learning • Learning Transferable Architectures for Scalable Image Recognition • Efficient Neural Architecture Search via Parameter Sharing • Everything you need to know about AutoML and Neural Architecture Search • Understanding AutoML and Neural Architecture Search • An Opinionated Introduction to AutoML and Neural Architecture Search • What do machine learning practitioners actually do? -
ࢀߟࢿྉ ϋΠύʔύϥϝʔλνϡʔχϯά • Random Search for Hyper-Parameter Optimization • A
Conceptual Explanation of Bayesian Hyperparameter Optimization for Machine Learning • ػցֶशϞσϧͷϋΠύύϥϝʔλ࠷దԽ • ػցֶशͷͨΊͷϕΠζ࠷దԽೖ