Automating Machine Learning

Automating Machine Learning

8ffe68e4b19092aab184e4aa09ca4bff?s=128

Andreas Mueller

July 15, 2016
Tweet

Transcript

  1. 2.
  2. 7.

    What? from automl import AutoClassifier clf = AutoClassifier().fit(X_train, y_train) >

    Current Accuracy: 70% (AUC .65) LinearSVC(C=1), 10sec > Current Accuracy: 76% (AUC .71) RandomForest(n_estimators=20) 30sec > Current Accuracy: 80% (AUC .74) RandomForest(n_estimators=500) 30sec
  3. 11.
  4. 18.
  5. 19.
  6. 20.
  7. 24.
  8. 25.
  9. 28.

    Meta-Learning optimization Algorithm + Parameters Dataset 3 optimization Algorithm +

    Parameters Dataset 2 optimization Algorithm + Parameters Dataset 1
  10. 29.

    Meta-Learning Meta-Features 1 optimization Algorithm + Parameters Dataset 3 optimization

    Algorithm + Parameters Dataset 2 optimization Algorithm + Parameters Dataset 1 Meta-Features 2 Meta-Features 3 ML model
  11. 30.

    Meta-Learning Meta-Features 1 optimization Algorithm + Parameters Dataset 3 optimization

    Algorithm + Parameters Dataset 2 optimization Algorithm + Parameters Dataset 1 Meta-Features 2 Meta-Features 3 ML model New Dataset ML model Algorithm + Parameters
  12. 34.
  13. 36.
  14. 39.

    Within Scikit-learn • GridSearchCV • RandomizedSearchCV • BayesianSearchCV (coming) •

    Searching over Pipelines (coming) • Built-in parameter ranges (coming)
  15. 40.

    TODO Clean separation of: • Model Search Space • Pipeline

    Search Space • Optimization Method • Meta-Learning • Exploit prior knowledge better! • Usability • Runtime consideration
  16. 41.

    TODO Clean separation of: • Model Search Space • Pipeline

    Search Space • Optimization Method • Meta-Learning • Exploit prior knowledge better! • Usability • Runtime consideration • Data subsampling
  17. 42.
  18. 47.

    47 Material • Random Search for Hyper-Parameter Optimization (Bergstra, Bengio)

    • Efficient and Robust Automated Machine Learning (Feurer et al) [autosklearn] • http://automl.github.io/auto-sklearn/stable/ • Efficient Hyperparameter Optimization and Infinitely Many Armed Bandits (Lie et. al) [hyperband] https://arxiv.org/abs/1603.06560 • Scalable Bayesian Optimization Using Deep Neural Networks [Snoek et al]