Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Gradient Boosting Machines (GBM): From Zero to Hero (with R and Python Code) - Budapest BI Forum, Budapest, Nov 2019

szilard
November 01, 2019

Gradient Boosting Machines (GBM): From Zero to Hero (with R and Python Code) - Budapest BI Forum, Budapest, Nov 2019

szilard

November 01, 2019
Tweet

More Decks by szilard

Other Decks in Technology

Transcript

  1. Gradient Boosting Machines (GBM):
    From Zero to Hero (with R and Python Code)
    Szilard Pafka, PhD
    Chief Scientist, Epoch (USA)
    Budapest BI Forum
    Nov 2019

    View full-size slide

  2. Disclaimer:
    I am not representing my employer (Epoch) in this talk
    I cannot confirm nor deny if Epoch is using any of the methods, tools,
    results etc. mentioned in this talk

    View full-size slide

  3. Source: Andrew Ng

    View full-size slide

  4. Source: Andrew Ng

    View full-size slide

  5. Source: Andrew Ng

    View full-size slide

  6. Source: https://twitter.com/iamdevloper/

    View full-size slide

  7. y = f(x
    1
    ,x
    2
    ,...,x
    n
    )
    “Learn” f from data

    View full-size slide

  8. y = f(x
    1
    ,x
    2
    ,...,x
    n
    )

    View full-size slide

  9. y = f(x
    1
    ,x
    2
    ,...,x
    n
    )

    View full-size slide

  10. Supervised Learning
    Data: X (n obs, p features), y (labels)
    Regression, classification
    Train/learn/fit f from data (model)
    Score: for new x, get f(x)
    Algos: LR, k-NN, DT, RF, GBM, NN/DL, SVM, NB…
    Goal: max acc/min err new data
    Metrics: MSE, AUC (ROC)
    Bad: measure on train set. Need: test set/cross-validation (CV)
    Hyperparameters, model capacity, overfitting
    Regularization
    Model selection
    Hyperparameter search (grid, random)
    Ensembles

    View full-size slide

  11. Supervised Learning
    Data: X (n obs, p features), y (labels)
    Regression, classification
    Train/learn/fit f from data (model)
    Score: for new x, get f(x)
    Algos: LR, k-NN, DT, RF, GBM, NN/DL, SVM, NB…
    Goal: max acc/min err new data
    Metrics: MSE, AUC (ROC)
    Bad: measure on train set. Need: test set/cross-validation (CV)
    Hyperparameters, model capacity, overfitting
    Regularization
    Model selection
    Hyperparameter search (grid, random)
    Ensembles

    View full-size slide

  12. Source: Hastie etal, ESL 2ed

    View full-size slide

  13. Source: Hastie etal, ESL 2ed

    View full-size slide

  14. no-one is using
    this crap

    View full-size slide

  15. Live Demo
    Summary of the demo for those reading just the
    slides (e.g. those who did not attend the talk):

    View full-size slide

  16. http://www.jmlr.org/papers/volume13/bergstra12a/bergstra12a.pdf

    View full-size slide