Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Gradient Boosting Machines (GBM): From Zero to Hero (with R and Python Code) - Data Con LA - Oct 2020

szilard
October 09, 2020
87

Gradient Boosting Machines (GBM): From Zero to Hero (with R and Python Code) - Data Con LA - Oct 2020

szilard

October 09, 2020
Tweet

More Decks by szilard

Transcript

  1. Gradient Boosting Machines (GBM):
    From Zero to Hero (with R and Python Code)
    Szilard Pafka, PhD
    Chief Scientist, Epoch
    Data Con LA (Online)
    Oct 2020

    View Slide

  2. View Slide

  3. Disclaimer:
    I am not representing my employer (Epoch) in this talk
    I cannot confirm nor deny if Epoch is using any of the methods, tools,
    results etc. mentioned in this talk

    View Slide

  4. Source: Andrew Ng

    View Slide

  5. Source: Andrew Ng

    View Slide

  6. Source: Andrew Ng

    View Slide

  7. View Slide

  8. View Slide

  9. View Slide

  10. View Slide

  11. View Slide

  12. Source: https://twitter.com/iamdevloper/

    View Slide

  13. View Slide

  14. View Slide

  15. ...

    View Slide

  16. View Slide

  17. View Slide

  18. View Slide

  19. View Slide

  20. View Slide

  21. View Slide

  22. y = f(x
    1
    ,x
    2
    ,...,x
    n
    )
    “Learn” f from data

    View Slide

  23. y = f(x
    1
    ,x
    2
    ,...,x
    n
    )

    View Slide

  24. y = f(x
    1
    ,x
    2
    ,...,x
    n
    )

    View Slide

  25. Supervised Learning
    Data: X (n obs, p features), y (labels)
    Regression, classification
    Train/learn/fit f from data (model)
    Score: for new x, get f(x)
    Algos: LR, k-NN, DT, RF, GBM, NN/DL, SVM, NB…
    Goal: max acc/min err new data
    Metrics: MSE, AUC (ROC)
    Bad: measure on train set. Need: test set/cross-validation (CV)
    Hyperparameters, model capacity, overfitting
    Regularization
    Model selection
    Hyperparameter search (grid, random)
    Ensembles

    View Slide

  26. Supervised Learning
    Data: X (n obs, p features), y (labels)
    Regression, classification
    Train/learn/fit f from data (model)
    Score: for new x, get f(x)
    Algos: LR, k-NN, DT, RF, GBM, NN/DL, SVM, NB…
    Goal: max acc/min err new data
    Metrics: MSE, AUC (ROC)
    Bad: measure on train set. Need: test set/cross-validation (CV)
    Hyperparameters, model capacity, overfitting
    Regularization
    Model selection
    Hyperparameter search (grid, random)
    Ensembles

    View Slide

  27. View Slide

  28. View Slide

  29. Source: Hastie etal, ESL 2ed

    View Slide

  30. Source: Hastie etal, ESL 2ed

    View Slide

  31. View Slide

  32. View Slide

  33. View Slide

  34. View Slide

  35. View Slide

  36. View Slide

  37. View Slide

  38. View Slide

  39. View Slide

  40. View Slide

  41. View Slide

  42. View Slide

  43. View Slide

  44. View Slide

  45. View Slide

  46. View Slide

  47. View Slide

  48. View Slide

  49. View Slide

  50. no-one is using
    this crap

    View Slide

  51. View Slide

  52. View Slide

  53. View Slide

  54. Live Demo
    Summary of the demo for those reading just the
    slides (e.g. those who did not attend the talk):

    View Slide

  55. View Slide

  56. View Slide

  57. View Slide

  58. View Slide

  59. View Slide

  60. View Slide

  61. View Slide

  62. View Slide

  63. View Slide

  64. View Slide

  65. View Slide

  66. View Slide

  67. View Slide

  68. http://www.jmlr.org/papers/volume13/bergstra12a/bergstra12a.pdf

    View Slide

  69. End of Demo

    View Slide

  70. View Slide

  71. View Slide

  72. View Slide

  73. View Slide

  74. View Slide

  75. View Slide

  76. View Slide

  77. View Slide