Gradient Boosting Machines (GBM): From Zero to Hero (with R and Python Code) - Data Con LA - Oct 2020

Transcript

1. Gradient Boosting Machines (GBM):
   From Zero to Hero (with R and Python Code)
   Szilard Pafka, PhD
   Chief Scientist, Epoch
   Data Con LA (Online)
   Oct 2020
   

   View Slide

2. View Slide

3. Disclaimer:
   I am not representing my employer (Epoch) in this talk
   I cannot confirm nor deny if Epoch is using any of the methods, tools,
   results etc. mentioned in this talk
   

   View Slide

4. Source: Andrew Ng
   

   View Slide

5. Source: Andrew Ng
   

   View Slide

6. Source: Andrew Ng
   

   View Slide

7. View Slide

8. View Slide

9. View Slide

10. View Slide

11. View Slide

12. Source: https://twitter.com/iamdevloper/
    

    View Slide

13. View Slide

14. View Slide

15. ...
    

    View Slide

16. View Slide

17. View Slide

18. View Slide

19. View Slide

20. View Slide

21. View Slide

22. y = f(x
    1
    ,x
    2
    ,...,x
    n
    )
    “Learn” f from data
    

    View Slide

23. y = f(x
    1
    ,x
    2
    ,...,x
    n
    )
    

    View Slide

24. y = f(x
    1
    ,x
    2
    ,...,x
    n
    )
    

    View Slide

25. Supervised Learning
    Data: X (n obs, p features), y (labels)
    Regression, classification
    Train/learn/fit f from data (model)
    Score: for new x, get f(x)
    Algos: LR, k-NN, DT, RF, GBM, NN/DL, SVM, NB…
    Goal: max acc/min err new data
    Metrics: MSE, AUC (ROC)
    Bad: measure on train set. Need: test set/cross-validation (CV)
    Hyperparameters, model capacity, overfitting
    Regularization
    Model selection
    Hyperparameter search (grid, random)
    Ensembles
    

    View Slide

26. Supervised Learning
    Data: X (n obs, p features), y (labels)
    Regression, classification
    Train/learn/fit f from data (model)
    Score: for new x, get f(x)
    Algos: LR, k-NN, DT, RF, GBM, NN/DL, SVM, NB…
    Goal: max acc/min err new data
    Metrics: MSE, AUC (ROC)
    Bad: measure on train set. Need: test set/cross-validation (CV)
    Hyperparameters, model capacity, overfitting
    Regularization
    Model selection
    Hyperparameter search (grid, random)
    Ensembles
    

    View Slide

27. View Slide

28. View Slide

29. Source: Hastie etal, ESL 2ed
    

    View Slide

30. Source: Hastie etal, ESL 2ed
    

    View Slide

31. View Slide

32. View Slide

33. View Slide

34. View Slide

35. View Slide

36. View Slide

37. View Slide

38. View Slide

39. View Slide

40. View Slide

41. View Slide

42. View Slide

43. View Slide

44. View Slide

45. View Slide

46. View Slide

47. View Slide

48. View Slide

49. View Slide

50. no-one is using
    this crap
    

    View Slide

51. View Slide

52. View Slide

53. View Slide

54. Live Demo
    Summary of the demo for those reading just the
    slides (e.g. those who did not attend the talk):
    

    View Slide

55. View Slide

56. View Slide

57. View Slide

58. View Slide

59. View Slide

60. View Slide

61. View Slide

62. View Slide

63. View Slide

64. View Slide

65. View Slide

66. View Slide

67. View Slide

68. http://www.jmlr.org/papers/volume13/bergstra12a/bergstra12a.pdf
    

    View Slide

69. End of Demo
    

    View Slide

70. View Slide

71. View Slide

72. View Slide

73. View Slide

74. View Slide

75. View Slide

76. View Slide

77. View Slide