Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Better than My Meetup/Conference Talks: Going Deeper in Various GBM Topics - GBM Advanced Workshop - Budapest, Nov 2019

Ce8e94cc306ba164175f693fb01aa8b0?s=47 szilard
November 09, 2019
0
37

Better than My Meetup/Conference Talks: Going Deeper in Various GBM Topics - GBM Advanced Workshop - Budapest, Nov 2019

Ce8e94cc306ba164175f693fb01aa8b0?s=128

szilard

November 09, 2019
Tweet

More Decks by szilard

See All by szilard
Gradient Boosting Machines (GBM): From Zero to Hero (with R and Python Code) - Data Con LA - Oct 2020
Ce8e94cc306ba164175f693fb01aa8b0?s=48 szilard
0
46
Make Machine Learning Boring Again: Best Practices for Using Machine Learning in Businesses - Albuquerque Machine Learning Meetup (Online) - Aug 2020
Ce8e94cc306ba164175f693fb01aa8b0?s=48 szilard
0
58
Better than Deep Learning: Gradient Boosting Machines (GBM) - eRum conference - invited talk - June 2020
Ce8e94cc306ba164175f693fb01aa8b0?s=48 szilard
0
41
Gradient Boosting Machines (GBM): From Zero to Hero (with R and Python Code) - LA Data Science Meetup - February 2020
Ce8e94cc306ba164175f693fb01aa8b0?s=48 szilard
0
31
A Random Walk in Data Science and Machine Learning in Practice - CEU, Business Analytics Masters - Budapest, Febr 2020
Ce8e94cc306ba164175f693fb01aa8b0?s=48 szilard
0
130
Gradient Boosting Machines (GBM): From Zero to Hero (with R and Python Code) - Budapest BI Forum, Budapest, Nov 2019
Ce8e94cc306ba164175f693fb01aa8b0?s=48 szilard
0
96
Make Machine Learning Boring Again: Best Practices for Using Machine Learning in Businesses - LA Data Science Meetup - Playa Vista, August 2019
Ce8e94cc306ba164175f693fb01aa8b0?s=48 szilard
0
38
Better than Deep Learning: Gradient Boosting Machines (GBM) / 2019 edition - Budapest R and Data Science Meetups - Budapest, June 2019
Ce8e94cc306ba164175f693fb01aa8b0?s=48 szilard
0
48
Better than Deep Learning: Gradient Boosting Machines (GBM) / 2019 edition - LA R Meetup - Santa Monica, May 2019
Ce8e94cc306ba164175f693fb01aa8b0?s=48 szilard
0
12

Featured

See All Featured
Large-scale JavaScript Application Architecture
96270e4c3e5e9806cf7245475c00b275?s=48 addyosmani
499
110k
Fashionably flexible responsive web design (full day workshop)
433acaea1012b25d97ae66da9b998dad?s=48 malarkey
396
62k
Fantastic passwords and where to find them - at NoRuKo
8ec1383b240b5ba15ffb9743fceb3c0e?s=48 philnash
27
1.6k
The Mythical Team-Month
E6c6e133e74c3b83f04d2861deaa1c20?s=48 searls
210
39k
Ruby is Unlike a Banana
6804f1775cb4babfcc3851298566fbce?s=48 tanoku
91
9.3k
The Invisible Side of Design
B3d6434763caa0ef5dc4b792662c49f7?s=48 smashingmag
290
48k
What's new in Ruby 2.0
7edec06b5435e0dac07a353b2cc26253?s=48 geeforr
335
30k
Faster Mobile Websites
C620790ae5bf5b50c245b2e0ef95f338?s=48 deanohume
294
29k
Automating Front-end Workflow
96270e4c3e5e9806cf7245475c00b275?s=48 addyosmani
1351
200k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
95d9f37115fbb0d13135d0f77132f856?s=48 morganepeng
107
16k
GraphQLとの向き合い方2022年版
893f54413c2bd9ba41d11d753aacaf2c?s=48 quramy
16
8.5k
Reflections from 52 weeks, 52 projects
E43f8dfd01057f8304f5f8fb9a294d7d?s=48 jeffersonlam
337
17k

Transcript

  1. Better than My Meetup/Conference Talks: Going Deeper in Various GBM

    Topics Szilard Pafka, PhD Chief Scientist, Epoch (USA) GBM Advanced Workshop Budapest Nov 2019

  2. Why GBMs

  3. None

  4. meetup/conference talks going deeper section dividers

  5. None

  6. Disclaimer: I am not representing my employer (Epoch) in this

    talk I cannot confirm nor deny if Epoch is using any of the methods, tools, results etc. mentioned in this talk

  7. Source: Andrew Ng

  8. Source: Andrew Ng

  9. Source: Andrew Ng

  10. None
  11. None
  12. None
  13. None
  14. None
  15. None

  16. ...

  17. None
  18. None
  19. None
  20. None
  21. None
  22. None
  23. None
  24. None

  25. http://lowrank.net/nikos/pubs/empirical.pdf http://www.cs.cornell.edu/~alexn/papers/empirical.icml06.pdf

  26. http://lowrank.net/nikos/pubs/empirical.pdf http://www.cs.cornell.edu/~alexn/papers/empirical.icml06.pdf

  27. None
  28. None
  29. None
  30. None

  31. top algos (RF, boosting), all features 2007

  32. top algos (RF, boosting), all features most algos (lin, tree,

    nnet) worst algos (knn, NB) 2007

  33. top algos (RF, boosting), all features most algos (lin, tree,

    nnet) worst algos (knn, NB) top algos, removed top feature(s) 2007
  34. None
  35. None
  36. None

  37. Source: Hastie etal, ESL 2ed

  38. Source: Hastie etal, ESL 2ed

  39. GBM libs

  40. None
  41. None
  42. None
  43. None

  44. 10x

  45. 10x

  46. None
  47. None
  48. None
  49. None
  50. None
  51. None
  52. None
  53. None
  54. None
  55. None
  56. None
  57. None
  58. None
  59. None
  60. None
  61. None
  62. None
  63. None
  64. None
  65. None
  66. None

  67. Scoring

  68. None
  69. None
  70. None
  71. None
  72. None

  73. * very first request not shown >500ms (JVM “warmup”)

  74. None
  75. None
  76. None
  77. None
  78. None
  79. None
  80. None
  81. None
  82. None
  83. None

  84. GBM-perf github repo

  85. None
  86. None
  87. None
  88. None
  89. None
  90. None
  91. None

  92. multi-core/socket

  93. None
  94. None
  95. None
  96. None
  97. None
  98. None

  99. CPU 1

  100. CPU 1 CPU 2

  101. CPU 1 CPU 2

  102. CPU 1 CPU 2

  103. CPU 1 CPU 2

  104. None

  105. 5x 3.5x

  106. None
  107. None
  108. None
  109. None
  110. None
  111. None
  112. None
  113. None
  114. None
  115. None
  116. None
  117. None
  118. None
  119. None

  120. zero

  121. None
  122. None

  123. Spark

  124. None
  125. None
  126. None
  127. None
  128. None
  129. None
  130. None
  131. None
  132. None
  133. None
  134. None
  135. None
  136. None
  137. None
  138. None
  139. None
  140. None
  141. None
  142. None
  143. None
  144. None
  145. None
  146. None
  147. None
  148. None
  149. None
  150. None
  151. None

  152. GPU

  153. None
  154. None
  155. None
  156. None

  157. catboost

  158. None
  159. None
  160. None
  161. None
  162. None
  163. None
  164. None
  165. None
  166. None
  167. None

  168. API / tuning

  169. None
  170. None
  171. None
  172. None
  173. None

  174. http://www.jmlr.org/papers/volume13/bergstra12a/bergstra12a.pdf

  175. http://www.argmin.net/2016/06/20/hypertuning/

  176. http://www.argmin.net/2016/06/20/hypertuning/

  177. None
  178. None
  179. None

  180. time ordered data time ordered data

  181. time ordered data time ordered data train sample

  182. time ordered data time ordered data train test sample sample

    (slightly different distribution)

  183. time ordered data time ordered data train test sample sample

    proper train early stopping Model selection resampled 80-10-10 (~CV) (slightly different distribution)

  184. time ordered data time ordered data train test sample sample

    proper train early stopping Model selection random search over lightgbm resampled 80-10-10 (~CV) (slightly different distribution)
  185. None
  186. None
  187. None
  188. None
  189. None
  190. None
  191. None
  192. None

  193. Closing

  194. None
  195. None

  196. Source: https://www.linkedin.com/pulse/winning-solution-kaggledays-2019-competition-san-francisco-mark-peng/

  197. Source: https://www.linkedin.com/pulse/winning-solution-kaggledays-2019-competition-san-francisco-mark-peng/

  198. Source: https://www.linkedin.com/pulse/winning-solution-kaggledays-2019-competition-san-francisco-mark-peng/

  199. Source: https://www.linkedin.com/pulse/winning-solution-kaggledays-2019-competition-san-francisco-mark-peng/

  200. Source: https://www.linkedin.com/pulse/winning-solution-kaggledays-2019-competition-san-francisco-mark-peng/

  201. Source: https://www.linkedin.com/pulse/winning-solution-kaggledays-2019-competition-san-francisco-mark-peng/

  202. None

  203. More:

  204. None