Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Machine Learning in Production with R or Python...

Avatar for szilard szilard
June 08, 2017
0
190

Machine Learning in Production with R or Python - Budapest Data Forum - June 2017

Avatar for szilard

szilard

June 08, 2017
Tweet

More Decks by szilard

See All by szilard
Gradient Boosting Machines (GBM): From Zero to Hero (with R and Python Code) - Data Con LA - Oct 2020
Avatar for szilard szilard
0
210
Make Machine Learning Boring Again: Best Practices for Using Machine Learning in Businesses - Albuquerque Machine Learning Meetup (Online) - Aug 2020
Avatar for szilard szilard
0
160
Better than Deep Learning: Gradient Boosting Machines (GBM) - eRum conference - invited talk - June 2020
Avatar for szilard szilard
0
140
Gradient Boosting Machines (GBM): From Zero to Hero (with R and Python Code) - LA Data Science Meetup - February 2020
Avatar for szilard szilard
0
130
A Random Walk in Data Science and Machine Learning in Practice - CEU, Business Analytics Masters - Budapest, Febr 2020
Avatar for szilard szilard
0
320
Better than My Meetup/Conference Talks: Going Deeper in Various GBM Topics - GBM Advanced Workshop - Budapest, Nov 2019
Avatar for szilard szilard
0
97
Gradient Boosting Machines (GBM): From Zero to Hero (with R and Python Code) - Budapest BI Forum, Budapest, Nov 2019
Avatar for szilard szilard
0
150
Make Machine Learning Boring Again: Best Practices for Using Machine Learning in Businesses - LA Data Science Meetup - Playa Vista, August 2019
Avatar for szilard szilard
0
140
Better than Deep Learning: Gradient Boosting Machines (GBM) / 2019 edition - Budapest R and Data Science Meetups - Budapest, June 2019
Avatar for szilard szilard
0
120

Featured

See All Featured
Tips & Tricks on How to Get Your First Job In Tech
Avatar for Honza Javorek honzajavorek
0
440
ラッコキーワード サービス紹介資料
Avatar for ラッコ株式会社 rakko
1
2.3M
Speed Design
Avatar for Sergey Chernyshev sergeychernyshev
33
1.5k
How to Build an AI Search Optimization Roadmap - Criteria and Steps to Take #SEOIRL
Avatar for Aleyda Solis aleyda
1
1.9k
Designing for humans not robots
Avatar for Tammie Lister tammielis
254
26k
The Organizational Zoo: Understanding Human Behavior Agility Through Metaphoric Constructive Conversations (based on the works of Arthur Shelley, Ph.D)
Avatar for Dr. Kim W Petersen kimpetersen
PRO
0
240
Utilizing Notion as your number one productivity tool
Avatar for Mfonobong Umondia mfonobong
3
220
Dominate Local Search Results - an insider guide to GBP, reviews, and Local SEO
Avatar for Greg Gifford greggifford
PRO
0
78
Rebuilding a faster, lazier Slack
Avatar for samanthasiow samanthasiow
85
9.4k
SEO for Brand Visibility & Recognition
Avatar for Aleyda Solis aleyda
0
4.2k
Building the Perfect Custom Keyboard
Avatar for Naoto Takai takai
2
690
Improving Core Web Vitals using Speculation Rules API
Avatar for Sergey Chernyshev sergeychernyshev
21
1.4k

Transcript

  1. Machine Learning in Production Szilárd Pafka, PhD Chief Scientist, Epoch

    Budapest Data Forum June 2017

  2. Machine Learning in Production with R Szilárd Pafka, PhD Chief

    Scientist, Epoch Budapest Data Forum June 2017

  3. Machine Learning in Production with R or Python Szilárd Pafka,

    PhD Chief Scientist, Epoch Budapest Data Forum June 2017

  4. Machine Learning in Production with R or maybe Python Szilárd

    Pafka, PhD Chief Scientist, Epoch Budapest Data Forum June 2017
  5. None

  6. Disclaimer: I am not representing my employer (Epoch) in this

    talk I cannot confirm nor deny if Epoch is using any of the methods, tools, results etc. mentioned in this talk
  7. None

  8. http://datascience.la/meetup-machine-learning-in-production-with-szilard-pafka/

  9. None
  10. None
  11. None
  12. None
  13. None
  14. None
  15. None
  16. None
  17. None
  18. None
  19. None
  20. None
  21. None

  22. Aggregation 100M rows 1M groups Join 100M rows x 1M

    rows time [s] time [s]

  23. Aggregation 100M rows 1M groups Join 100M rows x 1M

    rows time [s] time [s]
  24. None
  25. None
  26. None
  27. None

  28. binary classification, 10M records numeric & categorical features, non-sparse

  29. http://www.cs.cornell.edu/~alexn/papers/empirical.icml06.pdf http://lowrank.net/nikos/pubs/empirical.pdf

  30. http://www.cs.cornell.edu/~alexn/papers/empirical.icml06.pdf http://lowrank.net/nikos/pubs/empirical.pdf

  31. None
  32. None
  33. None
  34. None
  35. None

  36. EC2

  37. n = 10K, 100K, 1M, 10M, 100M Training time RAM

    usage AUC CPU % by core read data, pre-process, score test data

  38. n = 10K, 100K, 1M, 10M, 100M Training time RAM

    usage AUC CPU % by core read data, pre-process, score test data
  39. None
  40. None
  41. None
  42. None
  43. None
  44. None
  45. None

  46. 10x

  47. None
  48. None
  49. None
  50. None
  51. None

  52. http://datascience.la/benchmarking-random-forest-implementations/#comment-53599

  53. None
  54. None
  55. None
  56. None
  57. None
  58. None
  59. None

  60. Best linear: 71.1

  61. None
  62. None

  63. learn_rate = 0.1, max_depth = 6, n_trees = 300 learn_rate

    = 0.01, max_depth = 16, n_trees = 1000
  64. None
  65. None
  66. None
  67. None
  68. None
  69. None
  70. None
  71. None
  72. None
  73. None
  74. None
  75. None
  76. None
  77. None

  78. ...

  79. None
  80. None
  81. None
  82. None
  83. None
  84. None
  85. None
  86. None
  87. None
  88. None
  89. None
  90. None
  91. None
  92. None
  93. None
  94. None
  95. None
  96. None
  97. None
  98. None
  99. None
  100. None
  101. None
  102. None