Best Practices for Using Machine Learning in Businesses in 2018 - Keynote at Budapest BI Forum Conference - Budapest, November 2018

Transcript

1. Best Practices for Using Machine Learning in Businesses in 2018 Szilárd Pafka, PhD Chief Scientist, Epoch (USA) Budapest BI Forum Conference November 2018
3. Disclaimer: I am not representing my employer (Epoch) in this talk I cannot confirm nor deny if Epoch is using any of the methods, tools, results etc. mentioned in this talk
4. https://twitter.com/baroquepasa/
10. y = f (x1, x2, ... , xn) Source: Hastie etal, ESL 2ed
11. y = f (x1, x2, ... , xn)
13. Source: Yann LeCun
15. 2018?
16. 2018?
17. #1 Use the Right Algo
18. Source: Andrew Ng
36. *
37. #2 Use Open Source
43. in 2006 - cost was not a factor! - data.frame - [800] packages
49. #3 Simple > Complex
51. 10x
60. #4 Incorporate Domain Knowledge Do Feature Engineering (Still) Explore Your Data Clean Your Data
72. #5 Do Proper Validation Avoid: Overfitting, Data Leakage
87. #6 Batch or Real-Time Scoring?
89. https://medium.com/@HarlanH/patterns-for-connecting-predictive-models-to-software-products-f9b6e923f02d
90. https://medium.com/@dvelsner/deploying-a-simple-machine-learning-model-in-a-modern-web-application-flask-angular-docker-a657db075280 your app
93. R/Python: - Slow(er) - Encoding of categ. variables
94. #7 Do Online Validation as Well
96. https://www.oreilly.com/ideas/evaluating-machine-learning-models/page/2/orientation
97. https://www.oreilly.com/ideas/evaluating-machine-learning-models/page/2/orientation
98. https://www.oreilly.com/ideas/evaluating-machine-learning-models/page/2/orientation https://www.slideshare.net/FaisalZakariaSiddiqi/netflix-recommendations-feature-engineering-with-time-travel
99. #8 Monitor Your Models
101. https://www.retentionscience.com/blog/automating-machine-learning-monitoring-rs-labs/
102. https://www.retentionscience.com/blog/automating-machine-learning-monitoring-rs-labs/
104. 20% 80% (my guess)
105. 20% 80% (my guess)
106. #9 Business Value Seek / Measure / Sell
112. #10 Make it Reproducible
122. Cloud (servers)
123. ML training: lots of CPU cores lots of RAM limited time
124. ML training: lots of CPU cores lots of RAM limited time ML scoring: separated servers
125. ML (cloud) services (MLaaS)
127. “people that know what they’re doing just use open source [...] the same open source tools that the MLaaS services offer” - Bradford Cross
128. Kaggle
130. already pre-processed data less domain knowledge (or deliberately hidden) AUC 0.0001 increases "relevant" no business metric no actual deployment models too complex no online evaluation no monitoring data leakage
131. Tuning and Auto ML
132. Ben Recht, Kevin Jamieson: http://www.argmin.net/2016/06/20/hypertuning/
133. GPUs
134. Aggregation 100M rows 1M groups Join 100M rows x 1M rows time [s] time [s]
135. Aggregation 100M rows 1M groups Join 100M rows x 1M rows time [s] time [s] “Motherfucka!”
137. API and GUIs
140. AI?
144. How to Start?