Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Machine Learning in Production with R or Python - Budapest Data Forum - June 2017
Search
szilard
June 08, 2017
0
180
Machine Learning in Production with R or Python - Budapest Data Forum - June 2017
szilard
June 08, 2017
Tweet
Share
More Decks by szilard
See All by szilard
Gradient Boosting Machines (GBM): From Zero to Hero (with R and Python Code) - Data Con LA - Oct 2020
szilard
0
120
Make Machine Learning Boring Again: Best Practices for Using Machine Learning in Businesses - Albuquerque Machine Learning Meetup (Online) - Aug 2020
szilard
0
89
Better than Deep Learning: Gradient Boosting Machines (GBM) - eRum conference - invited talk - June 2020
szilard
0
83
Gradient Boosting Machines (GBM): From Zero to Hero (with R and Python Code) - LA Data Science Meetup - February 2020
szilard
0
76
A Random Walk in Data Science and Machine Learning in Practice - CEU, Business Analytics Masters - Budapest, Febr 2020
szilard
0
270
Better than My Meetup/Conference Talks: Going Deeper in Various GBM Topics - GBM Advanced Workshop - Budapest, Nov 2019
szilard
0
53
Gradient Boosting Machines (GBM): From Zero to Hero (with R and Python Code) - Budapest BI Forum, Budapest, Nov 2019
szilard
0
130
Make Machine Learning Boring Again: Best Practices for Using Machine Learning in Businesses - LA Data Science Meetup - Playa Vista, August 2019
szilard
0
98
Better than Deep Learning: Gradient Boosting Machines (GBM) / 2019 edition - Budapest R and Data Science Meetups - Budapest, June 2019
szilard
0
77
Featured
See All Featured
No one is an island. Learnings from fostering a developers community.
thoeni
17
2.8k
GraphQLとの向き合い方2022年版
quramy
36
13k
Making Projects Easy
brettharned
111
5.7k
4 Signs Your Business is Dying
shpigford
178
21k
From Idea to $5000 a Month in 5 Months
shpigford
377
46k
How GitHub (no longer) Works
holman
305
140k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
35
6.3k
A Modern Web Designer's Workflow
chriscoyier
689
190k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
44
4.7k
Building Effective Engineering Teams - LeadDev
addyosmani
47
2.2k
Building Better People: How to give real-time feedback that sticks.
wjessup
357
18k
How to train your dragon (web standard)
notwaldorf
79
5.5k
Transcript
Machine Learning in Production Szilárd Pafka, PhD Chief Scientist, Epoch
Budapest Data Forum June 2017
Machine Learning in Production with R Szilárd Pafka, PhD Chief
Scientist, Epoch Budapest Data Forum June 2017
Machine Learning in Production with R or Python Szilárd Pafka,
PhD Chief Scientist, Epoch Budapest Data Forum June 2017
Machine Learning in Production with R or maybe Python Szilárd
Pafka, PhD Chief Scientist, Epoch Budapest Data Forum June 2017
None
Disclaimer: I am not representing my employer (Epoch) in this
talk I cannot confirm nor deny if Epoch is using any of the methods, tools, results etc. mentioned in this talk
None
http://datascience.la/meetup-machine-learning-in-production-with-szilard-pafka/
None
None
None
None
None
None
None
None
None
None
None
None
None
Aggregation 100M rows 1M groups Join 100M rows x 1M
rows time [s] time [s]
Aggregation 100M rows 1M groups Join 100M rows x 1M
rows time [s] time [s]
None
None
None
None
binary classification, 10M records numeric & categorical features, non-sparse
http://www.cs.cornell.edu/~alexn/papers/empirical.icml06.pdf http://lowrank.net/nikos/pubs/empirical.pdf
http://www.cs.cornell.edu/~alexn/papers/empirical.icml06.pdf http://lowrank.net/nikos/pubs/empirical.pdf
None
None
None
None
None
EC2
n = 10K, 100K, 1M, 10M, 100M Training time RAM
usage AUC CPU % by core read data, pre-process, score test data
n = 10K, 100K, 1M, 10M, 100M Training time RAM
usage AUC CPU % by core read data, pre-process, score test data
None
None
None
None
None
None
None
10x
None
None
None
None
None
http://datascience.la/benchmarking-random-forest-implementations/#comment-53599
None
None
None
None
None
None
None
Best linear: 71.1
None
None
learn_rate = 0.1, max_depth = 6, n_trees = 300 learn_rate
= 0.01, max_depth = 16, n_trees = 1000
None
None
None
None
None
None
None
None
None
None
None
None
None
None
...
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None