Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Make Machine Learning Boring Again: Best Practices for Using Machine Learning in Businesses - Albuquerque Machine Learning Meetup (Online) - Aug 2020

szilard
August 16, 2020
87

Make Machine Learning Boring Again: Best Practices for Using Machine Learning in Businesses - Albuquerque Machine Learning Meetup (Online) - Aug 2020

szilard

August 16, 2020
Tweet

More Decks by szilard

Transcript

  1. Make Machine Learning Boring Again: Best Practices for Using Machine

    Learning in Businesses Szilard Pafka, PhD Chief Scientist, Epoch Albuquerque Machine Learning Meetup (Online) Aug 2020
  2. Disclaimer: I am not representing my employer (Epoch) in this

    talk I cannot confirm nor deny if Epoch is using any of the methods, tools, results etc. mentioned in this talk
  3. *

  4. 10x

  5. Intelligible Models for HealthCare: Predicting Pneumonia Risk and Hospital 30-day

    Readmission - Rich Caruana etal On one of the pneumonia datasets, the rule-based system learned the rule “HasAsthama(x) ⇒ LowerRisk(x)”, i.e., that patients who have a history of asthma have lower risk of dying from pneumonia than the general population patients with a history of asthma usually were admitted not only to the hospital but directly to the ICU (Intensive Care Unit). [...] the aggressive care received by asthmatic patients was so effective that it lowered their risk of dying from pneumonia compared to the general population models trained on the data incorrectly learn that asthma lowers risk, when in fact asthmatics have much higher risk (if not hospitalized) The logistic regression model also learned that having asthma lowered risk, but this could easily be corrected by changing the weight on the asthma feature from negative to positive (or to zero).
  6. ML training: lots of CPU cores lots of RAM limited

    time ML scoring: separated servers
  7. “ the people that know what they’re doing just use

    open source, and the people that don’t will not get anything to work, ever, even with APIs.” https://bradfordcross.com/five-ai-startup-predictions-for-2017/
  8. already pre-processed data less domain knowledge (or deliberately hidden) AUC

    0.0001 increases "relevant" no business metric no actual deployment models too complex no online evaluation no monitoring data leakage
  9. Aggregation 100M rows 1M groups Join 100M rows x 1M

    rows time [s] time [s] “Motherfucka!”
  10. “There is no AutoML system which consistently outperforms all others.

    On some datasets, the performance differences can be significant, but on others the AutoML methods are only marginally better than a Random Forest. On 2 datasets, all frameworks perform worse than a Random Forest.”