Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Machine Learning on Largish Data - Big Data Day LA - June 2015

szilard
June 27, 2015

Machine Learning on Largish Data - Big Data Day LA - June 2015

szilard

June 27, 2015
Tweet

More Decks by szilard

Other Decks in Technology

Transcript

  1. I usually use other people’s code [...] it is usually

    not “efficient” (from time budget perspective) to write my own algorithm [...] I can find open source code for what I want to do, and my time is much better spent doing research and feature engineering -- Owen Zhang http://blog.kaggle.com/2015/06/22/profiling-top-kagglers-owen-zhang-currently-1-in-the-world/
  2. - R packages 30% - Python scikit-learn 40% - Vowpal

    Wabbit 8% - H2O 10% - xgboost 8% - Spark MLlib 6%
  3. - R packages 30% - Python scikit-learn 40% - Vowpal

    Wabbit 8% - H2O 10% - xgboost 8% - Spark MLlib 6% - a few others
  4. - R packages 30% - Python scikit-learn 40% - Vowpal

    Wabbit 8% - H2O 10% - xgboost 8% - Spark MLlib 6% - a few others
  5. EC2

  6. Distributed computation generally is hard, because it adds an additional

    layer of complexity and [network] communication overhead. The ideal case is scaling linearly with the number of nodes; that’s rarely the case. Emerging evidence shows that very often, one big machine, or even a laptop, outperforms a cluster. http://fastml.com/the-emperors-new-clothes-distributed-machine-learning/
  7. n = 10K, 100K, 1M, 10M, 100M Training time RAM

    usage AUC CPU % by core read data, pre-process, score test data
  8. linear tops off more data & better algo random forest

    on 1% of data beats linear on all data
  9. linear tops off more data & better algo random forest

    on 1% of data beats linear on all data
  10. 10x

  11. I’m of course paranoid that the need for distributed learning

    is diminishing as individual computing nodes (augmented with GPUs) become increasingly powerful. So I was ready for Jure Leskovec’s workshop talk [at NIPS 2014]. Here is a killer screenshot. -- Paul Mineiro
  12. we will continue to run large [...] jobs to scan

    petabytes of [...] data to extract interesting features, but this paper explores the interesting possibility of switching over to a multi-core, shared-memory system for efficient execution on more refined datasets [...] e.g., machine learning http://openproceedings.org/2014/conf/edbt/KumarGDL14.pdf
  13. learn_rate = 0.1, max_depth = 6, n_trees = 300 learn_rate

    = 0.01, max_depth = 16, n_trees = 1000