Machine Learning and Hadoop: Present and Future

Machine Learning and Hadoop
Present and Future
Josh Wills
Cloudera Data Science Team

September 6th, 2012

View Slide

About Me
Copyright 2012 Cloudera Inc. All rights reserved

View Slide

Outline
•  Part 1: Industrial Machine Learning
•  Part 2: ML and Hadoop: The State of the World
•  Part 3: ML and Hadoop: Where Things are Headed

View Slide

(Academic) ML vs. (Academic) StaIsIcs

“Machine learning is sta/s/cs minus any checking of
models and assump/ons.”
-‐-‐ Brian Ripley, UseR! 2004
(provoca/vely paraphrased)

View Slide

Industrial Machine Learning: Truth #1

The thing that we are trying to predict is rarely the thing
that we are trying to opImize.


View Slide


Systems precede algorithms.


View Slide

Practice Over Theory Blog

View Slide

ImplicaIon

Data science requires predicIon-‐oriented machine
learning models AND classical, rigorous staIsIcal
analysis.


View Slide

Outline
•  Part 1: Industrial Machine Learning
•  Part 2: ML and Hadoop: The State of the World
•  Part 3: ML and Hadoop: Where Things are Headed

View Slide

“Hadoop. It’s Where The Data Is.”

View Slide

Hadoop PlaWorm: Substrate
•  Commodity servers
•  Open source operaFng system
•  “” ConfiguraFon Management
•  “” CoordinaFon Service
•  “” File System API
•  “” Efficient and Extensible File Formats
•  “” Efficient and Extensible RPC Libraries

View Slide

Hadoop PlaWorm: MapReduce Frameworks
•  Languages/Environments
•  PigLaFn (Apache)
•  HiveQL (Apache)
•  Jaql (IBM)
•  Java/Scala APIs
•  Crunch (Apache Incubator)
•  Scoobi (NICTA)
•  Cascading (Concurrent)
•  Pangool


View Slide

ML and Hadoop: The State of the World


View Slide

MapReduce
•  Great for:
•  Data PreparaFon
•  Feature Engineering
•  Model ValidaFon/EvaluaFon
•  Works Well For Certain Model Fi\ng Problems
•  CollaboraFve Filtering Algorithms
•  ExpectaFon MaximizaFon
•  Decision Trees (PLANET; Gradient Boosted Decision Trees)
•  Not A PracIcal OpIon for Many Kinds of Problems
•  Way More Detail in the KDD 2011 Talk

View Slide

Apache Mahout
•  The starFng place for MapReduce-‐based machine
learning algorithms
•  Not machine-‐learning-‐in-‐a-‐box
•  Custom tweaks/modiﬁcaFons are the rule
•  A disparate collecFon of algorithms for:
•  RecommendaFons
•  Clustering
•  ClassiﬁcaFon
•  Frequent Itemset Mining

View Slide

Apache Mahout (cont.)
•  Best Library: Taste Recommender
•  Oldest project, most widely-‐deployed in producFon
•  SVD implementaFon is parFcularly acFve
•  Good Libraries: Online SGD
•  Does not use MapReduce
•  Vowpal Rabbit is faster, has L-‐BFGS opFon
•  Roll Your Own Instead: Naïve Bayes


View Slide

The Ominous Challenges


View Slide

1. The Secret Sauce Eﬀect


View Slide

2. Delta Between Mahout and the Cu_ng Edge


View Slide

ML and Hadoop: Where Things are Headed


View Slide

Moving Beyond MapReduce

View Slide

The Contenders


View Slide

AllReduce
•  Developed at Yahoo! Research
•  Deﬁnes the allreduce operaFon
•  N machines each have a number => each machine has the
sum of the numbers
•  At the heart of Vowpal Wabbit’s performance
•  Implemented in C++
•  Can be patched into Apache Hadoop and used today

View Slide

Spark
•  Developed at Berkeley’s
AMP Lab
•  Deﬁnes operaFons on
distributed in-‐memory
collecFons
•  Wriken in Scala
•  Supports reading to and
wriFng from HDFS

View Slide

GraphLab
•  Developed at CMU
•  Lower-‐level primiFves
•  (but higher than MPI)
•  Map/Reduce =>
Update/Sort
•  Flexible, allows for
asynchronous
computaFons
•  Reads from HDFS

View Slide

Machine Learning and Hadoop: Present and Future

Machine Learning and Hadoop: Present and Future

Data Science London

More Decks by Data Science London

Other Decks in Technology

Featured

Transcript