Power of Ensembles

Transcript

1. How  you  win  ML  competitions:  “  you  take  other  people’s  work     and  ensemble  them  together”    -­‐  Vitaly  Kuznetsov  NIPS  2014     -­‐  Bargava  Subramanian  (@bargava)   -­‐  Nischal  HP  (@nischalhp)      
2. Increased  Accuracy   Robustness   Efficiency   Parallelization   Wider  search  of   solution  space   Reduces  over   fitting  
3. Model  human   readability  is  not   great   Time/Effort  required  to  build   complex  ensemble  models  might  not   be  directly  proportional  to  the   accuracy    
4. {   Building  Base  Models   Model  Aggregation   Different  Training  Sets   Different  Algorithms   Different  Parameter  Setups   Algorithm  Randomization   Feature  Sampling   Voting  /  Averaging   Weighted  Voting   Using  as  attributes   Stacking   Bagging  
5. Input  Data   Model  1   Model  2   Model  3   Model  4   Combine  the   models  using   some  logic   Final   Output  
6. Random  Forest   Gradient   Boosting   Logistic   Regression   Ensemble   Output   1   0   1   1   0   0   1   0   1   1   0   1   1   1   1   1   0   0   0   0   70%   70%   70%   90%  
7. Input   Data   Model  1   Model  2   Model  3   Model  4   Combine  the   models  using   some  logic   Final   Output   pre     processing     feature    extraction     model     [   [   - Pipeline identify   the  models   Assign  weights   to  models   [   [   - Hyperopt finding  randomized  hyper  parameters  for  models   [   [   - RandomizedSearchCV [   [   - joblib ( running models in parallel ) sklearn   keras   xgboost   [   [   - libraries used to build base models