Ensemble Package

Ensmeble Models and Ensmeble Package - Talk given at SciPy IIT Bombay 2016

Transcript

1. ENSEMBLE PACKAGE Prajwal Kailas (@prajwal967), National Institute Of Technology Karnataka
2. Ensemble Learning Ensemble learning is a machine learning paradigm where multiple learners are trained to solve the same problem. In contrast to ordinary machine learning approaches which try to learn one hypothesis from training data, ensemble methods try to construct a set of hypotheses and combine them to use.
3. Why Ensemble? Different models with same features mostly result in different predictions.
4. A Simple Ensemble - max count
5. Examples Some examples of ensemble models: ➔ Random Forest Constructing a multitude of decision trees at training time ➔ Gradient Boosting Produces a prediction model in the form of an ensemble of weak prediction models ➔ Bagging Special case of the model averaging approach.
6. ENSEMBLE PACKAGE in Python and has been registered on PyPI
7. Base Models • Linear Regression. • Gradient Boosting (XGBoost). • Random Forest Classifier. • Decision Tree. • Logistic Regression.
8. Model Aggregation • Blending. • Stacking. • Weighted Average.
9. S T A C K I N G
10. B L E N D I N G
11. Key Features • Hyper parameter optimisation for any of the models (Hyperopt and Gridsearch). • Running the models in parallel during training and testing (Joblib).
12. Advantages • Improved Accuracy • Robustness • Parallelization • Base Model Diversity
13. Disadvantages • Model human readability isn't great • Time/Effort trade-off to improve accuracy may not make sense
14. Ensembles Secret to winning a Kaggle competition
15. https://github.com/unnati-xyz/ense mble-package Ensemble Package Name: ensembles Version: 0.1.2 pip install ensembles TEAM: Amit Kapoor, Bargava Subramaniam, Nischal HP, Prajwal Kailas, Raghotam S prajwal967@gmail.com SciPy India 2016
16. Questions?