ISI Programming Course - 06 - Scikit Learn

Sample Code : https://github.com/thejungwon/ISI-programming-class/tree/master/exercises/week-06

Transcript

1. Jungwon Seo | University of Stavanger ISI Programming Class Scikit Learn
2. Machine Learning Machine Learning Supervised Learning Unsupervised Learning Classification Regression Clustering Association
3. Supervised vs. Unsupervised Supervised Learning Unsupervised Learning Known number of classes Unknown number of classes Uses training dataset Uses input dataset For prediction For analysis
4. Supervised Learning {features, class} Model {features} Model Class A? Class B? Class C? Training Testing Result Training Data Testing Data
5. Classifier vs. Regressor Trained Classifier Trained Regressor Pass / Fail Score 
 ( e.g. 95/100) Student info Student info
6. Unsupervised Learning {features} Algorithm Data Class A? Class B? Class C? Generating Classes
7. Clustering vs. Association Clustering Association
8. In this lecture, we will focus on classification!
9. Classification algorithms • Linear Classifiers: Logistic Regression, Naive Bayes Classifier. • Decision Trees • Random Forest • Support Vector Machines • Neural Networks • Nearest Neighbor • Genetic Algorithm • …….
10. Logistic Regression • To find weights (B) we train the model. • The goal is making function that divide the data the best.
11. Again! We won’t implement this algorithm. We will use implemented one.
12. Naive Bayes Classifier • In this case, training data is used for making P(Xn | c).
13. Decision Tree <Mansplaining Decision Tree> Did she ask? Not mansplaining Do you know better than her? Did you ask 
 if she needed it explained? Not mansplaining Mansplaining Mansplaining Mansplaining Yes Yes Yes No About the same No No
14. Decision Tree <With numerical data>
15. Decision Tree • The most important part is determining the split node. • e.g. : Gender ? F : M, Height ? over 170 : under 170. • There are several index that you can use. • e.g. : Entropy, Gini, Misclassification error • The training data is used for building the tree.
16. Random Forest
17. Random Forest Test data Class A Class B Class A Tree 1 Tree 2 Tree 3 Class A
18. Random Forest • The randomness affects while building each tree. • It is improved version of decision tree • But! You can’t explain about the process. (Blackbox)
19. Support Vector Machine 1. Find the border with the largest margin 2. For non-linear case, mapping inputs into high-dimensional feature spaces.
20. SVM vs. Logistic Reg. • Logistic regression focuses on maximizing the probability of the data. The farther the data lies from the separating hyperplane (on the correct side), the happier LR is. • An SVM tries to find the separating hyperplane that maximizes the distance of the closest points to the margin (the support vectors). If a point is not a support vector, it doesn’t really matter. http://www.cs.toronto.edu/~kswersky/wp-content/uploads/svm_vs_lr.pdf
21. K-Nearest Neighbor 1. How to determine K? 2. What kind of distance shall we use? Euclidean? Manhattan?
22. Things to be careful • Overfitting • Outlier
23. How can I choose the right algorithm?
24. Let’s get started!
25. 1. Scikit learn • Preprocessing • Cross validation • Classifiers Source Code
26. 2. Calculating similarities • Minkowski Distance • Manhattan Distance • Euclidean Distance • Chebyshev Distance • Cosine Distance Source Code
27. Thank you