SVM classifiers

Transcript

1. 「 Support Vector Machine (SVM) Classifiers 」 長岡技術科学大学 自然言語処理研究室 - VO HUYNH QUOC VIET 2018/2/27 　　 B4 ゼミ ～第 8 回～
2. • Input: Given training data (x i , y i ) for i = 1 ...N, with x i and y ∈ i {−1, 1} ∈ • Task: learn a classifier f(x) such that 2 Linear Classifiers
3. 3 Not Linearly Separable Linearly Separable Linear Classifiers
4. A linear classifier has the form 4 Linear Classifiers • w is the normal to the line, and b the bias • w is known as the weight vector
5. 5 Linear Support Vector Machine ( LSVM) m argin hyperplane
6. 6 Linear Support Vector Machine ( LSVM) m argin is + is - DECISION RULE guter
7. 7 Linear Support Vector Machine ( LSVM) Assuming that : m argin is + is - When:
8. Linear Support Vector Machine ( LSVM) • SVM can be formulated as an optimization: • Or equivalently:
9. 9 Linear Support Vector Machine ( LSVM) • Change to the Lagrange formulation: The Maximization depends only on dot products of pairs of vectors Maximize
10. 10 Linear Support Vector Machine ( LSVM) • Summary: DECISION RULE
11. Not Linearly Separable Data Not Linearly Separable Data transform to a higher demensional space, called feature space. 11 Support Vector Machine ( SVM) kernel Maximization depends only on dot products
12. 12 Maximization depends only on dot products Support Vector Machine ( SVM) Not Linearly Separable Data
13. 13 Support Vector Machine ( SVM) Not Linearly Separable Data
14. Multiple Classes 14 Support Vector Machine ( SVM) • There are 2 kind of comparision for doing multiple classes >> .SVC(kernel='linear', decision_function_shape='ovr’) OVO: One vs One OVR: One vs Rest Pros: less sensitive to imbalanced Cons: More classifications Pros: Fewer classifications Cons: Classes may be imbalanced
15. 15 • Pros: • Good at dealing with high dimentional data. • Works well on small data sets. • Different kernel functions for various decision functions or combine 2 different kernel functions for better result. • Cons: • Picking the right kernel and parameters can be computationally intensive. • SVM do not provide probability estimates. Support Vector Machine ( SVM)
16. 参考文献 16 • Support Vector Machines - Patrick Winston, MIT OpenCourseWare • Understanding Support Vector Machine algorithm from examples - Sunil Ray