Semidefinite Programming Intro

intro to SDP and how they connect to multikernel SVM problems

Transcript

1. Semidefinite Programming Matthew Barga 06 December 2012 1 / 9
2. 1 Intro 2 / 9
3. SDP Intro maximize yi bi subject to C = yi Ai + S where S 0 This can roughly be considered as maximizing to the objective function yi bi for multipliers yi where S has to be a semidefinite matrix 2 / 9
4. SDP Intro Dual problem minimize C.X subject to Ai .X = bi where X 0 3 / 9
5. Connection to Multikernel SVM In multikernel SVM construction, we have a linear objective to maximize in order to get a combined kernel matrix Target optimization problem is a set of linear combinations of fixed kernel matrices that are computed beforehand K = i µi ∗ Ki 4 / 9
6. Connection to Multikernel SVM Some implementations relax to a Quadratically Constrained Quadratic Programming problem by introducing some further constraints (SHOGUN, etc.) 5 / 9
7. High Performance General Solver for Extremely Large-Scale Semidifinite Programming Problems Takes advantage of two heavily studied elements of convex optimization Schur complement matrix Cholesky factorization 6 / 9
8. High Performance .. Size of the schur complement matrix (SCM) is m (= number of constraints) Size of the data matrix = n Computation of the SCM O(m2) Cholesky factorization of SCM O(m3) this dominates when SCM is large and dense 7 / 9
9. More Still problems arise in memory usage data reconstruction of SCM doubles memory usage in multikernel SVMs, the caching of kernel matrices becomes ineffective due to memory limitations sparsity / SMO 8 / 9
10. Cited Works 9 / 9