ML: regression March 2017 

Notations : Scalar (number) : Vector : Matrix : Transpose of matrix X : Mean of vectors x : Estimate of vector value x 

Simple linear regression 

Solving simple linear regression Goal : find the best and for Best ? Minimize the square residuals (least squares) 

Solving simple linear regression: finding the best estimates for a and b 

Demo 1 

Multivariate linear regression 

Demo 2 

Polynomial regression 

Trick : still a linear regression ! Just create additional columns, derived from pre-existing ones Then it comes back to a linear regression 

Demo 3 

Gradient descent 

Gradient descent 

Gradient descent 

Normal equation vs gradient descent Normal equation Gradient descent No additional parameters Need to choose a learning step No loop Needs to iterate : for inverse Slow if is large Works well when is large In practice : n < 10.000 ⇔ normal equation 

Logistic regression Used for binary classification Decision boundary : 0.5 - y < 0.5 : class A - y >= 0.5 : class B 

Logistic regression : cost function 

Logistic regression : cost function 

Demo 4: logistic regression + gradient descent 

Questions? March 2017