1 2 3 4 5 6 7 8 Cost function 1 def Cost(m): 2 return sum(((f(m, x)-y )**2 f o r x, y in z i p (X, Y))) / len (X ) Problem solved: cost = squared dierences between each y and f = m × x.
1 2 3 4 5 6 7 8 Cost function 1 def Cost(m): 2 return sum(((f(m, x)-y )**2 f o r x, y in z i p (X, Y))) / len (X ) We can now iterate through many values of m. What search algorithms could be used to improve this?
1 2 3 4 5 6 7 8 dCost function 1 def Cost(m): 2 return sum(((f(m, x)-y )**2 f o r x, y in z i p (X, Y))) / len (X ) 3 4 def dCost(m): 5 return sum((2*(f(m, x) -y)*x f o r x, y in z i p (X, Y))) / len (X ) Newton's optimization method: m i +1 = m i − Cost(m i ) dCost(m i ) .
1 2 3 4 5 6 7 8 A more complex model: neural network Linear regression X Y m Neural network X Y Furthermore, ReLU re neuron only when excited ReLU(x) = x if x ≥ b 0 if x < b
1 2 3 4 5 6 7 8 A more complex model: neural network We are now able to model the initial exponential... f (m1, m2, . . . , n1, n2, . . . , b1, b2, . . . , x) = n1σ(m1 x+b1)+n2σ(m2 x+b2)+. . . 1 def relu(x): 2 return x i f x >= 0 e l s e 0 3 4 def f(mm , nn, bb, x): 5 return sum((n*relu(m*x +b) f o r m, n, b in z i p (mm , nn , bb))) 1 mm = [1, 1, 1] 2 nn = [1e4 , 1e5 , 7.5e5] 3 bb = [0, -35, -40]