DevOpsPorto Meetup19: Behind Machine Learning by Ricardo Cruz

A linear regression and a neuronal network will be implemented using nothing but Python. This isn't a talk for the feeble. :)

Transcript

1. Behind Machine Learning Ricardo Cruz DevOps & Python Porto Meetup  August 1, 2018
2. 1 2 3 4 5 6 7 8 Data 1 X = [1971, 1972, 1974, 1979, 1982, 1985, ...] 2 Y = [2.31, 3.55, 6.10, 29.16, 135.77 , 273.84 , ...]
3. 1 2 3 4 5 6 7 8 Data 1 import math 2 X = [x-1970 f o r x in X] 3 Y = [math.log10(y) f o r y in Y]
4. 1 2 3 4 5 6 7 8 Create a model: f(m,x)=mx 1 def f(m, x): 2 return m*x Which is the best slope m?
5. 1 2 3 4 5 6 7 8 Cost function 1 def Cost(m): 2 return sum(((f(m, x)-y )**2 f o r x, y in z i p (X, Y))) / len (X ) Problem solved: cost = squared dierences between each y and f = m × x.
6. 1 2 3 4 5 6 7 8 Cost function 1 def Cost(m): 2 return sum(((f(m, x)-y )**2 f o r x, y in z i p (X, Y))) / len (X ) We can now iterate through many values of m. What search algorithms could be used to improve this?
7. 1 2 3 4 5 6 7 8 dCost function 1 def Cost(m): 2 return sum(((f(m, x)-y )**2 f o r x, y in z i p (X, Y))) / len (X ) 3 4 def dCost(m): 5 return sum((2*(f(m, x) -y)*x f o r x, y in z i p (X, Y))) / len (X ) Newton's optimization method: m i +1 = m i − Cost(m i ) dCost(m i ) .
8. 1 2 3 4 5 6 7 8 A more complex model: neural network Linear regression X Y m Neural network X Y Furthermore, ReLU  re neuron only when excited ReLU(x) = x if x ≥ b 0 if x < b
9. 1 2 3 4 5 6 7 8 A more complex model: neural network We are now able to model the initial exponential... f (m1, m2, . . . , n1, n2, . . . , b1, b2, . . . , x) = n1σ(m1 x+b1)+n2σ(m2 x+b2)+. . . 1 def relu(x): 2 return x i f x >= 0 e l s e 0 3 4 def f(mm , nn, bb, x): 5 return sum((n*relu(m*x +b) f o r m, n, b in z i p (mm , nn , bb))) 1 mm = [1, 1, 1] 2 nn = [1e4 , 1e5 , 7.5e5] 3 bb = [0, -35, -40]
10. Neural Networks are Everywhere ! My webpage: https://rpmcruz.github.io/