61

# DevOpsPorto Meetup19: Behind Machine Learning by Ricardo Cruz

A linear regression and a neuronal network will be implemented using nothing but Python. This isn't a talk for the feeble. :) August 01, 2018

## Transcript

1. Behind Machine Learning
Ricardo Cruz
DevOps & Python Porto Meetup August 1, 2018

2. 1
2
3
4
5
6
7
8
Data
1 X = [1971, 1972, 1974,
1979, 1982, 1985, ...]
2 Y = [2.31, 3.55, 6.10,
29.16, 135.77 , 273.84 ,
...]

3. 1
2
3
4
5
6
7
8
Data
1 import math
2 X = [x-1970 f o r x in X]
3 Y = [math.log10(y) f o r y
in Y]

4. 1
2
3
4
5
6
7
8
Create a model: f(m,x)=mx
1 def f(m, x):
2 return m*x
Which is the best slope m?

5. 1
2
3
4
5
6
7
8
Cost function
1 def Cost(m):
2 return sum(((f(m, x)-y
)**2 f o r x, y in
z i p (X, Y))) / len (X
)
Problem solved: cost = squared
dierences between each y and
f = m × x.

6. 1
2
3
4
5
6
7
8
Cost function
1 def Cost(m):
2 return sum(((f(m, x)-y
)**2 f o r x, y in
z i p (X, Y))) / len (X
)
We can now iterate through many
values of m.
What search algorithms could be
used to improve this?

7. 1
2
3
4
5
6
7
8
dCost function
1 def Cost(m):
2 return sum(((f(m, x)-y
)**2 f o r x, y in
z i p (X, Y))) / len (X
)
3
4 def dCost(m):
5 return sum((2*(f(m, x)
-y)*x f o r x, y in
z i p (X, Y))) / len (X
)
Newton's optimization method:
m
i
+1 = m
i

Cost(m
i
)
dCost(m
i
)
.

8. 1
2
3
4
5
6
7
8
A more complex model: neural network
Linear regression
X Y
m
Neural network
X Y
Furthermore, ReLU re neuron only when excited
ReLU(x) =
x if x ≥ b
0 if x < b

9. 1
2
3
4
5
6
7
8
A more complex model: neural network
We are now able to model the initial exponential...
f (m1, m2, . . . , n1, n2, . . . , b1, b2, . . . , x) =
n1σ(m1
x+b1)+n2σ(m2
x+b2)+. . .
1 def relu(x):
2 return x i f x >= 0
e l s e 0
3
4 def f(mm , nn, bb, x):
5 return sum((n*relu(m*x
+b) f o r m, n, b in
z i p (mm , nn , bb)))
1 mm = [1, 1, 1]
2 nn = [1e4 , 1e5 , 7.5e5]
3 bb = [0, -35, -40]

10. Neural Networks are Everywhere !
My webpage: https://rpmcruz.github.io/