Learn Neural Networks With Go—Not Math!_Ellen Körbes_Codemotion Berlin 2019

It’s common for even the most amazing programmers to not have the first clue about math. That makes learning neural networks particularly inaccessible, as an integral part of explaining it relies on mathematical formulas. Ah, the formulas… with all their lines and curves and ancient symbols, they are just as unintelligible as they are beautiful. What’s a better way for us to learn it instead? With a language we all speak: code! In this talk we’ll look at every component required to write a neural network from scratch. Things like network structure, activation functions and forward propagation.

About: Ellen Körbes, Developer Relations - Garden

Ellen’s a developer advocate at Garden, and also an avid gopher—responsible for the most comprehensive Go course in Portuguese. They first got acquainted with Kubernetes while writing code for kubectl, in a SIG-CLI internship. They've spoken at world-famous events, and at countless local meet-ups. Ellen is a proud recipient of a 'Best Hair' award.

Transcript

1. Title Text Title Text Learn Neural Networks With Go—Not Math! October 2019
3. The reason neural networks The reason neural networks seem difficult is math notation. seem difficult is math notation. Let's do without it. Let's do without it. @ellenkorbes
5. A neural network is a A neural network is a trainable predictor trainable predictor. . @ellenkorbes
6. input Predictor output ? predictor @ellenkorbes
7. Training @ellenkorbes ▪ Start with a random prediction. ▪ How wrong was it? ▪ "This much." ▪ Adjust parameters and try again. ▪ Rinse & repeat.
8. How do we How do we code code this? this? @ellenkorbes
9. ▪ Training: How wrong is my prediction? ▪ Training: How to improve the results? ▪ Training: How do I change it? ▪ Introducing non-linearity. ▪ Structure: Neurons. ▪ Structure: Neuron layers. ▪ Structure: Neural networks. @ellenkorbes In Practice
10. input Neuron output × weight + bias @ellenkorbes
11. input Loss predictor output input output - output = ... How wrong I am @ellenkorbes
12. Gradient/Change input output predictor +1 +? @ellenkorbes
13. @ellenkorbes func pred(input, weight int) int { return weight * input } 1 2 3
14. @ellenkorbes func pred(input, weight int) int { return weight * input } 1 2 3 input 2 3 | weight | 4 4 result 8 12 × × = = difference = 4 input 2 2 | weight | 4 5 result 8 10 × × = = difference = 2 Input Gradient Weight Gradient
15. @ellenkorbes func pred(input, weight int) int { return weight * input } 1 2 3 input 2 3 | weight | 4 4 result 8 12 × × = = difference = 4 input 2 2 | weight | 4 5 result 8 10 × × = = difference = 2 Input Gradient Weight Gradient Input Gradient = Weight Weight Gradient = Input Weight Input
16. So far input neuron × weight + bias output output ... bias: +1 +1 weight: +1 +1×input input: +1 +1×weight input @ellenkorbes
17. Flowchart Step @ellenkorbes Predict Loss
18. Flowchart Step Neuron Predict @ellenkorbes Loss Change Neuron Change Learning Rate Update Parameters
19. We've got the basics. Let's add We've got the basics. Let's add some some complexity complexity. . @ellenkorbes
20. Neuron Layers input outputs n n n n n @ellenkorbes
21. Activation function Before input output 0 1 ? ? After @ellenkorbes
22. Bias Sigmoid function 0 1 no bias bias @ellenkorbes
23. Flowchart Step Activation Change @ellenkorbes Loss Change Neuron Predict* Learning Rate Update Parameters Loop Neuron Change * Includes activation function.
24. Flowchart Step Activation Change @ellenkorbes Loss Change Learning Rate Update Parameters Loop Neuron Change What. Neuron Predict* * Includes activation function.
25. @ellenkorbes func add2(x int) int { return x + 2 } func times6(x int) int { return x * 6 } func half(x int) int { return x / 2 } Gradient Chain Rule x := 7 first := half(times6(add2(x))) fmt.Println(first) Output: 27
26. @ellenkorbes func add2(x int) int { return x + 2 } func times6(x int) int { return x * 6 } func half(x int) int { return x / 2 } Gradient Chain Rule x := 7 first := half(times6(add2(x))) fmt.Println(first) x = x + 3 second := half(times6(add2(x))) fmt.Println(second) Output: 27 Output: ?
27. @ellenkorbes func add2(x int) int { return x + 2 } func times6(x int) int { return x * 6 } func half(x int) int { return x / 2 } Gradient Chain Rule Gradient: change × 1 add2(7) = 9 add2(6) = 8 Gradient: change × 6 times6(3) = 18 times6(4) = 24 Gradient: change × 0.5 half(40) = 20 half(30) = 15
28. @ellenkorbes func add2(x int) int { return x + 2 } func times6(x int) int { return x * 6 } func half(x int) int { return x / 2 } Gradient Chain Rule Previous output: 27 Input change: 3 Chained gradients: change × 1 × 6 × 0.5 Total gradient: change × 3 New output: previous output + input change × 3 27 + 3 × 3 36.
29. @ellenkorbes func add2(x int) int { return x + 2 } func times6(x int) int { return x * 6 } func half(x int) int { return x / 2 } Gradient Chain Rule x := 7 first := half(times6(add2(x))) fmt.Println(first) x = x + 3 second := half(times6(add2(x))) fmt.Println(second) Output: 27 Output: 36
30. Flowchart Step Activation Change @ellenkorbes Loss Change Neuron Predict Learning Rate Update Parameters Loop Neuron Change
31. All the building blocks are in All the building blocks are in place. Let's make it a place. Let's make it a network network. . @ellenkorbes
32. Network input output (1-neuron) layer n n n n n n n n n n n @ellenkorbes
33. Network gradient +? n n n n n n n n n n n +1 @ellenkorbes
34. @ellenkorbes Assembly Line Tape it. Label it.
35. @ellenkorbes Assembly Line Local Gradient Tape it. Label it. Network Gradient
36. Network gradient +X n n n +1 × ∂i ∂i + + bias gradients weight gradients input gradients +? n ∂i ∂i n ∂i ∂i @ellenkorbes
37. Network input How much does changing this neuron affect the end result? Just look at its gradients. n n n n n n n n n n n @ellenkorbes
38. How much does changing this neuron affect the end result? Look at its gradients, and... Network input n n n n n n n n n n n @ellenkorbes Multiply them by the next neurons' input gradient.
39. How much does changing this neuron affect the end result? Look at its gradients, and... Network input n n n n n n n n n n n @ellenkorbes Multiply them by the next neurons' input gradient... (Which is made of a chain of gradients that goes all the way to the end of the network.)
40. Flowchart Step @ellenkorbes Loss Change Layer Predict Learning Rate Update Parameters Loop Layer Change Current Layer Next Layer
41. @ellenkorbes Thanks Thanks! ! And huuuuuuuuuge thanks And huuuuuuuuuge thanks to to Francesc Campoy Francesc Campoy! ! ✨ ✨ ✨ ✨
42. ▪ developer relations ▪ ellen@garden.io ▪ @ellenkorbes ▪ they/them ellen körbes
43. ellen@garden.io | @ellenkorbes