Neural networks basis

Presentation for Data Science Łódź

Transcript

1. Podstawy Sieci neuronowych Michal Gutowski Neural networks - quick overview Small characteristics Example Neuron model Real neuron Artificial neuron Learning Supervised learning Delta rule Activation function Perceptron Example time! Little conclusion 1/20 Podstawy Sieci neuronowych Michal Gutowski April 5, 2016
2. Podstawy Sieci neuronowych Michal Gutowski Neural networks - quick overview Small characteristics Example Neuron model Real neuron Artificial neuron Learning Supervised learning Delta rule Activation function Perceptron Example time! Little conclusion 2/20 Outline 1 Neural networks - quick overview Small characteristics Example 2 Neuron model Real neuron Artificial neuron 3 Learning Supervised learning Delta rule 4 Activation function 5 Perceptron 6 Example time! 7 Little conclusion 8 When to use vs. when not to use
3. Podstawy Sieci neuronowych Michal Gutowski Neural networks - quick overview Small characteristics Example Neuron model Real neuron Artificial neuron Learning Supervised learning Delta rule Activation function Perceptron Example time! Little conclusion 3/20 Quick overview Created in order to work like a brain Consists of several neurons Used where results depends on many inputs or where you are not sure what are you looking for
4. Podstawy Sieci neuronowych Michal Gutowski Neural networks - quick overview Small characteristics Example Neuron model Real neuron Artificial neuron Learning Supervised learning Delta rule Activation function Perceptron Example time! Little conclusion 4/20 Diet application You need to provide sufficient levels of carbohydrates proteins fats vitamins water (...) What you eat today depends on meals from a period of time - you won’t eat the same meals everyday Your meals should take under consideration you activities on a given day
5. Podstawy Sieci neuronowych Michal Gutowski Neural networks - quick overview Small characteristics Example Neuron model Real neuron Artificial neuron Learning Supervised learning Delta rule Activation function Perceptron Example time! Little conclusion 5/20 Real neuron Source: http://shajitheodore.com/neuroscience/drug- addiction/neuron-the-nerve-cell/neuron-the-nerve- cell/ obtains data from other neurons electrical charges are being transmitted through the axon sends data to another neuron if charge is high enough
6. Podstawy Sieci neuronowych Michal Gutowski Neural networks - quick overview Small characteristics Example Neuron model Real neuron Artificial neuron Learning Supervised learning Delta rule Activation function Perceptron Example time! Little conclusion 6/20 Neuron model x1w1 + x2w2 + x3w3
7. Podstawy Sieci neuronowych Michal Gutowski Neural networks - quick overview Small characteristics Example Neuron model Real neuron Artificial neuron Learning Supervised learning Delta rule Activation function Perceptron Example time! Little conclusion 6/20 Neuron model x1w1 + x2w2 + x3w3 y = i=N i=1 xi wi where N is number of inputs
8. Podstawy Sieci neuronowych Michal Gutowski Neural networks - quick overview Small characteristics Example Neuron model Real neuron Artificial neuron Learning Supervised learning Delta rule Activation function Perceptron Example time! Little conclusion 7/20 Supervised learning We must create a learning set Learning set consists of known inputs with known outputs Algorithm is as follows 1 calculate output for given training entry 2 calculate error 3 update weights We repeat the algorithm until results are OK Each full run over training set is called training epoch
9. Podstawy Sieci neuronowych Michal Gutowski Neural networks - quick overview Small characteristics Example Neuron model Real neuron Artificial neuron Learning Supervised learning Delta rule Activation function Perceptron Example time! Little conclusion 8/20 Example 2 inputs with weights w1 = 1, w2 = 2 expected output t = 3 inputs x1 = 1, x2 = 1 current output y = x1 ∗ w1 + x2 ∗ w2 = 1 ∗ 1 + 1 ∗ 1 = 2
10. Podstawy Sieci neuronowych Michal Gutowski Neural networks - quick overview Small characteristics Example Neuron model Real neuron Artificial neuron Learning Supervised learning Delta rule Activation function Perceptron Example time! Little conclusion 8/20 Example 2 inputs with weights w1 = 1, w2 = 2 expected output t = 3 inputs x1 = 1, x2 = 1 current output y = x1 ∗ w1 + x2 ∗ w2 = 1 ∗ 1 + 1 ∗ 1 = 2 error error = t − y = 3 − 2 = 1
11. Podstawy Sieci neuronowych Michal Gutowski Neural networks - quick overview Small characteristics Example Neuron model Real neuron Artificial neuron Learning Supervised learning Delta rule Activation function Perceptron Example time! Little conclusion 9/20 Delta rule ∆wi = α(t − y)xi ∆wi how should a weight change α small constant - learning rate t expected output from a training set y actual output xi input from training set
12. Podstawy Sieci neuronowych Michal Gutowski Neural networks - quick overview Small characteristics Example Neuron model Real neuron Artificial neuron Learning Supervised learning Delta rule Activation function Perceptron Example time! Little conclusion 10/20 Delta rule - example 2 inputs with weights w1 = 1, w2 = 2 expected output t = 4 inputs x1 = 1, x2 = 2 learning rate α = 0.1 current output y = x1 ∗ w1 + x2 ∗ w2 = 1 ∗ 1 + 1 ∗ 2 = 3 error error = t − y = 4 − 3 = 1 ∆wi = α(t − y)xi
13. Podstawy Sieci neuronowych Michal Gutowski Neural networks - quick overview Small characteristics Example Neuron model Real neuron Artificial neuron Learning Supervised learning Delta rule Activation function Perceptron Example time! Little conclusion 10/20 Delta rule - example 2 inputs with weights w1 = 1, w2 = 2 expected output t = 4 inputs x1 = 1, x2 = 2 learning rate α = 0.1 current output y = x1 ∗ w1 + x2 ∗ w2 = 1 ∗ 1 + 1 ∗ 2 = 3 error error = t − y = 4 − 3 = 1 ∆wi = α(t − y)xi ∆w1 = 0.1(4−3)∗1 = 0.1
14. Podstawy Sieci neuronowych Michal Gutowski Neural networks - quick overview Small characteristics Example Neuron model Real neuron Artificial neuron Learning Supervised learning Delta rule Activation function Perceptron Example time! Little conclusion 10/20 Delta rule - example 2 inputs with weights w1 = 1, w2 = 2 expected output t = 4 inputs x1 = 1, x2 = 2 learning rate α = 0.1 current output y = x1 ∗ w1 + x2 ∗ w2 = 1 ∗ 1 + 1 ∗ 2 = 3 error error = t − y = 4 − 3 = 1 ∆wi = α(t − y)xi ∆w1 = 0.1(4−3)∗1 = 0.1 ∆w2 = 0.1(4−3)∗2 = 0.2
15. Podstawy Sieci neuronowych Michal Gutowski Neural networks - quick overview Small characteristics Example Neuron model Real neuron Artificial neuron Learning Supervised learning Delta rule Activation function Perceptron Example time! Little conclusion 10/20 Delta rule - example 2 inputs with weights w1 = 1, w2 = 2 expected output t = 4 inputs x1 = 1, x2 = 2 learning rate α = 0.1 current output y = x1 ∗ w1 + x2 ∗ w2 = 1 ∗ 1 + 1 ∗ 2 = 3 error error = t − y = 4 − 3 = 1 ∆wi = α(t − y)xi ∆w1 = 0.1(4−3)∗1 = 0.1 ∆w2 = 0.1(4−3)∗2 = 0.2 w1 = 1 + 0.1 = 1.1 w2 = 1 + 0.2 = 1.2
16. Podstawy Sieci neuronowych Michal Gutowski Neural networks - quick overview Small characteristics Example Neuron model Real neuron Artificial neuron Learning Supervised learning Delta rule Activation function Perceptron Example time! Little conclusion 11/20 Neuron model y = F( i=N i=1 xi wi ) where N is number of inputs y = x y = 1 1+ex
17. Podstawy Sieci neuronowych Michal Gutowski Neural networks - quick overview Small characteristics Example Neuron model Real neuron Artificial neuron Learning Supervised learning Delta rule Activation function Perceptron Example time! Little conclusion 12/20 Delta rule ∆wi = α(t − y)g (h)xi ∆wi how should a weight change α small constant - learning rate t expected output from a training set y actual output h is what y used to be previously xi input from training set h = i=N i=1 xi wi g (h) derivative of activation function in given point
18. Podstawy Sieci neuronowych Michal Gutowski Neural networks - quick overview Small characteristics Example Neuron model Real neuron Artificial neuron Learning Supervised learning Delta rule Activation function Perceptron Example time! Little conclusion 13/20 Perceptron basics One of the best known neural networks Used mostly for classification Invented in 1957
19. Podstawy Sieci neuronowych Michal Gutowski Neural networks - quick overview Small characteristics Example Neuron model Real neuron Artificial neuron Learning Supervised learning Delta rule Activation function Perceptron Example time! Little conclusion 14/20 First perceptron Source: https://en.wikipedia.org/wiki/Perceptron
20. Podstawy Sieci neuronowych Michal Gutowski Neural networks - quick overview Small characteristics Example Neuron model Real neuron Artificial neuron Learning Supervised learning Delta rule Activation function Perceptron Example time! Little conclusion 15/20 3 layer perceptron Source: http://neuroph.sourceforge.net
21. Podstawy Sieci neuronowych Michal Gutowski Neural networks - quick overview Small characteristics Example Neuron model Real neuron Artificial neuron Learning Supervised learning Delta rule Activation function Perceptron Example time! Little conclusion 16/20 Classification
22. Podstawy Sieci neuronowych Michal Gutowski Neural networks - quick overview Small characteristics Example Neuron model Real neuron Artificial neuron Learning Supervised learning Delta rule Activation function Perceptron Example time! Little conclusion 17/20 Example Output - 1 everything ok, 0 possible problems input x1 1 tall 0 small input x2 1 big weight 0 small weight
23. Podstawy Sieci neuronowych Michal Gutowski Neural networks - quick overview Small characteristics Example Neuron model Real neuron Artificial neuron Learning Supervised learning Delta rule Activation function Perceptron Example time! Little conclusion 18/20 Example
24. Podstawy Sieci neuronowych Michal Gutowski Neural networks - quick overview Small characteristics Example Neuron model Real neuron Artificial neuron Learning Supervised learning Delta rule Activation function Perceptron Example time! Little conclusion 19/20 Example decide on inputs/outputs create a training set create the network observe output and error adjust parameters
25. Podstawy Sieci neuronowych Michal Gutowski Neural networks - quick overview Small characteristics Example Neuron model Real neuron Artificial neuron Learning Supervised learning Delta rule Activation function Perceptron Example time! Little conclusion 20/20 When to use vs when not to use When to use? a lot of inputs not sure what are you looking for When not to use?
26. Podstawy Sieci neuronowych Michal Gutowski Neural networks - quick overview Small characteristics Example Neuron model Real neuron Artificial neuron Learning Supervised learning Delta rule Activation function Perceptron Example time! Little conclusion 20/20 When to use vs when not to use When to use? a lot of inputs not sure what are you looking for When not to use? there exists any other solution