JGS594 Lecture 03

Software Engineering for Machine Learning
Introduction II
(202201)

Transcript

1. jgs SER 594 Software Engineering for Machine Learning Lecture 03: Introduction II Dr. Javier Gonzalez-Sanchez javiergs@asu.edu javiergs.engineering.asu.edu | javiergs.com PERALTA 230U Office Hours: By appointment
2. jgs Previously … Models
3. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 3 jgs Machine Learning
4. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 4 jgs Neural Networks
5. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 5 jgs Neural Networks Demystified https://medium.com/analytics-vidhya/neural-networks-in-a-nutshell-with-java-b4a635a2c4af
6. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 6 jgs Activation Function
7. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 7 jgs Neural Networks Demystified https://medium.com/analytics-vidhya/neural-networks-in-a-nutshell-with-java-b4a635a2c4af
8. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 8 jgs Neural Networks Demystified
9. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 9 jgs Neural Networks Demystified W [i][j] = W[weight][neuron]
10. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 10 jgs Neural Networks Demystified W [k][i][j] = W[layer][weight] [neuron]
11. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 11 jgs Neural Networks Demystified https://medium.com/analytics-vidhya/neural-networks-in-a-nutshell-with-java-b4a635a2c4af
12. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 12 jgs Data to Data
13. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 13 jgs Neural Network § And Where is the “learning” part? § It is in the calculation of the W values Calculate W values sig (W1*X1 + W2*X2 + W3*X3 + W4*X4)
14. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 14 jgs Cost Function | Example MSE
15. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 15 jgs Forward propagation § Use “random” values for W § Provide the network with one input and observing the output. § It is just linear algebra. § Calculate the error
16. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 16 jgs Backward propagation § Your output will probably have a high error. Minimize it. How? § A proper adjustment of weights ensures that the subsequent output will be closer to the expected output. Repeat until the network can produce results significantly close enough to the known output. § Partial derivative of the cost function with respect to any weight in the network. § We want to see how the function changes as we let just one of those variables change while holding all the others constant. § Black-box
17. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 17 jgs Problem § Your output will probably have a high error. Minimize it. How? § A proper adjustment of weights ensures that the subsequent output will be closer to the expected output. Repeat until the network can produce results significantly close enough to the known output. § How to modify the value of the weights? Short answer: use the gradient descent algorithm (1847), i.e., Partial derivatives of the error function with respect to each weight § We want to see how the function changes as we let just one of those variables change while holding all the others constant.
18. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 18 jgs Back propagation Did I mention something called learning rate? Review Details (Math) Here: https://medium.com/analytics-vidhya/neural-networks-in-a-nutshell-with-java-b4a635a2c4af
19. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 19 jgs Chain Rule Review Details (Math) Here: https://medium.com/analytics-vidhya/neural-networks-in-a-nutshell-with-java-b4a635a2c4af
20. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 20 jgs Gradient descent Review Details (Math) Here: https://medium.com/analytics-vidhya/neural-networks-in-a-nutshell-with-java-b4a635a2c4af
21. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 21 jgs A sneak peek at the Code
22. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 22 jgs A sneak peek at the Code
23. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 23 jgs Next CNN
24. jgs Summary Machine Learning
25. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 25 jgs Data
26. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 26 jgs Deep Learning
27. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 27 jgs Tree
28. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 28 jgs Forest | Bagging
29. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 29 jgs Clustering
30. jgs To be Continued …
31. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 31 jgs Questions
32. jgs SER 594 Software Engineering for Machine Learning Javier Gonzalez-Sanchez, Ph.D. javiergs@asu.edu Spring 2022 Copyright. These slides can only be used as study material for the class CSE205 at Arizona State University. They cannot be distributed or used for another purpose.