Upgrade to Pro — share decks privately, control downloads, hide ads and more …

JGS594 Lecture 03

JGS594 Lecture 03

Software Engineering for Machine Learning
Introduction II
(202201)

B546a9b97d993392e4b22b74b99b91fe?s=128

Javier Gonzalez
PRO

January 18, 2022
Tweet

More Decks by Javier Gonzalez

Other Decks in Programming

Transcript

  1. jgs SER 594 Software Engineering for Machine Learning Lecture 03:

    Introduction II Dr. Javier Gonzalez-Sanchez javiergs@asu.edu javiergs.engineering.asu.edu | javiergs.com PERALTA 230U Office Hours: By appointment
  2. jgs Previously … Models

  3. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 3

    jgs Machine Learning
  4. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 4

    jgs Neural Networks
  5. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 5

    jgs Neural Networks Demystified https://medium.com/analytics-vidhya/neural-networks-in-a-nutshell-with-java-b4a635a2c4af
  6. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 6

    jgs Activation Function
  7. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 7

    jgs Neural Networks Demystified https://medium.com/analytics-vidhya/neural-networks-in-a-nutshell-with-java-b4a635a2c4af
  8. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 8

    jgs Neural Networks Demystified
  9. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 9

    jgs Neural Networks Demystified W [i][j] = W[weight][neuron]
  10. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 10

    jgs Neural Networks Demystified W [k][i][j] = W[layer][weight] [neuron]
  11. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 11

    jgs Neural Networks Demystified https://medium.com/analytics-vidhya/neural-networks-in-a-nutshell-with-java-b4a635a2c4af
  12. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 12

    jgs Data to Data
  13. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 13

    jgs Neural Network § And Where is the “learning” part? § It is in the calculation of the W values Calculate W values sig (W1*X1 + W2*X2 + W3*X3 + W4*X4)
  14. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 14

    jgs Cost Function | Example MSE
  15. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 15

    jgs Forward propagation § Use “random” values for W § Provide the network with one input and observing the output. § It is just linear algebra. § Calculate the error
  16. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 16

    jgs Backward propagation § Your output will probably have a high error. Minimize it. How? § A proper adjustment of weights ensures that the subsequent output will be closer to the expected output. Repeat until the network can produce results significantly close enough to the known output. § Partial derivative of the cost function with respect to any weight in the network. § We want to see how the function changes as we let just one of those variables change while holding all the others constant. § Black-box
  17. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 17

    jgs Problem § Your output will probably have a high error. Minimize it. How? § A proper adjustment of weights ensures that the subsequent output will be closer to the expected output. Repeat until the network can produce results significantly close enough to the known output. § How to modify the value of the weights? Short answer: use the gradient descent algorithm (1847), i.e., Partial derivatives of the error function with respect to each weight § We want to see how the function changes as we let just one of those variables change while holding all the others constant.
  18. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 18

    jgs Back propagation Did I mention something called learning rate? Review Details (Math) Here: https://medium.com/analytics-vidhya/neural-networks-in-a-nutshell-with-java-b4a635a2c4af
  19. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 19

    jgs Chain Rule Review Details (Math) Here: https://medium.com/analytics-vidhya/neural-networks-in-a-nutshell-with-java-b4a635a2c4af
  20. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 20

    jgs Gradient descent Review Details (Math) Here: https://medium.com/analytics-vidhya/neural-networks-in-a-nutshell-with-java-b4a635a2c4af
  21. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 21

    jgs A sneak peek at the Code
  22. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 22

    jgs A sneak peek at the Code
  23. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 23

    jgs Next CNN
  24. jgs Summary Machine Learning

  25. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 25

    jgs Data
  26. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 26

    jgs Deep Learning
  27. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 27

    jgs Tree
  28. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 28

    jgs Forest | Bagging
  29. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 29

    jgs Clustering
  30. jgs To be Continued …

  31. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 31

    jgs Questions
  32. jgs SER 594 Software Engineering for Machine Learning Javier Gonzalez-Sanchez,

    Ph.D. javiergs@asu.edu Spring 2022 Copyright. These slides can only be used as study material for the class CSE205 at Arizona State University. They cannot be distributed or used for another purpose.