JGS594 Lecture 03

Software Engineering for Machine Learning
Introduction II
(202201)

Transcript

1. jgs
   SER 594
   Software Engineering for
   Machine Learning
   Lecture 03: Introduction II
   Dr. Javier Gonzalez-Sanchez
   javiergs@asu.edu
   javiergs.engineering.asu.edu | javiergs.com
   PERALTA 230U
   Office Hours: By appointment
   

   View Slide

2. jgs
   Previously …
   Models
   

   View Slide

3. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 3
   jgs
   Machine Learning
   

   View Slide

4. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 4
   jgs
   Neural Networks
   

   View Slide

5. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 5
   jgs
   Neural Networks Demystified
   https://medium.com/analytics-vidhya/neural-networks-in-a-nutshell-with-java-b4a635a2c4af
   

   View Slide

6. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 6
   jgs
   Activation Function
   

   View Slide

7. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 7
   jgs
   Neural Networks Demystified
   https://medium.com/analytics-vidhya/neural-networks-in-a-nutshell-with-java-b4a635a2c4af
   

   View Slide

8. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 8
   jgs
   Neural Networks Demystified
   

   View Slide

9. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 9
   jgs
   Neural Networks Demystified
   W [i][j] = W[weight][neuron]
   

   View Slide

10. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 10
    jgs
    Neural Networks Demystified
    W [k][i][j] = W[layer][weight] [neuron]
    

    View Slide

11. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 11
    jgs
    Neural Networks Demystified
    https://medium.com/analytics-vidhya/neural-networks-in-a-nutshell-with-java-b4a635a2c4af
    

    View Slide

12. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 12
    jgs
    Data to Data
    

    View Slide

13. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 13
    jgs
    Neural Network
    § And Where is the “learning” part?
    § It is in the calculation of the W values
    Calculate
    W values
    sig (W1*X1 + W2*X2 + W3*X3 + W4*X4)
    

    View Slide

14. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 14
    jgs
    Cost Function | Example MSE
    

    View Slide

15. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 15
    jgs
    Forward propagation
    § Use “random” values for W
    § Provide the network with one input and observing the output.
    § It is just linear algebra.
    § Calculate the error
    

    View Slide

16. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 16
    jgs
    Backward propagation
    § Your output will probably have a high error. Minimize it. How?
    § A proper adjustment of weights ensures that the subsequent output will be
    closer to the expected output. Repeat until the network can produce results
    significantly close enough to the known output.
    § Partial derivative of the cost function with respect to any weight in the
    network.
    § We want to see how the function changes as we let just one of those
    variables change while holding all the others constant.
    § Black-box
    

    View Slide

17. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 17
    jgs
    Problem
    § Your output will probably have a high error. Minimize it. How?
    § A proper adjustment of weights ensures that the subsequent
    output will be closer to the expected output. Repeat until the
    network can produce results significantly close enough to the
    known output.
    § How to modify the value of the weights? Short answer: use
    the gradient descent algorithm (1847), i.e., Partial
    derivatives of the error function with respect to each
    weight
    § We want to see how the function changes as we let just one
    of those variables change while holding all the others
    constant.
    

    View Slide

18. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 18
    jgs
    Back propagation
    Did I mention something called
    learning rate?
    Review Details (Math) Here:
    https://medium.com/analytics-vidhya/neural-networks-in-a-nutshell-with-java-b4a635a2c4af
    

    View Slide

19. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 19
    jgs
    Chain Rule
    Review Details (Math) Here:
    https://medium.com/analytics-vidhya/neural-networks-in-a-nutshell-with-java-b4a635a2c4af
    

    View Slide

20. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 20
    jgs
    Gradient descent
    Review Details (Math) Here:
    https://medium.com/analytics-vidhya/neural-networks-in-a-nutshell-with-java-b4a635a2c4af
    

    View Slide

21. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 21
    jgs
    A sneak peek at the Code
    

    View Slide

22. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 22
    jgs
    A sneak peek at the Code
    

    View Slide

23. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 23
    jgs
    Next
    CNN
    

    View Slide

24. jgs
    Summary
    Machine Learning
    

    View Slide

25. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 25
    jgs
    Data
    

    View Slide

26. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 26
    jgs
    Deep Learning
    

    View Slide

27. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 27
    jgs
    Tree
    

    View Slide

28. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 28
    jgs
    Forest | Bagging
    

    View Slide

29. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 29
    jgs
    Clustering
    

    View Slide

30. jgs
    To be Continued …
    

    View Slide

31. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 31
    jgs
    Questions
    

    View Slide

32. jgs
    SER 594 Software Engineering for Machine Learning
    Javier Gonzalez-Sanchez, Ph.D.
    javiergs@asu.edu
    Spring 2022
    Copyright. These slides can only be used as study material for the class CSE205 at Arizona State University.
    They cannot be distributed or used for another purpose.
    

    View Slide