$30 off During Our Annual Pro Sale. View Details »

JGS594 Lecture 03

JGS594 Lecture 03

Software Engineering for Machine Learning
Introduction II
(202201)

Javier Gonzalez-Sanchez
PRO

January 18, 2022
Tweet

More Decks by Javier Gonzalez-Sanchez

Other Decks in Programming

Transcript

  1. jgs
    SER 594
    Software Engineering for
    Machine Learning
    Lecture 03: Introduction II
    Dr. Javier Gonzalez-Sanchez
    [email protected]
    javiergs.engineering.asu.edu | javiergs.com
    PERALTA 230U
    Office Hours: By appointment

    View Slide

  2. jgs
    Previously …
    Models

    View Slide

  3. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 3
    jgs
    Machine Learning

    View Slide

  4. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 4
    jgs
    Neural Networks

    View Slide

  5. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 5
    jgs
    Neural Networks Demystified
    https://medium.com/analytics-vidhya/neural-networks-in-a-nutshell-with-java-b4a635a2c4af

    View Slide

  6. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 6
    jgs
    Activation Function

    View Slide

  7. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 7
    jgs
    Neural Networks Demystified
    https://medium.com/analytics-vidhya/neural-networks-in-a-nutshell-with-java-b4a635a2c4af

    View Slide

  8. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 8
    jgs
    Neural Networks Demystified

    View Slide

  9. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 9
    jgs
    Neural Networks Demystified
    W [i][j] = W[weight][neuron]

    View Slide

  10. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 10
    jgs
    Neural Networks Demystified
    W [k][i][j] = W[layer][weight] [neuron]

    View Slide

  11. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 11
    jgs
    Neural Networks Demystified
    https://medium.com/analytics-vidhya/neural-networks-in-a-nutshell-with-java-b4a635a2c4af

    View Slide

  12. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 12
    jgs
    Data to Data

    View Slide

  13. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 13
    jgs
    Neural Network
    § And Where is the “learning” part?
    § It is in the calculation of the W values
    Calculate
    W values
    sig (W1*X1 + W2*X2 + W3*X3 + W4*X4)

    View Slide

  14. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 14
    jgs
    Cost Function | Example MSE

    View Slide

  15. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 15
    jgs
    Forward propagation
    § Use “random” values for W
    § Provide the network with one input and observing the output.
    § It is just linear algebra.
    § Calculate the error

    View Slide

  16. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 16
    jgs
    Backward propagation
    § Your output will probably have a high error. Minimize it. How?
    § A proper adjustment of weights ensures that the subsequent output will be
    closer to the expected output. Repeat until the network can produce results
    significantly close enough to the known output.
    § Partial derivative of the cost function with respect to any weight in the
    network.
    § We want to see how the function changes as we let just one of those
    variables change while holding all the others constant.
    § Black-box

    View Slide

  17. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 17
    jgs
    Problem
    § Your output will probably have a high error. Minimize it. How?
    § A proper adjustment of weights ensures that the subsequent
    output will be closer to the expected output. Repeat until the
    network can produce results significantly close enough to the
    known output.
    § How to modify the value of the weights? Short answer: use
    the gradient descent algorithm (1847), i.e., Partial
    derivatives of the error function with respect to each
    weight
    § We want to see how the function changes as we let just one
    of those variables change while holding all the others
    constant.

    View Slide

  18. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 18
    jgs
    Back propagation
    Did I mention something called
    learning rate?
    Review Details (Math) Here:
    https://medium.com/analytics-vidhya/neural-networks-in-a-nutshell-with-java-b4a635a2c4af

    View Slide

  19. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 19
    jgs
    Chain Rule
    Review Details (Math) Here:
    https://medium.com/analytics-vidhya/neural-networks-in-a-nutshell-with-java-b4a635a2c4af

    View Slide

  20. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 20
    jgs
    Gradient descent
    Review Details (Math) Here:
    https://medium.com/analytics-vidhya/neural-networks-in-a-nutshell-with-java-b4a635a2c4af

    View Slide

  21. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 21
    jgs
    A sneak peek at the Code

    View Slide

  22. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 22
    jgs
    A sneak peek at the Code

    View Slide

  23. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 23
    jgs
    Next
    CNN

    View Slide

  24. jgs
    Summary
    Machine Learning

    View Slide

  25. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 25
    jgs
    Data

    View Slide

  26. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 26
    jgs
    Deep Learning

    View Slide

  27. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 27
    jgs
    Tree

    View Slide

  28. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 28
    jgs
    Forest | Bagging

    View Slide

  29. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 29
    jgs
    Clustering

    View Slide

  30. jgs
    To be Continued …

    View Slide

  31. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 31
    jgs
    Questions

    View Slide

  32. jgs
    SER 594 Software Engineering for Machine Learning
    Javier Gonzalez-Sanchez, Ph.D.
    [email protected]
    Spring 2022
    Copyright. These slides can only be used as study material for the class CSE205 at Arizona State University.
    They cannot be distributed or used for another purpose.

    View Slide