Javier Gonzalez-Sanchez
February 02, 2022
770

# JGS594 Lecture 07

Software Engineering for Machine Learning
Performance Measurements
(202202)

## Javier Gonzalez-SanchezPRO

February 02, 2022

## Transcript

1. ### jgs SER 594 Software Engineering for Machine Learning Lecture 07:

Performance Measurement Dr. Javier Gonzalez-Sanchez [email protected] javiergs.engineering.asu.edu | javiergs.com PERALTA 230U Office Hours: By appointment

3. ### Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 3

jgs ND4J Input
4. ### Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 4

jgs DL4J | Our Model
5. ### Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 5

jgs DL4J | Training

7. ### Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 7

jgs Definition A summary of prediction results on a classification problem
8. ### Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 8

jgs Definition A summary of prediction results on a classification problem TP FP FN TN positive 0 negative 1 positive 0 negative 1
9. ### Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 9

jgs Definition TP FP FN TN positive 1 negative 0 positive 1 negative 0 FP Type 1 Error It is FALSE Computer said TRUE FP Type 2 Error It is TRUE Computer said FALSE
10. ### Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 10

jgs Definition What about not binary classifiers? e.g. emotion recognition TP FP FN TN positive 😀 😡 . 🙁 negative positive 😀 😡. 🙁 negative FN TN FP TN TN

12. ### Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 12

jgs Definition § overall measure of how much the model is correctly predicting on the entire set of data § Addition of the elements in the main diagonal divide by the sum of all the entries of the confusion matrix at the denominator.
13. ### Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 13

jgs Accuracy Accuracy = TP + TN / TP + TN + FP + FN TP FP FN TN positive 1 negative 0 positive 1 negative 0
14. ### Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 14

jgs Definition positive negative TP FP FN TN positive negative FN X FP X TN Accuracy = TP + TN / TP + TN + FP + FN + X

16. ### Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 16

jgs Accuracy / / How much we can trust the model when predict a Positive Precision = TP / TP + FP / / Measure the ability of the model to find all Positive units Recall = TP / TP + FN TP FP FN TN positive 1 negative 0 positive 1 negative 0
17. ### Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 17

jgs Definition positive negative TP FP FN TN positive negative FN X FP X TN / / How much we can trust the model when predict a Positive Precision = TP / TP + FP / / Measure the ability of the model to find all Positive units Recall = TP / TP + FN

19. ### Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 19

jgs Definition § The harmonic mean of precision and recall. § Mixture of: How much we can trust the model when predict a Positive (Precision), and The ability of the model to find all Positive units (Recall)
20. ### Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 20

jgs Harmonic mean a = 7 b = 3 / / if same units (big) Arithmetic mean = 7+3 / 2 = 5 / / if diverse units (small) Geometric mean = sqrt (7*3) = 4.58 / / ratios of diverse units (smaller) Harmonic mean = pow (sqrt (7*3)) / (7+3 / 2) = 21 / 5 = 4.2
21. ### Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 21

jgs F1-score Precision = TP / TP + FP / / predicted Recall = TP / TP + FN / / real F1-score = 2 * Precision * Recall / Precision + Recall TP FP FN TN positive 1 negative 0 positive 1 negative 0
22. ### Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 22

jgs Definition positive negative TP FP FN TN positive negative FN X FP X TN Precision = TP / TP + FP / / predicted Recall = TP / TP + FN / / real F1-score = 2 * Precision * Recall / Precision + Recall
23. ### Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 23

jgs Evaluation

25. ### Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 25

jgs Assignment Using our own implementation of a Neural Network (The code you analyzed before): § Can you create a method output that returns all outputs? It is OK can return a Java array. § Can you create a class Evaluation with its method eval? i.e., create a report similar to the one reviewed before
26. ### Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 26

jgs Questions
27. ### Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 27

jgs Reference § Deeplearning4j Suite Overview https://deeplearning4j.konduit.ai § Source Code https://github.com/javiergs/Medium/blob/main/ NeuralNetwork/ExampleXORWithDL4J.java
28. ### jgs SER 594 Software Engineering for Machine Learning Javier Gonzalez-Sanchez,

Ph.D. [email protected] Spring 2022 Copyright. These slides can only be used as study material for the class CSE205 at Arizona State University. They cannot be distributed or used for another purpose.