JGS594 Lecture 06

Software Engineering for Machine Learning
DeepLearning4J
(202202)

Transcript

1. jgs SER 594 Software Engineering for Machine Learning Lecture 06: Deeplearning4j Overview Dr. Javier Gonzalez-Sanchez javiergs@asu.edu javiergs.engineering.asu.edu | javiergs.com PERALTA 230U Office Hours: By appointment
2. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 2 jgs Next From Scratch
3. jgs ML Frameworks and Libraries
4. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 4 jgs ML Frameworks and Libraries § Machine Learning and Deep Learning frameworks and libraries for large-scale data mining: a survey. (2019). Nguyen, G., Dlugolinsky, S., Bobak, M. et al. Artificial Intelligence Review. (52), 77–124 https://doi.org/10.1007/s10462-018-09679-z § It is posted on Canvas
5. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 5 jgs ML Frameworks and Libraries
6. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 6 jgs Eclipse DeepLearning4j § Widely adopted as a commercial, industry-focused, and distributed deep learning platform (library + framework), for running deep learning on the JVM. § Deeplearning4j has Java, Scala and Python APIs. § Deep network support through RBM (restricted Boltzmann machine ), DBN (deep belief network), Convolution Neural Networks (CNN), Recurrent Neural Networks (RNN), Recursive Neural Tensor Network (RNTN) and Long Short-Term Memory (LTSM). § Comes with built-in GPU support, which is an important feature for the training process and supports Hadoop and Spark.
7. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 7 jgs DL4J Usage § It shows matchless potential for image recognition, fraud detection, text- mining, parts of speech tagging, and natural language processing.
8. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 8 jgs DL4J Modules § Datavec: A data transformation library converting raw input data to tensors (N-dimensional array of data) suitable for running neural networks on. § Nd4j: numpy ++ for java. Contains a mix of numpy operations and tensorflow/pytorch operations. Native libraries support for matrix data processing on CPU and GPU § Python4j: A python script execution framework easing deployment of python scripts into production. § Samediff: a tensorflow/pytorch like framework for execution of complex graphs. This framework is lower level, but very flexible. It's also the base API for running onnx and tensorflow graphs.
9. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 9 jgs DL4J Modules § Libnd4j: A lightweight, standalone C++ library enable math code to run on different devices. Optimizable for running on a wide variety of devices. § Apache Spark Integration: An integration with the Apache Spark framework enabling execution of deep learning pipelines on clusters with implicit data parallelism and fault tolerance
10. jgs DeepLearning4J nuts and bolts
11. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 11 jgs Goal
12. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 12 jgs Goal
13. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 13 jgs Getting Started
14. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 14 jgs Prerequisites § datavec-api.jar —handle vectorizing and loading data § deeplearning4j-core.jar – handle neural network implementations. § nd4j-native-platform.jar – handle n-dimensional arrays (matrix) operations
15. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 15 jgs Note Do we know how to add libraries (JAR files) to our projects? https://deeplearning4j.konduit.ai /multi-project/explanation/build-tools
16. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 16 jgs ND4J Load Data § We need data outside the JVM § We need to initialize the data (no JVM, no default initialization)
17. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 17 jgs ND4J Input (coordinates, value)
18. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 18 jgs ND4J Output (coordinates, value)
19. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 19 jgs ND4J DataSet
20. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 20 jgs DL4J | Our Model * Dense layer – a layer that is deeply connected with its preceding layer which means the neurons of the layer are connected to every neuron of its preceding layer.
21. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 21 jgs DL4J | Our Model
22. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 22 jgs DL4J | Training
23. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 23 jgs DL4J | Testing
24. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 24 jgs Results The report includes: § Confusion matrix entries — in our example, there are two true positives (TP) and two true negatives (TN) predicted. § Accuracy — the measure of all the correctly identified cases. § Precision — the measure of the correctly identified positive cases from all the predicted positive cases. § F1-score — the harmonic mean of precision and recall.
25. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 25 jgs Confusion matrix
26. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 26 jgs Accuracy, Precision, Recall, F1-score Accuracy = TP + TN / TP + TN + FP + FN Precision = TP / TP + FP / / predicted Recall = TP / TP + FN / / real F1-score = 2 * Precision * Recall / Precision + Recall
27. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 27 jgs Harmonic mean a = 7 b = 3 / / if same units (big) Arithmetic mean = 7+3 / 2 = 5 / / if diverse units (small) Geometric mean = sqrt (7*3) = 4.58 / / ratios of diverse units (smaller) Harmonic mean = pow (sqrt (7*3)) / (7+3 / 2) = 21 / 5 = 4.2
28. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 28 jgs Questions
29. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 29 jgs Reference § Deeplearning4j Suite Overview https://deeplearning4j.konduit.ai § Source Code https://github.com/javiergs/Medium/blob/main/ NeuralNetwork/ExampleXORWithDL4J.java
30. jgs SER 594 Software Engineering for Machine Learning Javier Gonzalez-Sanchez, Ph.D. javiergs@asu.edu Spring 2022 Copyright. These slides can only be used as study material for the class CSE205 at Arizona State University. They cannot be distributed or used for another purpose.