Upgrade to Pro — share decks privately, control downloads, hide ads and more …

JGS594 Lecture 06

JGS594 Lecture 06

Software Engineering for Machine Learning
DeepLearning4J
(202202)

B546a9b97d993392e4b22b74b99b91fe?s=128

Javier Gonzalez
PRO

February 01, 2022
Tweet

More Decks by Javier Gonzalez

Other Decks in Programming

Transcript

  1. jgs SER 594 Software Engineering for Machine Learning Lecture 06:

    Deeplearning4j Overview Dr. Javier Gonzalez-Sanchez javiergs@asu.edu javiergs.engineering.asu.edu | javiergs.com PERALTA 230U Office Hours: By appointment
  2. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 2

    jgs Next From Scratch
  3. jgs ML Frameworks and Libraries

  4. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 4

    jgs ML Frameworks and Libraries § Machine Learning and Deep Learning frameworks and libraries for large-scale data mining: a survey. (2019). Nguyen, G., Dlugolinsky, S., Bobak, M. et al. Artificial Intelligence Review. (52), 77–124 https://doi.org/10.1007/s10462-018-09679-z § It is posted on Canvas
  5. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 5

    jgs ML Frameworks and Libraries
  6. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 6

    jgs Eclipse DeepLearning4j § Widely adopted as a commercial, industry-focused, and distributed deep learning platform (library + framework), for running deep learning on the JVM. § Deeplearning4j has Java, Scala and Python APIs. § Deep network support through RBM (restricted Boltzmann machine ), DBN (deep belief network), Convolution Neural Networks (CNN), Recurrent Neural Networks (RNN), Recursive Neural Tensor Network (RNTN) and Long Short-Term Memory (LTSM). § Comes with built-in GPU support, which is an important feature for the training process and supports Hadoop and Spark.
  7. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 7

    jgs DL4J Usage § It shows matchless potential for image recognition, fraud detection, text- mining, parts of speech tagging, and natural language processing.
  8. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 8

    jgs DL4J Modules § Datavec: A data transformation library converting raw input data to tensors (N-dimensional array of data) suitable for running neural networks on. § Nd4j: numpy ++ for java. Contains a mix of numpy operations and tensorflow/pytorch operations. Native libraries support for matrix data processing on CPU and GPU § Python4j: A python script execution framework easing deployment of python scripts into production. § Samediff: a tensorflow/pytorch like framework for execution of complex graphs. This framework is lower level, but very flexible. It's also the base API for running onnx and tensorflow graphs.
  9. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 9

    jgs DL4J Modules § Libnd4j: A lightweight, standalone C++ library enable math code to run on different devices. Optimizable for running on a wide variety of devices. § Apache Spark Integration: An integration with the Apache Spark framework enabling execution of deep learning pipelines on clusters with implicit data parallelism and fault tolerance
  10. jgs DeepLearning4J nuts and bolts

  11. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 11

    jgs Goal
  12. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 12

    jgs Goal
  13. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 13

    jgs Getting Started
  14. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 14

    jgs Prerequisites § datavec-api.jar —handle vectorizing and loading data § deeplearning4j-core.jar – handle neural network implementations. § nd4j-native-platform.jar – handle n-dimensional arrays (matrix) operations
  15. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 15

    jgs Note Do we know how to add libraries (JAR files) to our projects? https://deeplearning4j.konduit.ai /multi-project/explanation/build-tools
  16. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 16

    jgs ND4J Load Data § We need data outside the JVM § We need to initialize the data (no JVM, no default initialization)
  17. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 17

    jgs ND4J Input (coordinates, value)
  18. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 18

    jgs ND4J Output (coordinates, value)
  19. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 19

    jgs ND4J DataSet
  20. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 20

    jgs DL4J | Our Model * Dense layer – a layer that is deeply connected with its preceding layer which means the neurons of the layer are connected to every neuron of its preceding layer.
  21. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 21

    jgs DL4J | Our Model
  22. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 22

    jgs DL4J | Training
  23. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 23

    jgs DL4J | Testing
  24. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 24

    jgs Results The report includes: § Confusion matrix entries — in our example, there are two true positives (TP) and two true negatives (TN) predicted. § Accuracy — the measure of all the correctly identified cases. § Precision — the measure of the correctly identified positive cases from all the predicted positive cases. § F1-score — the harmonic mean of precision and recall.
  25. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 25

    jgs Confusion matrix
  26. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 26

    jgs Accuracy, Precision, Recall, F1-score Accuracy = TP + TN / TP + TN + FP + FN Precision = TP / TP + FP / / predicted Recall = TP / TP + FN / / real F1-score = 2 * Precision * Recall / Precision + Recall
  27. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 27

    jgs Harmonic mean a = 7 b = 3 / / if same units (big) Arithmetic mean = 7+3 / 2 = 5 / / if diverse units (small) Geometric mean = sqrt (7*3) = 4.58 / / ratios of diverse units (smaller) Harmonic mean = pow (sqrt (7*3)) / (7+3 / 2) = 21 / 5 = 4.2
  28. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 28

    jgs Questions
  29. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 29

    jgs Reference § Deeplearning4j Suite Overview https://deeplearning4j.konduit.ai § Source Code https://github.com/javiergs/Medium/blob/main/ NeuralNetwork/ExampleXORWithDL4J.java
  30. jgs SER 594 Software Engineering for Machine Learning Javier Gonzalez-Sanchez,

    Ph.D. javiergs@asu.edu Spring 2022 Copyright. These slides can only be used as study material for the class CSE205 at Arizona State University. They cannot be distributed or used for another purpose.