$30 off During Our Annual Pro Sale. View Details »

JGS594 Lecture 06

JGS594 Lecture 06

Software Engineering for Machine Learning
DeepLearning4J
(202202)

Javier Gonzalez-Sanchez
PRO

February 01, 2022
Tweet

More Decks by Javier Gonzalez-Sanchez

Other Decks in Programming

Transcript

  1. jgs
    SER 594
    Software Engineering for
    Machine Learning
    Lecture 06: Deeplearning4j Overview
    Dr. Javier Gonzalez-Sanchez
    [email protected]
    javiergs.engineering.asu.edu | javiergs.com
    PERALTA 230U
    Office Hours: By appointment

    View Slide

  2. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 2
    jgs
    Next
    From
    Scratch

    View Slide

  3. jgs
    ML Frameworks and Libraries

    View Slide

  4. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 4
    jgs
    ML Frameworks and Libraries
    § Machine Learning and Deep Learning frameworks and libraries for
    large-scale data mining: a survey. (2019). Nguyen, G., Dlugolinsky, S.,
    Bobak, M. et al. Artificial Intelligence Review. (52), 77–124
    https://doi.org/10.1007/s10462-018-09679-z
    § It is posted on Canvas

    View Slide

  5. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 5
    jgs
    ML Frameworks and Libraries

    View Slide

  6. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 6
    jgs
    Eclipse DeepLearning4j
    § Widely adopted as a commercial, industry-focused, and distributed deep
    learning platform (library + framework), for running deep learning on the
    JVM.
    § Deeplearning4j has Java, Scala and Python APIs.
    § Deep network support through RBM (restricted Boltzmann machine ), DBN
    (deep belief network), Convolution Neural Networks (CNN), Recurrent
    Neural Networks (RNN), Recursive Neural Tensor Network (RNTN) and Long
    Short-Term Memory (LTSM).
    § Comes with built-in GPU support, which is an important feature for the
    training process and supports Hadoop and Spark.

    View Slide

  7. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 7
    jgs
    DL4J Usage
    § It shows matchless potential for image recognition, fraud detection, text-
    mining, parts of speech tagging, and natural language processing.

    View Slide

  8. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 8
    jgs
    DL4J Modules
    § Datavec: A data transformation library converting raw input data to tensors
    (N-dimensional array of data) suitable for running neural networks on.
    § Nd4j: numpy ++ for java. Contains a mix of numpy operations and
    tensorflow/pytorch operations. Native libraries support for matrix data
    processing on CPU and GPU
    § Python4j: A python script execution framework easing deployment of
    python scripts into production.
    § Samediff: a tensorflow/pytorch like framework for execution of complex
    graphs. This framework is lower level, but very flexible. It's also the base API
    for running onnx and tensorflow graphs.

    View Slide

  9. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 9
    jgs
    DL4J Modules
    § Libnd4j: A lightweight, standalone C++ library enable math code to run on
    different devices. Optimizable for running on a wide variety of devices.
    § Apache Spark Integration: An integration with the Apache Spark
    framework enabling execution of deep learning pipelines on clusters with
    implicit data parallelism and fault tolerance

    View Slide

  10. jgs
    DeepLearning4J
    nuts and bolts

    View Slide

  11. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 11
    jgs
    Goal

    View Slide

  12. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 12
    jgs
    Goal

    View Slide

  13. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 13
    jgs
    Getting Started

    View Slide

  14. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 14
    jgs
    Prerequisites
    § datavec-api.jar —handle vectorizing and loading data
    § deeplearning4j-core.jar – handle neural network implementations.
    § nd4j-native-platform.jar – handle n-dimensional arrays (matrix) operations

    View Slide

  15. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 15
    jgs
    Note
    Do we know how to add libraries
    (JAR files) to our projects?
    https://deeplearning4j.konduit.ai
    /multi-project/explanation/build-tools

    View Slide

  16. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 16
    jgs
    ND4J Load Data
    § We need data outside the JVM
    § We need to initialize the data (no JVM, no default initialization)

    View Slide

  17. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 17
    jgs
    ND4J Input
    (coordinates, value)

    View Slide

  18. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 18
    jgs
    ND4J Output
    (coordinates, value)

    View Slide

  19. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 19
    jgs
    ND4J DataSet

    View Slide

  20. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 20
    jgs
    DL4J | Our Model
    * Dense layer – a layer that is deeply connected with its preceding layer which means the neurons of the layer are
    connected to every neuron of its preceding layer.

    View Slide

  21. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 21
    jgs
    DL4J | Our Model

    View Slide

  22. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 22
    jgs
    DL4J | Training

    View Slide

  23. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 23
    jgs
    DL4J | Testing

    View Slide

  24. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 24
    jgs
    Results
    The report includes:
    § Confusion matrix entries — in our
    example, there are two true positives (TP)
    and two true negatives (TN) predicted.
    § Accuracy — the measure of all the
    correctly identified cases.
    § Precision — the measure of the correctly
    identified positive cases from all the
    predicted positive cases.
    § F1-score — the harmonic mean of
    precision and recall.

    View Slide

  25. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 25
    jgs
    Confusion matrix

    View Slide

  26. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 26
    jgs
    Accuracy, Precision, Recall, F1-score
    Accuracy = TP + TN / TP + TN + FP + FN
    Precision = TP / TP + FP /
    / predicted
    Recall = TP / TP + FN /
    / real
    F1-score = 2 * Precision * Recall / Precision + Recall

    View Slide

  27. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 27
    jgs
    Harmonic mean
    a = 7
    b = 3
    /
    / if same units (big)
    Arithmetic mean = 7+3 / 2 = 5
    /
    / if diverse units (small)
    Geometric mean = sqrt (7*3) = 4.58
    /
    / ratios of diverse units (smaller)
    Harmonic mean = pow (sqrt (7*3)) / (7+3 / 2) = 21 / 5 = 4.2

    View Slide

  28. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 28
    jgs
    Questions

    View Slide

  29. Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 29
    jgs
    Reference
    § Deeplearning4j Suite Overview
    https://deeplearning4j.konduit.ai
    § Source Code
    https://github.com/javiergs/Medium/blob/main/
    NeuralNetwork/ExampleXORWithDL4J.java

    View Slide

  30. jgs
    SER 594 Software Engineering for Machine Learning
    Javier Gonzalez-Sanchez, Ph.D.
    [email protected]
    Spring 2022
    Copyright. These slides can only be used as study material for the class CSE205 at Arizona State University.
    They cannot be distributed or used for another purpose.

    View Slide