Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Deep Learning Introduction with KotlinDL

Deep Learning Introduction with KotlinDL

Alexey Zinoviev

February 23, 2021

More Decks by Alexey Zinoviev

Other Decks in Education


  1. Bio 1. Java & Kotlin developer 2. Distributed ML enthusiast

    3. Apache Ignite PMC 4. TensorFlow Contributor 5. ML engineer at JetBrains 6. Happy father and husband 7. https://github.com/zaleslaw
  2. Motivation 1. Kotlin took a course to become a convenient

    language for data science 2. No modern data science without Neural Networks 3. All deep learning frameworks are good enough at image recognition 4. Convolutional neural networks (CNNs) are the gold standard for image recognition 5. Training, Transfer Learning, and Inference are now available for different CNN architectures on Kotlin with KotlinDL library
  3. Agenda 1. Neural Network Intro 2. Deep Learning 3. Required

    and optional math knowledge 4. Primitives or building blocks a. Activation Functions b. Loss Functions c. Initializers d. Optimizers e. Layers 5. 5 Major Scientific breakthroughs in DL 6. Kotlin DL Demo
  4. Some basic terms 1. Model 2. Inference 3. Training 4.

    Transfer Learning 5. Evaluation 6. Train/validation/test datasets
  5. Need to keep in mind • Some trigonometry, exponentials and

    logarithms; • Linear Algebra: vectors, vector space; • Linear Algebra: inverse and transpose matrices, matrix decomposition, eigenvectors, Kronecker-Capelli’s theorem; • Mathematical Analysis: continuous, monotonous, differentiable functions; • Mathematical Analysis: derivative, partial derivative, Jacobian; • Methods of one-dimensional and multidimensional optimization; • Gradient Descent and all its variations; • Optimization methods and convex analysis will not be superfluous in your luggage
  6. Loss Functions • Each loss function could be reused as

    metric • Should be differentiable • Not every metric could be a loss function ( metrics could have not the derivative ) • Loss function could be very complex • Are different for regression and classification tasks
  7. Activation functions • Activation functions change the outputs coming out

    of each layer of a neural network. • Required to add non-linearity • Should have a derivative (to be used in backward propagation)
  8. KotlinDL Limitations 1. Now useful for Image Recognition task and

    Regression ML 2. Limited number of layers is supported 3. Tiny number of preprocessing methods 4. Only VGG-like architectures are supported 5. No Android support
  9. KotlinDL Roadmap 1. New models: Inception, ResNet, DenseNet 2. Rich

    Dataset API 3. GPU settings 4. Maven Central Availability 5. Functional API 6. New layers: BatchNorm, Add, Concatenate, DepthwiseConv2d 7. Regularization for layers will be added 8. New metrics framework 9. Conversion to TFLite (for mobile devices) 10. ONNX support 11. ML algorithms
  10. Useful links 1. https://github.com/JetBrains/KotlinDL 2. https://kotlinlang.org/docs/data-science-overview.html#kotlin-libraries 3. #deeplearning channel on

    Kotlin slack (join it, if you are not yet) 4. Feel free to join discussions on Github 5. Follow @zaleslaw and @KotlinForData on Twitter
  11. 5 Major Scientific breakthroughs 1. Non-linearity in early 90’s (could

    solve new problems) 2. ReLU and simplest and cheap non-linearity (could converge on new tasks) 3. Batch Normalization (could help convergence and give more acceleration) 4. Xe Initialization (solves vanishing/exploding gradient problem) 5. Adam optimizer (give more performance)