Deep Learning with KotlinDL

I think that many of you know the Keras library, which runs on top of Tensorflow. Tensorflow itself has stable C API, and therefore there are several high-level deep learning frameworks in other languages ​​inspired by Keras, such as TFJS, Keras.NET, Tensorflow in Go.

Today I want to introduce you to a Keras-like machine learning framework written in Kotlin.

Currently, the KotlinDL framework can boast that this is the only way to construct and train complex neural networks on JVM, such as VGG, ResNet, or MobileNet. There is also support for transfer learning for the popular models trained in Keras (or available in Keras. applications).

For image preprocessing, several functions are available that allow you to avoid complex and routine work on the JVM.

Finally, I’ll show you how you can build, train, and use a convolutional neural network (CNN) entirely in Kotlin with the help of KotlinDL, and we discuss how to use KotlinDL models in production.

Transcript

1. Deep Learning with KotlinDL
   Alexey Zinovyev, ML Engineer, Apache Ignite PMC
   JetBrains
   

   View Slide

2. Bio
   1. Java & Kotlin developer
   2. Distributed ML enthusiast
   3. Apache Ignite PMC
   4. TensorFlow Contributor
   5. ML engineer at JetBrains
   6. Happy father and husband
   7. https://github.com/zaleslaw
   8. https://twitter.com/zaleslaw
   

   View Slide

3. Motivation
   1. Kotlin took a course to become a convenient language for data science
   2. No modern data science without Neural Networks
   3. All deep learning frameworks are good enough at image recognition
   4. Convolutional neural networks (CNNs) are the gold standard for image
   recognition
   5. Training, Transfer Learning, and Inference are now available for different CNN
   architectures on Kotlin with KotlinDL library
   

   View Slide

4. Kotlin DL = Keras - Python
   

   View Slide

5. Kotlin DL = Keras - Python
   

   View Slide

6. How it works?
   

   View Slide

7. How it works?
   

   View Slide

8. How it works?
   

   View Slide

9. How it works?
   

   View Slide

10. The possible future
    

    View Slide

11. Kotlin for Data Science
    

    View Slide

12. Kotlin Data Science libraries
    1. Multik - Multidimensional array library for Kotlin
    2. Kotlin for Apache Spark
    3. KMath - multiplatform math library
    4. DataFrame library is in progress
    5. Lets-Plot for visualisation
    

    View Slide

13. Jupyter Kotlin kernel
    

    View Slide

14. Keras ideology
    

    View Slide

15. The Keras API philosophy
    Progressive disclosure of complexity
    

    View Slide

16. Model building: from simple to arbitrarily flexible
    

    View Slide

17. Model building: from simple to arbitrarily flexible
    

    View Slide

18. Model training: from simple to arbitrarily flexible
    

    View Slide

19. Model training: from simple to arbitrarily flexible
    

    View Slide

20. Low-level Tensor API
    1. Java API 1.15
    2. Low-level LeNet-5 on Kotlin
    3. Java API 2.x
    4. Low-level LeNet-5 examples on Java
    

    View Slide

21. Kotlin DL features
    

    View Slide

22. 0.1 release features
    1. Sequential API
    2. VGG-like Sequential models for CV domain
    3. Training on TensorFlow
    4. Transfer Learning for VGG’16 and VGG’19 models exported from Keras
    5. Loading Keras Sequential models from JSON config + weights in h5 file format
    6. Model Export to a few formats
    

    View Slide

23. 0.2 release features: ModelZoo
    Model ZOO with the following models
    1. ResNet50
    2. ResNet101
    3. ResNet152
    4. ResNet50v2
    5. ResNet101v2
    6. ResNet152v2
    7. MobileNet
    8. MobileNetv2
    

    View Slide

24. 0.2 release features: Layers
    A lot of new layers required for ResNet and MobileNet architectures
    1. BatchNorm
    2. ActivationLayer
    3. DepthwiseConv2D
    4. SeparableConv2D
    5. Merge (Add, Subtract, Multiply, Average, Concatenate, Maximum, Minimum)
    6. GlobalAvgPool2D
    7. Cropping2D
    8. Reshape
    9. ZeroPadding2D
    

    View Slide

25. 0.2 release features: embedded datasets
    1. Mnist
    2. FashionMnist
    3. Cifar’10
    4. Dogs vs. Cats (catdogs) dataset from Kaggle
    5. Sub-sample from Dogs vs. Cats dataset (1000 images per class)
    

    View Slide

26. 0.2 release features: Image Preprocessing DSL
    1. Load
    2. Crop
    3. Resize
    4. Rotate
    5. Rescale
    6. Save
    

    View Slide

27. 0.2 release features: Datasets
    1. OnHeapDataset
    2. OnFlyDataset
    3. CustomDataset
    

    View Slide

28. KotlinDL Limitations
    1. Now useful for Image Recognition task and Regression ML
    2. Limited number of layers is supported
    3. No Android support
    

    View Slide

29. KotlinDL Roadmap
    1. New models in the ModelZoo: Inception, DenseNet, EfficientNet
    2. Rich Dataset API
    3. GPU settings
    4. Regularization for layers will be added
    5. New metrics framework
    6. Conversion to TFLite (for mobile devices)
    7. ONNX support
    8. ML algorithms
    9. PyTorch Runtime for training
    

    View Slide

30. Demo
    

    View Slide

31. Useful links
    1. https://github.com/JetBrains/KotlinDL
    2. https://kotlinlang.org/docs/data-science-overview.html#kotlin-libraries
    3. #deeplearning channel on Kotlin slack (join it, if you are not yet)
    4. Feel free to join discussions on Github
    5. Follow @zaleslaw and @KotlinForData on Twitter
    

    View Slide

32. Use KotlinDL!
    

    View Slide