Deep Learning with KotlinDL

I think that many of you know the Keras library, which runs on top of Tensorflow. Tensorflow itself has stable C API, and therefore there are several high-level deep learning frameworks in other languages ​​inspired by Keras, such as TFJS, Keras.NET, Tensorflow in Go.

Today I want to introduce you to a Keras-like machine learning framework written in Kotlin.

Currently, the KotlinDL framework can boast that this is the only way to construct and train complex neural networks on JVM, such as VGG, ResNet, or MobileNet. There is also support for transfer learning for the popular models trained in Keras (or available in Keras. applications).

For image preprocessing, several functions are available that allow you to avoid complex and routine work on the JVM.

Finally, I’ll show you how you can build, train, and use a convolutional neural network (CNN) entirely in Kotlin with the help of KotlinDL, and we discuss how to use KotlinDL models in production.

Transcript

1. Deep Learning with KotlinDL Alexey Zinovyev, ML Engineer, Apache Ignite PMC JetBrains
2. Bio 1. Java & Kotlin developer 2. Distributed ML enthusiast 3. Apache Ignite PMC 4. TensorFlow Contributor 5. ML engineer at JetBrains 6. Happy father and husband 7. https://github.com/zaleslaw 8. https://twitter.com/zaleslaw
3. Motivation 1. Kotlin took a course to become a convenient language for data science 2. No modern data science without Neural Networks 3. All deep learning frameworks are good enough at image recognition 4. Convolutional neural networks (CNNs) are the gold standard for image recognition 5. Training, Transfer Learning, and Inference are now available for different CNN architectures on Kotlin with KotlinDL library
4. Kotlin DL = Keras - Python
5. Kotlin DL = Keras - Python
6. How it works?
7. How it works?
8. How it works?
9. How it works?
10. The possible future
11. Kotlin for Data Science
12. Kotlin Data Science libraries 1. Multik - Multidimensional array library for Kotlin 2. Kotlin for Apache Spark 3. KMath - multiplatform math library 4. DataFrame library is in progress 5. Lets-Plot for visualisation
13. Jupyter Kotlin kernel
14. Keras ideology
15. The Keras API philosophy Progressive disclosure of complexity
16. Model building: from simple to arbitrarily flexible
17. Model building: from simple to arbitrarily flexible
18. Model training: from simple to arbitrarily flexible
19. Model training: from simple to arbitrarily flexible
20. Low-level Tensor API 1. Java API 1.15 2. Low-level LeNet-5 on Kotlin 3. Java API 2.x 4. Low-level LeNet-5 examples on Java
21. Kotlin DL features
22. 0.1 release features 1. Sequential API 2. VGG-like Sequential models for CV domain 3. Training on TensorFlow 4. Transfer Learning for VGG’16 and VGG’19 models exported from Keras 5. Loading Keras Sequential models from JSON config + weights in h5 file format 6. Model Export to a few formats
23. 0.2 release features: ModelZoo Model ZOO with the following models 1. ResNet50 2. ResNet101 3. ResNet152 4. ResNet50v2 5. ResNet101v2 6. ResNet152v2 7. MobileNet 8. MobileNetv2
24. 0.2 release features: Layers A lot of new layers required for ResNet and MobileNet architectures 1. BatchNorm 2. ActivationLayer 3. DepthwiseConv2D 4. SeparableConv2D 5. Merge (Add, Subtract, Multiply, Average, Concatenate, Maximum, Minimum) 6. GlobalAvgPool2D 7. Cropping2D 8. Reshape 9. ZeroPadding2D
25. 0.2 release features: embedded datasets 1. Mnist 2. FashionMnist 3. Cifar’10 4. Dogs vs. Cats (catdogs) dataset from Kaggle 5. Sub-sample from Dogs vs. Cats dataset (1000 images per class)
26. 0.2 release features: Image Preprocessing DSL 1. Load 2. Crop 3. Resize 4. Rotate 5. Rescale 6. Save
27. 0.2 release features: Datasets 1. OnHeapDataset 2. OnFlyDataset 3. CustomDataset
28. KotlinDL Limitations 1. Now useful for Image Recognition task and Regression ML 2. Limited number of layers is supported 3. No Android support
29. KotlinDL Roadmap 1. New models in the ModelZoo: Inception, DenseNet, EfficientNet 2. Rich Dataset API 3. GPU settings 4. Regularization for layers will be added 5. New metrics framework 6. Conversion to TFLite (for mobile devices) 7. ONNX support 8. ML algorithms 9. PyTorch Runtime for training
30. Demo
31. Useful links 1. https://github.com/JetBrains/KotlinDL 2. https://kotlinlang.org/docs/data-science-overview.html#kotlin-libraries 3. #deeplearning channel on Kotlin slack (join it, if you are not yet) 4. Feel free to join discussions on Github 5. Follow @zaleslaw and @KotlinForData on Twitter
32. Use KotlinDL!