IMF and tensor interchange system

tokyorubykaigi11

Transcript

1. IMF and tensor interchange system Kenta Murata Tokyo Ruby Kaigi 11 2016.05.28
2. self.inspect • name: Kenta Murata • affiliation: ‣ Ruby committer ‣ Julia Tokyo ‣ Recruit Holdings, MTL
3. self.gems • bigdecimal • daru-td • enumerable-statistics
4. Today's summary • 「Ruby でも機械学習やりたい！」 • Everything Tensor • My approach • Demonstration
5. Ruby でも機械学習やりたい！
6. SciRuby http://gems.sciruby.com
7. OMG
8. OK, use scikit-learn この流れを止めたい
9. Image Processing on Ruby • RMagick • ruby-opencv • hornetseye
10. My case • Image Classification • Deep Learning ‣ Convolutional Neural Network • How to do it by Ruby?
11. Answer • Use Python • Use Pillow for processing images • Use Chainer for building CNN • NumPy bridges Pillow and Chainer
12. Use Python !! この流れを止めたい
13. What are missing?
14. What are missing? • Processing images ‣ Loading / saving ‣ Resizing ‣ Horizontal flipping ‣ Average image calculation ‣ Multi-dimensional array conversion RMagick ruby-opencv hornetseye
15. What are missing? • Processing images ‣ Loading / saving ‣ Resizing ‣ Horizontal flipping ‣ Average image calculation ‣ Multi-dimensional array conversion hornetseye
 (MultiArray) ruby-opencv
 (CvMat) NArray (Numo) NMatrix
16. What are missing? • Building CNN ‣ Tensor calculation ‣ Convolutional filter calculation ‣ GPGPU ‣ Neural network builder ‣ Gradient descent optimizer hornetseye
 (MultiArray) ruby-opencv
 (CvMat) NArray (Numo) NMatrix
17. What are missing? • Building CNN ‣ Tensor calculation ‣ Convolutional filter calculation ‣ GPGPU ‣ Neural network builder ‣ Gradient descent optimizer
18. What are missing? • Building CNN ‣ Tensor calculation ‣ Convolutional filter calculation ‣ GPGPU ‣ Neural network builder ‣ Gradient descent optimizer
19. Library relationship Image RMagick ruby-opencv hornetseye Tensor NArray (Numo) NMatrix MultiArray
 (hornetseye) CvMat
 (ruby-opencv) Others GPGPU DL OpenCV ML
 (ruby-opencv) decisiontree liblinear gsl_matrix
 (rb-gsl)
20. A lot of multi-dimensional array data types • RMagick, OpenCV, multiarray, ... • NArray, NMatrix, StatSample, ... • gsl, tensorflow, ... • Julia, Python, R, ...
21. Everything tensor • 1-tensor (vector) ‣ e.g. 1ch sound, text, time series scalar data • 2-tensor (matrix) ‣ e.g. 2ch sound, grayscale image, time series vector data • 3-tensor ‣ e.g. RGB image, grayscale animation • 4-tensor ‣ e.g. RGB animation
22. Image Animation Sound Tensor Function Statistical
 Model Neural
 Network
23. What is best for me? • RMagick • ruby-opencv • hornetseye
24. Different gems have different types for the same purpose • RMagick: Magick::Image • ruby-opencv: CvMat • horneteyes: MultiArray
25. What data type is best? • RMagick: Magick::Image • ruby-opencv: CvMat • horneteyes: MultiArray
26. Generic Tensor • 4 parameters ‣ component type (size) ‣ dimension ‣ shape ‣ steps / strides
27. A B G R component_size shape[2] = pixel_channels A B G R A B G R A B G R shape[1] = image_width ... ... ... ... shape[0] = im age_height shape = (image_height, image_width, pixel_channels)
28. B G R component_size shape[2] = pixel_channels B G R B G R B G R shape[1] = image_width ... ... ... shape[0] = im age_height shape = (pixel_channels, image_width, image_height)
29. 0 B G R shape[2] = pixel_channels = 3 0 B G R 0 B G R 0 B G R shape[1] = image_width = 17 ... ... ... ... shape = (19, 17, 3) steps[1] = 4 shape[0] = im age_height = 19 0 0 0 0 steps = (18, 4) steps[0] = 18
30. Generic Tensor • 4 parameters ‣ component type (size) ‣ dimension ‣ shape ‣ steps / strides
31. Image Animation Sound Tensor Function Statistical
 Model Neural
 Network
32. With Standard Tensor Representation • We can separate to develop tensor serializer and tensor operators ‣ Image, Audio, DB I/O ‣ Tensor transformation • and easy to integrate with each other
33. I tried to make it • IMF -- Image I/O • NumBuffer -- Tensor data type
34. IMF • Image Manipulation Framework • https://github.com/mrkn/IMF • I've aimed it to reproduce PIL • Pluggable design
35. Current status Detect Load Save JPEG ◦ ◦ PNG ◦ ◦ △ GIF ◦ WEBP ◦
36. NumBuffer • Tensor allocation and manipulation • Data type conversion • I/F for Ruby and C • I want support both dense and sparse tensors • I will release a gem until RubyKaigi
37. Demonstration • Integrating with tensorflow
38. tf-waifu2x • Waifu2x on tensorflow • It works, but it's incomplete
39. IMF tf-wife2x tensorflow MPBE
    JNBHF XSBQ
    X
 /VN#V⒎FS $POWFSU
    UP
    qPBU
    UFOTPS
 GPS
    UFOTPSqPX
    GSPN
    /VN#V⒎FS MPBE
    UFOTPSqPX
 NPEFM BQQMZ
    NPEFM $POWFSU
    UP
 VJOU
    /VN#V⒎FS
 GPS
    PVUQVU NBLF
    JNBHF
 GSPN
    /VN#V⒎FS TBWF
    JNBHF
40. waifu2x 7-step
 CNN Magnify x2
 Nearest
 Neighbor Pixel value
 adjustment 7-pixel padding insertion ↑ prepared ↑ omitted ↑ incomplete ↑ OK https://github.com/mrkn/tf-waifu2x/blob/master/convert_waifu2x.py
41. Demonstration
42. IMF tf-wife2x tensorflow MPBE
    JNBHF XSBQ
    X
 /VN#V⒎FS $POWFSU
    UP
    qPBU
    UFOTPS
 GPS
    UFOTPSqPX
    GSPN
    /VN#V⒎FS MPBE
    UFOTPSqPX
 NPEFM BQQMZ
    NPEFM $POWFSU
    UP
 VJOU
    /VN#V⒎FS
 GPS
    PVUQVU NBLF
    JNBHF
 GSPN
    /VN#V⒎FS TBWF
    JNBHF
43. Machine Learning on Ruby • We need standard tensor interchanging system like numpy • It's enough to provide the common data type for dense tensors ‣ I can't select one library
44. If you're interested in • SciRuby-jp ‣ https://groups.google.com/forum/#! forum/sciruby-jp ‣ Japanese de OK • Slack ‣ https://sciruby-jp.herokuapp.com