Data Augmentation and Transfer Learning For Convolutional Neural Network

Convolutional Neural Network changed the world of computer vision. Eventhough It's really powerful, It needs a lot of data to be used efficiently. Data Augmentation and Transfer Learning are few ways to handle this problem. By creating new data from our past data, and use a pre-trained problem we can make a powerful CNN model even when our data are tiny.

Transcript

1. Data Augmentation and Transfer Learning for Convolutional Neural Network By Ersa Christian
2. What is Neural Network? Convolutional Neural Network?
3. Who use it?
4. How does it work?
5. Just that? Of course not, many other things are used: - Other layers -> Pooling, ReLu, Fully-connected, Dropout - Loss Function -> cross-entropy, MSE, MAE, cosine similarity - Optimizer -> SGD, RMV, Adagrad But, we will not talk about it right now
7. What are its weaknesses? 1. The computional cost is huge Will take forever to train 1 network. 1. Need A TONS OF DATA Without it, Prone to overfitting.
8. How to solve that problems? 1. A. Good CPU/GPU (another reason to buy GPU, Woo~hoo) B. Use a pre-trained network (Transfer Learning) 1. A. Get more data B. Make new one yourself (Data Augmentation)
9. Transfer Learning? The idea is use a pre-trained network as an initialization or feature extractor for task we want it to do. Pre-trained network previously trained with very big dataset with a lot of classes Many people has published their network for the benefit of others Actually help too, if your dataset is too small
10. How and when to do it? Common rule of thumbs: 1. New dataset is small and similar to original dataset -> Only train the linear classifier. 2. New dataset is big and similar to original dataset -> fine tune through full network 3. New dataset is small and very different to original dataset -> cut some layers on the top, and train a SVM classifier there 4. New dataset is big and very different to original dataset -> train from scratch, or use the weight as initialization
11. Any Pre-Trained Network we can use? - VGG16 - VGG19 - XCEPTION - Inceptionv3 - ResNet50 Trained using ImageNet datasets If you use caffe, you can explore caffe zoo
12. Data Augmentation? How to do it? From each image that we have, we simply: - Rotate it - Crop it - Distort it - Put noise on it - Etc etc And give it the same label as the original
13. Can you give us some example? How many data augmentation should we do? As many as your computer can handle
14. Wait, There’s more... Simple program demo: Create a network to classify images of dog and cat using inceptionV3 pre-trained network I use Keras -> high-level neural network library that capable to run on top of tensorflow and theano