Making Sense of Neural Network Training

Presented at Pivotal Labs, Tokyo (2018/2/20)

Transcript

1. Making Sense of Neural Network Training Pivotal (2018/02/20)
2. @JohnEstropia Developer since 2008 (mostly Mobile apps) Principal Engineer @ Other hobby projects http://github.com/JohnEstropia/CoreStore
3. Today's talk My motivation with Machine Learning Rundown of Neural Networks in image recognition Some interesting insights
4. Why I started using ML
5. Who's that Pokemon? PokeRater's image processing
6. Optical Character Recognition (Tesseract) PokeRater's image processing
7. Current solution
8. Current (incomplete) solution
9. Other issues with traditional OCRs Infinite possibilities of misreads PLKACHUʢPIKACHUʣ ϏΨνϡϫʢϐΧνϡ΢ʣ Non-English OCRs are not reliable Pokemon names are in 9 languages
10. Neural Networks
11. "Charmander" Neurons = Cells
12. "Pikachu" Neurons = Cells
13. "Pikachu" "Charmander" Neurons = Weights (of features)
14. Neurons = Weights (of features) *Clip: The Game Theorists (Youtube channel)
15. Neurons = Weights (of features) Features extracted using Convolution filters
16. Training a Neural Network
17. Common Neural Network Creation Flow Front-end Back-end Model
18. Common Neural Network Creation Flow Front-end: - Training code (usually Python) - Loads and processes all training images - Template codes are abundant! (most NNs are set up very similarly)
19. Common Neural Network Creation Flow Back-end: Computes and builds the "weights" network
20. Common Neural Network Creation Flow Model file: What apps will use Example: Core ML
21. Insights on Neural Network Concepts
22. Training a Neural Network Teaching a kid From here on we’ll call Neural Network “N-chan”
23. Tons of images (100~ each) "Pikachu" "Charmander"
24. Training data =~ Flash cards
25. Teaching = Repetition Takes about a day on decent-sized data size GPU hardware is recommended
26. Repetitions → Misunderstandings Depending on our training data (or lack thereof), N-chan may misunderstand some things “Overfitting” Three
27. Countering Overfitting: “Dropout” Randomly force N-chan to “forget” a learned item Good example: Math Exams memorizing is not necessarily a good thing
28. Countering Overfitting: Optimizers Tweak the "learning rate" Example: N-chan is studying for an exam Read all book chapters then take a mock exam (slow but extensive) Take a mock exam then check the answers (trial and error)
29. Countering Overfitting: Optimizers 0% accuracy 100% accuracy loss (noise) loss (noise) loss (noise) speed = learning rate
30. Today's Key Points Neural Networks are better at analyzing unknown data than traditional image recognition systems (ex: OCR) Many template projects for training Neural Networks exist (esp. Keras) Training Neural Networks is like teaching a kid
31. References https://shibberu.com/2016/04/26/ma-490-deep-learning/ https://www.youtube.com/watch?v=ZCPauvMxV7Q&t=568s https://blog.keras.io/building-powerful-image-classification- models-using-very-little-data.html https://adeshpande3.github.io/A-Beginner%27s-Guide-To- Understanding-Convolutional-Neural-Networks/ http://cs231n.github.io/convolutional-networks/#overview
32. Thanks!