Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Making Sense of Neural Network Training

Making Sense of Neural Network Training

Presented at Pivotal Labs, Tokyo (2018/2/20)

Avatar for John Estropia

John Estropia

February 20, 2018
Tweet

More Decks by John Estropia

Other Decks in Technology

Transcript

  1. @JohnEstropia Developer since 2008 (mostly Mobile apps) Principal Engineer @

    Other hobby projects http://github.com/JohnEstropia/CoreStore
  2. Today's talk My motivation with Machine Learning Rundown of Neural

    Networks in image recognition Some interesting insights
  3. Other issues with traditional OCRs Infinite possibilities of misreads PLKACHUʢPIKACHUʣ

    ϏΨνϡϫʢϐΧνϡ΢ʣ Non-English OCRs are not reliable Pokemon names are in 9 languages
  4. Common Neural Network Creation Flow Front-end: - Training code (usually

    Python) - Loads and processes all training images - Template codes are abundant! (most NNs are set up very similarly)
  5. Training a Neural Network Teaching a kid From here on

    we’ll call Neural Network “N-chan”
  6. Repetitions → Misunderstandings Depending on our training data (or lack

    thereof), N-chan may misunderstand some things “Overfitting” Three
  7. Countering Overfitting: “Dropout” Randomly force N-chan to “forget” a learned

    item Good example: Math Exams memorizing is not necessarily a good thing
  8. Countering Overfitting: Optimizers Tweak the "learning rate" Example: N-chan is

    studying for an exam Read all book chapters then take a mock exam (slow but extensive) Take a mock exam then check the answers (trial and error)
  9. Today's Key Points Neural Networks are better at analyzing unknown

    data than traditional image recognition systems (ex: OCR) Many template projects for training Neural Networks exist (esp. Keras) Training Neural Networks is like teaching a kid