Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Making Sense of Neural Network Training
Search
John Estropia
February 20, 2018
Technology
0
76
Making Sense of Neural Network Training
Presented at Pivotal Labs, Tokyo (2018/2/20)
John Estropia
February 20, 2018
Tweet
Share
More Decks by John Estropia
See All by John Estropia
WWDC Party Intro to CoreStore
johnestropia
0
62
Making an Efficient Deploy Bot
johnestropia
0
440
Creating Keyboard Extensions
johnestropia
0
73
My journey taming Core Data: An intro to the CoreStore library
johnestropia
0
160
Fun with Swift 4 KeyPaths
johnestropia
1
650
Pairs JP Team's iOS Deployment
johnestropia
0
940
Making Slackbots deploy iOS apps for you
johnestropia
2
200
OSSの作法(本題)
johnestropia
0
79
Other Decks in Technology
See All in Technology
KubeCon + CloudNativeCon Japan 2025 Recap Opening & Choose Your Own Adventureシリーズまとめ
mmmatsuda
0
140
Kotlin Coroutine Mechanisms: A Surprisingly Deep Rabbithole
amanda_hinchman
2
100
生成AIで小説を書くためにプロンプトの制約や原則について学ぶ / prompt-engineering-for-ai-fiction
nwiizo
4
2.7k
Tech-Verse 2025 Global CTO Session
lycorptech_jp
PRO
0
760
登壇ネタの見つけ方 / How to find talk topics
pinkumohikan
5
540
AIエージェント最前線! Amazon Bedrock、Amazon Q、そしてMCPを使いこなそう
minorun365
PRO
15
5.4k
Javaで作る RAGを活用した Q&Aアプリケーション
recruitengineers
PRO
1
130
AIのAIによるAIのための出力評価と改善
chocoyama
2
580
MapStore at geOcom 2025: A Year in Review
simboss
PRO
0
100
"サービスチーム" での技術選定 / Making Technology Decisions for the Service Team
kaminashi
1
190
低レイヤを知りたいPHPerのためのCコンパイラ作成入門 完全版 / Building a C Compiler for PHPers Who Want to Dive into Low-Level Programming - Expanded
tomzoh
4
3.3k
PHPでWebブラウザのレンダリングエンジンを実装する
dip_tech
PRO
0
210
Featured
See All Featured
Chrome DevTools: State of the Union 2024 - Debugging React & Beyond
addyosmani
7
710
How to Ace a Technical Interview
jacobian
277
23k
Docker and Python
trallard
44
3.4k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
29
1.8k
A Tale of Four Properties
chriscoyier
160
23k
Writing Fast Ruby
sferik
628
62k
Embracing the Ebb and Flow
colly
86
4.7k
Building Adaptive Systems
keathley
43
2.6k
A better future with KSS
kneath
239
17k
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
panda_program
107
19k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
281
13k
Build your cross-platform service in a week with App Engine
jlugia
231
18k
Transcript
Making Sense of Neural Network Training Pivotal (2018/02/20)
@JohnEstropia Developer since 2008 (mostly Mobile apps) Principal Engineer @
Other hobby projects http://github.com/JohnEstropia/CoreStore
Today's talk My motivation with Machine Learning Rundown of Neural
Networks in image recognition Some interesting insights
Why I started using ML
Who's that Pokemon? PokeRater's image processing
Optical Character Recognition (Tesseract) PokeRater's image processing
Current solution
Current (incomplete) solution
Other issues with traditional OCRs Infinite possibilities of misreads PLKACHUʢPIKACHUʣ
ϏΨνϡϫʢϐΧνϡʣ Non-English OCRs are not reliable Pokemon names are in 9 languages
Neural Networks
"Charmander" Neurons = Cells
"Pikachu" Neurons = Cells
"Pikachu" "Charmander" Neurons = Weights (of features)
Neurons = Weights (of features) *Clip: The Game Theorists (Youtube
channel)
Neurons = Weights (of features) Features extracted using Convolution filters
Training a Neural Network
Common Neural Network Creation Flow Front-end Back-end Model
Common Neural Network Creation Flow Front-end: - Training code (usually
Python) - Loads and processes all training images - Template codes are abundant! (most NNs are set up very similarly)
Common Neural Network Creation Flow Back-end: Computes and builds the
"weights" network
Common Neural Network Creation Flow Model file: What apps will
use Example: Core ML
Insights on Neural Network Concepts
Training a Neural Network Teaching a kid From here on
we’ll call Neural Network “N-chan”
Tons of images (100~ each) "Pikachu" "Charmander"
Training data =~ Flash cards
Teaching = Repetition Takes about a day on decent-sized data
size GPU hardware is recommended
Repetitions → Misunderstandings Depending on our training data (or lack
thereof), N-chan may misunderstand some things “Overfitting” Three
Countering Overfitting: “Dropout” Randomly force N-chan to “forget” a learned
item Good example: Math Exams memorizing is not necessarily a good thing
Countering Overfitting: Optimizers Tweak the "learning rate" Example: N-chan is
studying for an exam Read all book chapters then take a mock exam (slow but extensive) Take a mock exam then check the answers (trial and error)
Countering Overfitting: Optimizers 0% accuracy 100% accuracy loss (noise) loss
(noise) loss (noise) speed = learning rate
Today's Key Points Neural Networks are better at analyzing unknown
data than traditional image recognition systems (ex: OCR) Many template projects for training Neural Networks exist (esp. Keras) Training Neural Networks is like teaching a kid
References https://shibberu.com/2016/04/26/ma-490-deep-learning/ https://www.youtube.com/watch?v=ZCPauvMxV7Q&t=568s https://blog.keras.io/building-powerful-image-classification- models-using-very-little-data.html https://adeshpande3.github.io/A-Beginner%27s-Guide-To- Understanding-Convolutional-Neural-Networks/ http://cs231n.github.io/convolutional-networks/#overview
Thanks!