Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Making Sense of Neural Network Training
Search
John Estropia
February 20, 2018
Technology
0
76
Making Sense of Neural Network Training
Presented at Pivotal Labs, Tokyo (2018/2/20)
John Estropia
February 20, 2018
Tweet
Share
More Decks by John Estropia
See All by John Estropia
WWDC Party Intro to CoreStore
johnestropia
0
62
Making an Efficient Deploy Bot
johnestropia
0
440
Creating Keyboard Extensions
johnestropia
0
73
My journey taming Core Data: An intro to the CoreStore library
johnestropia
0
160
Fun with Swift 4 KeyPaths
johnestropia
1
650
Pairs JP Team's iOS Deployment
johnestropia
0
940
Making Slackbots deploy iOS apps for you
johnestropia
2
200
OSSの作法(本題)
johnestropia
0
79
Other Decks in Technology
See All in Technology
mrubyと micro-ROSが繋ぐロボットの世界
kishima
2
380
asken AI勉強会(Android)
tadashi_sato
0
140
AIとともに進化するエンジニアリング / Engineering-Evolving-with-AI_final.pdf
lycorptech_jp
PRO
0
140
FOSS4G 2025 KANSAI QGISで点群データをいろいろしてみた
kou_kita
0
220
Core Audio tapを使ったリアルタイム音声処理のお話
yuta0306
0
150
タイミーのデータモデリング事例と今後のチャレンジ
ttccddtoki
4
1.4k
Connect 100+を支える技術
kanyamaguc
0
150
rubygem開発で鍛える設計力
joker1007
2
270
強化されたAmazon Location Serviceによる新機能と開発者体験
dayjournal
3
250
AWS テクニカルサポートとエンドカスタマーの中間地点から見えるより良いサポートの活用方法
kazzpapa3
2
610
解析の定理証明実践@Lean 4
dec9ue
1
200
さくらのIaaS基盤のモニタリングとOpenTelemetry/OSC Hokkaido 2025
fujiwara3
2
250
Featured
See All Featured
How to Think Like a Performance Engineer
csswizardry
24
1.7k
What's in a price? How to price your products and services
michaelherold
246
12k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
46
9.6k
Fantastic passwords and where to find them - at NoRuKo
philnash
51
3.3k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
252
21k
Done Done
chrislema
184
16k
Making Projects Easy
brettharned
116
6.3k
Bootstrapping a Software Product
garrettdimon
PRO
307
110k
Statistics for Hackers
jakevdp
799
220k
How to Ace a Technical Interview
jacobian
277
23k
Unsuck your backbone
ammeep
671
58k
Principles of Awesome APIs and How to Build Them.
keavy
126
17k
Transcript
Making Sense of Neural Network Training Pivotal (2018/02/20)
@JohnEstropia Developer since 2008 (mostly Mobile apps) Principal Engineer @
Other hobby projects http://github.com/JohnEstropia/CoreStore
Today's talk My motivation with Machine Learning Rundown of Neural
Networks in image recognition Some interesting insights
Why I started using ML
Who's that Pokemon? PokeRater's image processing
Optical Character Recognition (Tesseract) PokeRater's image processing
Current solution
Current (incomplete) solution
Other issues with traditional OCRs Infinite possibilities of misreads PLKACHUʢPIKACHUʣ
ϏΨνϡϫʢϐΧνϡʣ Non-English OCRs are not reliable Pokemon names are in 9 languages
Neural Networks
"Charmander" Neurons = Cells
"Pikachu" Neurons = Cells
"Pikachu" "Charmander" Neurons = Weights (of features)
Neurons = Weights (of features) *Clip: The Game Theorists (Youtube
channel)
Neurons = Weights (of features) Features extracted using Convolution filters
Training a Neural Network
Common Neural Network Creation Flow Front-end Back-end Model
Common Neural Network Creation Flow Front-end: - Training code (usually
Python) - Loads and processes all training images - Template codes are abundant! (most NNs are set up very similarly)
Common Neural Network Creation Flow Back-end: Computes and builds the
"weights" network
Common Neural Network Creation Flow Model file: What apps will
use Example: Core ML
Insights on Neural Network Concepts
Training a Neural Network Teaching a kid From here on
we’ll call Neural Network “N-chan”
Tons of images (100~ each) "Pikachu" "Charmander"
Training data =~ Flash cards
Teaching = Repetition Takes about a day on decent-sized data
size GPU hardware is recommended
Repetitions → Misunderstandings Depending on our training data (or lack
thereof), N-chan may misunderstand some things “Overfitting” Three
Countering Overfitting: “Dropout” Randomly force N-chan to “forget” a learned
item Good example: Math Exams memorizing is not necessarily a good thing
Countering Overfitting: Optimizers Tweak the "learning rate" Example: N-chan is
studying for an exam Read all book chapters then take a mock exam (slow but extensive) Take a mock exam then check the answers (trial and error)
Countering Overfitting: Optimizers 0% accuracy 100% accuracy loss (noise) loss
(noise) loss (noise) speed = learning rate
Today's Key Points Neural Networks are better at analyzing unknown
data than traditional image recognition systems (ex: OCR) Many template projects for training Neural Networks exist (esp. Keras) Training Neural Networks is like teaching a kid
References https://shibberu.com/2016/04/26/ma-490-deep-learning/ https://www.youtube.com/watch?v=ZCPauvMxV7Q&t=568s https://blog.keras.io/building-powerful-image-classification- models-using-very-little-data.html https://adeshpande3.github.io/A-Beginner%27s-Guide-To- Understanding-Convolutional-Neural-Networks/ http://cs231n.github.io/convolutional-networks/#overview
Thanks!