Outline • ML, DL & Artificial Intelligence • Deep Learning • Computer Vision • Natural Language Understanding and Machine Translation • Other possible applications
Deep Learning • Neural Networks from the 90’s rebranded in 2006+ • « Neuron » is a loose inspiration (not important) • Stacked architecture of modules that compute internal abstract representations from the data • Parameters are tuned from labeled examples
Recent success • 2009: state of the art acoustic model for speech recognition • 2011: state of the art road sign classification • 2012: state of the art object classification • 2013/14: end-to-end speech recognition, object detection • 2014/15: state of the art machine translation, getting closer for Natural Language Understanding in general
ImageNet Challenge 2012 • 1.2M images labeled with 1000 object categories • AlexNet from the deep learning team of U. of Toronto wins with 15% error rate vs 26% for the second (traditional CV pipeline)
GoogLeNet vs Andrej • Andrej Karpathy evaluated human performance (himself): ~5% error rate • "It is clear that humans will soon only be able to outperform state of the art image classification models by use of significant effort, expertise, and time.” source: What I learned from competing against a ConvNet on ImageNet
Applications of RNNs • Natural Language Processing (e.g. Language Modeling, Sentiment Analysis) • Machine Translation (e.g. English to French) • Speech recognition: audio to text • Speech synthesis: text to audio • Biological sequence modeling (DNA, Proteins)
Challenges for NeuroImaging • DL need many labeled images • Few subjects per studies (costly) • Poor labels: low inter-agreement (e.g. autism) • fMRI: low SNR of input data it-self • 3D data: huge GPU memory requirements
Conclusion • ML and DL progress is fast paced • Many applications already in production (e.g. speech, image indexing, translation, face recognition) • Machine Learning is now moving from pattern recognition to higher level reasoning • Lack of high quality labeled data still a limitation for some applications