Share
Embed
Deep Learning in Neural Networks
Emerging Technologies for Higher Education Post COVID-19 Webinar organized by CJU
Shadab Hussain
May 17, 2020
Transcript
34 slides
-
Shadab Hussain https://shadabhussain.com 3 Agenda • Introduction to Deep Learning
• Artificial Neural Networks • Activation function • CNN • Confusion matrix • RNN • LSTM • Applications • Tensor-flow playground • Deep Learning Frameworks • Learning ML/DL • Research Labs and Funding • AI Fellowships -
Shadab Hussain https://shadabhussain.com 5 The neuron collects signals from input
channels named dendrites, processes information in its nucleus, and then generates an output in a long thin branch called axon. NUCLEUS AXON DENDRITES X1 X2 X3 NEURON W1 W2 W3 ANN -
Shadab Hussain https://shadabhussain.com 6 INPUTS/INDEPENDENT VARIABLES 1 F b =
(1 1 + 2 2 + 3 3 + ) 2 3 Activation Function -
Shadab Hussain https://shadabhussain.com 10 Convolutional Neural Network WHAT ARE CONVOLUTIONAL
NEURAL NETWORKS (CNNS) AND HOW DO THEY LEARN? T-SHIRT/TOP TROUSER PULLOVER DRESS COAT SANDAL SHIRT SNEAKER BAG ANKLE BOOT KERNELS/ FEATURE DETECTORS POOLING FILTERS CONVOLUTIONAL LAYER POOLING LAYER (DOWNSAMPLING) CONVOLUTION POOLING FLATTENING -
Shadab Hussain https://shadabhussain.com 12 CNN 0 1 1 0 1
1 0 0 0 1 1 1 0 0 0 0 1 1 1 0 0 0 1 0 1 1 0 0 0 1 0 0 1 0 FEATURE DETECTOR IMAGE 1 1 1 3 1 1 2 3 1 FEATURE MAP Live Convolution: http://setosa.io/ev/image-kernels/ -
Shadab Hussain https://shadabhussain.com 13 Applying ReLU T-SHIRT/TOP TROUSER PULLOVER DRESS
COAT SANDAL SHIRT SNEAKER BAG ANKLE BOOT KERNELS/ FEATURE DETECTORS POOLING FILTERS CONVOLUTIONAL LAYER POOLING LAYER (DOWNSAMPLING) CONVOLUTION POOLING FLATTENING -
Shadab Hussain https://shadabhussain.com 14 Pooling • Live illustration : http://scs.ryerson.ca/~aharley/vis/conv/flat.html
1 1 3 4 3 6 2 8 3 9 1 0 1 3 3 4 6 8 9 4 MAX POOLING 2x2 STRIDE = 2 6 8 9 4 FLATTENING -
Shadab Hussain https://shadabhussain.com 15 HOW TO IMPROVE CNN KERNELS/ FEATURE
DETECTORS 64 INSTEAD OF 32 -
Shadab Hussain https://shadabhussain.com 16 Confusion Matrix PREDICTIONS TRUE CLASS TRUE
+ TRUE - + - + - FALSE + FALSE - TYPE II ERROR TYPE I ERROR -
Shadab Hussain https://shadabhussain.com 19 LSTM NETWORKS RNN PERFORMS WELL SINCE
THE GAP BETWEEN THE PREDICTION “GREEN” AND THE NECESSARY CONTEXT INFORMATION “TREE” IS SMALL 0 1 2 3 ℎ0 ℎ1 ℎ2 ℎ3 The tree color is “green” 0 1 2 3 ℎ0 ℎ1 ℎ2 ℎ3 I live in Quebec in Northern Canada……where I live, the weather is generally “cold” most of the year +1 +2 ℎ ℎ+1 ℎ+2 RNN PERFORMS POORLY WHEN THE GAP BETWEEN THE PREDICTION “COLD” AND THE NECESSARY CONTEXT INFORMATION “CANADA” IS LARGE -
Shadab Hussain https://shadabhussain.com 20 LSTM Intuition VANILLA RECURRENT NEURAL NETWORK
LONG SHORT TERM MEMORY NETWORK THIS HORIZONTAL LINE (MEMORY) OR CELL STATE ENABLES LSTM TO REMEMBER VERY OLD INFORMTION -
Shadab Hussain https://shadabhussain.com 30 Learning ML/DL • Faculty Awards to
Support Machine Learning Courses, Diversity, and Inclusion • Deep Learning Specialization • Fast.ai • Deep Learning A-Z • Deep Learning Nanodegree Program • MIT Deep Learning and Self Driving Cars -
Shadab Hussain https://shadabhussain.com 31 Research Labs & Funding • MARIE
SKŁODOWSKA-CURIE ACTIONS • NATIONAL CENTER FOR SUPERCOMPUTING APPLICATIONS • Salesforce Research Deep Learning Grant • AWS Machine Learning Research Awards • AI and deep learning Skilling and Research • Berkeley Artificial Intelligence Research (BAIR) • USC Information Sciences Institute • USC Center for AI in Society • NASA Jet Propulsion Laboratory • MIT CSAIL • The Alan Turing Institute • Fraunhofer Heinrich Hertz Institute • Oxford ML Research Reference: https://medium.com/swlh/what-are-a-few-ai-research-labs-on-the-west-coast-c0434996ad64 https://www.mariecuriealumni.eu/newsletter/10-websites-you-need-know-european-funding-opportunities -
Shadab Hussain https://shadabhussain.com 32 AI Fellowship Programs OpenAI 2020 Spring
Scholars Google AI Residency Program Microsoft AI Residency Program Facebook AI Residency For more fellowships https://github.com/dangkhoasdc/awesome-ai-residency -
Shadab Hussain https://shadabhussain.com 33 • https://commons.wikimedia.org/wiki/File:Logistic-curve.svg • https://colah.github.io/posts/2015-08-Understanding-LSTMs/ • https://fr.wikipedia.org/wiki/Fichier:Recurrent_neural_network_unfold.svg
• http://karpathy.github.io/2015/05/21/rnn-effectiveness/ • https://commons.wikimedia.org/wiki/File:RecurrentLayerNeuralNetwork_english.png • https://fr.m.wikipedia.org/wiki/Fichier:MultiLayerNeuralNetworkBigger_english.png • https://commons.wikimedia.org/wiki/File:Artificial_neural_network.svg • https://commons.wikimedia.org/wiki/File:Hyperbolic_Tangent.svg • https://commons.wikimedia.org/wiki/File:ReLU_and_Nonnegative_Soft_Thresholding_Functions.svg • https://commons.wikimedia.org/wiki/File:Sigmoid-function.svg • https://developer.nvidia.com/deep-learning-frameworks • https://towardsdatascience.com/top-10-best-deep-learning-frameworks-in-2019-5ccb90ea6de • https://www.predictiveanalyticstoday.com/deep-learning-software-libraries/ References